[ { "path": ".gitattributes", "content": "* text=auto\n\n*.cs text diff=csharp\n*.fs text diff=csharp\n*.fsi text diff=csharp\n*.fsx text diff=csharp\n\n*.sln text eol=crlf\n*.csproj text\n*.fsproj text\n*.config text\n*.json text\n\n*.txt text\n*.html text linguist-documentation\n*.css text linguist-documentation\n\n*.jpg binary\n*.png binary\n*.gif binary\n" }, { "path": ".github/workflows/ci.yml", "content": "name: Build and test\n\non:\n push:\n branches: [master]\n pull_request:\n\njobs:\n build-and-test-low-trust:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - name: Setup .NET\n uses: actions/setup-dotnet@v3\n with:\n global-json-file: global.json\n - name: Build Low-Trust version\n run: dotnet build Test/Test-LowTrust.fsproj -c Release-LowTrust -p:Platform=AnyCPU\n - name: Test Low-Trust version\n run: dotnet run --no-build --project Test/Test-LowTrust.fsproj -c Release-LowTrust -p:Platform=AnyCPU\n - name: Build samples for Low-Trust version\n run: dotnet build FParsec-LowTrust.sln -c Release-LowTrust -p:Platform=AnyCPU\n\n build-and-test-non-low-trust:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - name: Setup .NET\n uses: actions/setup-dotnet@v3\n - name: Build non-Low-Trust version\n run: dotnet build Test/Test.fsproj -c Release -p:Platform=AnyCPU\n - name: Test non-Low-Trust version\n run: dotnet run --no-build --project Test/Test.fsproj -c Release -p:Platform=AnyCPU\n - name: Build samples for non-Low-Trust version\n run: dotnet build FParsec.sln -c Release -p:Platform=AnyCPU\n" }, { "path": ".gitignore", "content": "Thumbs.db\nBuild/bin/\nUnused/*\nbin/\nobj/\n*.user\n*.suo\n*.pyc\n*~\n*.asv\n*.swp\n*.pdb\npackages/\nversion.props\n/.vs/\n.idea\n.DS_Store\n" }, { "path": ".vscode/tasks.json", "content": "{\n // See https://go.microsoft.com/fwlink/?LinkId=733558\n // for the documentation about the tasks.json format\n \"version\": \"2.0.0\",\n \"tasks\": [\n {\n \"label\": \"Build Test (Config: Debug-LowTrust)\",\n \"command\": \"dotnet build Test/Test.fsproj -c Debug-LowTrust -p:Platform=AnyCPU\",\n \"type\": \"shell\",\n \"group\": \"build\",\n \"presentation\": {\n \"reveal\": \"silent\"\n },\n \"problemMatcher\": \"$msCompile\"\n },\n {\n \"label\": \"Build All (Config: Debug-LowTrust)\",\n \"command\": \"dotnet build FParsec.sln -c Debug-LowTrust -p:Platform=AnyCPU\",\n \"type\": \"shell\",\n \"group\": \"build\",\n \"presentation\": {\n \"reveal\": \"silent\"\n },\n \"problemMatcher\": \"$msCompile\"\n },\n {\n \"label\": \"Test (Config: Debug-LowTrust)\",\n \"dependsOn\": [\"Build Test (Config: Debug-LowTrust)\"],\n \"command\": \"'Test/bin/Any CPU/Debug-LowTrust/net6/Test'\",\n \"type\": \"shell\",\n \"group\": \"test\",\n \"presentation\": {\n \"reveal\": \"always\"\n },\n \"problemMatcher\": \"$msCompile\"\n }\n ]\n}" }, { "path": "Build/FParsec.Common.targets", "content": "\n\n \n AnyCPU\n \n\n \n $(DefineConstants);SMALL_STATETAG\n \n \n $(DefineConstants);AGGRESSIVE_INLINING\n \n \n $(DefineConstants);UNALIGNED_READS\n \n\n \n LICENSE.txt\n \n \n \n \n\n \n true\n \n\n \n $(MSBuildThisFileDirectory)\\fparsec.snk\n $(DefineConstants);STRONG_NAME\n \n\n" }, { "path": "Build/fparsec-license.txt", "content": "The FParsec library in source and binary form is distributed under the Simplified BSD License. The Simplified BSD License (a.k.a. “2‐clause BSD License”) is a simple, permissive license that is OSI‐compliant.\n\nFParsec incorporates data derived from the Unicode Character Database v. 8.0.0, Copyright (c) 1991‒2015 Unicode, Inc., which is distributed under the following terms:\nhttp://www.unicode.org/terms_of_use.html#Exhibit1\n\n--\n\nFParsec Simplified BSD License\n\nCopyright (c) 2007‒2022, Stephan Tolksdorf. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n\nRedistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\nRedistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\nThis software is provided by the copyright holders “as is” and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage." }, { "path": "Directory.Build.props", "content": "\n\n \n Debug;Release;Debug-LowTrust;Release-LowTrust\n \n\n \n false\n true\n \n \n true\n $(DefineConstants);LOW_TRUST\n false\n \n\n \n true\n \n \n false\n $(DefineConstants);DEBUG\n \n \n true\n $(DefineConstants);RELEASE\n \n\n \n $(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb\n \n\n \n 2.0.0\n\n Stephan Tolksdorf\n Copyright © Stephan Tolksdorf\n FParsec\n http://www.quanttec.com/fparsec/\n https://github.com/stephan-tolksdorf/fparsec\n \n\n\n" }, { "path": "Doc/html/about/changelog.html", "content": "\n\n\n \n Changelog\n \n \n \n \n \n\n\n
\n
\n
\n \n \n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > About FParsec > Changelog\n \n
\n
\n

Changelog

\n
\n

Version 2.0, 2022‒11‒01

\n
\n
\n
    \n
  • Dropped .NET Framework 4.5 support and switched to using .NET 6.
    • \n
  • \n Changed NuGet build to always enable code signing. Contributed by Radek\n Krahl – thanks Radek!\n
    • \n
  • \n Norman Krämer fixed an error in the CharStream.Skip documentation – thanks Norman!\n
    • \n
  • \n Nathan Adams fixed a typo in the User’s Guide – thanks Nathan!\n
    • \n
\n
\n
\n
\n
\n

Version 1.1.1, 2020‒02‒01

\n
\n
\n
    \n
  • \n Fixed NuGet build to target the AnyCPU platform instead of the default platform of the build machine. Vadim Slynko and tpisciotta reported this\n issue – thanks Vadim and tpisciotta!\n
    • \n
\n
\n
\n
\n
\n

Version 1.1.0, 2020‒01‒05

\n
\n
\n
    \n
  • \n Behaviour change: pfloat now parses out‐of‐range finite values as plus or minus infinity instead of returning an error. This unifies\n the pfloat behaviour on all\n platforms after the\n behaviour change of System.Double.Parse on .NET Core 3.\n
    • \n
  • \n Enrico Sada modernized the F# and C# project files and the build script for\n the NuGet packages – thanks Enrico!\n
    • \n
  • Added SourceLink support, which was prepared and championed by Cameron Taggart – thanks Cameron!
    • \n
  • \n Maxime Didier fixed a bug in the Low‐Trust version of the CharStream constructors that accept a file\n path argument: The stream’s Name property wasn’t initialized. Thanks Maxime!\n
    • \n
  • \n Fixed missing parser definitions in the Parsing JSON section of the tutorial spotted by Josh\n Quintus – thanks Josh!\n
    • \n
  • \n Andre Wesseling fixed a parser definition in the Where is the monad section of the User’s Guide – thanks Andre!\n
    • \n
  • \n Frederik K. fixed an error in the ErrorMessage documentation – thanks Frederik!\n
    • \n
  • \n Jonathan Roeber fixed an error in the previousCharSatisfiesNot – thanks\n Jonathan!\n
    • \n
  • \n Vegard Løkken fixed an error in the unicodeSpaces documentation – thanks Vegard!\n
    • \n
\n
\n
\n
\n
\n

Version 1.0.3, 2017‒08‒20

\n
\n
\n
    \n
  • \n Modern solution and project files for .NET Core and VS 2017 were added to the source folders. Huge thanks to Marcus Griep for spearheading the effort to make FParsec .NET Standard compatible and contributing\n the new project and solution files!\n
    • \n
  • The old build script for the NuGet packages was replaced by a PowerShell script that uses the new project files.
    • \n
  • The FParsec NuGet package now contains assemblies for .NET Standard 1.6.
    • \n
  • \n The non‐netstandard assemblies of FParsec now reference the FSharp.Core 4.0.0.1 NuGet package, which should maximize compatibility when binding\n redirects aren’t available.\n
    • \n
  • \n A .vscode/tasks.json file\n with some task definitions for Visual Studio Code was added.\n
    • \n
  • The source repository was moved to GitHub.
    • \n
  • \n Added a stringsSepBy1\n parser (contributed by Robin Munn – thanks Robin!).\n
    • \n
  • \n Added a link to the Russian translation of the tutorial by Dmitry Vlasov –\n thanks Dmitry!\n
    • \n
  • Fixed documentation typos. One was spotted by Brandon Dimperio, another by ZelteHonor – thanks Brandon and ZelteHonor!
    • \n
  • \n Renamed CLR45 to AGGRESSIVE_INLINING to\n better match its purpose.\n
    • \n
\n
\n
\n
\n
\n

Version 1.0.2, 2015‒09‒27

\n
\n
\n
    \n
  • \n replaced all uses of Char.GetUnicodeCategory with CharCodeInfo.GetUnicodeCategory, since the former may or may not track the current Unicode standard and the latter is the only\n one supported by the PCL API subset\n
    • \n
  • updated the case folding, whitespace and XID property data tables to Unicode 8.0.0
    • \n
  • added a PCL Profile 259 version to the FParsec NuGet package
    • \n
  • removed the Silverlight, VS9 and VS10 solution files and the Mono Makefile
    • \n
  • \n updated the Lex & Yacc version of the FSharpParsingSample to use the FsLexYacc NuGet packages\n
    • \n
  • fixed documentation typos (two were spotted by Francois Nardon and Patrick McDonald – thanks Francois and Patrick!)
    • \n
\n
\n
\n
\n
\n

Version 1.0.1, 2013‒06‒25

\n
\n
\n
    \n
  • \n The maintainership of the FParsec NuGet package(s) was handed over from Ryan Riley, Huw Simpson, Cameron Taggart and Khan Thompson to Stephan\n Tolksdorf. Thanks Ryan, Huw, Cameron and Khan for creating and maintaining the previous versions of the NuGet package!\n
    • \n
  • \n FParsec now has two NuGet packages, built with a new fsx script\n
    • \n
  • \n fixed a bug in one of the CharStream constructors (reported and patched by Andrew Smith – thanks Andrew!)\n
    • \n
  • \n added USE_STATIC_MAPPING_FOR_IS_ANY_OF and UNALIGNED_READS as default\n compilation options in the Visual Studio projects (the default options now match the ones used by the “Big Data edition” NuGet\n package)\n
    • \n
  • some minor code tweaking / micro‐optimizations
    • \n
  • fixed some minor documentation issues
    • \n
\n
\n
\n
\n
\n

Version 1.0.0, 2012‒07‒19

\n
\n
\n
    \n
  • \n disabled code generation in isAnyOf, isNoneOf, anyOf, skipAnyOf, noneOf and skipNoneOf by default (you can reenable it using the new USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option)\n
    • \n
  • \n annotated some CharStream methods\n with the .NET 4.5 AggressiveInlining option (see the new CLR45 compilation option)\n
    • \n
  • updated case folding and XID property tables to Unicode 6.1.0
    • \n
  • fixed two documentation typos (spotted by Rasmus Meldgaard and Kurt Schelfthout – thanks Rasmus and Kurt!)
    • \n
\n
\n
\n
\n
\n

Version 0.9.2, 2012‒03‒09

\n
\n
\n
    \n
  • fixed compilation in Visual Studio 11 Beta
    • \n
  • \n added missing ReturnFrom member to parse builder object (reported by Kurt Schelfthout and\n Tomas Petricek – thanks Kurt and Tomas!)\n
    • \n
  • \n added workaround for .NET ConsoleStream issue (reported by Alexander Kahl – thanks\n Alexander!)\n
    • \n
  • \n set AllowPartiallyTrustedCallers and SecurityTransparent assembly attributes in LOW_TRUST NET4 build (as suggested by hammett – thanks hammett!)\n
    • \n
  • \n changed encoding of FParsecCS/Strings.cs to UTF‐8 (with signature) to fix Visual Studio build on machines with Japanese\n locale (the encoding issue was reported on http://d.hatena.ne.jp/ZOETROPE – thank you!)\n
    • \n
  • \n fixed some documentation issues (incorporating feedback from Alexander Gelkin, Antoine Latter and Stephen Swensen – thanks Alexander, Antoine\n and Stephen!)\n
    • \n
  • \n add link to the Japanese translation of the tutorial by Gab_km (thanks\n Gab_km!)\n
    • \n
\n
\n
\n
\n
\n

Version 0.9.1, 2011‒05‒22

\n
\n
\n
    \n
  • \n added /nooptimizationdata compiler flag as a workaround for an F# comiler issue (reported by Michael Giagnocavo – thanks\n Michael!)\n
    • \n
  • \n fixed an issue in the JSON sample (reported by Ryan Riley\n – thanks Ryan!)\n
    • \n
  • \n fixed the error message formatting when an error line contains unaccounted newlines or ends with a combining character sequence\n
    • \n
  • \n added warning to installation notes that the regex parser doesn’t work on Mono (reported by Laurent Le Brun – thanks Laurent!)\n
    • \n
  • fixed some documentation issues (one of which was reported by Michael Giagnocavo – thanks Michael!)
    • \n
\n
\n
\n
\n
\n

Version 0.9.0, 2011‒04‒26

\n
\n
\n \n
\n
\n
\n
\n
Highlights
\n
\n
\n
    \n
  • a new tutorial and user’s guide
    • \n
  • 2x performance improvements due to a refactored low‐level API
    • \n
  • \n new identifier parser\n for parsing identifiers based on Unicode XID syntax\n
    • \n
  • \n new StaticMapping module for compiling\n static key to value mappings into optimized functions (supports char, int and string as key types)\n
    • \n
\n
\n
\n
Changes to high‐level API
\n
\n
\n \n
\n
\n
    \n
  • \n

    \n OperatorPrecedenceParser has changed:\n

    \n
      \n
    • \n all types have been moved from the FParsec.OperatorPrecedenceParser module into the main FParsec\n namespace\n
      • \n
    • \n the operator types InfixOp, PrefixOp, … classes have been renamed to InfixOperator, PrefixOperator, …\n
      • \n
    • \n Assoc has been renamed to Associativity\n
      • \n
    • \n the “whitespace‐parser” argument of the operator types has been generalized into an “after‐string‐parser” argument\n
      • \n
    • \n if you previously used the InfixOp', PrefixOp', … constructors to supply a mapping that read the text position of the parsed operator from the passed\n State instance, read this\n
      • \n
    • \n the AddOperators method has been removed, call AddOperator instead\n
      • \n
    • \n the OperatorConflictErrorFormatter replaced the OperatorConflictHandler\n
      • \n
    \n
    • \n
\n
\n
\n
\n Removed variants of many, sepBy, sepEndBy and manyTill\n
\n
\n
\n

\n The ...Rev, ...Fold and ...Reduce variants of many, sepBy, sepEndByand manyTill have been removed.\n

\n
\n
\n

\n If you previously used these variants, you can easily define them in your own code using the |>> combinator, as documented in the\n reference documentation for the previous version. For example:\n

\n
let manyRev p = many p |>> List.rev\nlet manyFold acc0 f p = many p |>> List.fold f acc0\nlet manyReduce f defVal p = (many1 p |>> List.reduce f) <|>% defVal.\n
\n
\n
\n

\n If you need optimized implementations, you can define them using the new Inline helper class. The file Doc/misc/removed‐many‐variants.fs contains optimized definitions for all removed variants.\n

\n
\n
\n
\n Details on changes to manyChars, manyCharsTill and their variants\n
\n
\n
\n

\n The behaviour of all variants of manyChars and manyCharsTill has slightly changed. Now manyChars cp is equivalent\n to many cp, except that it returns a string instead of char list. Previously, manyChars cp behaved like\n many (attempt cp), i.e. it automatically backtracked if the char parser had failed after consuming input. The same change has been\n made to the behaviour of all other variants of manyChars and manyCharsTill. The new behaviour is more consistent with the rest of the libary and allows a faster\n implementation with the new low‐level API.\n

\n
\n
\n

There probably aren’t many parsers that relied on the old behaviour.

\n
\n
\n

\n The behaviour change made the skip variants of manyChar and manyCharsTill obsolete, since e.g. skipManyChars would do\n exactly the same as skipMany cp. Hence, the skip variants have been removed.\n

\n
\n
\n
Changes to low‐level API
\n
\n
\n
    \n
  • \n The old CharStream and State\n classes have been merged into a single CharStream class with a mutable interface.\n
    • \n
  • \n Parser functions now take a\n CharStream<'u> instance as the input argument.\n
    • \n
  • \n The Reply type has been moved to the main FParsec namespace and no longer has a State member.\n
    • \n
  • \n Parser state comparisons are now done with the help of the CharStream’s StateTag.\n
    • \n
  • \n

    \n Various methods from the old CharStream.Iterator and State types have been renamed in the new CharStream class and have new\n signatures:\n

    \n
      \n
    • \n

      When you adapt old code, the following changes require particular attention:

      \n
        \n
      • \n the old Iterator.Read methods\n did not advance the input stream position, but the new CharStream.Read now do (as is the expected\n behaviour in traditional stream classes)\n
        • \n
      • \n the old Read() is equivalent to the new Peek()\n
        • \n
      • \n the old Read(int) is equivalent to the new PeekString(int)\n
        • \n
      • \n the old Peek() is equivalent to the new (and old) Peek(1)\n
        • \n
      \n
      • \n
    • \n

      More renamings:

      \n
        \n
      • \n Next, Advance ? Skip\n
        • \n
      • \n _Increment, _Decrement ? SkipAndPeek\n
        • \n
      • \n SkipRestOfLine with string output ? ReadRestOfLine\n
        • \n
      • \n SkipCharOrNewlines with string output ? ReadCharsOrNewlines\n
        • \n
      • \n SkipToString ? SkipCharsOrNewlinesUntilString\n
        • \n
      • \n SkipToStringCI ? SkipCharsOrNewlinesUntilCaseFoldedString\n
        • \n
      • \n ReadUntil ? ReadFrom\n
        • \n
      • \n CharStream.FoldCase ? Text.FoldCase\n
        • \n
      • \n CharStream.NormalizeNewlines ?\n Text.NormalizeNewlines\n
        • \n
      \n
      • \n
    \n
    • \n
  • \n

    New CharStream methods:

    \n \n
    • \n
  • \n The ErrorMessage and\n ErrorMessageList types are now defined\n in the C# library part. This allows us to implement full parsers in C#. The FParsec.Error module contains type abbreviations and active\n patters that provide the familiar interface to F# clients.\n
    • \n
  • \n All error messages used by built‐in FParsec parsers are now defined in the C# classes FParsec.Strings and FParsec.Errors. This should simplify customization and\n internationalization efforts.\n
    • \n
\n
\n
\n
Background on low‐level API changes
\n
\n
\n

\n Previously parsers were implemented as functions operating on an immutable parser state in the form of a State instance. A parser function received\n a State instance\n as the input and returned a State instance as part of its return value. Since State instances were immutable, a parser\n function had to create a new State instance to advance the input stream, e.g. by calling state.Advance(2).\n

\n
\n
\n

\n This architecture was motivated by the desire to provide an API as “functional” as possible, an API that shields users from\n the underlying imperative/mutable nature of input streams. When FParsec originally started as a relatively close port of Haskell’s Parsec\n library, this design felt like a natural fit for a functional parser library. However, later, when FParsec moved away from its Parsec roots\n (to improve performance and provide more features), it became increasingly clear that the immutable CharStream‐State‐design was the main obstacle\n preventing FParsec from reaching the performance of hand‐optimized recursive‐descent parsers.\n

\n
\n
\n

\n Initial tests with some quick prototypes revealed that the allocation and garbage collection of temporary State instances took up to 50% or more of\n the run time of typical parsers – even though the State class was already heavily optimized.\n These tests also indicated that consolidating the stream and state classes into a classical imperative stream class simplified the overall\n library implementation and made the library source code more accessible to new users.\n

\n
\n
\n

\n The main drawback of the API change is that it requires modifications to practically all low‐level parser code. Another drawback is that\n backtracking is slightly less convenient with the new low‐level API (as the parser state has to be explicitly saved and restored, while\n previously one could just continue with an old state instance).\n

\n
\n
\n

Since FParsec’s high‐level API is only minimally affected by the change, the advantages seem to outweigh the costs.

\n
\n
\n
\n
\n
\n
\n
\n
\n

Version 0.8.x, no release

\n
\n
\n
\n
\n
New features/ improvements
\n
\n
\n
    \n
  • \n case‐insensitive matching with pstringCI, charsTillStringCI, etc. (using the Unicode 1‐to‐1 case folding mappings for chars in the BMP)\n
    • \n
  • \n various new parsers and combinators, including restOfLine, skipToString, manySatisfyMinMax, manyStrings, withSkippedString\n
    • \n
  • \n new functions runParserOnSubstring and runParserOnSubstream\n
    • \n
  • various performance improvements
    • \n
  • Silverlight support
    • \n
  • F# 1.9.6.16 compatibility
    • \n
\n
\n
\n
Design changes
\n
\n
\n
    \n
  • \n standardized on a single input stream type (FParsec.CharStream) and a single concrete parser state type (FParsec.State)\n
    • \n
  • \n

    \n refactored the Reply<_,_>, ErrorMessage and ParserError types:\n

    \n
      \n
    • error replies now also contain a complete State
      • \n
    • \n whether a parser has changed the state is now determined by checking the input and the output state for equality, instead of testing the\n Consumed flag\n
      • \n
    • \n replaced the Reply<_,_>.Flags\n with a Status field\n
      • \n
    • \n replaced the various helper functions for constructing a Reply with three overloaded\n Reply<_,_> constructors (with different arities)\n
      • \n
    \n
    • \n
  • \n all char parsers are now “newline aware”, i.e. they normalize any of the three standard newline representations (\"\\n\", \"\\r\\n\", \"\\r\") to “n” and they properly increment the line count whenever they parse a newline;\n this means that the behaviour of almost all char parsers has changed with regard to how newline chars are handled\n
    • \n
\n
\n
\n
Bug fixes
\n
\n
\n
    \n
  • \n The CharStream class now uses the serialization API to persist the decoder state for\n backtracking purposes. Previously it relied on the decoder loosing its state at block boundaries after a certain sequence of method calls.\n The previous approach works in practice for the .NET decoders of the standard unicode encodings and for simple stateless encodings like\n ASCII and ANSI, but it relies on undocumented behaviour and it does not work reliably for encodings like GB18030, ISO‐2022 or ISCII.\n
    • \n
  • \n In previous FParsec versions the CharStream file path/System.IO.Stream constructors\n failed with an IndexOutOfRange exception when the file/stream was empty and encoding\n detection was not turned off (reported by Vesa Karvonen ‐ thanks Vesa!).\n
    • \n
  • \n In previous FParsec versions the NumberLiteral.String returned by the numberLiteral parser included parsed\n suffix chars despite the documentation claiming the opposite. (The testing code was buggy too.) Applications that rely on this behaviour\n can now use the new NumberLiteralOptions.IncludeSuffixCharsInString to force the numberLiteral parser\n to include any suffix chars in the returned string.\n
    • \n
  • \n Fixed behaviour of >>=?, >>? and .>>? when second parser fails with fatal\n error without changing the parser state.\n
    • \n
  • \n Fixed behaviour of nextCharSatisfies[Not] when current “char” is a \"\\r\\n\" newline.\n
    • \n
\n
\n
\n
Other breaking changes
\n
\n
\n
    \n
  • \n renamed the module CharParser to CharParsers\n
    • \n
  • \n moved CharParser.OperatorPrecedenceParser into separate module\n
    • \n
\n
\n
\n
    \n
  • \n

    FParsec.Primitives:

    \n
      \n
    • \n subtle change: renamed message to fail and fail to failFatally\n
      • \n
    • \n renamed pair, triple and quad to tuple2, tuple3 and tuple4\n
      • \n
    • \n renamed manyFoldLeft to manyFold\n and changed the argument order of the accumulator and function argument\n
      • \n
    • removed manyFoldRight
      • \n
    • \n renamed count to parray and\n changed the return type, renamed skipCount to skipArray\n
      • \n
    • \n renamed followedBy and notFollowedBy to followedByL and notFollowedByL and introduced followedBy and notFollowedBy functions that take no second argument\n
      • \n
    • \n moved ParserResult<_> to CharParsers and changed constructor arguments\n
      • \n
    • removed applyParser
      • \n
    • \n removed |>>=, now >>=\n automatically uses an optimized branch for uncurried functions\n
      • \n
    • \n removed endBy and endBy1 (endBy p sep can be replaced with many (p .>> sep) and endBy1 p sep with many1 (p .>> sep))\n
      • \n
    \n
    • \n
\n
\n
\n
    \n
  • \n

    FParsec.CharParsers:

    \n
      \n
    • \n renamed manyTillString to charsTillString\n
      • \n
    • removed applyParser from the public interface
      • \n
    • \n removed getIndex, skip, registerNL, extract, regexp (these low‐level operations should be done directly through the State<_>/CharStream.Iterator interface)\n
      • \n
    • removed anyCharOrNL (no longer needed, see design changes above)
      • \n
    • \n removed nSatisfy (can be replaced with manySatisfyMinMax)\n
      • \n
    • \n removed unicodeDigit and unicodeNumber (can be replaced with satisfy System.Char.IsDigit and satisfy System.Char.IsNumber)\n
      • \n
    • \n moved the helper functions expectedError, unexpectedError etc. into the Error module\n
      • \n
    \n
    • \n
\n
\n
\n
    \n
  • \n

    FParsec.CharStream:

    \n
      \n
    • string constructor takes more arguments
      • \n
    • \n Iterator.Peek(i) now returns the EndOfStreamChar\n char instead of throwing an exception if the char peeked at lies before the beginning of the stream\n
      • \n
    \n
    • \n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Version 0.7.3.1, 2009‒02‒26

\n
\n
\n
    \n
  • \n
    \n

    \n Fixed a bug in CharParser.normalizeNewlines/CharStream.NormalizeNewlines. This bug also affected the skipped and manyTillString parsers, which internaly call normalizeNewlines to normalize the returned string.\n

    \n
    \n
    \n

    The bug was reported by Greg Chapman ‐ thanks Greg!

    \n
    \n
    \n

    \n When given a multi‐line string in which the lines are delimited by \"\\r\\n\" but the last line does not end in a newline, the buggy\n normalizeNewlines replaced the chars on the last line with '\\n' chars.\n

    \n
    \n
    • \n
\n
\n
\n
    \n
  • \n Changed the signature of Helper.SkipOverWhitespace.\n
    • \n
\n
\n
\n
\n
\n

Version 0.7.3, 2008‒12‒08

\n
\n
\n

Breaking changes (all of which should have little or no impact on existing code bases):

\n
    \n
  • \n CharStream.Iterator instances now compare\n equal if and only if they belong to the same CharStream and point to the same index\n (previously they compared only equal if their internal representations were identical)\n
    • \n
  • \n the constructor argument of Error.otherError is now expected to be comparable with F#’s structural comparison function compare, see http://research.microsoft.com/fsharp/manual/spec2.aspx#_Toc207785725\n
    • \n
  • \n the signature of the second ParserError.ToString overload has changed\n
    • \n
  • \n CharParser.errorToString and printErrorLine have been deprecated\n
    • \n
\n
\n
\n

New features:

\n
    \n
  • \n reimplemented the error formatting code in FParsec.Error\n
    • \n
  • \n added new State<_>.AdvanceTo and CharStream.Iterator.Advance overloads\n
    • \n
  • \n slightly modified the error reporting in Primitives.sepEndBy\n
    • \n
  • some documentation fixes
    • \n
\n
\n
\n
\n
\n

Version 0.7.2, 2008‒11‒17

\n
\n
\n
    \n
  • \n added CharParser.OperatorPrecedenceParser\n
    • \n
  • \n changed the overflow checking in pint32 such that it will not be affected by an expected\n future change in F#’s int32 -> uint64\n conversion behaviour\n
    • \n
  • \n added CharParser.pint16, puint16, pint8, puint8\n
    • \n
  • \n changed the signatures in CharParser.fsi to use the Parser<_,_> type abbreviation\n
    • \n
  • \n fixed outdated documentation of CharParser.expectedError\n
    • \n
  • some minor optimizations
    • \n
\n
\n
\n
\n
\n

Version 0.7.1, 2008‒09‒29

\n
\n
\n

Breaking changes:

\n
    \n
  • \n renamed Primitives.Reply._tag member to Flags and gave it a proper\n enumeration type\n
    • \n
  • \n CharParser.State is now a reference type\n
    • \n
  • \n Removed CharParser.State.Flags member\n
    • \n
  • \n deprecated Primitives.reconstructError\n
    • \n
\n
\n
\n
\n
\n

Version 0.7.0.1, 2008‒09‒23

\n
\n
\n

Breaking change:

\n
    \n
  • \n changed the case of the FParsec.Error.Pos members (This wasn’t already done in 0.7 because of an oversight.)\n
    • \n
\n
\n
\n
\n
\n

Version 0.7.0, 2008‒09‒13

\n
\n
\n

Bugfixes:

\n
    \n
  • \n made FParsec.Error.Pos IComparable to prevent ParserError.ToString from throwing an exception under rare circumstances\n
    • \n
  • \n corrected the argument checking for some CharStream.Iterator methods for very large arguments\n
    • \n
\n
\n
\n

New features:

\n
    \n
  • compatibility with the F# CTP release
    • \n
  • \n a configurable parser for number literals: CharParser.numberLiteral\n
    • \n
  • \n CharParser.pfloat now also parses NaN, Infinity and hexadecimal floating point\n literals as supported by IEEE754r, C99 and Java (but different from the hex representation supported by F#)\n
    • \n
  • \n new helper functions CharParser.floatToHexString, floatOfHexString, float32ToHexString and float32OfHexString\n
    • \n
  • \n integer parsers: Charparser.pint32, puint64, puint32, puint64\n
    • \n
  • new sample: a JSON parser
    • \n
  • various optimizations and some code cleanup
    • \n
  • \n new CharStream.Iterator members ReadUntil, Increment and Decrement\n
    • \n
  • \n new State member AdvanceTo\n
    • \n
  • \n new function Primitives.createParserForwardedToRef\n
    • \n
  • \n new combinator |>>= in Primitives\n
    • \n
\n
\n
\n

Breaking changes:

\n
    \n
  • \n renamed the parsers char and string to\n pchar and pstring (This is in deference\n to the built‐in F# functions char and string, which weren’t yet around when the first version of FParsec was released.)\n
    • \n
  • \n changed the case of the properties of the Reply and State types (This reflects the emerging consensus in the F# community that all public members of types should be named\n in PascalCase.)\n
    • \n
  • \n deprecated State.AdvanceNL (use the 3\n parameter Advance overload instead)\n
    • \n
  • \n deprecated the Primitives helper functions isOk, isEmpty, … (the Reply properties IsOk, IsEmpty,… should be used instead)\n
    • \n
  • \n deprecated the CharParser helper functions matchChar, readChar, … (the State.Iter methods Match, Read, … should be used instead)\n
    • \n
  • \n deprecated Primitives.option, <|>$ should be used instead\n
    • \n
  • \n made CharParser.CharList internal (If you\n need this helper class for your code, just copy the implementation to your source.)\n
    • \n
  • \n State.Flags() now\n has more bits (and less bits are reset on a position change)\n
    • \n
\n
\n
\n
\n
\n

Version 0.6.0, 2008‒05‒20

\n
\n
\n
    \n
  • \n fixed a bug in manyTillString (the code keeping track of newlines was buggy)\n
    • \n
  • \n fixed a bug in CharParser.<?> (the error reporting was\n inconsistent with Primitives.<?> in the rare case where <?> is applied inside an attempt (...) <?> label\n clause to a parser that returns an EmptyOk reply)\n
    • \n
  • various changes for F# 1.9.4.15
    • \n
  • \n added skipped parser to CharParser\n
    • \n
  • \n added nextCharSatifiesNot, prevCharSatifiesNot, currCharSatisfies, currCharSatisfiesNot to CharParser module; the behaviours of the\n existing nextCharSatisfies and prevCharSatisfies were slightly changed (see fparsec.html for more details)\n
    • \n
  • \n added TryWith and TryFinally members to\n Primitivs.ParserCombinator\n
    • \n
  • \n added triple and quad parsers to Primitives module\n
    • \n
  • \n set CompilationRepresentationFlags.PermitNull for Error.ParserError\n
    • \n
  • various optimizations
    • \n
  • \n some documentation fixes, including corrections for the docs of the CharParser error\n generation helper functions (expectedError etc.)\n
    • \n
\n
\n
\n
\n
\n

Version 0.5.1, 2008‒01‒20

\n
\n
\n
    \n
  • \n added pipe2, pipe3 and pipe4 primitives\n
    • \n
  • \n replaced count and skipCount primitives\n with optimized versions\n
    • \n
  • \n minor optimizations in spaces and spaces1\n
    • \n
  • added pfloat char parser
    • \n
  • minor documentation fixes
    • \n
\n
\n
\n
\n
\n

Version 0.5.0, 2008‒01‒15

\n
\n
\n
    \n
  • \n Major design change: all lazy computations were removed and the types Output and Reply unified. The new implementation is considerably simpler and also compiles with F# 1.9.3.7.\n
    • \n
  • Fixed a bug in build.bat (reported by Santosh Zachariah ‐ thanks Santosh!)
    • \n
\n
\n
\n
\n
\n

Version 0.4.4, 2008‒01‒13

\n
\n
\n
    \n
  • fixed a minor issue in CharParser.attempt
    • \n
  • \n added .>>! and >>.!\n primitives\n
    • \n
  • \n added skipManySatisfy and skipMany1Satisfy char parsers\n
    • \n
\n
\n
\n
\n
\n

Version 0.4.3, 2008‒01‒12

\n
\n
\n
    \n
  • \n fixed bugs in the CharParser versions of <?> and attempt.\n
    • \n
  • added >>? primitive
    • \n
  • \n added skipSatisfy and skipSatisfyL char\n parsers\n
    • \n
  • minor documentation fixes
    • \n
\n
\n
\n
\n
\n

Version 0.4.2, 2008‒01‒04

\n
\n
\n
    \n
  • \n performance improvements in CharStream.Iterator\n
    • \n
  • minor documentation fixes
    • \n
\n
\n
\n
\n
\n

Version 0.4.1, 2008‒01‒02

\n
\n
\n
    \n
  • documentation fixes
    • \n
  • new sample application: a parser for Parsing Expression Grammars
    • \n
  • \n newline and unicodeNewline now return\n '\\n', instead of\n 1 or 2\n
    • \n
  • \n added whitespace parser and changed unicodeWhitespace\n
    • \n
  • \n added spaces parser (equivalent to skipManyChars whitespace)\n
    • \n
  • \n removed newlineRepl parameter from manyTillString\n
    • \n
  • \n added skipManyTill and skipManyCharsTill\n
    • \n
  • generalized types of skipManyChars and skipManyChars1
    • \n
\n
\n
\n
\n
\n

Version 0.4.0, 2007‒12‒30

\n
\n
\n

Initial public release

\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/about/contact.html", "content": "\n\n\n \n Contact\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n FParsec vs alternatives
\n 2\n Status and roadmap
3Changelog
4Contact
\n \n \n \n \n \n \n \n \n \n \n \n
1Contact
2Impressum
\n
\n
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > About FParsec > Contact\n \n
\n
\n

1.4 Contact

\n
\n

1.4.1 Contact

\n
\n
\n

FParsec currently doesn’t have its own discussion forum or mailing list.
(Please let me know if you’d like that to be changed.)

\n
\n
\n

\n Currently the best place to get a quick answer to any FParsec‐related question is: StackOverflow.com.\n

\n
\n
\n

\n You can also email me (Stephan) directly at: fparsec [at] quanttec.com. Please don’t hesitate to\n contact me with any feedback or question regarding FParsec. I’m always happy to hear from FParsec users.\n

\n
\n
\n
\n
\n

1.4.2 Impressum

\n
\n
\n

Author: Stephan Tolksdorf

\n
\n
\n

Address:
Geschwister‐Scholl‐Allee 253
25524 Itzehoe
Germany

\n
\n
\n

fparsec [at] quanttec.com

\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/about/fparsec-vs-alternatives.html", "content": "\n\n\n \n FParsec vs alternatives\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1FParsec vs alternatives
\n 2\n Status and roadmap
3Changelog
4Contact
\n
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > About FParsec > FParsec vs alternatives\n \n
\n
\n

1.1 FParsec vs alternatives

\n
\n
\n

\n The following tables contain a bullet‐point comparison between FParsec and the two main alternatives for parsing with F#: parser generator tools\n (e.g. fslex & fsyacc) and “hand‐written” recursive descent parsers.\n

\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 1.1.1: Relative advantages\n
Parser‐generator toolsFParsecHand‐written
recursive‐descent parser
\n
    \n
  • Declarative and easy‐to‐read syntax
    • \n
  • Ensures adherence to grammar formalism
    • \n
  • Can check for certain kinds of ambiguity in grammar
    • \n
  • \n You don’t have to think about performance. Either the generated parser is fast enough, or not. There’s not much you can do about it.\n
    • \n
\n 		\n
    \n
  • Implemented as F# library, so no extra tools or build steps
    • \n
  • Parsers are first‐class values within the language
    • \n
  • Succinct and expressive syntax
    • \n
  • Modular and easily extensible
    • \n
  • Extensive set of predefined parsers and combinators
    • \n
  • Semi‐automatically generated, highly readable error messages
    • \n
  • Supports arbitrary lookahead and backtracking
    • \n
  • Runtime‐configurable operator‐precedence parser component
    • \n
  • Does not require a pre‐parsing tokenization phase
    • \n
  • Comprehensive documentation
    • \n
  • Extensively unit‐tested
    • \n
\n 		\n
    \n
  • No extra tools or build steps
    • \n
  • Most amenable to individual requirements
    • \n
  • Potentially as fast as technically possible
    • \n
  • Parsers are relatively portable if you stick to simple language features and keep library dependencies to a minimum
    • \n
\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 1.1.2: Relative\n disadvantages\n
Parser‐generator toolsFParsecHand‐written
recursive‐descent parser
\n
    \n
  • Restricted to features of grammar formalism
    • \n
  • Extra tools and compilation steps
    • \n
  • Reliance on opaque generator tool, that is often hard to debug, optimize or extend
    • \n
  • Static grammar that can’t be changed at runtime
    • \n
  • Often hard to generate good error messages
    • \n
  • Many tools generate comparatively slow parsers
    • \n
  • Some tools have only limited Unicode support
    • \n
  • Portability problems
    • \n
\n 		\n
    \n
  • Tradeoff between declarativeness and performance
    • \n
  • Syntax less readable than PEG or Regular Expression syntax
    • \n
  • Left‐recursive grammar rules have to be rewritten
    • \n
  • Does not support a pre‐parsing tokenization phase
    • \n
  • You have to learn the API
    • \n
  • Limited to F#
    • \n
  • Code‐dependence on FParsec
    • \n
  • Aggressive performance optimizations add complexity to parts of the lower‐level FParsec source code
    • \n
\n 		\n
    \n
  • You have to write everything yourself, which can take a lot of effort
    • \n
  • Implementing (fast) parsers requires some experience
    • \n
  • \n Expression (sub)grammars with infix operators can be ugly and inefficient to parse with a pure recursive‐descent parser, so you might also\n have to write some kind of embedded operator precedence parser\n
    • \n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/about/index.html", "content": "\n\n\n \n About FParsec\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n FParsec vs alternatives
\n 2\n Status and roadmap
3Changelog
4Contact
\n
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > About FParsec\n \n
\n
\n

1 About FParsec

\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1FParsec vs alternatives
2Status and roadmap
3Changelog
4Contact
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/about/status-and-roadmap.html", "content": "\n\n\n \n Status and roadmap\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n FParsec vs alternatives
2Status and roadmap
\n \n \n \n \n \n \n \n \n \n \n \n
1Status
\n 2\n Future development
\n
3Changelog
4Contact
\n
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > About FParsec > Status and roadmap\n \n
\n
\n

1.2 Status and roadmap

\n
\n

1.2.1 Status

\n
\n
\n

FParsec has been in development for several years and can now be considered “stable”.

\n
\n
\n

Version 1.0 of FParsec was released on 19 July 2012.

\n
\n
\n
\n
Note
\n
\n
\n

\n Although FParsec has rather comprehensive unit tests (with code coverage close to 100% for many components), it likely still contains bugs.\n If you want to use FParsec in a production environment, you need to test your parsers thoroughly.\n

\n
\n
\n
\n
\n
\n
\n
\n

1.2.2 Future development

\n
\n
\n

There are no firm plans for any major new features yet.

\n
\n
\n

\n One goal for the future development of FParsec is to support a more declarative parser definition syntax without compromising on FParsec’s\n performance or language‐integrated nature.\n

\n
\n
\n

\n For example, it would be nice if FParsec provided a way to automatically create optimized lexer functions from a series of typed regular\n expressions and associated mapping functions, ideally at compile time. Using such a feature could maybe look similar to\n

\n
\n
\n
let lexer : Parser<AstNode, 'u> = \n    lex [\"regex-with-1-capture-group\", (fun x -> AstNode1(x))\n         \"regex-with-2-capture-groups\", (fun x y -> AstNode2(x, y))\n         (* ... *)]\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/css/print.css", "content": "/*\n#wrapper {\n max-width: none;\n}\n*/\n\na, a code {\n color: #000000;\n text-decoration: underline;\n}\n\ncode, .code {\n background-color: transparent;\n}\n\ncode a, .code a {\n color: #000000;\n text-decoration: none;\n border: 0;\n}\n\nspan.ck /* keyword */ /*, span.cb, span.cnu */ {\n color: #000000;\n}\n\nspan.cr /* right arrow */ {\n color: #000000;\n}\n\nspan.cc /* char literal */, span.cs /* string literal */ {\n color: #000000;\n}\n\nspan.ce /* escaped char */ {\n color: #000000;\n}\n\nspan.clc /* line comment*/, span.cbc /* block comment */ {\n color: #000000;\n}\n\n.interface-member-code pre {\n overflow: visible;\n white-space: pre;\n word-wrap: normal;\n}\n\n.interface-member-description {\n padding-left: 1em;\n}\n\n.interface-member-backlink {\n display: none;\n}" }, { "path": "Doc/html/css/screen-sidebar.css", "content": "\nbody {\n background-color: #e9e9e9;\n}\n\nhtml {\n height: 100%;\n overflow-y: scroll;\n}\n\nbody {\n height: 100%;\n}\n\n\n#fixed-layer {\n display: block;\n position: fixed;\n left: 0px;\n top: 0px;\n width: 100%;\n height: 100%;\n overflow: hidden;\n}\n\n#wrapper {\n height: 100%;\n min-height: 100%;\n max-width: 73em;\n margin-left: auto;\n margin-right: auto;\n}\n\n#fixed-wrapper {\n position: relative;\n max-width: 73em;\n width: 100%;\n height: 100%;\n margin-left: auto;\n margin-right: auto;\n\tbackground-color: #ffffff;\n\tborder: solid 1px #d9d9d9;\n}\n\n#sidebar {\n position: absolute;\n z-index: 2;\n left: 0;\n top: 0;\n width: 22.5em;\n height: 100%;\n overflow: auto;\n}\n\n#main {\n position: relative;\n z-index: 3;\n left: 0;\n top: 0;\n min-height: 100%;\n min-width: 40em;\n margin-left: 22.5em;\n background-color: #ffffff;\n border-left: solid 1px #d9d9d9;\n}\n\n#main-content {\n padding: 0 1.5em 1.5em 1.5em;\n}\n\n#top-links {\n margin-left: 1.5em;\n padding-top: 0.5em;\n margin-bottom: 0.5em; \n margin-top: -0.07143em;\n}\n#top-links span {\n font-size: 0.9286em;\n color: #333;\n}\n#top-links a {\n color: #333;\n}\n\n#nav-tree {\n margin-left: 1.5em;\n margin-right: 1.5em;\n margin-top: 1.5em;\n}\n\n#nav-tree > table {\n margin-top: -0.07143em;\n}\n\n#nav-tree table,\n#nav-tree td.nav-title {\n width: 100%;\n}\n\n.nav-entry td.n1 {\n font-size: 1.5em;\n padding-bottom: 0.6667em;\n line-height: 1em;\n}\n\n.nav-entry td.n1 a {\n color: #000000;\n}\n\n.nav-number {\n white-space: nowrap;\n min-width: 1em;\n}\n\n.nav-number.n1 {\n min-width: 0;\n}\n\n.nav-number a {\n padding-right: 0.5em;\n}\n\n.nav-number {\n text-align: right; /* must be right, even for left aligned number (for a continuous underline between number and title) */\n}\n\n.nav-space {\n display: none;\n /*uncomment for left-aligned numbers*/\n /*\n display: inline-block;\n width: 100%;\n height: 0px;\n vertical-align: middle;\n */\n}\n\ntbody.selected {\n background-color: #f8f8f8;\n}\n\n.nav-entry a {\n outline: none;\n}\n\n.nav-entry .selected a {\n color: #000000;\n}\n\n.nav-entry:hover a {\n text-decoration: none;\n border-bottom: 1px solid;\n}\n\n#copyright {\n margin-left: 1.5em;\n margin-top: 0.8em;\n}\n\n#copyright span {\n font-size: 0.9286em;\n margin-left: 0.1em;\n color: #333; \n}\n#copyright a {\n color: #333;\n}\n\n" }, { "path": "Doc/html/css/style-ie.css", "content": "\ncode {\n white-space: nowrap; /* pre breaks the layout, even in ie8 */\n}" }, { "path": "Doc/html/css/style-ie6.css", "content": "\n#fixed-layer {\n display: none;\n}\n\n#main {\n margin-left: 0;\n border-right: solid 1px #d9d9d9;\n}" }, { "path": "Doc/html/css/style.css", "content": "/* reset */\n\nhtml, body, div, span, h1, h2, h3, h4, h5, h6, p,\npre, a, code, em, strong, img, dl, dt, dd, ol, ul, li,\ntable, caption, thead, tbody, tfoot, th, tr, td {\n margin: 0;\n padding: 0;\n border: 0;\n font-family: inherit;\n font-size: 100%;\n font-weight: inherit;\n font-style: inherit;\n vertical-align: baseline;\n text-align:left;\n text-decoration: none;\n}\n\ntable {\n border-collapse: collapse;\n border-spacing: 0;\n}\n\n/* layout */\n\n#fixed-layer {\n display: none;\n}\n\n#wrapper {\n max-width: 47.5em;\n margin-left: auto;\n margin-right: auto;\n}\n\npre {\n overflow:auto;\n}\n\n.interface-member-code pre {\n overflow: visible;\n white-space: pre-wrap; /* css-3 */\n white-space: -moz-pre-wrap; /* Mozilla, since 1999 */\n white-space: -pre-wrap; /* Opera 4-6 */\n white-space: -o-pre-wrap; /* Opera 7 */\n word-wrap: break-word; /* Internet Explorer 5.5+ */\n}\n\n/* typography */\n\nbody {\n font-family: \"Cambria\", \"Georgia\", \"Palatino\", \"Palatino Linotype\", \"Times\", \"Times New Roman\", serif;\n font-size: 87.5%; /* 14px */\n line-height: 1.429em;\n}\n\npre, code, .tt {\n font-family: Consolas, \"DejaVu Sans Mono\", \"Bitstream Vera Sans Mono\", \"Lucida Sans Typewriter\", \"Courier New\", \"Courier\"; /* can't put monospace in there because otherwise WebKit browsers would change the default size */\n font-size: 0.9286em; /* 13px */\n font-size-adjust: 0.461; /* at the same font size Consolas and Courier New have a considerably smaller x-height than the other fonts*/\n line-height: 1.25em;\n}\n\n.title .section-number {\n /* Georgia's old-style figures don't look good in section numbers like 1.2.3.4 */\n font-family: \"Cambria\", /* \"Georgia\", */ \"Palatino\", \"Palatino Linotype\", \"Times\", \"Times New Roman\", serif;\n}\n\nh1, h2, h3, h4, h5, h6 {\n font-weight: normal; /* for IE */\n}\n\nh1 {\n font-size: 1.7143em;\n line-height: 1em;\n margin-top: 0.75em;\n margin-bottom: 0.5833em;\n}\n\nh2, h3 {\n font-size: 1.5em;\n line-height: 1em;\n margin-top: 1em;\n margin-bottom: 0.6666em;\n}\n\n.para {\n margin-bottom: 1em;\n}\n\nul, ol {\n padding-left: 2em;\n}\n\nul { list-style-type: disc; }\nol { list-style-type: decimal; }\n\nul.l2 { list-style-type: circle; }\nul.l3 { list-style-type: square; }\nul.l4 { list-style-type: circle; }\nul.l5 { list-style-type: square; }\n\ndiv.dl {\n margin-top: 1em;\n margin-bottom: 1em;\n}\n\n.dl-title {\n font-size: 1.3em;\n line-height: 1em;\n \n}\n\ndt {\n margin-top: 1em;\n font-style: italic;\n}\n\ndd {\n padding-left: 2em;\n}\n\ndd > .para > ul:first-child {\n padding-left: 0em;\n}\n\n.fn-title {\n font-weight: bold;\n margin-top: 3em;\n margin-bottom: 0.5em;\n}\n\nth.fn {\n padding-right: 1em;\n}\n\nth.fn, td.fn {\n padding-bottom: 1em;\n}\n\nem, .i {\n font-style: italic;\n}\n\nstrong, .b {\n font-weight: bold;\n}\n\n.s {\n text-decoration: line-through;\n}\n\n.small {\n font-size: 0.9em;\n}\n\nsup {\n font-size: 0.85em;\n line-height: 1em;\n}\n\nsup.fn-mark {\n font-size: 0.8em;\n line-height: 1em;\n}\n\n/* design */\n\n#breadcrumbs {\n padding-top: 0.5em;\n margin-bottom: 0.5em;\n}\n\n.toc-toc-title {\n font-size: 1.5em;\n line-height: 1em;\n margin-top: 1em;\n margin-bottom: 0.3333em;\n}\n\n.local-toc ol {\n list-style-type:none;\n padding-left: 0;\n}\n\ntd.toc-title {\n width: 100%;\n}\n\n.toc-number {\n white-space: nowrap;\n min-width: 1em;\n}\n\n.toc-number a {\n padding-right: 0.5em;\n}\n\n.toc-number {\n text-align: right; /* must be right, even for left aligned number (for a continuous underline between number and title) */\n}\n\n.toc-space {\n display: none;\n /*uncomment for left-aligned numbers*/\n /*\n display: inline-block;\n width: 100%;\n height: 0px;\n vertical-align: middle;\n */\n}\n.toc-entry .selected a {\n color: #000000;\n}\n\n.toc-entry a {\n outline: none;\n}\n\n.toc-entry:hover a {\n text-decoration: none;\n border-bottom: 1px solid;\n}\n\na {\n color: #003399;\n}\n\na:hover {\n border-bottom-style: solid;\n border-bottom-width: 1px;\n}\n\na:target, span:target, a:target code, span:target code,\n.interface-member:target > .interface-member-code .interface-member-marker,\n.table:target > table > caption .table-title {\n background-color: #dddddd;\n}\n\ndiv:target > .title > span, div:target > .title > span > code {\n background-color: #e4e4e4;\n}\n\n\n.para.lcinp + .para>.admonition {\n margin-top: 1.36em;\n}\n\n.admonition {\n border-top: 1px solid #999999;\n border-bottom: 1px solid #999999;\n padding-top: 0.2em;\n padding-bottom: 0.3em;\n margin-top: 1em;\n margin-bottom: 1em;\n}\n\n.admonition-title {\n font-weight: bold;\n font-style: italic;\n}\n\n.admonition .para {\n margin-top: 1em;\n margin-bottom: 0em;\n}\n\n.admonition .para._1 {\n margin-top: 0em;\n}\n\ncode {\n white-space: pre;\n margin-left: 1px;\n margin-right: 1px;\n background-color: #f8f8f8;\n}\n\n.code {\n background-color : #f8f8f8;\n padding-bottom: 4px;\n}\n\n.code + .code {\n border-top: 1px solid #dcdcdc;\n padding-top: 3px;\n}\n\np + .code {\n margin-top: 1px;\n}\n\n.code + p {\n margin-top: 1px;\n}\n\n.interface-member-code .code, .interface-code .code {\n padding: 2px;\n border-top: 1px solid #dcdcdc;\n border-bottom: 1px solid #dcdcdc;\n}\n\ncode a, .code a {\n color: #000000;\n text-decoration: none;\n border-bottom: 1px dashed #c8c8c8;\n}\n\ncode a:hover, .code a:hover {\n text-decoration: none;\n border-bottom-style: solid;\n}\n\na code, .title code {\n color: inherit;\n background-color: transparent;\n}\n\n.title code a {\n border-bottom-style:none;\n}\n\n.title code a:hover {\n border-bottom-style:solid;\n}\n\na code span, .title code span {\n color: inherit !important; \n}\n\n/*\n.cb: boolean literal\n.cbc: block comment\n.cc: char literal\n.ce: escaped char\n.cei: escaped identifier\n.ci: identifier\n.cin: invalid\n.ck: keyword\n.clc: line comment\n.cm: member access/scope resolution operator\n.cn: number literal\n.cnu: null literal\n.co: operator\n.col: other literal\n.cp: punctuation\n.cpr: preprocessor\n.cr: right arrow (in functional languages)\n.cra: range operator/ ellipsis\n.cre: reserved\n.cs: string literal\n.ctv: type variable\n.cv: void\n*/\n\nspan.ci /* identifier */,\nspan.tv /* type variable */,\nspan.cb /* boolean literal */,\n\nspan.cn /* number literal */,\nspan.cnu /* null literal */,\nspan.cp /* punctuation */,\nspan.cra /* range operator/ ellipsis */,\nspan.cm /* member access/scope resolution operator */,\nspan.co /* operator */ {\n color: #000000;\n}\n\n\nspan.ck /* keyword */, span.cbt, span.cv {\n color: #0048a0;\n}\n\nspan.cr /* right arrow */ {\n color: #0048a0;\n}\n\nspan.cc /* char literal */, span.cs /* string literal */ {\n color: #940c00;\n}\n\nspan.ce /* escaped char */ {\n color: #d0532c;\n}\n\nspan.clc /* line comment*/, span.cbc /* block comment */ {\n color: #007e1e;\n}\n\nspan.cin /* invalid */ {\n color: White;\n background-color: red;\n}\n\nspan.cpr /* preprocessor */ {\n color: #009293;\n}\n\n/* the following definitions are for debugging the syntax highlighter */\n\n/*\nspan.cp {\n color: red;\n}\n\nspan.co {\n color: Fuchsia;\n}\n*/\n\n/* interface reference */\n\n.interface-member {\n margin-bottom: 2em;\n}\n\n.interface-member-code {\n margin-bottom: 0.5em;\n}\n\n.interface-member-backlink {\n float: right;\n color: #000000;\n font-family: \"Lucida Sans Unicode\", \"Lucida Grande\", \"Lucida Sans\", \"DejaVu Sans\", sans-serif;\n text-decoration: none;\n font-size: 16px;\n line-height: 20px;\n margin-bottom: -2px;\n}\n\n.interface-member-backlink:hover {\n border-bottom: 0;\n}\n\n.interface-member-description {\n padding-left: 2.075em;\n}\n\n.interface-member-marker {\n font-weight: bold;\n}\n\n.interface-member-marker .ck {\n color: #003C85;\n}\n\n\n/* tables */\n\n.table {\n margin-top: 0.2em;\n}\n\n.table table {\n width: 100%;\n border-top: 1px solid #000000;\n border-bottom: 1px solid #000000;\n}\n\n.table thead th {\n font-size: 1.1em;\n font-weight: bold;\n border-top: 1px solid #000000;\n border-bottom: 1px solid #000000;\n}\n\n.table .table-caption-prefix {\n font-style: normal;\n margin-right: 0.2em;\n font-weight: bold;\n}\n\n.table caption {\n font-style: italic;\n padding-bottom: 0.3em;\n}\n\n.table td, .api-table th {\n padding-top: 0.3em;\n padding-bottom: 0.2em;\n}\n\n.table tbody td {\n border-top: 1px solid #dddddd;\n}\n\n.table td._1 {\n padding-right: 1em;\n}\n\n.api-table {\n margin-top: 2em;\n}\n\n#v0_9\\.renamings caption {\n display: none;\n}\n\n.table td > ul {\n padding-left: 1em;\n}\n\n#relative-advantages thead th { padding-left: 1em; }\n#relative-advantages td._1 { width: 34%; padding-right: 1em; }\n#relative-advantages td._2 { width: 35%; padding-right: 1em; }\n\n#relative-disadvantages { margin-top: 1.5em; }\n#relative-disadvantages thead th { padding-left: 1em; }\n#relative-disadvantages td._1 { width: 34%; padding-right: 1em; }\n#relative-disadvantages td._2 { width: 35%; padding-right: 1em; }\n\n/* parser overview tables */\n#parsing-single-chars td._1 { width: 35%; }\n#parsing-strings-directly td._1 { width: 38.5%; }\n#parsing-strings-with-the-help-of-other-parsers td._1 { width: 36%; }\n#parsing-whitespace td._1 { width: 35%; }\n#chaining-and-piping-parsers td._1 { width: 27%; }\n#parsing-sequences td._1 { width: 23%; }\n#parsing-sequences td._2 { width: 22%; }\n#parsing-alternatives-and-recovering-from-errors td._1 { width: 28%; }\n#conditional-parsing-and-looking-ahead td._1 { width: 35%; }\n#customizing-error-messages td._1 { width: 28%; }\n#user-state-handling-and-getting-the-input-stream-position td._1 { width: 24%; }\n" }, { "path": "Doc/html/download-and-installation.html", "content": "\n\n\n \n Download and installation\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
3Download and installation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1NuGet packages
2Getting the source
\n 3\n FParsec is built as two DLLs
\n 4\n Building FParsec from source
5The Low‐Trust version of FParsec
6Configuration options
\n
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Download and installation\n \n
\n
\n

3 Download and installation

\n
\n
\n

FParsec is distributed in source code form and as NuGet packages.

\n
\n
\n

\n If you’re new to FParsec, I’d recommend to start by downloading the source code package and experimenting a bit with the included sample\n projects. With the project and solution files building the library and the samples is as easy as clicking a button.\n

\n
\n
\n

The source package also includes a complete copy of the HTML documentation for offline viewing.

\n
\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1NuGet packages
2Getting the source
3FParsec is built as two DLLs
4Building FParsec from source
5The Low‐Trust version of FParsec
6Configuration options
\n
\n
\n
\n
\n

3.1 NuGet packages

\n
\n
\n

There are two NuGet packages of FParsec, which are built with different configuration options.

\n
\n
\n

\n The basic package uses the Low‐Trust version of FParsec, which\n uses no unverifiable code and is optimized for maximum portability. The\n main limitation of this version is that any input stream is completely read into a string before parsing, which limits the maximum practical\n input size. This package also contains assemblies for .NET Standard 2.0.\n

\n
\n
\n

\n The “Big Data edition” package uses the non‐Low‐Trust version\n of FParsec that is optimized for maximum performance and supports extremely large input streams. Since this configuration is also the default\n configuration of the solution files included with the source code, it is sometimes referred to as the “normal” version of FParsec.\n This version of FParsec does use “unsafe” (i.e. unverifiable) code involving unmanaged pointers. It also uses code generation in\n the implementation of isAnyOf,\n isNoneOf, anyOf, skipAnyOf, noneOf and skipNoneOf. Unfortunately, this version is currently\n not compatible with .NET Standard/.NET Core.\n

\n
\n
\n

\n Should you measure a significant performance degradation when switching to the Big Data edition, you’re probably\n inadvertently recreating the same isAnyOf‐ or isNoneOf‐based parsers again and again, as explained here and here.\n

\n
\n
\n

\n The .NET Framework assemblies in the NuGet packages are strongly signed. Their assembly version numbers will only be incremented for breaking\n changes. The .NET Standard assembly in the FParsec package is not signed.\n

\n
\n
\n

The NuGet packages include PDBs and SourceLink support, which should allow you to step through FParsec code in the debugger of your IDE.

\n
\n
\n
\n
\n

3.2 Getting the source

\n
\n
\n

\n FParsec’s source code repository is hosted on GitHub at: github.com/stephan‐tolksdorf/fparsec\n

\n
\n
\n

\n You can clone the source code using Git or you can download it as a\n zip‐file.\n

\n
\n
\n

\n It’s an FParsec project policy to check only stable and tested code into the master branch of the GitHub repository, so you can normally just\n work with the master version of FParsec.\n

\n
\n
\n
\n
Tip
\n
\n
\n

Fork is a great free GUI for Git for Windows and MacOS.

\n
\n
\n
\n
\n
\n
\n
\n

3.3 FParsec is built as two DLLs

\n
\n
\n

\n FParsec’s source code is written in both C# and F#. Since neither the C# nor the F# compiler directly support the other language, the respective\n components need to be built separately.\n

\n
\n
\n

\n Hence, FParsec is built as two DLLs. The C# bits are compiled into the FParsecCS.dll and the F# bits (which depend on\n the C# bits) are compiled into FParsec.dll.\n

\n
\n
\n

Projects that use FParsec thus have to reference both DLLs.

\n
\n
\n

\n If you reference the DLLs in the F# Interactive console, you need to\n reference FParsecCS.dll before you reference\n FParsec.dll.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If you don’t want to distribute the FParsec DLLs together with the assembly of your project, you can use the staticlink command‐line option of the F# compiler\n to merge the FParsec DLLs into your assembly.\n

\n
\n
\n

\n Unfortunately, the same option cannot be used to merge FParsecCS.dll into the FParsec.dll, as\n the public definitions in FParsecCS.dll wouldn’t be reexported by FParsec.dll. For similar\n reasons it also doesn’t seem to be possible to use tools like ILMerge or il‐repack to obtain a merged FParsec.dll that can be properly consumed\n by F# programs.\n

\n
\n
\n
\n
\n
\n
\n
\n

3.4 Building FParsec from source

\n
\n
\n

\n The solution file FParsec.sln in the root source folder and the associated project files in the subfolders can be used\n to build FParsec from the command line or with IDEs such as Visual Studio 2019 or JetBrains Rider.\n

\n
\n
\n

\n To build the Low‐Trust version of FParsec, you have to specifiy either Debug-LowTrust or Release-LowTrust as the configuration. The Debug and Release configurations build the non‐Low‐Trust version of FParsec, which currently is not\n compatible with the .NET Core runtime.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n In contrast to JetBrains Rider, Visual Studio 2019 currently does not support setting the supported target frameworks depending on the\n configuration. Due to this issue one currently has to use the separate FParsec-LowTrust.sln solution for building the\n Low‐Trust version of FParsec in VS 2019.\n

\n
\n
\n
\n
\n
\n

The Test project in the solution files contains the unit tests for FParsec.

\n
\n
\n

\n The file .vscode/tasks.json\n contains some convenient task definitions for Visual Studio Code.\n

\n
\n
\n

\n The NuGet packages are built with the pack.ps1 PowerShell script.\n

\n
\n
\n
\n
\n

3.5 The Low‐Trust version of FParsec

\n
\n
\n

\n For optimization reasons the normal implementation (the “Big Data edition”) of FParsec involves unverifiable code using unmanaged pointers and runtime code generation.\n

\n
\n
\n

\n If you compile FParsec with the LOW_TRUST conditional compiler symbol, the unverifiable code\n is replaced with a “safe” alternative. This allows FParsec to be run in environments with “reduced trust”, such as\n medium trust ASP.NET applications, and it also allows FParsec to be compiled against reduced subsets of the .NET API.\n

\n
\n
\n

\n In the Debug-LowTrust and Release-LowTrust configurations of the FParsec.sln solution file in the root source folder, LOW_TRUST is\n automatically defined as true.\n

\n
\n
\n

The Low‐Trust version of FParsec has the following two major limitations:

\n
    \n
  • \n A CharStream that is constructed from a\n System.IO.Stream or a file path reads the\n complete file into a single string during construction. This severely limits the maximum practical input stream size.\n
    • \n
  • \n The StaticMapping module is not supported.\n
    • \n
\n
\n
\n
\n
\n

3.6 Configuration options

\n
\n
\n

\n You can configure FParsec’s source code with a number of conditional compilation symbols (a.k.a. preprocessor defines). Besides the Low‐Trust option, these symbols mostly serve tuning purposes.\n

\n
\n
\n
\n
Options for FParsecCS.dll
\n
\n
LOW_TRUST
\n
\n
\n

See above.

\n
\n
\n
\n AGGRESSIVE_INLINING\n
\n
\n
\n

Requires a version of NET ≥ 4.5.

\n
\n
\n

\n Annotates some functions with the MethodImplOptions.AggressiveInlining attribute.\n

\n
\n
\n
PCL
\n
\n
\n

Compile for a PCL subset of the .NET API.

\n
\n
\n
\n SMALL_STATETAG\n
\n
\n
\n

\n Use a 32‐bit StateTag in the CharStream class instead of the default 64‐bit one.\n

\n
\n
\n

\n This is an optimization for 32‐bit runtimes. You can find more information about the state tag in section 5.4.3 of the user’s guide.\n

\n
\n
\n
\n UNALIGNED_READS\n
\n
\n
\n

\n This option does not affect the Low‐Trust version of\n FParsec.
\n

\n
\n
\n

Optimize for CPUs that support fast unaligned memory reads, i.e. any modern x86‐based CPU.

\n
\n
\n

This option only makes a noticeable difference is some specific situations.

\n
\n
\n
\n
\n
\n
\n
\n
Options for FParsec.dll
\n
\n
LOW_TRUST
\n
\n
\n

See above.

\n
\n
\n
UNALIGNED_READS
\n
\n
\n

See above.

\n
\n
\n
NOINLINE
\n
\n
\n

Do not force inlining of certain parser combinators.

\n
\n
\n

This option enables you to step through the respective combinators during debugging.

\n
\n
\n
\n USE_STATIC_MAPPING_FOR_IS_ANY_OF\n
\n
\n
\n

\n This option does not affect the Low‐Trust version of\n FParsec.
\n

\n
\n
\n

\n Use StaticMapping.createStaticCharIndicatorFunction for the implementation of isAnyOf, isNoneOf, anyOf, skipAnyOf, noneOf and skipNoneOf for generating optimized char predicate\n functions using runtime code generation.\n

\n
\n
\n

\n Runtime code generation is a relatively expensive operation, so this optimization is primarily meant for parsers that are applied to large\n (or lots of) input streams. Please see the remarks for the StaticMapping module for more information.\n

\n
\n
\n

\n If you run into noticeable performance problems or memory leaks when enabling this option, you’re probably inadvertently recreating the same\n isAnyOf‐ or isNoneOf‐based parser again and\n again, as explained here and here.\n

\n
\n
\n
\n DEBUG_STATIC_MAPPING\n
\n
\n
\n

\n This option does not affect the Low‐Trust version of\n FParsec.
\n

\n
\n
\n

\n See StaticMapping\n documentation.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/index.html", "content": "\n\n\n \n FParsec Documentation\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation\n \n
\n
\n

FParsec Documentation

\n
\n
\n

\n FParsec is a parser combinator library for F#.\n

\n
\n
\n

\n With FParsec you can implement recursive‐descent text parsers for formal grammars.\n

\n
\n
\n

FParsec’s features include:

\n
    \n
  • support for context‐sensitive, infinite look‐ahead grammars,
    • \n
  • automatically generated, highly readable error messages,
    • \n
  • Unicode support,
    • \n
  • efficient support for very large files,
    • \n
  • \n an embeddable, runtime‐configurable operator‐precedence parser component,\n
    • \n
  • a simple, efficient and easily extensible API,
    • \n
  • an implementation thoroughly optimized for performance,
    • \n
  • comprehensive documentation,
    • \n
  • a permissive open source license.
    • \n
\n
\n
\n

\n FParsec is an F# adaptation of Parsec, the popular parser combinator library for Haskell\n by Daan Leijen. While the implementations of Parsec and FParsec are completely\n different, they share a similar top‐level API.\n

\n
\n
\n

\n Latest release: FParsec 2.0.0, 2022‒11‒01, Download, NuGet packages, Changes\n

\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
3Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/license.html", "content": "\n\n\n \n License\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n Simplified BSD License
\n 2\n \n Creative Commons Attribution‐NonCommercial 3.0 Unported\n License\n
\n
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > License\n \n
\n
\n

2 License

\n
\n
\n

\n Except where noted otherwise, the FParsec library in source and binary form is distributed under the Simplified\n BSD License. The Simplified BSD License (a.k.a. “2‐clause BSD License”) is a simple, permissive license that is OSI‐compliant.\n

\n
\n
\n

\n FParsec incorporates data derived from the Unicode Character Database v. 8.0.0, Copyright (c) 1991‒2015\n Unicode, Inc., which is distributed under the following terms:
http://www.unicode.org/terms_of_use.html#Exhibit1\n

\n
\n
\n

\n The documentation in the Doc folder is licensed under the Creative Commons Attribution‐NonCommercial 3.0 Unported License. This\n Creative Commons license does not allow you to use the documentation for commercial purposes without permission. This means, for example, that\n you cannot sell the documentation in book form for profit or put it on a web content farm in order to earn money with ads. However, you can\n of course use the documentation in a commercial context (e.g. put it on the intranet of a commercial corporation), as long as you’re not\n trying to directly earn money from the text of the documentation.\n

\n
\n
\n
\n

2.1 Simplified BSD License

\n
\n
\n

Copyright (c) 2007‒2022, Stephan Tolksdorf. All rights reserved.

\n
\n
\n

\n Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n

\n
    \n
  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
    • \n
  • \n Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the\n documentation and/or other materials provided with the distribution.\n
    • \n
\n
\n
\n

\n This software is provided by the copyright holders “as is” and any express or implied warranties, including, but not limited\n to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders be\n liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of\n substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in\n contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of\n the possibility of such damage.\n

\n
\n
\n
\n
\n

2.2 Creative Commons Attribution‐NonCommercial 3.0 Unported License

\n
\n
\n

Summary: https://creativecommons.org/licenses/by-nc/3.0/

\n
\n
\n

\n The full license text: https://creativecommons.org/licenses/by-nc/3.0/legalcode\n

\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/charparsers.html", "content": "\n\n\n \n FParsec.CharParsers\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Members
\n
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.CharParsers\n \n
\n
\n

6.3 FParsec.CharParsers

\n
\n

6.3.1 Interface

\n
\n
\n
\n
// FParsec.dll\n\n[<AutoOpen>] // module is automatically opened when FParsec namespace is opened\nmodule FParsec.CharParsers\n\nopen FParsec.Error\nopen FParsec.Primitives\n\n// Running parsers on input\n// ========================\ntype ParserResult<'Result,'UserState>=\n     | Success of 'Result * 'UserState * Position\n     | Failure of string * ParserError * 'UserState\n\nval runParserOnString:\n         Parser<'a,'u> -> 'u -> streamName: string -> string\n      -> ParserResult<'a,'u>\n\nval runParserOnSubstring:\n        Parser<'a,'u> -> 'u -> streamName: string -> string -> int -> int\n     -> ParserResult<'a,'u>\n\nval runParserOnStream:\n        Parser<'a,'u> -> 'u -> streamName: string\n     -> System.IO.Stream -> System.Text.Encoding\n     -> ParserResult<'a,'u>\n\nval runParserOnFile:\n        Parser<'a,'u> -> 'u -> path: string -> System.Text.Encoding\n     -> ParserResult<'a,'u>\n\nval run: Parser<'a, unit> -> string -> ParserResult<'a,unit>\n\n// Reading the input stream position and handling the user state\n// =============================================================\nval getPosition: Parser<Position,'u>\n\nval getUserState: Parser<'u,'u>\nval setUserState: 'u -> Parser<unit,'u>\nval updateUserState: ('u -> 'u) -> Parser<unit,'u>\n\nval userStateSatisfies: ('u -> bool) -> Parser<unit,'u>\n\n// Parsing single chars\n// ====================\nval pchar:      char ->       Parser<char,'u>\nval skipChar:   char ->       Parser<unit,'u>\nval charReturn: char -> 'a -> Parser<'a,'u>\n\nval anyChar:     Parser<char,'u>\nval skipAnyChar: Parser<unit,'u>\n\nval satisfy:      (char -> bool)           -> Parser<char,'u>\nval skipSatisfy:  (char -> bool)           -> Parser<unit,'u>\nval satisfyL:     (char -> bool) -> string -> Parser<char,'u>\nval skipSatisfyL: (char -> bool) -> string -> Parser<unit,'u>\n\nval anyOf:      seq<char> -> Parser<char,'u>\nval skipAnyOf:  seq<char> -> Parser<unit,'u>\nval noneOf:     seq<char> -> Parser<char,'u>\nval skipNoneOf: seq<char> -> Parser<unit,'u>\n\nval asciiLower:  Parser<char,'u>\nval asciiUpper:  Parser<char,'u>\nval asciiLetter: Parser<char,'u>\n\nval lower:  Parser<char,'u>\nval upper:  Parser<char,'u>\nval letter: Parser<char,'u>\n\nval digit: Parser<char,'u> // parses '0'-'9'\nval hex:   Parser<char,'u> // parses '0'-'9', 'a'-'f', 'A'-'F'\nval octal: Parser<char,'u> // parses '0'-'7'\n\n// predicate functions corresponding to the above parsers\nval isAnyOf:  seq<char> ->  (char -> bool)\nval isNoneOf: seq<char> ->  (char -> bool)\nval inline isAsciiUpper:  char -> bool\nval inline isAsciiLower:  char -> bool\nval inline isAsciiLetter: char -> bool\nval inline isUpper:       char -> bool\nval inline isLower:       char -> bool\nval inline isLetter:      char -> bool\nval inline isDigit:       char -> bool\nval inline isHex:         char -> bool\nval inline isOctal:       char -> bool\n\n// Parsing whitespace\n// ==================\nval tab:                 Parser<char,'U>\n\nval newline:             Parser<char,'u>\nval skipNewline:         Parser<unit,'u>\nval newlineReturn: 'a -> Parser<'a,'u>\n\nval unicodeNewline:             Parser<char,'u>\nval skipUnicodeNewline:         Parser<unit,'u>\nval unicodeNewlineReturn: 'a -> Parser<'a,'u>\n\nval spaces:  Parser<unit,'u>\nval spaces1: Parser<unit,'u>\n\nval unicodeSpaces:  Parser<unit,'u>\nval unicodeSpaces1: Parser<unit,'u>\n\nval eof: Parser<unit,'u>\n\n// Parsing strings directly\n// ========================\nval pstring:      string ->       Parser<string,'u>\nval skipString:   string ->       Parser<unit,'u>\nval stringReturn: string -> 'a -> Parser<'a,'u>\n\nval pstringCI:      string ->       Parser<string,'u>\nval skipStringCI:   string ->       Parser<unit,'u>\nval stringCIReturn: string -> 'a -> Parser<'a,'u>\n\nval anyString:     int32 -> Parser<string,'u>\nval skipAnyString: int32 -> Parser<unit,'u>\n\nval restOfLine:     skipNewline: bool -> Parser<string,'u>\nval skipRestOfLine: skipNewline: bool -> Parser<unit,'u>\n\nval charsTillString:\n    string -> skipString: bool -> maxCount: int -> Parser<string,'u>\nval skipCharsTillString:\n    string -> skipString: bool -> maxCount: int -> Parser<unit,'u>\n\nval charsTillStringCI:\n    string -> skipString: bool -> maxCount: int -> Parser<string,'u>\nval skipCharsTillStringCI:\n    string -> skipString: bool -> maxCount: int -> Parser<unit,'u>\n\nval manySatisfy:       (char -> bool)                   -> Parser<string,'u>\nval manySatisfy2:      (char -> bool) -> (char -> bool) -> Parser<string,'u>\nval skipManySatisfy:   (char -> bool)                   -> Parser<unit,'u>\nval skipManySatisfy2:  (char -> bool) -> (char -> bool) -> Parser<unit,'u>\n\nval many1Satisfy:      (char -> bool)                   -> Parser<string,'u>\nval many1Satisfy2:     (char -> bool) -> (char -> bool) -> Parser<string,'u>\nval skipMany1Satisfy:  (char -> bool)                   -> Parser<unit,'u>\nval skipMany1Satisfy2: (char -> bool) -> (char -> bool) -> Parser<unit,'u>\n\nval many1SatisfyL:\n    (char -> bool)                   -> string -> Parser<string,'u>\nval many1Satisfy2L:\n    (char -> bool) -> (char -> bool) -> string -> Parser<string,'u>\nval skipMany1SatisfyL:\n    (char -> bool)                   -> string -> Parser<unit,'u>\nval skipMany1Satisfy2L:\n    (char -> bool) -> (char -> bool) -> string -> Parser<unit,'u>\n\nval manyMinMaxSatisfy:\n    int -> int -> (char -> bool)                   -> Parser<string,'u>\nval manyMinMaxSatisfy2:\n    int -> int -> (char -> bool) -> (char -> bool) -> Parser<string,'u>\nval skipManyMinMaxSatisfy:\n    int -> int -> (char -> bool)                   -> Parser<unit,'u>\nval skipManyMinMaxSatisfy2:\n    int -> int -> (char -> bool) -> (char -> bool) -> Parser<unit,'u>\nval manyMinMaxSatisfyL:\n    int -> int -> (char -> bool)                   -> string -> Parser<string,'u>\nval manyMinMaxSatisfy2L:\n    int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser<string,'u>\nval skipManyMinMaxSatisfyL:\n    int -> int -> (char -> bool)                   -> string -> Parser<unit,'u>\nval skipManyMinMaxSatisfy2L:\n    int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser<unit,'u>\n\nval regex:  string -> Parser<string,'u>\n\ntype IdentifierOptions =\n    new: ?isAsciiIdStart: (char -> bool) *\n         ?isAsciiIdContinue: (char -> bool) *\n         ?normalization: System.Text.NormalizationForm *\n         ?normalizeBeforeValidation: bool *\n         ?allowJoinControlChars: bool *\n         ?preCheckStart: (char -> bool) *\n         ?preCheckContinue: (char -> bool) *\n         ?allowAllNonAsciiCharsInPreCheck: bool *\n         ?label: string *\n         ?invalidCharMessage: string -> IdentifierOptions\n\nval identifier: IdentifierOptions -> Parser<string, 'u>\n\n// Parsing strings with the help of other parsers\n// ==============================================\n\nval manyChars:   Parser<char,'u>                    -> Parser<string,'u>\nval manyChars2:  Parser<char,'u> -> Parser<char,'u> -> Parser<string,'u>\n\nval many1Chars:  Parser<char,'u>                    -> Parser<string,'u>\nval many1Chars2: Parser<char,'u> -> Parser<char,'u> -> Parser<string,'u>\n\nval manyCharsTill:\n       Parser<char,'u>                    -> Parser<'b,'u> -> Parser<string,'u>\nval manyCharsTill2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\nval manyCharsTillApply:\n       Parser<char,'u>                    -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\nval manyCharsTillApply2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n\nval many1CharsTill:\n       Parser<char,'u>                    -> Parser<'b,'u> -> Parser<string,'u>\nval many1CharsTill2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\nval many1CharsTillApply:\n       Parser<char,'u>                    -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\nval many1CharsTillApply2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n\nval manyStrings:   Parser<string,'u>                      -> Parser<string,'u>\nval manyStrings2:  Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\nval many1Strings:  Parser<string,'u>                      -> Parser<string,'u>\nval many1Strings2: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n\nval stringsSepBy:  Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\nval stringsSepBy1: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n\nval skipped: Parser<unit,'u> -> Parser<string,'u>\n\nval withSkippedString: (string -> 'a -> 'b) -> Parser<'a,'u> -> Parser<'b,'u>\n\n// Parsing numbers\n// ===============\ntype NumberLiteralOptions = //...\n\ntype NumberLiteral = //...\n\nval numberLiteral:  NumberLiteralOptions -> string -> Parser<NumberLiteral,'u>\nval numberLiteralE:\n       NumberLiteralOptions -> errorInCaseNoLiteralFound: ErrorMessageList\n    -> CharStream<'u> -> Reply<NumberLiteral>\n\nval pfloat: Parser<float,'u>\n\nval pint64: Parser<int64,'u>\nval pint32: Parser<int32,'u>\nval pint16: Parser<int16,'u>\nval pint8:  Parser<int8,'u>\n\nval puint64: Parser<uint64,'u>\nval puint32: Parser<uint32,'u>\nval puint16: Parser<uint16,'u>\nval puint8:  Parser<uint8,'u>\n\n// Conditional parsing\n// ===================\n\nval notFollowedByEof: Parser<unit,'u>\n\nval followedByNewline: Parser<unit,'u>\nval notFollowedByNewline: Parser<unit,'u>\n\nval followedByString:      string -> Parser<unit,'u>\nval followedByStringCI:    string -> Parser<unit,'u>\nval notFollowedByString:   string -> Parser<unit,'u>\nval notFollowedByStringCI: string -> Parser<unit,'u>\n\nval nextCharSatisfies:        (char -> bool)         -> Parser<unit,'u>\nval nextCharSatisfiesNot:     (char -> bool)         -> Parser<unit,'u>\nval next2CharsSatisfy:        (char -> char -> bool) -> Parser<unit,'u>\nval next2CharsSatisfyNot:     (char -> char -> bool) -> Parser<unit,'u>\nval previousCharSatisfies:    (char -> bool)         -> Parser<unit,'u>\nval previousCharSatisfiesNot: (char -> bool)         -> Parser<unit,'u>\n\n// Helper functions\n// ================\n[<Literal>]\nval EOS: char = CharStream.Iterator.EndOfStreamChar\n\nval foldCase: string -> string\n\nval normalizeNewlines: string -> string\n\nval floatToHexString:   float -> string\nval floatOfHexString:   string -> float\nval float32ToHexString: float32 -> string\nval float32OfHexString: string -> float32\n
\n
\n
\n
\n
\n
\n

6.3.2 Members

\n
\n
\n
\n
\n
\n ⇧\n
type ParserResult<'Result,'UserState>
\n
\n
\n
\n

\n Values of this union type are returned by the runParser functions (not by Parser<_,_> functions).\n

\n
\n
\n
\n
\n
\n ⇧\n
| Success of 'Result * 'UserState * Position\n
\n
\n
\n
\n

\n Success(result, userState, endPos) holds the result and the user state returned by a successful parser, together with the position where the\n parser stopped.\n

\n
\n
\n
\n
\n
\n ⇧\n
| Failure of string * ParserError * 'UserState\n
\n
\n
\n
\n

\n Failure(errorAsString, error, userState) holds the parser error and the user state returned by a failing parser, together with the string\n representation of the parser error. The ParserError value error contains an ErrorMessageList and the position and user state value\n associated with the error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val runParserOnString:\n         Parser<'a,'u> -> 'u -> streamName: string -> string\n      -> ParserResult<'a,'u>\n
\n
\n
\n
\n

\n runParserOnString p ustate streamName str runs the parser p on the\n content of the string str, starting with the initial user state ustate. The streamName is used in error\n messages to describe the source of the input (e.g. a file path) and may be empty. The parser’s Reply is captured and returned as a ParserResult value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val runParserOnSubstring:\n        Parser<'a,'u> -> 'u -> streamName: string -> string -> int -> int\n     -> ParserResult<'a,'u>\n
\n
\n
\n
\n

\n runParserOnSubstring p ustate streamName str index count runs the parser\n p directly on the content of the string str between the indices index (inclusive) and index + count (exclusive), starting with the\n initial user state ustate. The streamName is used in error messages to describe the source of the input (e.g. a file path) and may be empty. The\n parser’s Reply is captured and returned as a ParserResult value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val runParserOnStream:\n        Parser<'a,'u> -> 'u -> streamName: string\n     -> System.IO.Stream -> System.Text.Encoding\n     -> ParserResult<'a,'u>\n
\n
\n
\n
\n

\n runParserOnStream p ustate streamName stream encoding runs the parser p on the content of the System.IO.Stream stream, starting with the initial user state ustate. The streamName is used in error messages to describe the source of the input (e.g. a file path)\n and may be empty. In case no Unicode byte order mark is found, the stream data is assumed to be encoded with the given encoding. The parser’s Reply is captured and returned as a ParserResult value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val runParserOnFile:\n        Parser<'a,'u> -> 'u -> path: string -> System.Text.Encoding\n     -> ParserResult<'a,'u>\n
\n
\n
\n
\n

\n runParserOnFile p ustate path encoding runs the parser p on the\n content of the file at the given path, starting with the initial user state ustate. In case no Unicode byte order mark is found, the file data is assumed to be encoded\n with the given encoding. The parser’s Reply is captured and returned as a ParserResult value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val run: Parser<'a, unit> -> string -> ParserResult<'a,unit>\n
\n
\n
\n
\n

\n run parser str is a convenient\n abbreviation for runParserOnString parser () \"\" str.\n

\n
\n
\n
\n
\n
\n ⇧\n
val getPosition: Parser<Position,'u>\n
\n
\n
\n
\n

The parser getPosition returns the current position in the input stream.

\n
\n
\n

\n getPosition is defined as fun stream -> Reply(stream.Position).\n

\n
\n
\n
\n
\n
\n ⇧\n
val getUserState: Parser<'u,'u>\n
\n
\n
\n
\n

The parser getUserState returns the current user state.

\n
\n
\n

\n getUserState is defined as fun stream -> Reply(stream.UserState).\n

\n
\n
\n
\n
\n
\n ⇧\n
val setUserState: 'u -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser setUserState u sets the user state to u.\n

\n
\n
\n

setUserState u is defined as

\n
fun stream ->\n    stream.UserState <- u\n    Reply(())\n
\n
\n
\n
\n
\n
\n ⇧\n
val updateUserState: ('u -> 'u) -> Parser<unit,'u>\n
\n
\n
\n
\n

updateUserState f is defined as

\n
fun stream ->\n    stream.UserState <- f stream.UserState\n    Reply(())\n
\n
\n
\n
\n
\n
\n ⇧\n
val userStateSatisfies: ('u -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser userStateSatisfies f succeeds if the predicate\n function f returns true when applied\n to the current UserState, otherwise it fails.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If the parser userStateSatisfies f fails, it returns no\n descriptive error message; hence it should only be used together with other parsers that take care of a potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val pchar: char -> Parser<char,'u>\n
\n
\n
\n
\n

\n pchar c parses the char c and returns c. If c = '\\r' or c = '\\n' then pchar c will parse any one newline (\"\\n\", \"\\r\\n\" or \"\\r\") and return\n c.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipChar: char -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipChar c is an optimized implementation of pchar c |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val charReturn: char -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n charReturn c result is an optimized\n implementation of pchar c >>% result.\n

\n
\n
\n
\n
\n
\n ⇧\n
val anyChar: Parser<char,'u>\n
\n
\n
\n
\n

\n anyChar parses any single char or newline (\"\\n\", \"\\r\\n\" or \"\\r\"). Returns the\n parsed char, or '\\n' in case a newline was parsed.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipAnyChar: Parser<unit,'u>\n
\n
\n
\n
\n

\n skipAnyChar is an optimized implementation of anyChar |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val satisfy: (char -> bool) -> Parser<char,'u>\n
\n
\n
\n
\n

\n satisfy f parses any one char or newline for which the predicate\n function f returns true. It returns\n the parsed char. Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'. Thus, to accept a newline f '\\n' must\n return true. f will never be called\n with '\\r' and\n satisfy f will never return the result '\\r'.\n

\n
\n
\n

\n For example, satisfy (fun c -> '0' <= c && c <= '9') parses\n any decimal digit.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If the parser satisfy f fails, it returns no descriptive\n error message (because it does not know what chars f accepts); hence it should only\n be used together with other parsers that take care of a potential error. Alternatively, satisfyL f label can be used\n to ensure a more descriptive error message.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val skipSatisfy: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipSatisfy f is an optimized implementation of satisfy f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val satisfyL: (char -> bool) -> string -> Parser<char,'u>\n
\n
\n
\n
\n

\n satisfy f label is an optimized implementation of satisfy f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipSatisfyL: (char -> bool) -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipSatisfyL f label is an optimized\n implementation of skipSatisfy f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val anyOf: seq<char> -> Parser<char,'u>\n
\n
\n
\n
\n

\n anyOf str parses any char contained in the char sequence chars. It returns the parsed char. If chars contains the char '\\n', anyOf chars parses any newline (\"\\n\", \"\\r\\n\" or \"\\r\") and returns it as '\\n'. (Note that it does not make a\n difference whether or not chars contains '\\r' and that anyOf chars will never return '\\r'.)\n

\n
\n
\n

\n For example, anyOf \". \\t\\n\" will parse any of the chars '.', ' ', '\\t' or any newline.\n

\n
\n
\n

\n anyOf chars is defined as satisfy (isAnyOf chars).\n

\n
\n
\n

\n For performance critical parsers it might be worth replacing instances of anyOf in loops\n with a manySatisfy‐based parser. For example, manyChars (anyOf \". \\t\\n\") could be replaced with manySatisfy (function '.'|' '|'\\t'|'\\n' -> true | _ -> false).\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipAnyOf: seq<char> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipAnyOf chars is an optimized implementation of anyOf chars |>> ignore.\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val noneOf: seq<char> -> Parser<char,'u>\n
\n
\n
\n
\n

\n noneOf chars parses any char not contained in the char sequence\n chars. It returns the parsed char. If chars does not contain the char '\\n', noneOf chars parses any newline (\"\\n\", \"\\r\\n\" or \"\\r\") and returns it as as '\\n'. (Note that it does not make a\n difference whether or not chars contains '\\r' and that noneOf chars will never return '\\r'.)\n

\n
\n
\n

\n For example, noneOf \". \\t\\n\" will parse any char other than '.', ' ', '\\t', '\\r' or '\\n'.\n

\n
\n
\n

\n noneOf chars is defined as satisfy (isNoneOf chars).\n

\n
\n
\n

\n For performance critical parsers it might be worth replacing instances of noneOf in\n loops with a manySatisfy‐based parser. For example,\n manyChars (noneOf \". \\t\\n\") could be replaced with manySatisfy (function '.'|' '|'\\t'|'\\n' -> false | _ -> true).\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipNoneOf: seq<char> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipNoneOf chars is an optimized implementation of noneOf chars |>> ignore.\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val asciiLower: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any char in the range 'a' ‐\n 'z'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val asciiUpper: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any char in the range 'A' ‐\n 'Z'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val asciiLetter: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any char in the range 'a' ‐\n 'z' and 'A' ‐ 'Z'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val lower: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any UTF‐16 lowercase letter char identified by System.Char.IsLower. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val upper: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any UTF‐16 uppercase letter char identified by System.Char.IsUpper. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val letter: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses any UTF‐16 letter char identified by System.Char.IsLetter. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val digit: Parser<char,'u>
\n
\n
\n
\n

\n Parses any char in the range '0' ‐\n '9'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val hex: Parser<char,'u>
\n
\n
\n
\n

\n Parses any char in the range '0' ‐\n '9', 'a' ‐ 'f' and 'A' ‐ 'F'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val octal: Parser<char,'u>
\n
\n
\n
\n

\n Parses any char in the range '0' ‐\n '7'. Returns the parsed char.\n

\n
\n
\n
\n
\n
\n ⇧\n
val isAnyOf: seq<char> -> (char -> bool)\n
\n
\n
\n
\n

\n isAnyOf chars returns a predicate function. When this predicate\n function is applied to a char, it returns true if and only if the char is contained in\n the char sequence chars.\n

\n
\n
\n

\n For example, the function isAnyOf \".,;\" returns true when applied to the chars '.', ',' or ';', and false for all other\n chars.\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val isNoneOf: seq<char> -> (char -> bool)\n
\n
\n
\n
\n

\n isNoneOf chars returns a predicate function. When this predicate\n function is applied to a char, it returns true if and only if the char is not contained\n in char sequence chars.\n

\n
\n
\n

\n For example, the function isNoneOf \".,;\" returns false when applied to the chars '.', ',' or ';', and true for all other\n chars.\n

\n
\n
\n

\n This function is affected by the USE_STATIC_MAPPING_FOR_IS_ANY_OF compilation option.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isAsciiUpper: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range 'A' ‐ 'Z' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isAsciiLower: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range 'a' ‐ 'z' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isAsciiLetter: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range 'a' ‐ 'z', 'A' ‐ 'Z' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isUpper: char -> bool\n
\n
\n
\n
\n

\n isUpper is equivalent to System.Char.IsUpper.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isLower: char -> bool\n
\n
\n
\n
\n

\n isLower is equivalent to System.Char.IsLower.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isLetter: char -> bool\n
\n
\n
\n
\n

\n isLetter is equivalent to System.Char.IsLetter.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isDigit: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range '0' ‐ '9' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isHex: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range '0' ‐ '9', 'a' ‐ 'f', 'A' ‐ 'F' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val inline isOctal: char -> bool\n
\n
\n
\n
\n

\n Returns true for any char in the range '0' ‐ '7' and false for all other chars.\n

\n
\n
\n
\n
\n
\n ⇧\n
val tab: Parser<char,'U>\n
\n
\n
\n
\n

\n Parses the tab char '\\t' and returns '\\t'.\n

\n
\n
\n
\n
Note
\n
\n
\n

A tab char is treated like any other non‐newline char: the column number is incremented by (only) 1.

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val newline: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses a newline (\"\\n\", \"\\r\\n\" or \"\\r\"). Returns '\\n'. Is equivalent to pchar '\\n'.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipNewline: Parser<unit,'u>\n
\n
\n
\n
\n

\n skipNewline is an optimized implementation of newline |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val newlineReturn: 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n newlineReturn result is an optimized implementation of newline >>% result.\n

\n
\n
\n
\n
\n
\n ⇧\n
val unicodeNewline: Parser<char,'u>\n
\n
\n
\n
\n

\n Parses a Unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\", or \"\\u2029\"). Returns '\\n'. In contrast to all other parsers in FParsec except\n unicodeWhitespace this parser also increments the internal line count for Unicode\n newline characters other than '\\n' and '\\r'.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This method does not recognize the form feed char '\\f' ('\\u000C') as a newline character.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

\n This parser is included only for the sake of completeness. If you design your own parser grammar, we recommend not to accept any\n character sequence other than \"\\n\", \"\\r\\n\" or \"\\r\" for a newline. The three usual newline representations already make text\n parsing complicated enough.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val skipUnicodeNewline: Parser<unit,'u>\n
\n
\n
\n
\n

\n skipUnicodeNewline is an optimized implementation of newline |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val unicodeNewlineReturn: 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n unicodeNewlineReturn result is an optimized implementation of\n newline >>% result.\n

\n
\n
\n
\n
\n
\n ⇧\n
val spaces: Parser<unit,'u>\n
\n
\n
\n
\n

\n Skips over any sequence of zero or more whitespaces (space (' '), tab ('\\t') or newline (\"\\n\", \"\\r\\n\" or \"\\r\")).\n

\n
\n
\n
\n
\n
\n ⇧\n
val spaces1: Parser<unit,'u>\n
\n
\n
\n
\n

\n Skips over any sequence of one or more whitespaces (space (' '), tab('\\t') or newline (\"\\n\", \"\\r\\n\" or \"\\r\")).\n

\n
\n
\n
\n
\n
\n ⇧\n
val unicodeSpaces: Parser<unit,'u>\n
\n
\n
\n
\n

\n Skips over any sequence of zero or more Unicode whitespace chars and registers any Unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\"or \"\\u2029\") as a\n newline.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This method does not recognize the form feed char '\\f' ('\\u000C') as a newline character.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

\n This parser is included only for the sake of completeness. If you design your own parser grammar, we recommend not to accept any\n whitespace character other than ' ', '\\t', '\\r' and '\\n'. There is no need to make whitespace parsing unnecessary complicated and slow.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val unicodeSpaces1: Parser<unit,'u>\n
\n
\n
\n
\n

\n Skips over any sequence of one or more Unicode whitespace char and registers any Unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\"or \"\\u2029\") as a\n newline.\n

\n
\n
\n

See also the notes above for unicodeSpaces.

\n
\n
\n
\n
\n
\n ⇧\n
val eof: Parser<unit,'u>\n
\n
\n
\n
\n

The parser eof only succeeds at the end of the input. It never consumes input.

\n
\n
\n
\n
\n
\n ⇧\n
val pstring: string -> Parser<string,'u>\n
\n
\n
\n
\n

\n pstring str parses the string str and returns str. It is an atomic parser: either it succeeds\n or it fails without consuming any input.\n

\n
\n
\n

\n str may not contain newline chars ('\\n' or '\\r'), otherwise pstring str raises an ArgumentException.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipString: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipString str is an optimized implementation of pstring str |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val stringReturn: string -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n stringReturn str result is an optimized\n implementation of pstring str >>% result.\n

\n
\n
\n
\n
\n
\n ⇧\n
val pstringCI: string -> Parser<string,'u>\n
\n
\n
\n
\n

\n pstringCI str parses any string that case‐insensitively matches\n the string str. It returns the parsed string. pstringCI str is an atomic parser: either it succeeds or it fails without consuming any\n input.\n

\n
\n
\n

\n str may not contain newline chars ('\\n' or '\\r'), otherwise pstringCI str raises an ArgumentException.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipStringCI: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipStringCI str is an optimized implementation of pstringCI str |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val stringCIReturn: string -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n stringCIReturn str result is an\n optimized implementation of pstringCI str >>% result.\n

\n
\n
\n
\n
\n
\n ⇧\n
val anyString: int32 -> Parser<string,'u>\n
\n
\n
\n
\n

\n anyString n parses any sequence of n chars or newlines (\"\\n\", \"\\r\\n\" or \"\\r\"). It returns the parsed string. In the returned\n string all newlines are normalized to \"\\n\". anyString n is an atomic\n parser: either it succeeds or it fails without consuming any input.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipAnyString: int32 -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipAnyString n is an optimized implementation of anyString n |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val restOfLine: skipNewline: bool -> Parser<string,'u>\n
\n
\n
\n
\n

\n restOfLine skipNewline parses any chars before the end of the\n line and, if skipNewline is true,\n skips to the beginning of the next line (if there is one). It returns the parsed chars before the end of the line as a string (without a\n newline). A line is terminated by a newline (\"\\n\", \"\\r\\n\" or \"\\r\") or the end of the input stream.\n

\n
\n
\n

\n For example, sepBy (restOfLine false) newline will parse an input file and split it into lines:\n

\n
\n
\n
> run (sepBy (restOfLine false) newline) \"line1\\nline2\\n\";;\nval it : ParserResult<string list,unit> = Success: [\"line1\"; \"line2\"; \"\"]\n
\n
\n
\n

\n Note that you could not use many (restOfLine true) in this example, because\n at the end of the input restOfLine succeeds without consuming input, which would cause\n many to throw an exception.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipRestOfLine: skipNewline: bool -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipRestOfLine skipNewline is an optimized implementation of\n restOfLine skipNewline |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val charsTillString:\n    string -> skipString: bool -> maxCount: int -> Parser<string,'u>\n
\n
\n
\n
\n

\n charsTillString skipString maxCount\n parses all chars before the first occurance of the string str and, if skipString is true, skips over str. It returns the parsed chars before the string. If more than maxCount chars come before the first occurance of str, the\n parser fails after consuming maxCount chars.\n

\n
\n
\n

\n Newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are counted as single chars and in the returned string all newlines are\n normalized to \"\\n\", but str may not contain any newline.\n

\n
\n
\n

\n charsTillString str maxCount raises\n

\n \n
\n
\n
\n
\n
\n ⇧\n
val skipCharsTillString:\n    string -> skipString: bool -> maxCount: int -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipCharsTillString str maxCount is an\n optimized implementation of charsTillString str maxCount |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val charsTillStringCI:\n    string -> skipString: bool -> maxCount: int -> Parser<string,'u>\n
\n
\n
\n
\n

\n charsTillStringCI str maxCount parses\n all chars before the first case‐insensitive occurance of the string str and, if skipString is true, skips over it. It\n returns the parsed chars before the string. If more than maxCount chars come before the\n first case‐insensitive occurance of str the parser fails after consuming maxCount chars.\n

\n
\n
\n

\n Newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are counted as single chars, but str may not contain any newline.\n

\n
\n
\n

\n charsTillStringCI str maxCount raises\n

\n \n
\n
\n
\n
\n
\n ⇧\n
val skipCharsTillStringCI:\n    string -> skipString: bool -> maxCount: int -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipCharsTillStringCI str maxCount is an\n optimized implementation of charsTillStringCI str maxCount |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manySatisfy: (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n manySatisfy f parses a sequence of zero or more chars\n that satisfy the predicate function f (i.e. chars for which f returns true). It returns the parsed chars as a string.\n

\n
\n
\n

\n Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'. Thus, to accept a newline f '\\n' must\n return true. f will never be called\n with '\\r' and\n the string returned by manySatisfy f will never contain an '\\r'.\n

\n
\n
\n

\n For example, manySatisfy (function ' '|'\\t'|'\\n' -> true | _ -> false) parses zero or more whitespaces and returns them as a string.\n

\n
\n
\n
\n
Caution
\n
\n
\n

\n The function predicate f must not access the currently used CharStream itself, because manySatisfy relies on f not having any side‐effect on the\n internal state of the stream.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val manySatisfy2: (char -> bool) -> (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n manySatisfy2 f1 f behaves like manySatisfy f, except that the\n first char of the parsed string must satisfy f1 instead of f.\n

\n
\n
\n

\n For example, manySatisfy ((=) '.') isDigit will parse a dot\n followed by zero or more decimal digits. If there is no dot, the parser succeeds with an empty string.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManySatisfy: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManySatisfy f is an optimized implementation of manySatisfy f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManySatisfy2: (char -> bool) -> (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManySatisfy2 f1 f is an optimized\n implementation of manySatisfy2 f1 f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Satisfy: (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Satisfy f parses a sequence of one or more chars\n that satisfy the predicate function f (i.e. chars for which f returns true). It returns the parsed chars as a string. If\n the first char does not satisfy f, this parser fails without consuming input.\n

\n
\n
\n

\n Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'. Thus, to accept a newline f '\\n' must\n return true. f will never be called\n with '\\r' and\n the string returned by many1Satisfy f will never contain an\n '\\r'.\n

\n
\n
\n

\n For example, many1Satisfy isDigit\n parses a number consisting of one or more decimal digits and returns it as a string.\n

\n
\n
\n
\n
Caution
\n
\n
\n

\n The function predicate f must not access the currently used CharStream itself, because many1Satisfy relies on f not having any side‐effect on the\n internal state of the stream.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

\n If the parser many1Satisfy f fails, it returns no descriptive\n error message (because it does not know what chars f accepts); hence it should only\n be used together with other parsers that take care of a potential error. Alternatively, many1SatisfyL f label\n can be used to ensure a more descriptive error message.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val many1Satisfy2: (char -> bool) -> (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Satisfy2 f1 f behaves like many1Satisfy f, except that the\n first char of the parsed string must satisfy f1 instead of f.\n

\n
\n
\n

\n For example, many1Satisfy2 isLetter (fun c -> isLetter c || isDigit c) will parse any string consisting of one letter followed by zero or more letters or digits.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1Satisfy: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipMany1Satisfy f is an optimized implementation of many1Satisfy f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1Satisfy2: (char -> bool) -> (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipMany1Satisfy2 f1 f is an optimized\n implementation of many1Satisfy2 f1 f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1SatisfyL:\n    (char -> bool) -> string -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1SatisfyL f label is an optimized\n implementation of many1Satisfy f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Satisfy2L:\n    (char -> bool) -> (char -> bool) -> string -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Satisfy2L f1 f label is an optimized implementation of many1Satisfy2 f1 f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1SatisfyL:\n    (char -> bool)     -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipMany1SatisfyL f label is an\n optimized implementation of skipMany1Satisfy f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1Satisfy2L:\n    (char -> bool) -> (char -> bool) -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipMany1Satisfy2L f1 f label is an optimized implementation of skipMany1Satisfy2 f1 f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyMinMaxSatisfy:\n    int -> int -> (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyMinMaxSatisfy minCount maxCount f parses a sequence of minCount or more chars that satisfy the\n predicate function f (i.e. chars for which f returns true), but not more than maxCount chars. It returns the parsed chars as a string. This parser is atomic, i.e. if the first minCount chars do not all satisfy f, the\n parser fails without consuming any input.\n

\n
\n
\n

\n Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'. Thus, to accept a newline f '\\n' must\n return true. f will never be called\n with '\\r' and\n the string returned by manyMinMaxSatisfy minCount maxCount f will never contain an '\\r'.\n

\n
\n
\n

\n manyMinMaxSatisfy minCount maxCount f raises an ArgumentOutOfRangeException if maxCount is negative.\n

\n
\n
\n

\n For example, manyMinMaxSatisfy 4 8 isHex parses a string that consists of at least 4 hexadecimal digits. If there are\n 8 or more hex chars, this parser stops after the 8th.\n

\n
\n
\n
\n
Caution
\n
\n
\n

\n The function predicate f must not access the currently used CharStream itself, because manyMinMaxSatisfy relies on f not having any side‐effect on\n the internal state of the stream.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

\n If the parser manyMinMaxSatisfy minCount maxCount f fails, it returns no descriptive error message (because it does not know what\n chars f accepts); hence it should only be used together with other parsers that take\n care of a potential error. Alternatively, manyMinMaxSatisfyL f label can be used to ensure a more\n descriptive error message.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val manyMinMaxSatisfy2:\n    int -> int -> (char -> bool) -> (char -> bool) -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyMinMaxSatisfy2 minCount maxCount f1 f behaves like manyMinMaxSatisfy minCount maxCount f,\n except that the first char of the parsed string must satisfy f1 instead of f.\n

\n
\n
\n

\n For example, manyMinMaxSatisfy2 3 5 ((=) '.') isDigit parses a dot followed by 2‒4 decimal digits.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManyMinMaxSatisfy:\n    int -> int -> (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManyMinMaxSatisfy minCount maxCount f is an optimized implementation of manyMinMaxSatisfy minCount maxCount f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManyMinMaxSatisfy2:\n    int -> int -> (char -> bool) -> (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManyMinMaxSatisfy2 minCount maxCount f1 f is an optimized implementation of manyMinMaxSatisfy2 minCount maxCount f1 f |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyMinMaxSatisfyL:\n    int -> int -> (char -> bool) -> string -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyMinMaxSatisfyL minCount maxCount f label is an optimized implementation of manyMinMaxSatisfy minCount maxCount f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyMinMaxSatisfy2L:\n    int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyMinMaxSatisfy2L minCount maxCount f1 f label is an optimized implementation of manyMinMaxSatisfy2 minCount maxCount f1 f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManyMinMaxSatisfyL:\n    int -> int -> (char -> bool) -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManyMinMaxSatisfyL minCount maxCount f label is an optimized implementation of skipManyMinMaxSatisfy minCount maxCount f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManyMinMaxSatisfy2L:\n    int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n skipManyMinMaxSatisfy2L minCount maxCount f1 f label is an optimized implementation of skipManyMinMaxSatisfy2 minCount maxCount f1 f <?> label.\n

\n
\n
\n
\n
\n
\n ⇧\n
val regex: string -> Parser<string,'u>\n
\n
\n
\n
\n

\n regex pattern matches the .NET regular expression given by the string pattern on the chars beginning at the current index in the input stream. If the regular expression matches, the\n parser skips the matched chars and returns them as a string. If the regular expression does not match, the parser fails without consuming\n input.\n

\n
\n
\n

\n The System.Text.RegularExpressions.Regex object that is internally used to\n match the pattern is constructed with the RegexOptions MultiLine and ExplicitCapture. In order to ensure that the regular expression can only match at the beginning of a string, \"\\\\A\" is\n automatically prepended to the pattern. You should avoid the use of greedy expressions like \".*\", because these might trigger a scan of the complete input every time the\n regex is matched.\n

\n
\n
\n

\n Newline chars ('\\r' and '\\n') in the pattern are interpreted literally. For example, an '\\n' char in the pattern will only match \"\\n\", not \"\\r\" or \"\\r\\n\". However, in the returned string all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are normalized\n to \"\\n\".\n

\n
\n
\n

\n For large files the regular expression is not applied to a string containing all the remaining chars in the stream. The\n number of chars that are guaranteed to be visible to the regular expression is specified during construction of the CharStream. If one of the runParser functions is used to run the parser, this number is 43690.\n

\n
\n
\n
\n
\n
\n ⇧\n
type IdentifierOptions =\n    new: ?isAsciiIdStart: (char -> bool) *\n         ?isAsciiIdContinue: (char -> bool) *\n         ?normalization: System.Text.NormalizationForm *\n         ?normalizeBeforeValidation: bool *\n         ?allowJoinControlChars: bool *\n         ?preCheckStart: (char -> bool) *\n         ?preCheckContinue: (char -> bool) *\n         ?allowAllNonAsciiCharsInPreCheck: bool *\n         ?label: string *\n         ?invalidCharMessage: string -> IdentifierOptions\n
\n
\n
\n
\n

\n The configuration options for the identifier parser.\n

\n
\n
\n
\n
\n
isAsciiIdStart
\n
\n
\n

\n Specifies the ASCII characters that are valid as the first character of an identifier. This predicate function is called once for each\n char in the range '\\u0001'–'\\u007f' during construction of the IdentifierOptions\n object. By default, the ASCII chars 'A'–'Z' and 'a'–'z' can start an identifier.\n

\n
\n
\n
isAsciiIdContinue
\n
\n
\n

\n Specifies the ASCII characters that are valid as non‐first characters of an identifier. This predicate function is called once for each\n char in the range '\\u0001'–'\\u007f' during construction of the IdentifierOptions\n object. Normally the chars for which isAsciiIdContinue returns true should include all chars for which isAsciiIdStart returns true. By default, the ASCII chars\n 'A'–'Z', 'a'–'z', '0'–'9' and '_' are accepted at non‐start positions.\n

\n
\n
\n
normalization
\n
\n
\n

\n This option is not supported in the Silverlight version of FParsec.
The normalization form to which identifier strings are normalized. The value must be one of\n the four enum values of System.Text.NormalizationForm. If no normalization parameter is\n given, no normalization is performed.\n

\n
\n
\n

\n The normalization is performed with the System.String.Normalize method provided by the Base Class Library.\n

\n
\n
\n
normalizeBeforeValidation
\n
\n
\n

\n This option is not supported in the Silverlight version of FParsec.
Indicates\n whether the identifier string should be normalized before validation (but after the pre‐check). By default, identifiers are normalized\n after they have been validated. Normalization before validation will only work properly with non‐default pre‐check options.\n

\n
\n
\n
allowJoinControlChars
\n
\n
\n

\n Indicates whether the two join control characters (zero‐width\n non‐joiner and zero‐width joiner) are allowed at any non‐start\n character position in the identifier.\n

\n
\n
\n
\n preCheckStart, preCheckContinue\n
\n
\n
\n

\n These two char predicates are used to identify potential identifier strings in the input. The first UTF‐16 char of an identifier must\n satisfy preCheckStart, the following chars must satify preCheckContinue. Input chars that don’t pass the pre‐check aren’t included in the identifier string, while\n characters that pass the pre‐check but not the identifier validation trigger a parser error. For the identifier parser to work properly, the pre‐check functions must accept a\n superset of valid identifier characters.\n

\n
\n
\n

\n If you specify no preCheckStart (preCheckContinue) parameter, a default function will be used that accepts all chars that satisfy isAsciiIdStart (isAsciiIdContinue) as\n well as all non‐ASCII characters in the Basic Multilingual Plane with the XID_Start (XID_Continue) property and all surrogate chars. preCheckContinue\n by default also accepts the two join control characters.\n

\n
\n
\n

\n If you pass the option allowAllNonAsciiCharsInPreCheck = true, the pre‐check predicates are only called once for each char in the range '\\u0001' ‐ '\\u007f' during construction of the\n IdentifierOptions object (in order to construct a lookup table).\n

\n
\n
\n
allowAllNonAsciiCharsInPreCheck
\n
\n
\n

\n Indicates whether all non‐ASCII chars should be accepted in the pre‐check, irrespective of whether the (default) pre‐check functions\n return true for these chars.\n

\n
\n
\n
label
\n
\n
\n

\n The string label that is used in error messages if no identifier is found. The default is \"identifier\".\n

\n
\n
\n
invalidCharMessage
\n
\n
\n

\n The error message that is reported when an invalid char is found during validation of an identifier (after the pre‐check). The default\n is \"The identifier contains an invalid character at the indicated position.\".\n

\n
\n
\n
\n
\n
\n
\n

\n The following example implements a parser for Python identifiers as described in PEP‐3131:\n

\n
let pythonIdentifier =\n    let isAsciiIdStart    = fun c -> isAsciiLetter c || c = '_'\n    let isAsciiIdContinue = fun c -> isAsciiLetter c || isDigit c || c = '_'\n\n    identifier (IdentifierOptions(\n                    isAsciiIdStart = isAsciiIdStart,\n                    isAsciiIdContinue = isAsciiIdContinue,\n                    normalization = System.Text.NormalizationForm.FormKC,\n                    normalizeBeforeValidation = true,\n                    allowAllNonAsciiCharsInPreCheck = true))\n
\n
\n
\n
\n
\n
\n ⇧\n
val identifier: IdentifierOptions -> Parser<string, 'u>\n
\n
\n
\n
\n

\n The identifier parser is a configurable parser for the XID identifier syntax specified\n in the Unicode Standard Annex #31.\n

\n
\n
\n

\n By default, a valid identifier string must begin with a Unicode character with the XID_Start property and continue\n with zero or more characters with the XID_Continue property. The specification of which characters have these\n properties can be found in the DerivedCoreProperties file in\n the Unicode Character Database. Currently FParsec implements the XID specification of Unicode\n 8.0.0.\n

\n
\n
\n

\n Within the ASCII character range '\\u0001'–'\\u007f' you can customize the set of accepted characters through the isAsciiIdStart and isAsciiIdContinue parameters (the XID\n default allows 'a'–'z' and 'A'–'Z' at any position and '_' and '0'–'9'\n only in non‐start positions). For example, to accept the same ASCII characters that are valid in F# identifiers, you could use the following\n IdentifierOptions:\n

\n
let isAsciiIdStart c =\n    isAsciiLetter c || c = '_'\n\nlet isAsciiIdContinue c =\n    isAsciiLetter c || isDigit c || c = '_' || c = '\\''\n\nidentifier (IdentifierOptions(isAsciiIdStart    = isAsciiIdStart,\n                              isAsciiIdContinue = isAsciiIdContinue))\n
\n
\n
\n

\n By default, identifiers cannot contain the two join control characters zero‐width non‐joiner and zero‐width joiner. While these characters can be abused to create distinct\n identifiers that look confusingly similar or even identical, they are also necessary to create identifiers with the correct visual\n appearance for common words or phrases in certain languages. Section 2.3 of the Unicode Standard Annex #31\n recommends to accept join control characters if the identifier system is supposed to support “natural representations of terms in\n modern, customary use”. However, in order to minimize the potential for abuse it also recommends accepting these characters only in\n some very specific contexts.\n

\n
\n
\n

\n Unfortunately, the proposed rules describing the contexts in which join control character should be allowed are rather difficult to\n implement, especially with the limited Unicode support in .NET. For this reason the identifier parser currently only supports a simpler option: if you set the parameter allowJoinControlChars to true, the parser accepts the two join\n control characters in any non‐start position. Whether this setting is a reasonable compromise between not supporting join control characters\n at all and implementing the complicated rules proposed in Annex #31 obviously depends on the individual requirements of your project. An\n example of a programming language that adopted the same\n compromise is ECMAScript 5.\n

\n
\n
\n

\n Apart from the joint control characters, no layout or format control characters are allowed in identifiers. This is in accordance to the\n recommendation of the Unicode Standard Annex #31, but contrary to what Annex #15 recommended prior to Unicode version 4.1.\n Programming languages whose identifier syntax is based on the recommendations of earlier versions of the Unicode standard may require that\n layout and format control characters are ignored or filtered out, as for example is the case for C#. However, since the identifier syntax of\n these languages isn’t based on the XID properties, one can’t parse their identifiers with this parser anyway.\n

\n
\n
\n

\n By providing a value for the normalization parameter, you can ensure that identifiers\n are returned in a particular Unicode normalization form. By default, an identifier is\n normalized after it has been validated. Since XID identifiers are “closed under normalization”, a valid identifier is\n guaranteed to stay valid after normalization. The reverse, however, is not true, since not all identifier strings that are valid after\n normalization are also valid prior to normalization. If you want the identifier string to be normalized before validation, you have to set\n the normalizeBeforeValidation parameter to true and specify appropriate preCheckStart and preCheckContinue parameters.\n

\n
\n
\n

\n Silverlight does not support Unicode normalization, so the Silverlight version of FParsec does not support the normalization and normalizeBeforeValidation parameters.\n

\n
\n
\n

\n The identifier parser uses the preCheckStart and preCheckContinue predicate functions to\n identify potential identifier strings in the input. The first UTF‐16 char of the identifier must satisfy preCheckStart, the following chars must satifsy preCheckContinue. Input chars that don’t pass the pre‐check aren’t included in the identifier string, while\n characters that pass the pre‐check but not the identifier validation trigger a parser error (FatalError). For the identifier parser to work properly, the preCheck functions must\n accept a superset of valid identifier characters.\n

\n
\n
\n

\n If you specify no preCheckStart (preCheckContinue) parameter, a default function will be used that accepts all chars that satisfy isAsciiIdStart (isAsciiIdContinue) as well\n as all non‐ASCII characters in the Basic Multilingual Plane with the XID_Start (XID_Continue) property and all surrogate chars. preCheckContinue by\n default also accepts the two join control characters. If you set the parameter allowAllNonAsciiCharsInPreCheck to true, all non‐ASCII chars\n will be accepted in the pre‐check, irrespective of whether the (default) pre‐check functions return true for these chars.\n

\n
\n
\n

\n By passing custom preCheckStart and preCheckContinue functions you can modify the error reporting behaviour and support identifier strings that are\n only valid after normalization. You can also exclude specific UTF‐16 chars that would otherwise be valid in identifiers, though you’d have\n to be careful to cover all (pre‐)normalization forms.\n

\n
\n
\n

\n In the following examples we will demonstrate the effect of custom pre‐check functions on identifier parsing. For this we first set up two\n identifier parsers, ident and identP, with differing sets of options. Both parsers accept the same ASCII chars in identifiers. In particular,\n both do not accept the underscore char '_' in identifiers. However, only identP lets underscores\n through the pre‐check.\n

\n
\n
\n
// we don't allow underscores in identifiers ...\nlet isAsciiIdStart c    = isAsciiLetter c\nlet isAsciiIdContinue c = isAsciiLetter c || isDigit c\n\n// ... but accept them in in the pre-check\nlet preCheckStart c    = isAsciiLetter c || c = '_'\nlet preCheckContinue c = isAsciiLetter c || isDigit c || c = '_'\n\ntype NF = System.Text.NormalizationForm\n\nlet opts = IdentifierOptions(isAsciiIdStart    = isAsciiIdStart,\n                             isAsciiIdContinue = isAsciiIdContinue,\n                             normalization = NF.FormKC,\n                             // The following option isn't really useful without\n                             // modified pre-check options. We only set the\n                             // option here to prove this point in an example below.\n                             normalizeBeforeValidation = true)\n\nlet optsWithPreCheck = IdentifierOptions(isAsciiIdStart = isAsciiIdStart,\n                                         isAsciiIdContinue = isAsciiIdContinue,\n                                         preCheckStart = preCheckStart,\n                                         preCheckContinue = preCheckContinue,\n                                         allowAllNonAsciiCharsInPreCheck = true,\n                                         normalization = NF.FormKC,\n                                         normalizeBeforeValidation = true)\n\nlet ident  : Parser<string, unit> = identifier opts\nlet identP : Parser<string, unit> = identifier optsWithPreCheck\n
\n
\n
\n

\n Both ident and identP parse simple\n identifiers without a problem:\n

\n
> run (ident .>> eof) \"täst1\";;\nval it : ParserResult<string,unit> = Success: \"täst1\"\n> run (identP .>> eof) \"täst2\";;\nval it : ParserResult<string,unit> = Success: \"täst2\"\n
\n
\n
\n

\n The identifier parser with the default pre‐check functions will treat underscores just like whitespace or any other non‐identifier\n character:\n

\n
> run (ident .>> eof) \"test_id\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 5\ntest_id\n    ^\nExpecting: end of input\n
\n
\n
\n

\n Since ident only consumed the \"test\" part of the input string, the eof parser complained that it was expecting to be applied at the end of the input.\n

\n
\n
\n

When we use identP instead, we get a different error message:

\n
> run (identP .>> eof) \"test_id\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 5\ntest_id\n    ^\nThe identifier contains an invalid character at the indicated position.\n
\n

This time the underscore passed the pre‐check, but not the identifier validation.

\n
\n
\n

\n As mentioned above, a custom pre‐check is also neccessary to make the normalizeBeforeValidation option work properly. With the default pre‐check options the identifier parser doesn’t\n accept \"MC²\" as an identifier, even\n with the normalization set to NFKC:\n

\n
> run (ident .>> eof) \"MC²\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 3\nMC²\n  ^\nExpecting: end of input\n
\n
\n
\n

\n identP on the other hand doesn’t have this issue, because it accepts all non‐ASCII chars\n in the pre‐check:\n

\n
> run (identP .>> eof) \"MC²\";;\nval it : ParserResult<string,unit> = Success: \"MC2\"\n
\n
\n
\n
\n
\n
\n ⇧\n
val manyChars: Parser<char,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyChars cp parses a sequence of zero or more chars\n with the char parser cp. It returns the parsed chars as a string.\n

\n
\n
\n

\n manyChars cp is an optimized implementation of many cp that returns the\n chars as a string instead of a char list.\n

\n
\n
\n

\n Many string parsers can be conveniently implemented with both manyChars and manySatisfy. In these cases you should generally prefer the\n faster manySatisfy. For example, the parser manySatisfyL isHex \"hex integer\" is more efficient than manyChars hex.\n

\n
\n
\n

\n If you are using manyChars for a parser similar to manyChars (notFollowedBy endp >>. p), you should check whether this use of manyChars can be replaced with the more specialized manyCharsTill parser.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyChars2: Parser<char,'u> -> Parser<char,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyChars2 cp1 cp behaves like manyChars2 cp, except that it parses the first char with cp1 instead of cp.\n

\n
\n
\n

\n For example, manyChars2 letter (letter <|> digit) will parse a\n letter followed by letters or digits and return the chars as a string. If the first char is not a letter, the parser succeeds with an empty\n string. Note, however, that this parser could be more efficiently implemented using manySatisfy2L.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Chars: Parser<char,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Chars cp parses a sequence of one or more chars\n with the char parser cp. It returns the parsed chars as a string.\n

\n
\n
\n

\n many1Chars cp is an optimized implementation of many1 cp that returns the\n chars as a string instead of a char list.\n

\n
\n
\n

\n Many string parsers can be conveniently implemented with both many1Chars and many1Satisfy. In these cases you should generally prefer\n the faster many1Satisfy. For example, the parser\n many1SatisfyL isHex \"hex integer\" is more efficient\n than many1Chars hex.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Chars2: Parser<char,'u> -> Parser<char,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Chars2 cp1 cp behaves like many1Chars2 cp, except that it parses the first char with cp1 instead of cp.\n

\n
\n
\n

\n For example, many1Chars2 letter (letter <|> digit) will parse a\n letter followed by letters or digits and return the chars as a string. Note, however, that this parser could be more efficiently implemented\n using many1Satisfy2L.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyCharsTill:\n       Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyCharsTill cp endp parses chars with\n the char parser cp until the parser endp succeeds. It stops after endp and returns the parsed chars\n as a string.\n

\n
\n
\n

\n manyCharsTill cp endp is an optimized\n implementation of manyTill cp endp that returns the chars as a string instead of a char list.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyCharsTill2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyCharsTill2 cp1 cp endp behaves like manyCharsTill cp endp, except that it parses the first char\n with cp1 instead of cp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyCharsTillApply:\n       Parser<char,'u>  -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n
\n
\n
\n
\n

\n manyCharsTillApply cp endp f behaves like manyCharsTill cp endp, except that it returns the result of the\n function application f str b, where\n str is the parsed string and b is\n result returned by endp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyCharsTillApply2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n
\n
\n
\n
\n

\n manyCharsTillApply2 cp1 cp endp f behaves like manyCharsTillApply cp endp f, except\n that it parses the first char with cp1 instead of cp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1CharsTill:\n       Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1CharsTill cp endp parses one char\n with the char parser cp. Then it parses more chars with cp until the parser endp succeeds. It stops after endp and returns the parsed chars as a string.\n

\n
\n
\n

\n many1CharsTill cp endp is an optimized\n implementation of many1Till cp endp that returns the chars as a string instead of a char list.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1CharsTill2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1CharsTill2 cp1 cp endp behaves like many1CharsTill cp endp, except that it parses the first char\n with cp1 instead of cp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1CharsTillApply:\n       Parser<char,'u>   -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n
\n
\n
\n
\n

\n many1CharsTillApply cp endp f behaves like many1CharsTill cp endp, except that it returns the result of\n the function application f str b, where\n str is the parsed string and b is\n result returned by endp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1CharsTillApply2:\n       Parser<char,'u> -> Parser<char,'u> -> Parser<'b,'u> -> (string -> 'b -> 'c)\n    -> Parser<'c,'u>\n
\n
\n
\n
\n

\n many1CharsTillApply2 cp1 cp endp f behaves like many1CharsTillApply cp endp f, except\n that it parses the first char with cp1 instead of cp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyStrings: Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyStrings sp parses a sequence of zero or more\n strings with the string parser sp. It returns the strings in concatenated form.\n

\n
\n
\n

\n manyStrings sp is an optimized implementation of many sp |>> List.fold (fun acc s -> acc + s) \"\".\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyStrings2: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n manyStrings2 sp1 sp behaves like manyStrings sp, except that it\n parses the first string with sp1 instead of sp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Strings: Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Strings sp parses a sequence of one or more\n strings with the string parser sp. It returns the strings in concatenated form. Note\n that many1Strings sp does not require the first string to be\n non‐empty.\n

\n
\n
\n

\n many1Strings sp is an optimized implementation of many1 sp |>> List.reduce (+).\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Strings2: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n many1Strings2 sp1 sp behaves like many1Strings sp, except that it\n parses the first string with sp1 instead of sp.\n

\n
\n
\n
\n
\n
\n ⇧\n
val stringsSepBy: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n stringsSepBy sp sep parses zero\n or more occurrences of the string parser sp separated by sep (in EBNF: (sp (sep sp)*)?). It returns the strings parsed by sp and sep in concatenated form.\n

\n
\n
\n

\n stringsSepBy behaves like sepBy, except that instead of returning a list of the results of\n only the first argument parser it returns a concatenated string of all strings returned by both argument parsers (in the sequence they\n occurred).\n

\n
\n
\n

\n With stringsSepBy you can for example implement an efficient parser for the following\n string literal format:\n

\n
\n
\n
  stringLiteral: '\"' (normalChar|escapedChar)* '\"'\n  normalChar:    any char except '\\' and '\"'\n  escapedChar:   '\\\\' ('\\\\'|'\"'|'n'|'r'|'t')\n
\n
\n
\n

The parser implementation exploits the fact that two (possibly empty) normal char snippets must be separated by an escaped char:

\n
\n
\n
let stringLiteral =\n    let str s = pstring s\n    let normalCharSnippet = manySatisfy (fun c -> c <> '\\\\' && c <> '\"')\n    let escapedChar = str \"\\\\\" >>. (anyOf \"\\\\\\\"nrt\" |>> function\n                                                        | 'n' -> \"\\n\"\n                                                        | 'r' -> \"\\r\"\n                                                        | 't' -> \"\\t\"\n                                                        | c   -> string c)\n    between (str \"\\\"\") (str \"\\\"\")\n            (stringsSepBy normalCharSnippet escapedChar)\n
\n
\n
\n
\n
\n
\n ⇧\n
val stringsSepBy1: Parser<string,'u> -> Parser<string,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n stringsSepBy1 sp sep parses one\n or more occurrences of the string parser sp separated by sep (in EBNF: (sp (sep sp)*)). It returns the strings parsed by sp and sep in concatenated form.\n

\n
\n
\n

\n stringsSepBy1 behaves like stringsSepBy, except that it fails without consuming input if sp does not succeed at least once.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipped: Parser<unit,'u> -> Parser<string,'u>\n
\n
\n
\n
\n

\n skipped p applies the parser p and returns the chars skipped over by p as a string. All\n newlines (\"\\r\\n\", \"\\r\" or \"\\n\") are normalized to \"\\n\".\n

\n
\n
\n
\n
\n
\n ⇧\n
val withSkippedString: (string -> 'a -> 'b) -> Parser<'a,'u> -> Parser<'b,'u>\n
\n
\n
\n
\n

\n p |> withSkippedString f applies the parser p and returns the result f str x, where str is the string skipped over by p and x is the result returned by p.\n

\n
\n
\n
\n
\n
\n ⇧\n
type NumberLiteralOptions
\n
\n
\n
\n

\n An enum type that encodes the various options of the numberLiteral parser:\n

\n
\n
\n
type NumberLiteralOptions =\n| None                       = 0\n| AllowSuffix                = 0b000000000001\n| AllowMinusSign             = 0b000000000010\n| AllowPlusSign              = 0b000000000100\n| AllowFraction              = 0b000000001000\n| AllowFractionWOIntegerPart = 0b000000010000\n| AllowExponent              = 0b000000100000\n| AllowHexadecimal           = 0b000001000000\n| AllowBinary                = 0b000010000000\n| AllowOctal                 = 0b000100000000\n| AllowInfinity              = 0b001000000000\n| AllowNaN                   = 0b010000000000\n\n| IncludeSuffixCharsInString = 0b100000000000\n\n| DefaultInteger             = 0b000111000110\n| DefaultUnsignedInteger     = 0b000111000000\n| DefaultFloat               = 0b011001101110\n
\n
\n
\n

If all flags are set any literal matching the following regular expression is accepted:

\n
\n
\n
[+-]?((([0-9]+(\\.[0-9]*)?|\\.[0-9]+)([eE][+-]?[0-9]+)?\n      |0[xX]([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n      |0[oO][0-7]+\n      |0[bB][01]+\n      )[a-zA-Z]?[a-zA-Z]?[a-zA-Z]?[a-zA-Z]?\n     |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n     |[nN][aA][nN]\n     )\n
\n
\n
\n

\n Hexadecimal literals must begin with 0x or 0X, octal literals with 0o or 0O and binary literals with 0b or 0B. If the respective flags are set, hexadecimal floating‐point literals as supported by IEEE 754r, C99 and\n Java are accepted.\n

\n
\n
\n

Some remarks on the individual flags:

\n
\n
\n
AllowSuffix
\n
\n
\n

\n Allows up to 4 suffix chars. Such chars are used in many programming languages to determine the type of a number. For example, in F# the\n literal \"123UL\" represents the\n unsigned 64‐bit integer 123.\n

\n
\n
\n
AllowFraction
\n
\n
\n

Allows a fraction in decimal and hexadecimal literals.

\n
\n
\n
AllowFractionWOIntegerPart
\n
\n
\n

\n Allows number literals with a fraction but no integer part, e.g. \".123\" or \"0x.abc\". This flag can only be used together with AllowFraction.\n

\n
\n
\n
AllowExponent
\n
\n
\n

\n Allows exponents in decimal literals (beginning with an \"e\" or \"E\") and in hexadecimal literals (beginning with a \"p\" or \"P\").\n

\n
\n
\n
AllowInfinity
\n
\n
\n

\n Allows \"Inf\" or \"Infinity\" literals (case‐insensitive).\n

\n
\n
\n
AllowNaN
\n
\n
\n

\n Allows \"NaN\" literals\n (case‐insensitive).\n

\n
\n
\n
IncludeSuffixCharsInString
\n
\n
\n

\n Instructs the numberLiteral parser to include\n any parsed suffix chars in the NumberLiteral.String member.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
type NumberLiteral
\n
\n
\n
\n

\n The return type of the numberLiteral parser. An\n instance contains the parsed number literal and various bits of information about it. Note that the String member contains the string literal without the suffix chars, except if the NumberLiteralOptions passed to the numberLiteral parser have the IncludeSuffixCharsInString flag set. Any parsed suffix chars are always available through the SuffixChar1 ‐ 4 members.\n

\n
\n
\n
type NumberLiteral =\n    member String: string\n\n    member SuffixLength: int\n    member SuffixChar1: char // EOS if no suffix char was parsed\n    member SuffixChar2: char // EOS if less than 2 suffix chars were parsed\n    member SuffixChar3: char ...\n    member SuffixChar4: char\n\n    member Info: NumberLiteralResultFlags\n    member HasMinusSign: bool\n    member HasPlusSign: bool\n    member HasIntegerPart: bool\n    member HasFraction: bool\n    member HasExponent: bool\n    member IsInteger: bool // not (HasFraction || HasExponent)\n    member IsDecimal: bool\n    member IsHexadecimal: bool\n    member IsBinary: bool\n    member IsOctal: bool\n    member IsNaN: bool\n    member IsInfinity: bool\n\nand NumberLiteralResultFlags =\n    | None             = 0\n    | SuffixLengthMask = 0b0000000000001111\n    | HasMinusSign     = 0b0000000000010000\n    | HasPlusSign      = 0b0000000000100000\n    | HasIntegerPart   = 0b0000000001000000\n    | HasFraction      = 0b0000000010000000\n    | HasExponent      = 0b0000000100000000\n    | IsDecimal        = 0b0000001000000000\n    | IsHexadecimal    = 0b0000010000000000\n    | IsBinary         = 0b0000100000000000\n    | IsOctal          = 0b0001000000000000\n    | BaseMask         = 0b0001111000000000\n    | IsInfinity       = 0b0010000000000000\n    | IsNaN            = 0b0100000000000000\n
\n
\n
\n
\n
\n
\n ⇧\n
val numberLiteral: NumberLiteralOptions -> string -> Parser<NumberLiteral,'u>\n
\n
\n
\n
\n

\n numberLiteral options label parses a\n number literal and returns the result in form of a NumberLiteral value. The given NumberLiteralOptions argument determines the kind of number literals accepted. The string label is used in the Expected error message that is generated when the parser fails without consuming input.\n

\n
\n
\n

\n The parser fails without consuming input if not at least one digit (including the 0 in the format specifiers \"0x\" etc.) can be parsed. It fails after\n consuming input, if no decimal digit comes after an exponent marker or no valid digit comes after a format specifier.\n

\n
\n
\n

\n The parser in the following example employs numberLiteral to parse decimal numbers as\n either integer or float values:\n

\n
open FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\ntype Number = Int   of int64\n            | Float of float\n\n                    // -?[0-9]+(\\.[0-9]*)?([eE][+-]?[0-9]+)?\nlet numberFormat =     NumberLiteralOptions.AllowMinusSign\n                   ||| NumberLiteralOptions.AllowFraction\n                   ||| NumberLiteralOptions.AllowExponent\n\nlet pnumber : Parser<Number, unit> =\n    numberLiteral numberFormat \"number\"\n    |>> fun nl ->\n            if nl.IsInteger then Int (int64 nl.String)\n            else Float (float nl.String)\n
\n
\n
\n

Some test runs:

\n
> run pnumber \"123\";;\nval it : ParserResult<Number,unit> = Success: Int 123L\n\n> run pnumber \"-123.456E-7\";;\nval it : ParserResult<Number,unit> = Success: Float -1.23456e-05\n\n> run pnumber \"-\";;\nval it : ParserResult<Number,unit> = Failure:\nError in Ln: 1 Col: 1\n-\n^\nExpecting: number\n\n> run pnumber \"123.456E-a\";;\nval it : ParserResult<Number,unit> = Failure:\nError in Ln: 1 Col: 10\n123.456E-a\n         ^\nExpecting: decimal digit\n\n> run pnumber \"1E9999\";;\nSystem.OverflowException:\n   Value was either too large or too small for a Double.\n   at (... stack trace ...)\nstopped due to error\n
\n
\n
\n

\n The next example improves on the error reporting in case of overflows. It also demonstrates how to support hexadecimal numbers and a suffix\n to indicate the integer format:\n

\n
open FParsec\nopen FParsec.Error\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\ntype Number = Int32 of int32\n            | Int64 of int64\n            | Float of float\n\n// We want to support decimal or hexadecimal numbers with an optional minus\n// sign. Integers may have an 'L' suffix to indicate that the number should\n// be parsed as a 64-bit integer.\nlet numberFormat =     NumberLiteralOptions.AllowMinusSign\n                   ||| NumberLiteralOptions.AllowFraction\n                   ||| NumberLiteralOptions.AllowExponent\n                   ||| NumberLiteralOptions.AllowHexadecimal\n                   ||| NumberLiteralOptions.AllowSuffix\n\nlet pnumber : Parser<Number, unit> =\n    let parser = numberLiteral numberFormat \"number\"\n    fun stream ->\n        let reply = parser stream\n        if reply.Status = Ok then\n            let nl = reply.Result // the parsed NumberLiteral\n            if nl.SuffixLength = 0\n               || (   nl.IsInteger\n                   && nl.SuffixLength = 1 && nl.SuffixChar1 = 'L')\n            then\n                try\n                    let result = if nl.IsInteger then\n                                     if nl.SuffixLength = 0 then\n                                         Int32 (int32 nl.String)\n                                     else\n                                         Int64 (int64 nl.String)\n                                 else\n                                     if nl.IsHexadecimal then\n                                         Float (floatOfHexString nl.String)\n                                     else\n                                         Float (float nl.String)\n                    Reply(result)\n                with\n                | :? System.OverflowException as e ->\n                    stream.Skip(-nl.String.Length)\n                    Reply(FatalError, messageError e.Message)\n            else\n                stream.Skip(-nl.SuffixLength)\n                Reply(Error, messageError \"invalid number suffix\")\n        else // reconstruct error reply\n            Reply(reply.Status, reply.Error)\n
\n
\n
\n

Some test runs:

\n
> run pnumber \"123\";;\nval it : ParserResult<Number,unit> = Success: Int32 123\n\n> run pnumber \"-0xffL\";;\nval it : ParserResult<Number,unit> = Success: Int64 -255L\n\n> run pnumber \"123.123\";;\nval it : ParserResult<Number,unit> = Success: Float 123.123\n\n> run pnumber \"0xabc.defP-4\";;\nval it : ParserResult<Number,unit> = Success: Float 171.8044281\n\n> run pnumber \"-0x\";;\nval it : ParserResult<Number,unit> = Failure:\nError in Ln: 1 Col: 4\n-0x\n   ^\nNote: The error occurred at the end of the input stream.\nExpecting: hexadecimal digit\n\n> run pnumber \"0x123UL\";;\nval it : ParserResult<Number,unit> = Failure:\nError in Ln: 1 Col: 6\n0x123UL\n     ^\ninvalid number suffix\n\n> run pnumber \"1E9999\";;\nval it : ParserResult<Number,unit> = Failure:\nError in Ln: 1 Col: 1\n1E9999\n^\nValue was either too large or too small for a Double.\n
\n
\n
\n
\n
\n
\n ⇧\n
val numberLiteralE:\n       NumberLiteralOptions -> errorInCaseNoLiteralFound: ErrorMessageList\n    -> CharStream<'u> -> Reply<NumberLiteral>\n
\n
\n
\n
\n

\n numberLiteralE is an uncurried version of numberLiteral that can be used to implement number parsers without having\n to construct a numberLiteral closure.\n

\n
\n
\n
\n
\n
\n ⇧\n
val pfloat: Parser<float,'u>\n
\n
\n
\n
\n

Parses a floating point number in the decimal format (in regular expression notation)

\n
[0-9]+(\\.[0-9]*)?([eE][+-]?[0-9]+)?\n
\n

or the hexadecimal format

\n
0[xX][0-9a-fA-F]+(\\.[0-9a-fA-F]*)?([pP][+-]?[0-9]+)?\n
\n

(as supported by IEEE 754r, C99 and Java, where e.g. 0x1f.cP-5 represents 31.75 * 2‒5).

\n
\n
\n

\n The special values NaN and Inf(inity)? (case‐insensitive) are also recognized. All\n recognized numbers may be prefixed with a plus or minus sign.\n

\n
\n
\n

Fractions without a leading digit, as for example “.5”, are not supported.

\n
\n
\n

The parser fails

\n
    \n
  • \n without consuming input, if not at least one digit (including the 0 in 0x) can be\n parsed,\n
    • \n
  • \n after consuming input, if no digit comes after an exponent marker or no hex digit comes after 0x.\n
    • \n
\n
\n
\n
\n
Note
\n
\n
\n

\n Values that can’t be represented as a finite float after rounding are parsed as plus\n or minus infinity. This behaviour changed between FParsec versions 1.0.3 and 1.0.10, following the respective\n behaviour change of System.Double.Parse on .NET Core 3.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

\n The pfloat parser is based on the configurable numberLiteral parser. If you’d like to support a different\n floating‐point format, there’s a good chance you can implement a parser for that format by some simple changes to a copy of the pfloat source.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val pint64: Parser<int64,'u>\n
\n
\n
\n
\n

\n Parses a 64‐bit signed integer number in the decimal, hexadecimal (0[xX]), octal (0[oO]) and binary (0[bB]) formats (in regular expression notation):\n

\n
[+-]?([0-9]+\n     |0[xX][0-9a-fA-F]+\n     |0[oO][0-7]+\n     |0[bB][01]+\n     )\n
\n
\n
\n

The parser fails

\n
    \n
  • \n without consuming input, if not at least one digit (including the 0 in the format specifiers 0x etc.) can be parsed,\n
    • \n
  • after consuming input, if no digit comes after an exponent marker or no digit comes after a format specifier,
    • \n
  • \n after consuming input, if the value represented by the input string is greater than System.Int64.MaxValue or less than System.Int64.MinValue.\n
    • \n
\n
\n
\n
\n
\n
\n ⇧\n
val pint32: Parser<int32,'u>\n
\n
\n
\n
\n

\n pint32 parses a 32‐bit signed integer and behaves like pint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val pint16: Parser<int16,'u>\n
\n
\n
\n
\n

\n pint16 parses a 16‐bit signed integer and behaves like pint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val pint8: Parser<int8,'u>\n
\n
\n
\n
\n

\n pint8 parses an 8‐bit signed integer and behaves like pint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val puint64: Parser<uint64,'u>\n
\n
\n
\n
\n

\n Parses numbers in the decimal, hexadecimal (0[xX]), octal (0[oO]) and binary (0[bB]) formats (in regular expression notation):\n

\n
[0-9]+\n|0[xX][0-9a-fA-F]+\n|0[oO][0-7]+\n|0[bB][01]+\n
\n

Note that the parser does not accept a leading plus sign.

\n
\n
\n

The parser fails

\n
    \n
  • \n without consuming input, if not at least one digit (including the 0 in the format specifiers 0x etc.) can be parsed,\n
    • \n
  • after consuming input, if no digit comes after an exponent marker or no digit comes after a format specifier,
    • \n
  • \n after consuming input, if the value represented by the input string is greater than System.UInt64.MaxValue.\n
    • \n
\n
\n
\n
\n
\n
\n ⇧\n
val puint32: Parser<uint32,'u>\n
\n
\n
\n
\n

\n puint32 parses a 32‐bit unsigned integer and behaves like puint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val puint16: Parser<uint16,'u>\n
\n
\n
\n
\n

\n puint16 parses a 16‐bit unsigned integer and behaves like puint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val puint8: Parser<uint8,'u>\n
\n
\n
\n
\n

\n puint8 parses an 8‐bit unsigned integer and behaves like puint64, except for the different return type and smaller integer range.\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedByEof: Parser<unit,'u>\n
\n
\n
\n
\n

\n notFollowedByEof is an optimized implementation of notFollowedByL eof \"end of input\".\n

\n
\n
\n
\n
\n
\n ⇧\n
val followedByNewline: Parser<unit,'u>\n
\n
\n
\n
\n

\n followedByNewline is an optimized implementation of followedByL newline \"newline\".\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedByNewline: Parser<unit,'u>\n
\n
\n
\n
\n

\n notFollowedByNewline is an optimized implementation of notFollowedByL newline \"newline\".\n

\n
\n
\n
\n
\n
\n ⇧\n
val followedByString: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n followedByString str is an optimized implementation of followedByL (pstring str) (\"'\" + str + \"'\").\n

\n
\n
\n
\n
\n
\n ⇧\n
val followedByStringCI: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n followedByStringCI str is an optimized implementation of followedByL (pstringCI str) (\"'\" + str + \"'\").\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedByString: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n notFollowedByString str is an optimized implementation of notFollowedByL (pstring str) (\"'\" + str + \"'\").\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedByStringCI: string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n notFollowedByStringCI str is an optimized implementation of\n notFollowedByL (pstringCI str) (\"'\" + str + \"'\").\n

\n
\n
\n
\n
\n
\n ⇧\n
val nextCharSatisfies: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n nextCharSatisfies f is an optimized implementation of followedBy (satisfy f).\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val nextCharSatisfiesNot: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n nextCharSatisfiesNot f is an optimized implementation of notFollowedBy (satisfy f).\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val next2CharsSatisfy: (char -> char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n next2CharsSatisfy f succeeds if the predicate function f returns true when applied to the next 2\n chars in the input stream, otherwise it fails. If there aren’t 2 chars remaining in the input stream, this parser fails (as opposed to next2CharsSatisfyNot). This parser never changes\n the parser state. Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val next2CharsSatisfyNot: (char -> char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n next2CharsSatisfy f\n succeeds if the predicate function f returns false when applied to the next 2 chars in the input stream, otherwise it fails. If there aren’t 2 chars remaining\n in the input stream, this parser succeeds (as opposed to next2CharsSatisfy). This parser never changes the parser state. Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input\n is interpreted as a single char '\\n'.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val previousCharSatisfies: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n previousCharSatisfies f succeeds if the predicate function f returns true when applied to the\n previous char in the stream, otherwise it fails. If there is no previous char (because the input stream is at the beginning), this parser\n fails (as opposed to previousCharSatisfiesNot). This parser never changes the parser state. Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input\n is interpreted as a single char '\\n'.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val previousCharSatisfiesNot: (char -> bool) -> Parser<unit,'u>\n
\n
\n
\n
\n

\n previousCharSatisfiesNot f succeeds if the predicate function\n f returns false when applied to the\n previous char in the stream, otherwise it fails. If there is no previous char (because the stream is at the beginning),If this parser fails,\n it returns no descriptive error message; hence it should only be used this parser succeeds (as opposed to previousCharSatisfies). This parser never changes the parser state.\n Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a\n potential error.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val foldCase: string -> string\n
\n
\n
\n
\n

\n Forwards all calls to FParsec.Text.FoldCase.\n

\n
\n
\n
\n
\n
\n ⇧\n
val normalizeNewlines: string -> string\n
\n
\n
\n
\n

\n Forwards all calls to FParsec.Text.NormalizeNewlines.\n

\n
\n
\n
\n
\n
\n ⇧\n
val floatToHexString: float -> string\n
\n
\n
\n
\n

\n Returns a hexadecimal string representation of the float argument. The hexadecimal\n format is the one supported by IEEE 754r, C99 and Java. This function produces the same output as the Double.toHexString method in Java.\n

\n
\n
\n
\n
\n
\n ⇧\n
val floatOfHexString: string -> float\n
\n
\n
\n
\n

\n Returns the float value represented by the given string in hexadecimal format. The supported input format is (in regular expression\n notation):\n

\n
[+-]?((0[xX])?([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n     |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n     |[nN][aA][nN]\n     )\n
\n
\n
\n

\n Note that no leading or trailing whitespace is allowed, neither are trailing format specifiers such as f or d.\n

\n
\n
\n

For example, a valid input string is 0x1f.cP-5, which represents the value 31.75 * 2‒5.

\n
\n
\n

\n The numerical value represented by the input string is conceptually converted to an “infinitely precise” binary value that is\n then rounded to type float by the usual round‐to‐nearest (and ties‐to‐even) rule of IEEE\n 754 floating‐point arithmetic. The special values NaN and Inf(inity)? (case\n insensitive) are also recognized. Signs of zero and Infinity values are preserved.\n

\n
\n
\n

\n A System.FormatException is raised if\n the string representation is invalid. A System.OverflowException is raised, if the value represented by the input string (after rounding) is greater than System.Double.MaxValue or less than System.Double.MinValue.\n

\n
\n
\n
\n
\n
\n ⇧\n
val float32ToHexString: float32 -> string\n
\n
\n
\n
\n

\n Returns a hexadecimal string representation of the float32 argument. The hexadecimal\n format is the one supported by IEEE 754r, C99 and Java. This function produces the same output as the Float.toHexString method in Java.\n

\n
\n
\n
\n
\n
\n ⇧\n
val float32OfHexString: string -> float32\n
\n
\n
\n
\n

\n Returns the float32 value represented by the given string in hexadecimal format. The\n supported input format is (in regular expression notation):\n

\n
\n
\n
[+-]?((0[xX])?([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n     |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n     |[nN][aA][nN]\n     )\n
\n
\n
\n

\n Note that no leading or trailing whitespace is allowed, neither are trailing format specifiers such as f or d.\n

\n
\n
\n

For example, a valid input string is 0x1f.cP-5, which represents the value 31.75 * 2‒5.

\n
\n
\n

\n The numerical value represented by the input string is conceptually converted to an “infinitely precise” binary value that is\n then rounded to type float32 by the usual round‐to‐nearest (and ties‐to‐even) rule of\n IEEE 754 floating‐point arithmetic. The special values NaN and Inf(inity)? (case insensitive) are also recognized. Signs of zero and Infinity values are preserved.\n

\n
\n
\n

\n Note that in general float32OfHexString(str) is not equivalent to float32 (floatOfHexString(str)), because the latter version rounds\n twice.\n

\n
\n
\n

\n A System.FormatException is raised if\n the string representation is invalid. A System.OverflowException is raised, if the value represented by the input string (after rounding) is greater than System.Float.MaxValue or less than System.Float.MinValue.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/charstream.html", "content": "\n\n\n \n FParsec.CharStream\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1CharStream
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n Interface
\n 2\n Remarks
\n 3\n I/O exceptions
\n 4\n Members
\n
2CharStream<TUserState>
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n Interface
\n 2\n Remarks
\n 3\n Members
\n
\n 3\n CharStreamIndexToken
4CharStreamState
5TwoChars
\n
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.CharStream\n \n
\n
\n

6.11 FParsec.CharStream

\n
\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1CharStream
2CharStream<TUserState>
3CharStreamIndexToken
4CharStreamState
5TwoChars
\n
\n
\n
\n
\n

6.11.1 CharStream

\n
\n
\n

Provides read‐access to a sequence of UTF‐16 chars.

\n
\n
\n
\n

6.11.1.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype CharStream =\n  interface System.IDisposable\n\n  new:    chars: string * index: int * length: int\n       -> CharStream\n  new:    chars: string * index: int * length: int * streamBeginIndex: int64\n       -> CharStream\n\n  new:    chars: char[] * index: int * length: int\n       -> CharStream\n  new:    chars: char[] * index: int * length: int * streamBeginIndex: int64\n       -> CharStream\n\n  new:    chars: NativePtr<char> * length: int\n       -> CharStream\n  new:    chars: NativePtr<char> * length: int * streamBeginIndex: int64\n       -> CharStream\n\n  new:    path: string * encoding: System.Text.Encoding\n       -> CharStream\n  new:    path: string\n        * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n       -> CharStream\n  new:    path: string\n        * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n        * blockSize: int * blockOverlap: int * minRegexSpace: int\n        * byteBufferLength: int\n       -> CharStream\n\n  new:    stream: System.IO.Stream * encoding: System.Text.Encoding\n       -> CharStream\n  new:    stream: System.IO.Stream * leaveOpen: bool\n        * encoding: System.Text.Encoding\n       -> CharStream\n  new:    stream: System.IO.Stream * leaveOpen: bool\n        * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n       -> CharStream\n  new:    stream: System.IO.Stream * leaveOpen: bool\n        * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n        * blockSize: int * blockOverlap: int * minRegexSpace: int\n        * byteBufferLength: int\n       -> CharStream\n\n  member Dispose: unit -> unit\n\n  member BlockOverlap: int\n\n  member IndexOfFirstChar: int64\n  member IndexOfLastCharPlus1: int64\n\n  member IsBeginOfStream: bool\n  member IsEndOfStream: bool\n\n  member Index: int64\n  member IndexToken: CharStreamIndexToken\n  member Line: int64\n  member LineBegin: int64\n  member Column: int64\n  member Name: string with get, set\n  member Position: Position\n\n  val mutable StateTag: uint64\n\n  member Seek: index: int64 -> unit\n  member Seek: indexToken: CharStreamIndexToken -> unit\n\n  static val EndOfStreamChar: char\n\n  member Peek:  unit -> char\n  member Peek2: unit -> TwoChars\n  member Peek:  utf16Offset: int    -> char\n  member Peek:  utf16Offset: uint32 -> char\n\n  member PeekString: length: int -> string\n  member PeekString: buffer: char[] * bufferIndex: int * length: int -> int\n  member PeekString: buffer: NativePtr<char> * length: int -> int\n\n  member Match: char -> bool\n  member Match: chars: string -> bool\n  member Match: chars: char[] * charsIndex: int * length: int -> bool\n  member Match: chars: NativePtr<char> * length: int -> bool\n\n  member MatchCaseFolded: caseFoldedChar: char -> bool\n  member MatchCaseFolded: caseFoldedChars: string -> bool\n  member MatchCaseFolded: caseFoldedChars: NativePtr<char> * length:int -> bool\n\n  member Match: System.Text.RegularExpressions.Regex\n                -> System.Text.RegularExpressions.Match\n  member MinRegexSpace: int with get, set\n\n  member RegisterNewline: unit -> bool\n  member RegisterNewlines: lineOffset: int   -> newColumnMinus1: int   -> bool\n  member RegisterNewlines: lineOffset: int64 -> newColumnMinus1: int64 -> bool\n\n  // The following methods require manual registration of skipped newlines\n\n  member Skip: unit -> unit\n  member Skip: utf16Offset: int    -> unit\n  member Skip: utf16Offset: uint32 -> unit\n  member Skip: utf16Offset: int64  -> unit\n\n  member SkipAndPeek: unit   -> char\n  member SkipAndPeek: utf16Offset: int    -> char\n  member SkipAndPeek: utf16Offset: uint32 -> char\n\n  member Skip: char -> bool\n  member Skip: TwoChars -> bool\n  member Skip: chars: string -> bool\n  member Skip: chars: char[] * charsIndex: int * length: int -> bool\n  member Skip: chars: NativePtr<char> * length: int -> bool\n\n  member SkipCaseFolded: caseFoldedChar: char -> bool\n  member SkipCaseFolded: caseFoldedChars: string -> bool\n  member SkipCaseFolded: caseFoldedChars: NativePtr<char> * length:int -> bool\n\n  member Read: unit -> char\n  member Read: length: int -> string\n  member Read: buffer: char[] * bufferIndex: int * length: int -> int\n  member Read: buffer: NativePtr<char> * length: int -> int\n\n  member ReadFrom: indexOfFirstChar: CharStreamIndexToken -> string\n\n  // The following methods automatically register skipped newlines\n\n  member SkipWhitespace: unit -> bool\n  member SkipUnicodeWhitespace: unit -> bool\n\n  member SkipNewline: unit -> bool\n  member SkipUnicodeNewline: unit -> bool\n\n  member SkipNewlineThenWhitespace:\n      powerOf2TabStopDistance: int * allowFormFeed: bool -> int\n\n  member SkipRestOfLine: skipNewline: bool -> unit\n  member ReadRestOfLine: skipNewline: bool -> string\n\n  member ReadCharOrNewline: unit -> char\n\n  member SkipCharsOrNewlines: maxCount: int -> int\n  member ReadCharsOrNewlines: maxCount: int * normalizeNewlines: bool -> string\n\n  member SkipCharsOrNewlinesWhile:\n      predicate: (char -> bool) -> int\n  member SkipCharsOrNewlinesWhile:\n      predicateForFirstChar: (char -> bool) * predicate: (char -> bool) -> int\n  member SkipCharsOrNewlinesWhile:\n      predicate: (char -> bool) * minCount: int * maxCount: int -> int\n  member SkipCharsOrNewlinesWhile:\n      predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n    * minCount: int * maxCount: int -> int\n\n  member ReadCharsOrNewlinesWhile:\n      predicate: (char -> bool)\n    * normalizeNewlines: bool -> string\n  member ReadCharsOrNewlinesWhile:\n      predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n    * normalizeNewlines: bool -> string\n  member ReadCharsOrNewlinesWhile:\n      predicate: (char -> bool)\n    * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n  member ReadCharsOrNewlinesWhile:\n      predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n    * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n\n  member SkipCharsOrNewlinesUntilString:\n      str: string * maxCount: int\n    * foundString: out<bool> -> int\n  member SkipCharsOrNewlinesUntilString:\n      str: string * maxCount: int * normalizeNewlines: bool\n    * skippedCharsIfStringFoundOtherwiseNull: out<string> -> int\n\n  member SkipCharsOrNewlinesUntilCaseFoldedString:\n      caseFoldedString: string * maxCount: int\n    * foundString: out<bool> -> int\n  member SkipCharsOrNewlinesUntilCaseFoldedString:\n      caseFoldedString: string * maxCount: int * normalizeNewlines: bool\n    * skippedCharsIfStringFoundOtherwiseNull: out<string> -> int\n
\n
\n
\n
\n
\n
\n

6.11.1.2 Remarks

\n
\n
\n

\n The CharStream class provides a unified interface for efficiently reading UTF‐16 chars from\n a binary stream or an in‐memory char buffer (e.g. a string). It is optimized for the use in backtracking parser applications and supports\n arbitrary char‐based seeking, even for streams larger than the addressable memory (on 32‐bit platforms).\n

\n
\n
\n

\n The CharStream class is the base class of CharStream<'TUserState>, which adds a user‐definable state component and some convenience methods for working with the state\n of a CharStream instance.\n

\n
\n
\n

\n A CharStream constructed from a System.IO.Stream or a file path reads the stream\n block‐wise and only holds the most recently accessed block in memory. The blocks overlap in order to provide efficient access on the\n boundary between blocks.\n

\n
\n
\n

\n If the char content is already available as a string or a char array, a CharStream can be\n directly constructed from the char buffer (without needing to copy the buffer). The overhead of accessing an in‐memory char buffer through a\n CharStream is minimal.\n

\n
\n
\n
\n
\n
Position information
\n
\n
\n

The position of the next char in the stream is described by the following 4 properties:

\n
    \n
  • \n Index, the index of the UTF‐16 char in the\n stream,\n
    • \n
  • \n Line, the line number for the next char,\n
    • \n
  • \n LineBegin, the index of the first char of\n the line that also contains the next char,\n
    • \n
  • \n Name, a description or identifier for the\n stream.\n
    • \n
\n
\n
\n

\n The LineBegin can be combined with the\n Index to calculate a Column number.\n

\n
\n
\n

\n Among these properties the char index is the most important one, as the CharStream uses\n it to uniquely identify a UTF‐16 char in the stream.\n

\n
\n
\n

\n The other 3 properties further describe the text location of the char identified by the index, but they are not necessary for the core\n functionality of the CharStream class. The CharStream class keeps track of this additional position information to provide a more convenient interface to\n higher‐level library functions, in particular to assist debugging and error reporting purposes.\n

\n
\n
\n
Newlines
\n
\n
\n

\n For performance reasons the most basic stream operations do not automatically recognize newlines (end‐of‐line markers) in the stream content. If you skip any newline with these\n methods, you have to manually register the newline afterwards with one of the RegisterNewline methods (otherwise the line and column count becomes incorrect).\n

\n
\n
\n

\n In order to provide a convenient interface for parser routines, the CharStream class\n also provides some more advanced methods that automatically register any\n skipped standard newline (\"\\n\", \"\\r\\n\" and \"\\r\"). Additionally, it provides two methods that automatically register any\n Unicode newline (SkipUnicodeWhitespace and SkipUnicodeNewline).\n

\n
\n
\n

It should be obvious from the method names which methods automatically register newlines and which don’t.

\n
\n
\n
Case‐insensitive matching
\n
\n
\n

\n The MatchCaseFolded and SkipCaseFolded members match the content of\n the stream “case‐insensitively” with a reference string. In this instance “case‐insensitive” means that before\n the chars are matched with the reference string they are mapped to\n a canonical form where case differences are erased. For performance reasons MatchCaseFolded only applies the (non‐Turkic) 1‐to‐1 case folding mappings (v. 8.0.0) for Unicode code points in the Basic\n Multilingual Plane, i.e. code points below 0x10000. These mappings are sufficient for many case‐insensitive parser grammars encountered in\n practice, but they are not appropriate for matching arbitrary natural language content. Please also note that the CharStream class performs no Unicode normalization.\n

\n
\n
\n
Non‐sequential access
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
If you construct a CharStream from a System.IO.Stream or a file path and you\n backtrack over a distance long enough to require the CharStream to reread a previous\n block, then the underlying byte stream needs to support seeking, otherwise a NotSupportedException\n is thrown. Furthermore, the Decoder for the input\n Encoding must be serializable if you backtrack to\n a block other than the first in the stream. Note that file streams created for regular disk files are always seekable and all the .NET\n standard decoders are serializable. In order to support non‐seekable streams for applications which don’t require extensive\n backtracking, no exception will be thrown before an operation actually requires backtracking and the necessary capabilities of the stream\n or decoder are not available.\n

\n
\n
\n
Decoder errors
\n
\n
\n

\n A CharStream constructed from a binary input stream decodes the input data with the\n help of a Decoder instance obtained via the Encodings’s GetDecoder method. Depending on the configuration of the encoding the decoder might throw an\n exception if it encounters invalid byte sequences, usually a System.Text.DecoderFallbackException or a System.IO.ArgumentException. [1]\n

\n
\n
\n
Disposable interface
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
A CharStream holds managed and unmanaged resources that\n need to be explicitly released. Hence, it is very important that CharStream objects are\n promptly disposed after use. Where possible CharStream objects should only be used\n within a “using” block (C#), a “use” expression( F#) or similar constructs in other languages.\n

\n
\n
\n
Thread safety
\n
\n
\n

CharStream members are not thread‐safe.

\n
\n
\n
Low‐Trust version
\n
\n
\n

\n If you compile FParsec with the LOW_TRUST conditional compiler symbol, the CharStream class differs from the normal version as follows:\n

\n
    \n
  • \n No unverifiable code involving pointers is used. This allows\n FParsec to be executed in an environment with reduced trust, such as medium trust ASP.NET applications or Silverlight applications.\n
    • \n
  • \n A CharStream that is constructed from a System.IO.Stream or a file path reads the complete file into a\n single string during construction. This severely limits the maximum practical stream size.\n
    • \n
  • \n Although the CharStream class still supports the IDisposable interface, disposing the CharStream instances is\n no longer necessary, since no resources are hold that need to be explicitly released.\n
    • \n
\n

See also section 3.5.

\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.1.3 I/O exceptions

\n
\n
\n

\n If you construct a CharStream from a System.IO.Stream or a file path, the constructor and any CharStream operation that requires reading chars from the underlying byte stream may throw one of\n the following exceptions.\n

\n
\n
\n

\n In the Low‐Trust version, the constructor decodes the complete byte stream and hence only the\n constructor may throw one of these exceptions.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n Doing actual work in a constructor and potentially throwing exceptions seems to be a somewhat controversial design. We think it’s the right\n choice for the CharStream class, because this way you can a have a reasonable\n expectation that the CharStream actually works after you’ve successfully constructed it.\n

\n
\n
\n
\n
\n
\n

\n In general it is not safe to continue to use a CharStream instance after one of\n these exceptions was thrown, though calling Dispose() is always safe.\n

\n
\n
\n
\n
\n
NotSupportedException
\n
\n
\n

\n Seeking of the underlying byte stream is required, but the byte stream does not support seeking or the Encoding’s Decoder is\n not serializable. See also the remarks above on non‐sequential access.\n

\n
\n
\n
IOException
\n
\n
\n

An I/O occurred while reading data from the underlying byte stream.

\n
\n
\n
ArgumentException
\n
\n
\n

\n The underlying byte stream contains invalid bytes and the Encoding was constructed with\n the throwOnInvalidBytes option.\n

\n
\n
\n
DecoderFallbackException
\n
\n
\n

The underlying byte stream contains invalid bytes for which the decoder fallback threw this exception.

\n
\n
\n

\n The byte index of the invalid bytes in the stream is stored as a boxed System.Int64 in the \"Stream.Position\" entry of the Data member of the exception\n instance. The precision of the index depends on the precision of the DecoderFallbackException’s Index\n member. If the underlying System.IO.Stream is not seekable, the byte index only takes into account the bytes read by the CharStream, but not any bytes read before the CharStream was constructed.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.1.4 Members

\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: string * index: int * length: int\n     -> CharStream\n
\n
\n
\n
\n

\n Is equivalent to new CharStream(chars, index, length, 0L).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: string * index: int * length: int * streamBeginIndex: int64\n     -> CharStream\n
\n
\n
\n
\n

\n Constructs a CharStream from the chars in the string argument between the indices index (inclusive) and index + length (exclusive). By directly referencing the chars in the string this constructor\n avoids any copy of the string content.\n

\n
\n
\n

\n The first char in the stream is assigned the index streamBeginIndex. A positive streamBeginIndex allows you for example to create a substream of another CharStream, i.e. a CharStream instance\n that only contains a sub‐segment of another char stream but is accessible through the same char indices.\n

\n
\n
\n

\n chars must not be null. An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions:\n

\n
    \n
  • \n index ≥ 0, length ≥ 0, index + length ≤ chars.Length and\n
    • \n
  • 0 ≤ streamBeginIndex < 260.
    • \n
\n
\n
\n
\n
Important
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
The given string is “pinned” until the CharStream is disposed. Pinning the string prevents the GC from moving it around in memory during garbage\n collection. On .NET (at least in versions up to and including 4.0) the pinning has no effect if the string is large enough to be\n allocated on the Large Object Heap, i.e. has a length of about 42500 chars or more. However, pinning smaller strings does constrain the\n normal operations of the GC. Thus, to minimize the negative impact on the GC, you should dispose CharStream instances constructed from small strings as soon as you’re done parsing it. If you keep a\n large number of CharStream instances constructed from small strings around for an\n extended period of time, you risk fragmenting the heap.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: char[] * index: int * length: int\n     -> CharStream\n
\n
\n
\n
\n

\n This constructor is not available in the Low‐Trust version\n of FParsec.\n

\n
\n
\n

\n Is equivalent to new CharStream(chars, index, length, 0L).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: char[] * index: int * length: int * streamBeginIndex: int64\n     -> CharStream\n
\n
\n
\n
\n

\n This constructor is not available in the Low‐Trust version\n of FParsec.\n

\n
\n
\n

\n Constructs a CharStream from the chars in the char array argument between the indices\n index (inclusive) and index + length (exclusive). By directly referencing the chars in the char array this constructor\n avoids any copy of the char array content.\n

\n
\n
\n

\n The first char in the stream is assigned the index streamBeginIndex. A positive streamBeginIndex allows you for example to create a substream of another CharStream, i.e. a CharStream instance\n that only contains a sub‐segment of another char stream but is accessible through the same char indices.\n

\n
\n
\n

\n chars must not be null. An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions:\n

\n
    \n
  • \n index ≥ 0, length ≥ 0, index + length ≤ chars.Length and\n
    • \n
  • 0 ≤ streamBeginIndex < 260.
    • \n
\n
\n
\n
\n
Note
\n
\n
\n

\n A CharStream constructed from a char array does not support .NET regex matching via\n the Match method.\n

\n
\n
\n
\n
\n
\n
\n
Important
\n
\n
\n

\n The given char array is “pinned” until the CharStream is disposed. Pinning the char array prevents the GC from moving it around in\n memory during garbage collection. On .NET (at least in versions up to and including 4.0) the pinning has no effect if the char array is\n large enough to be allocated on the Large Object Heap, i.e. has a length of about 42500 chars or more. However, pinning smaller char\n arrays does constrain the normal operations of the GC. Thus, to minimize the negative impact on the GC, you should dispose CharStream instances constructed from small char arrays as soon as you’re done parsing\n it. If you keep a large number of CharStream instances constructed from\n small char arrays around for an extended period of time, you risk fragmenting the heap.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: NativePtr<char> * length: int\n     -> CharStream\n
\n
\n
\n
\n

\n This constructor is not available in the Low‐Trust version\n of FParsec.
\n

\n
\n
\n

\n Is equivalent to new CharStream(chars, length, 0L).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    chars: NativePtr<char> * length: int * streamBeginIndex: int64\n     -> CharStream\n
\n
\n
\n
\n

\n This constructor is not available in the Low‐Trust version\n of FParsec.\n

\n
\n
\n

\n Constructs a CharStream from the length chars at the pointer address. By directly referencing the chars at the pointer address this constructor\n avoids any copy of the char buffer.\n

\n
\n
\n

\n The first char in the stream is assigned the index streamBeginIndex. A positive streamBeginIndex allows you for example to create a substream of another CharStream, i.e. a CharStream instance\n that only contains a sub‐segment of another char stream but is accessible through the same char indices.\n

\n
\n
\n

\n chars must not be null. An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions:\n

\n
    \n
  • \n length ≥ 0, chars + length must not overflow and\n
    • \n
  • 0 ≤ streamBeginIndex < 260.
    • \n
\n
\n
\n
\n
Note
\n
\n
\n

\n A CharStream constructed from a pointer does not support .NET regex matching via the\n Match method.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    path: string * encoding: System.Text.Encoding\n     -> CharStream\n
\n
\n
\n
\n

\n Is equivalent to new CharStream(path, encoding, true).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    path: string\n      * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n     -> CharStream\n
\n
\n
\n
\n

Is equivalent to

\n
new CharStream(\n    path, encoding, detectEncodingFromByteOrderMarks,\n    blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n    blockOverlap = DefaultBlockSize/3,\n    minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n    byteBufferLength = DefaultByteBufferLength\n)\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    path: string\n      * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n      * blockSize: int * blockOverlap: int * minRegexSpace: int\n      * byteBufferLength: int\n     -> CharStream\n
\n
\n
\n
\n

\n Constructs a CharStream from a FileStream as if by calling\n

\n
new CharStream(\n    new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096,\n                   FileOptions.SequentialScan),\n    leaveOpen = false,\n    encoding = encoding,\n    detectEncoding = true,\n    blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n    blockOverlap = DefaultBlockSize/3,\n    minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n    byteBufferLength = DefaultByteBufferLength\n)\n
\n
\n
\n

\n If an exception occurs after the FileStream is constructed but before the CharStream\n constructor is finished, the FileStream is disposed.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The FileStream constructor might throw an exception, too.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    stream: System.IO.Stream * encoding: System.Text.Encoding\n     -> CharStream\n
\n
\n
\n
\n

\n Is equivalent to new CharStream(stream, false, encoding, true).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    stream: System.IO.Stream * leaveOpen: bool\n      * encoding: System.Text.Encoding\n     -> CharStream\n
\n
\n
\n
\n

\n Is equivalent to new CharStream(stream, leaveOpen, encoding, true).\n

\n
\n
\n
\n
\n
\n ⇧\n
new:    stream: System.IO.Stream * leaveOpen: bool\n      * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n     -> CharStream\n
\n
\n
\n
\n

Is equivalent to

\n
new CharStream(\n    stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks,\n    blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n    blockOverlap = DefaultBlockSize/3,\n    minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n    byteBufferLength = DefaultByteBufferLength\n)\n
\n
\n
\n
\n
\n
\n ⇧\n
new:    stream: System.IO.Stream * leaveOpen: bool\n      * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n      * blockSize: int * blockOverlap: int * minRegexSpace: int\n      * byteBufferLength: int\n     -> CharStream\n
\n
\n
\n
\n

\n Constructs a CharStream from a System.IO.Stream.\n

\n
\n
\n

\n The normal version of the CharStream class supports stream sizes up to approximately\n (231/p)×(blockSize ‐ blockOverlap) chars, where p is 4 on a 32‐bit CLR and 8 on a 64‐bit CLR.
The Low‐Trust version only supports streams small enough that the complete content can be read into a\n single string.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This constructor reads the first block of chars from the input stream and hence can throw any of the I/O related exceptions detailed in\n the exceptions section above.\n

\n
\n
\n
\n
\n
\n

Arguments:

\n
\n
\n
stream
\n
\n
\n

\n The byte stream providing the input. If stream.CanRead returns false, an ArgumentException is\n thrown.\n

\n
\n
\n
leaveOpen
\n
\n
\n

\n Indicates whether the stream should be left open when the CharStream has finished reading it.\n

\n
\n
\n
encoding
\n
\n
\n

\n The default Encoding used for decoding the byte stream into chars.\n

\n
\n
\n

\n If the preamble returned by encoding.GetPreamble() is present at the beginning of the stream, the CharStream will skip over it.\n

\n
\n
\n
detectEncodingFromByteOrderMarks
\n
\n
\n

\n Indicates whether the constructor should detect the encoding from a unicode byte‐order mark at the beginning of the stream. An encoding detected from a\n byte‐order mark overrides the default encoding. The standard byte‐order marks for\n the following encodings are supported: UTF‐8, UTF‐16 LE/BE and UTF‐32 LE/BE.\n

\n
\n
\n
blockSize
\n
\n
\n

\n The number of chars per block. The value is rounded up to the first positive multiple of 1536. The default is 3×216 ≈ 200k.\n

\n
\n
\n
\n blockOverlap\n
\n
\n
\n

\n The number of chars at the end of a block that are preserved when reading the next block into into its internal char buffer. If this\n value is less than encoding.GetMaxCharCount(1) or not less than\n blockSize/2, the default value is\n used instead. The default is blockSize/3.\n

\n
\n
\n
byteBufferLength
\n
\n
\n

The size of the byte buffer used for decoding purposes. The default is 212 = 4KB.

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member Dispose: unit -> unit\n
\n
\n
\n
\n

\n Releases all resources used by the CharStream. If the CharStream was constructed from a System.IO.Stream or a file path and the constructor was not called\n with leaveOpen = true, the byte stream\n is closed.\n

\n
\n
\n
\n
\n
\n ⇧\n
member BlockOverlap: int\n
\n
\n
\n
\n

\n The number of chars at the end of a block that are preserved when the CharStream reads\n the next block into its internal char buffer.\n

\n
\n
\n

\n This value is only relevant for optimization purposes and as the maximum value for MinRegexSpace.\n

\n
\n
\n

\n This value can only be set at construction time with the respective constructor\n parameter.\n

\n
\n
\n

\n If the CharStream is constructed from a string, char array or char pointer or only\n contains 1 block, then this value is 0. In the Low‐Trust version this value is always 0.\n

\n
\n
\n
\n
\n
\n ⇧\n
member IndexOfFirstChar: int64\n
\n
\n
\n
\n

\n The index of the first char in the stream. This value is determined by the streamIndexOffset argument of some of the CharStream\n constructors. By default this value is 0.\n

\n
\n
\n
\n
\n
\n ⇧\n
member IndexOfLastCharPlus1: int64\n
\n
\n
\n
\n

\n The index of the last char of the stream plus 1, or Int64.MaxValue if the end of the stream has not yet been detected.\n

\n
\n
\n
\n
\n
\n ⇧\n
member IsBeginOfStream: bool\n
\n
\n
\n
\n

\n Indicates whether the next char in the stream is the first char, i.e. whether Index equals IndexOfFirstChar.\n

\n
\n
\n

If the stream is empty, this value is always true.

\n
\n
\n
\n
\n
\n ⇧\n
member IsEndOfStream: bool\n
\n
\n
\n
\n

\n Indicates whether there is no char remaining in the stream, i.e. whether Index equals IndexOfLastCharPlus1.\n

\n
\n
\n

If the stream is empty, this value is always true.

\n
\n
\n
\n
\n
\n ⇧\n
member Index: int64\n
\n
\n
\n
\n

The stream index of the next char.

\n
\n
\n
\n
\n
\n ⇧\n
member IndexToken: CharStreamIndexToken\n
\n
\n
\n
\n

\n A CharStreamIndexToken value representing the\n current Index value.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Line: int64\n
\n
\n
\n
\n

The line number for the next char. (The line count starts with 1.)

\n
\n
\n
\n
\n
\n ⇧\n
member LineBegin: int64\n
\n
\n
\n
\n

The stream index of the first char of the line that also contains the next char.

\n
\n
\n
\n
\n
\n ⇧\n
member Column: int64\n
\n
\n
\n
\n

\n The UTF‐16 column number of the next char, i.e. Index ‐ LineBegin + 1.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Name: string with get, set\n
\n
\n
\n
\n

This string is used in error messages to describe the input stream.

\n
\n
\n

\n If the CharStream is constructed from a file path, the constructor initializes the\n Name value with the file path value. Otherwise, Name is initialized to null.\n

\n
\n
\n

\n If the stream content is the concatenated content of multiple input files, you can improve error messages and help debugging by setting the\n name and resetting the line and column count at the transitions between the different content pieces.\n

\n
\n
\n

\n Setting the Name value increments the StateTag by 1, independent of whether the new value is different\n from the previous one.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Position: Position\n
\n
\n
\n
\n

\n Returns new Position(Name, Index, Line, Column).\n

\n
\n
\n
\n
\n
\n ⇧\n
val mutable StateTag: uint64\n
\n
\n
\n
\n

\n The StateTag’s purpose is to provide an efficient way to determine whether the\n publically visible state of the CharStream has changed after a series of method calls.\n For the purpose of this property, the state is defined as the aggregate of the Index, Line, LineBegin and Name\n values. The UserState value of CharStream<'UserState> instances is also part of the CharStream state. If a\n method or property setter changes one or more of these state values it increments the StateTag’s by 1. Thus, to determine whether a series of method calls has changed the CharStream, it is often enough to compare the StateTag values\n from before and after the method calls.\n

\n
\n
\n

\n The StateTag property is primarily meant for use in the implementation of parser\n combinators. If you directly call CharStream methods, you normally don’t need the StateTag to determine whether the state has changed, because that is usually obvious from\n either the method’s return value or the context in which it was called. Please see section 5.4.3 for more details on the design rationale behind the\n StateTag.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Seek: index: int64 -> unit\n
\n
\n
\n
\n

Seeks the CharStream to the char with the specified index in the stream.

\n
\n
\n

\n If you pass an index larger than the index of the last char in the stream, this method seeks the stream to the end of the stream, i.e. to\n one char past the last char in the stream.\n

\n
\n
\n

\n The index is zero‐based, except if the CharStream was constructed with a positive streamIndexOffset argument, in which case the index of the first char equals the value of the\n streamIndexOffset argument (and the IndexOfFirstChar value).\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1. When it does not change the position, it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the index is less than the IndexOfFirstChar. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Seek: indexToken: CharStreamIndexToken -> unit\n
\n
\n
\n
\n

\n This method is an optimized implementation of Seek(GetIndex(indexToken)).\n

\n
\n
\n
\n
\n
\n ⇧\n
static val EndOfStreamChar: char\n
\n
\n
\n
\n

\n The char returned by Peek and Read at the end of the stream.\n

\n
\n
\n

\n The value is '\\uFFFF'.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Peek: unit -> char\n
\n
\n
\n
\n

Returns the next char without changing the state of the CharStream.

\n
\n
\n

\n At the end of the CharStream the EndOfStreamChar ('\\uFFFF') is returned.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Peek2: unit -> TwoChars\n
\n
\n
\n
\n

\n Peek2() is an optimized implementation of new TwoChars(Peek(), Peek(1)).\n

\n
\n
\n
\n
\n
\n ⇧\n
member Peek: utf16Offset: int -> char\n
\n
\n
\n
\n

\n Returns the char at the stream index Index + utf16Offset, without changing the state of the CharStream.\n

\n
\n
\n

\n If Index + utf16Offset is smaller than the index of the first char in the stream or larger than the index of the last char in\n the stream, the EndOfStreamChar\n ('\\uFFFF') is\n returned.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Peek: utf16Offset: uint32 -> char\n
\n
\n
\n
\n

\n This method is an optimized implementation of Peek(int) for uint32 arguments.\n

\n
\n
\n
\n
\n
\n ⇧\n
member PeekString: length: int -> string\n
\n
\n
\n
\n

\n Returns a string with the next length stream chars, without changing the state of the\n CharStream.\n

\n
\n
\n

\n If less than length chars are remaining in the stream, only the remaining chars are\n returned.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
If length is greater than the number of remaining chars\n in the stream, a temporary string with length chars may be allocated. For very large\n length values this might lead to an OutOfMemoryException even though a string with only the remaining chars in the stream would comfortably fit\n into memory.\n

\n
\n
\n

\n Please also note that the maximum length of a string on .NET is less than 230. Allocating a string larger than the maximum\n length will always yield an OutOfMemoryException, even on 64‐bit systems with enough\n physical memory.\n

\n
\n
\n
\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member PeekString: buffer: char[] * bufferIndex: int * length: int -> int\n
\n
\n
\n
\n

\n Copies the next length stream chars into buffer, without changing the state of the CharStream. Returns\n the number of chars copied.\n

\n
\n
\n

\n The chars are written into buffer beginning at the index bufferIndex. If less than length chars are remaining in the\n stream, only the remaining chars are copied.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions: bufferIndex ≥ 0, length ≥ 0 and bufferIndex + length ≤ buffer.Length. This method may also throw any\n of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member PeekString: buffer: NativePtr<char> * length: int -> int\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Copies the next length stream chars into the buffer at the specified pointer address,\n without changing the state of the CharStream. Returns the number of chars copied.\n

\n
\n
\n

\n If less than length chars are remaining in the stream, only the remaining chars are\n copied.\n

\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Match: char -> bool\n
\n
\n
\n
\n

\n Returns true if the next char in the stream matches the specified char. At the end of\n the stream Match always returns false.\n

\n
\n
\n

This method does not change the state of the CharStream.

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Match: chars: string -> bool\n
\n
\n
\n
\n

\n Returns true if the passed string chars matches the next chars.Length stream chars.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, false is returned. If chars is empty, true is returned. chars must not be null.\n

\n
\n
\n

This method does not change the state of the CharStream.

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Match: chars: char[] * charsIndex: int * length: int -> bool\n
\n
\n
\n
\n

\n Returns true if the next length\n stream chars match the chars in the array chars at the indices charIndex to charsIndex + length - 1.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, false is returned. If length is 0, true is returned. chars must not be null.\n

\n
\n
\n

This method does not change the state of the CharStream.

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions: charsIndex ≥ 0, length ≥ 0 and charsIndex + length ≤ chars.Length. This method may also throw any\n of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Match: chars: NativePtr<char> * length: int -> bool\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Returns true if the next length\n stream chars match the chars at the specified pointer address.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, false is returned. If length is 0, true is returned.\n

\n
\n
\n

This method does not change the state of the CharStream.

\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member MatchCaseFolded: caseFoldedChar: char -> bool\n
\n
\n
\n
\n

\n Behaves like Match(caseFoldedChar), except that the next char in the stream is case‐folded before it is\n compared with caseFoldedChar.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the char in the stream is case‐folded before it is matched, the char caseFoldedChar is assumed to already be case‐folded (e.g. with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member MatchCaseFolded: caseFoldedChars: string -> bool\n
\n
\n
\n
\n

\n Behaves like Match(caseFoldedChars), except that the chars in the stream are\n case‐folded before they are compared with caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the CharStream are case‐folded before they are matched, the chars\n in the string argument caseFoldedChars are assumed to already be case‐folded (e.g.\n with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member MatchCaseFolded: caseFoldedChars: NativePtr<char> * length:int -> bool\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Behaves like Match(caseFoldedChars, length), except that the chars in the stream are case‐folded before they are compared with the chars at the pointer\n address caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the CharStream are case‐folded before they are matched, the chars\n at the pointer address caseFoldedChars are assumed to already be case‐folded (e.g.\n with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member Match: System.Text.RegularExpressions.Regex\n              -> System.Text.RegularExpressions.Match\n
\n
\n
\n
\n

\n Applies the given regular expression to the stream chars beginning with the next char. Returns the resulting Match object.\n

\n
\n
\n

\n For performance reasons you should specify the regular expression such that it can only match at the beginning of a string, for example by\n prepending \"\\\\A\".\n

\n
\n
\n

\n For CharStream instances constructed from strings the regular expression is applied to\n a string containing all the remaining chars in the stream.\n

\n
\n
\n

\n For CharStream instances constructed from large binary streams (with more than 1 block)\n the regular expression is not applied to a string containing all the remaining chars in the stream. Here the MinRegexSpace value determines the minimum number of\n chars that are guaranteed to be visible to the regular expression (assuming there are still enough chars remaining in the stream). The\n exact number of chars visible to the regular expression may be affected even by calls to CharStream methods like Peek or Match that otherwise guarantee to not change the (outwardly visible) state of the CharStream.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
Important
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
This method is not supported by CharStream instances\n constructed directly from char arrays or pointers. A NotSupportedException is thrown if this method is called on such a CharStream instance.\n

\n
\n
\n
\n
\n
\n
\n
Important
\n
\n
\n

\n This note does not apply to the Low‐Trust version of\n FParsec.
If the CharStream was constructed from a System.IO.Stream or a file path, the\n regular expression is applied to an internal mutable buffer. Since the Match object may work lazily, i.e. compute return values not before they are needed, you need to retrieve\n all the required information from the Match object before you continue to access the\n CharStream, otherwise you might get back invalid match results. Note that all\n strings returned by the Match object are, of course, immutable.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member MinRegexSpace: int with get, set\n
\n
\n
\n
\n

\n The number of chars that are guaranteed to be visible to a regular expression when it is matched by Match (assuming there are enough chars remaining in the stream).\n

\n
\n
\n

\n The value must be non‐negative and not greater than BlockOverlap. The default value is 2/3 of BlockOverlap.\n

\n
\n
\n

\n If the CharStream is constructed from a string, char array or char pointer or has only\n 1 block, then this value has no relevance and calling the property setter has no effect. (No Low‐Trust version CharStream instance has more\n than 1 block.)\n

\n
\n
\n

\n The MinRegexSpace value is not recorded in CharStreamState instances and setting its value does not affect the StateTag.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if you try to set the property on a multi‐block CharStream instance to a negative value or a value larger than the BlockOverlap.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RegisterNewline: unit -> bool\n
\n
\n
\n
\n

\n Registers a newline (an end‐of‐line character) at the previous stream char, i.e. increments the Line value by 1 and sets the LineBegin to Index.\n

\n
\n
\n

\n The previous LineBegin value must not equal\n Index. (For performance reasons this condition\n is only checked by an assert check in the debug build).\n

\n
\n
\n

\n This method also increments the StateTag by\n 1.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RegisterNewlines: lineOffset: int -> newColumnMinus1: int -> bool\n
\n
\n
\n
\n

\n Increments the Line value by lineOffset and sets the LineBegin value to Index - newColumnMinus1 (so that the Column value becomes newColumnMinus1 + 1).\n

\n
\n
\n

\n The lineOffset must not be 0, the new Line value must be greater than 0 and and the new LineBegin value must be different from the\n previous one. (For performance reasons these conditions are only checked by assert checks in the debug build).\n

\n
\n
\n

\n This method also increments the StateTag by\n 1.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RegisterNewlines: lineOffset: int64 -> newColumnMinus1: int64 -> bool\n
\n
\n
\n
\n

\n This method is a variant of RegisterNewlines for int64 arguments.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: unit -> unit\n
\n
\n
\n
\n

Advances the position within the stream by 1 char, except at the end of the stream, where it does nothing.

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: utf16Offset: int -> unit\n
\n
\n
\n
\n

Advances the position within the stream by utf16Offset chars.

\n
\n
\n

\n The new position within the stream will be min(Index + utf16Offset, IndexOfLastCharPlus1). This means you can’t move past the end of the stream, because any position beyond the last char in the stream\n is interpreted as precisely one char beyond the last char.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the new position would lie before the beginning of the CharStream, i.e. if the new index would be less than IndexOfFirstChar. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1. When it does not change the position (because the given offset is 0 or because the stream has\n already reached the end and the offset is positive), it may or may not increment the StateTag by 1.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: utf16Offset: uint32 -> unit\n
\n
\n
\n
\n

\n This method is an optimized implementation of Skip for uint32 offsets.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: utf16Offset: int64 -> unit\n
\n
\n
\n
\n

\n This method is a variant of Skip for int64 offsets.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipAndPeek: unit -> char\n
\n
\n
\n
\n

\n c <- SkipAndPeek() is an optimized implementation of Skip(); c <- Peek().\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipAndPeek: utf16Offset: int -> char\n
\n
\n
\n
\n

\n c <- SkipAndPeek(utf16Offset) is an optimized implementation of Skip(utf16Offset); c <- Peek(), with the following exception for\n negative offsets n:
If the new position would lie before the beginning of\n the CharStream, i.e. if the new index would be less than IndexOfFirstChar, then SkipAndPeek(n) does not throw an exception\n like stream.Skip(n) would do. Instead it sets the\n position of the stream to IndexOfFirstChar and returns the EndOfStreamChar ('\\uFFFF').\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipAndPeek: utf16Offset: uint32 -> char\n
\n
\n
\n
\n

\n c <- SkipAndPeek(utf16Offset) is an optimized implementation of Skip(utf16Offset); c <- Peek().\n

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: char -> bool\n
\n
\n
\n
\n

\n Skips over the next char in the stream if this char matches the passed argument char. Returns true if the chars match; otherwise, false. At the end of the\n stream this method always returns false.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: TwoChars -> bool\n
\n
\n
\n
\n

\n Skips over the next two chars in the stream if these chars match the two chars in the passed TwoChars value. Returns true if the chars match.\n

\n
\n
\n

\n If not both chars match or if there are less than 2 chars remaining in the stream, no char is skipped and false is returned.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: chars: string -> bool\n
\n
\n
\n
\n

\n Skips over the next chars.Length chars\n in the stream if these chars match the passed string chars. Returns true if the chars match.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, no char is skipped and false is returned. If chars is empty, true is returned. chars must not be null.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if chars is empty, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: chars: char[] * charsIndex: int * length: int -> bool\n
\n
\n
\n
\n

\n Skips over the next length chars in the stream if these chars match the chars in the\n passed array chars at the indices charIndex to charsIndex + length - 1. Returns true if the chars match.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, false is returned and the position within the CharStream is\n not changed. If length is 0, true\n is returned. chars must not be null.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if length is 0, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions: charsIndex ≥ 0, length ≥ 0 and charsIndex + length ≤ chars.Length. This method may also throw any\n of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Skip: chars: NativePtr<char> * length: int -> bool\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Skips over the next length chars in the stream if these chars match the chars at the\n pointer address chars. Returns true\n if the chars match.\n

\n
\n
\n

\n If not all the chars match or if there are not enough chars remaining in the stream, false is returned and the position within the CharStream is\n not changed. If length is 0, true\n is returned.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if length is 0, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCaseFolded: caseFoldedChar: char -> bool\n
\n
\n
\n
\n

\n Behaves like Skip(caseFoldedChar), except that the next char in the stream is case‐folded before it is\n compared with caseFoldedChar.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the char in the stream is case‐folded before it is matched, the char caseFoldedChar is assumed to already be case‐folded (e.g. with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member SkipCaseFolded: caseFoldedChars: string -> bool\n
\n
\n
\n
\n

\n Behaves like Skip(caseFoldedChars), except that the chars in the stream are case‐folded before they are\n compared with caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the CharStream are case‐folded before they are matched, the chars\n in the string argument caseFoldedChars are assumed to already be case‐folded (e.g.\n with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member SkipCaseFolded: caseFoldedChars: NativePtr<char> * length:int -> bool\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Behaves like Skip(caseFoldedChars), except that the chars in the stream are\n case‐folded before they are compared with the chars at the pointer address caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the CharStream are case‐folded before they are matched, the chars\n at the pointer address caseFoldedChars are assumed to already be case‐folded (e.g.\n with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member Read: unit -> char\n
\n
\n
\n
\n

Skips over the next char in the stream. Returns the skipped char.

\n
\n
\n

\n At the end of the stream Read() does not change the stream\n position and returns the EndOfStreamChar ('\\uFFFF').\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member Read: length: int -> string\n
\n
\n
\n
\n

\n Skips over the next length chars in the stream. Returns the skipped chars as a string.\n

\n
\n
\n

\n If less than length chars are remaining in the stream, only the remaining chars are\n skipped and returned.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if length is 0, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Read: buffer: char[] * bufferIndex: int * length: int -> int\n
\n
\n
\n
\n

\n Skips over the next length stream chars and copies the skipped chars into buffer. Returns the number of copied and skipped chars.\n

\n
\n
\n

\n The chars are written into buffer beginning at the index bufferIndex. If less than length chars are remaining in the\n stream, only the remaining chars are copied and skipped.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if length is 0, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if the arguments do not satisfy the following conditions: bufferIndex ≥ 0, length ≥ 0 and bufferIndex + length ≤ buffer.Length. This method may also throw any\n of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Read: buffer: NativePtr<char> * length: int -> int\n
\n
\n
\n
\n

\n This method is not available in the Low‐Trust version of\n FParsec.\n

\n
\n
\n

\n Skips over the next length stream chars and copies the skipped chars into the buffer at\n the given pointer address. Returns the number of copied and skipped chars.\n

\n
\n
\n

\n If less than length chars are remaining in the stream, only the remaining chars are\n copied and skipped.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag, except if length is 0, in which case it may or may not increment the StateTag by 1.\n

\n
\n
\n

\n If length is negative, an ArgumentOutOfRangeException is thrown. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadFrom: indexOfFirstChar: CharStreamIndexToken -> string\n
\n
\n
\n
\n

\n Returns a string with the chars between the stream index indexOfFirstChar (inclusive)\n and the current Index of the stream\n (exclusive).\n

\n
\n
\n

This method trows

\n \n

It may also throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
Note
\n
\n
\n

\n You may only pass CharStreamToken values that were retrieved from the CharStream instance on which you’re calling ReadFrom. Passing a CharStreamToken value that was created\n for another CharStream instance triggers an assert exception in debug builds and\n will otherwise lead to undefined behaviour.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member SkipWhitespace: unit -> bool\n
\n
\n
\n
\n

\n Skips over any sequence of space (' '), tab ('\\t') or newline ('\\r', '\\n') chars. Returns true if it\n skips at least one char, otherwise false.\n

\n
\n
\n

\n This method registers any skipped standard newline (\"\\n\", \"\\r\\n\" or \"\\r\").\n

\n
\n
\n

\n When this method skips at least one char, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipUnicodeWhitespace: unit -> bool\n
\n
\n
\n
\n

\n Skips over any sequence of unicode whitespace chars (as identified by System.Char.IsWhiteSpace). Returns true if it skips at least one char, otherwise false.\n

\n
\n
\n

\n This method registers any skipped unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u000C\", \"\\u2028\" or \"\\u2029\").\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This method recognizes the form feed char '\\f' ('\\u000C') as a Unicode whitespace character, but not as a newline character.\n

\n
\n
\n
\n
\n
\n

\n When this method skips at least one char, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipNewline: unit -> bool\n
\n
\n
\n
\n

\n Skips over a standard newline (\"\\n\", \"\\r\\n\" or \"\\r\"). Returns true if a\n newline is skipped, otherwise false.\n

\n
\n
\n

When this method skips a newline, it also registers it.

\n
\n
\n

\n When this method skips a newline, it increments the StateTag by 1, otherwise it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipUnicodeNewline: unit -> bool\n
\n
\n
\n
\n

\n Skips over a unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\", or \"\\u2029\"). Returns true if a\n newline is skipped, otherwise false.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n This method does not recognize the form feed char '\\f' ('\\u000C') as a newline character.\n

\n
\n
\n
\n
\n
\n

When this method skips a newline, it also registers it.

\n
\n
\n

\n When this method skips a newline, it increments the StateTag by 1, otherwise it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipNewlineThenWhitespace:\n    powerOf2TabStopDistance: int * allowFormFeed: bool -> int\n
\n
\n
\n
\n

\n Skips over a newline (\"\\n\", \"\\r\\n\" or \"\\r\") followed by any (possibly empty) sequence of whitespace chars (' ', '\\t', '\\r', '\\n' and optionally '\\f').\n

\n
\n
\n

\n If this method skips no chars because the next stream char is no newline char, it returns ‒1. Otherwise it returns the indentation of the\n first line with non‐whitespace characters.\n

\n
\n
\n

The indentation is calculated as follows:

\n
    \n
  • \n Any newline char ('\\r' or '\\n') or form feed char ('\\f') resets the indentation to 0.\n
    • \n
  • \n Any space char (' ') increments the\n indentation by 1.\n
    • \n
  • \n Any tab char ('\\t') increments the indentation by
    powerOf2TabStopDistance ‐ (indentation modulo powerOf2TabStopDistance).\n
    • \n
\n
\n
\n

\n The maximum indentation is 231 ‐ 1. If skipping a whitespace char would cause the indentation to overflow, the char is not\n skipped and the method returns the indentation up to that char.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if powerOf2TabStopDistance is not a positive power of 2.\n

\n
\n
\n

\n The value of the allowFormFeed argument determines whether this method accepts the form\n feed char '\\f' as a whitespace char.\n

\n
\n
\n

\n This method registers all skipped standard newlines (\"\\n\", \"\\r\\n\" or \"\\r\").\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipRestOfLine: skipNewline: bool -> unit\n
\n
\n
\n
\n

\n Skips over any chars before the next newline (\"\\n\", \"\\r\\n\" or \"\\r\") or the end of the stream. If skipNewline is true and a newline is\n present, the newline is also skipped.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member ReadRestOfLine: skipNewline: bool -> string\n
\n
\n
\n
\n

\n ReadRestOfLine(skipNewline) behaves like SkipRestOfLine(skipNewline), except\n that it returns a string with the skipped chars (without a newline).\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharOrNewline: unit -> char\n
\n
\n
\n
\n

\n Skips over any single char or standard newline (\"\\n\", \"\\r\\n\" or \"\\r\").\n

\n
\n
\n

\n This method returns '\\n' when it skips a newline. Otherwise, it returns the skipped char, except at the end of the stream, where\n it returns the EndOfStreamChar ('\\uffff').\n

\n
\n
\n

When this method skips a newline, it also registers it.

\n
\n
\n

\n When this method skips a char or newline, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlines: maxCount: int -> int\n
\n
\n
\n
\n

Skips over up to maxCount chars. Returns the number of skipped chars.

\n
\n
\n

\n The number of actually skipped chars is less than maxCount if the end of the stream is\n reached after less than maxCount chars.\n

\n
\n
\n

\n This method counts standard newlines (\"\\n\", \"\\r\\n\" or \"\\r\") as single chars. When this method skips a newline, it also registers it.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if maxCount is\n negative. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharsOrNewlines: maxCount: int * normalizeNewlines: bool -> string\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlines(maxCount), except that it returns a string with the skipped\n chars.\n

\n
\n
\n

\n The normalizeNewlines parameter determines whether all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") in the returned string are normalized to '\\n' or whether they are preserved in the original form\n they are encountered in the input.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesWhile:\n    predicate: (char -> bool) -> int\n
\n
\n
\n
\n

\n Skips over a sequence of chars that satisfy the predicate function. Stops at the first\n char for which predicate returns false. Returns the number of skipped chars.\n

\n
\n
\n

\n This method counts standard newlines (\"\\n\", \"\\r\\n\" or \"\\r\") as single chars and passes them to the predicate function as single '\\n' chars. When\n this method skips a newline, it also registers it.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n
\n
Caution
\n
\n
\n

\n The predicate function must not access the CharStream instance itself, because SkipCharsOrNewlinesWhile relies on predicate not having any\n side‐effect on the internal state of the stream.\n

\n
\n
\n
\n
\n
\n

This method may throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesWhile:\n    predicateForFirstChar: (char -> bool) * predicate: (char -> bool) -> int\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesWhile(predicate),\n except that the first char to be skipped must satisfy predicateForFirstChar instead of\n predicate.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesWhile:\n    predicate: (char -> bool) * minCount: int * maxCount: int -> int\n
\n
\n
\n
\n

\n Skips over a sequence of up to maxCount chars that satisfy the predicate function, but backtracks to the start if it can only skip less than minCount chars. Returns the number of skipped chars.\n

\n
\n
\n

\n This method counts standard newlines (\"\\n\", \"\\r\\n\" or \"\\r\") as single chars and passes them to the predicate function as single '\\n' chars. When\n this method skips a newline, it also registers it.\n

\n
\n
\n

\n An ArgumentOutOfRangeException is thrown if maxCount is\n negative. This method may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
Caution
\n
\n
\n

\n The predicate function must not access the CharStream instance itself, because SkipCharsOrNewlinesWhile relies on predicate not having any\n side‐effect on the internal state of the stream.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesWhile:\n    predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n  * minCount: int * maxCount: int -> int\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesWhile(predicate, nMin, nMax), except that the first char to\n be skipped must satisfy predicateForFirstChar instead of predicate.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharsOrNewlinesWhile:\n    predicate: (char -> bool)\n  * normalizeNewlines: bool -> string\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesWhile(predicate),\n except that it returns a string with the skipped chars.\n

\n
\n
\n

\n The normalizeNewlines parameter determines whether all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") in the returned string are normalized to '\\n' or whether they are preserved in the original form\n they are encountered in the input.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharsOrNewlinesWhile:\n    predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n  * normalizeNewlines: bool -> string\n
\n
\n
\n
\n

\n Behaves like ReadCharsOrNewlinesWhile(predicate, normalizeNewlines), except that the first char to be skipped must satisfy predicateForFirstChar instead of predicate.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharsOrNewlinesWhile:\n    predicate: (char -> bool)\n  * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesWhile(predicate, minCount, maxCount), except that it\n returns a string with the skipped chars.\n

\n
\n
\n

\n The normalizeNewlines parameter determines whether all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") in the returned string are normalized to '\\n' or whether they are preserved in the original form\n they are encountered in the input.\n

\n
\n
\n
\n
\n
\n ⇧\n
member ReadCharsOrNewlinesWhile:\n    predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n  * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n
\n
\n
\n
\n

\n Behaves like ReadCharsOrNewlinesWhile(predicate, minCount, maxCount, normalizeNewlines), except that the first char to be skipped must satisfy predicateForFirstChar instead of predicate.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesUntilString:\n    str: string * maxCount: int\n  * foundString: out<bool> -> int\n
\n
\n
\n
\n

\n Skips over all stream chars before the first occurrence of the specified string or the end of the stream, but not over more than maxCount chars. Assigns true to the\n output parameter if the string is found, otherwise false.\n

\n
\n
\n

\n This method registers skipped newlines (\"\\n\", \"\\r\\n\" or \"\\r\") and counts them as single chars. However, no newline normalization takes\n place when the argument string str is matched with the stream chars. Hence, str should either contain no newlines or only in the form they occur in the stream. If str starts with '\\n', then SkipCharsOrNewlinesUntilString will not find occurences of str\n in the stream that start in the middle of an \"\\r\\n\" newline.\n

\n
\n
\n

\n When this method changes the stream position, it increments the StateTag by 1; otherwise, it does not change the StateTag.\n

\n
\n
\n

This method throws

\n
    \n
  • \n an ArgumentException, if the string argument is empty, and\n
    • \n
  • \n an ArgumentOutRangeException, if nMax is negative.\n
    • \n
\n

It may also throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesUntilString:\n    str: string * maxCount: int * normalizeNewlines: bool\n  * skippedCharsIfStringFoundOtherwiseNull: out<string> -> int\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesUntilString(str, maxCount, outBool), except that its output\n parameter is a string instead of a boolean. If str is found, a string with the skipped\n chars is assigned to this output parameter; otherwise, null is assigned to the output\n parameter.\n

\n
\n
\n

\n The normalizeNewlines parameter determines whether all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") in the output string are normalized to '\\n' or are preserved in the original form they are\n encountered in the input.\n

\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesUntilCaseFoldedString:\n    caseFoldedString: string * maxCount: int\n  * foundString: out<bool> -> int\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesUntilString(caseFoldedString, maxCount, foundString), except that the chars in the stream are case‐folded before they are compared with caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the CharStream are case‐folded before they are matched, the chars\n in the string argument caseFoldedString are assumed to already be case‐folded (e.g.\n with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member SkipCharsOrNewlinesUntilCaseFoldedString:\n    caseFoldedString: string * maxCount: int * normalizeNewlines: bool\n  * skippedCharsIfStringFoundOtherwiseNull: out<string> -> int\n
\n
\n
\n
\n

\n Behaves like SkipCharsOrNewlinesUntilString(caseFoldedString, maxCount, normalizeNewlines, skippedCharsIfStringFoundOtherwiseNull), except that the chars in the stream are case‐folded before they are compared with caseFoldedChars.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n While the chars in the stream are case‐folded before they are matched, the chars in the string argument caseFoldedString are assumed to already be case‐folded (e.g. with the help of FParsec.Text.FoldCase). Please also see the above remarks on case‐insensitive matching.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.2 CharStream<TUserState>

\n
\n
\n

Provides read‐access to a sequence of UTF‐16 chars.

\n
\n
\n
\n

6.11.2.1 Interface

\n
\n
\n
\n
[<Sealed>]\ntype CharStream<'TUserState> =\n  inherit CharStream\n\n  // has the same constructors as CharStream\n\n  member UserState: 'TUserState with get, set\n\n  member State: CharStreamState<'TUserState>\n\n  member BacktrackTo: CharStreamState<'TUserState> -> unit\n\n  member ReadFrom:\n      stateWhereStringBegins: CharStreamState<'TUserState>\n    * normalizeNewlines: bool\n   -> string\n\n  member CreateSubstream<'TSubStreamUserState>:\n      stateWhereSubstreamBegins: CharStreamState<'TUserState>\n   -> CharStream<'TSubStreamUserState>\n
\n
\n
\n
\n
\n
\n

6.11.2.2 Remarks

\n
\n
\n

\n The CharStream<'TUserState> class adds a user definable state component to its base class CharStream.\n

\n
\n
\n

\n The user state is accessible through the property UserState. It has the type 'TUserState.\n

\n
\n
\n

\n You can retrieve a snapshot of the complete stream state, including the user state, from the State property. The value returned from the State property has the type CharStreamState<'TUserState>. You can pass a CharStreamState value to the BacktrackTo method in order to restore a previous state of the CharStream.\n

\n
\n
\n
\n
Important
\n
\n
\n

\n 'TUserState must be an immutable type or at least be treated as an immutable type if\n you want BacktrackTo to completely\n restore old values of the user state. Hence, when you need to change the user state, you should set a new 'TUserState value to the UserState property of the CharStream instance, not mutate the existing 'TUserState value.\n

\n
\n
\n
\n
\n
\n
\n
\n

6.11.2.3 Members

\n
\n
\n
\n
\n
\n ⇧\n
member UserState: 'TUserState with get, set\n
\n
\n
\n
\n

The current user state value.

\n
\n
\n

\n Setting the UserState value increments the StateTag by 1, independent of whether the new value is different\n from the previous one.\n

\n
\n
\n
\n
\n
\n ⇧\n
member State: CharStreamState<'TUserState>\n
\n
\n
\n
\n

\n Returns a snapshot of the current StateTag,\n Index, Line, LineBegin, Name,\n and UserState values in the form of an\n immutable CharStreamState value.\n

\n
\n
\n
\n
\n
\n ⇧\n
member BacktrackTo: CharStreamState<'TUserState> -> unit\n
\n
\n
\n
\n

\n Restores the stream to the state represented by the given CharStreamState value.\n

\n
\n
\n

For example:

\n
fun (stream: CharStream<'u>) ->\n    let state = stream.State\n    // ... (do something with stream that might change the state)\n    stream.BacktrackTo(state) // restores stream to previous state\n    // ...\n
\n
\n
\n

\n This method throws an ArgumentException if the CharStreamState instance is zero‐initialized (i.e. constructed with the default value type constructor). It\n may also throw any of the I/O related exceptions detailed above.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n You may only pass CharStreamState values that were\n retrieved from the CharStream instance on which you’re\n calling BacktrackTo. Passing a CharStreamState value that was created for another CharStream instance triggers an assert exception in debug builds and will otherwise lead to undefined\n behaviour.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member ReadFrom:\n    stateWhereStringBegins: CharStreamState<'TUserState>\n  * normalizeNewlines: bool\n -> string\n
\n
\n
\n
\n

\n Returns a string with the chars between the index of the stateWhereStringBegins\n (inclusive) and the current Index of the stream\n (exclusive).\n

\n
\n
\n

\n The normalizeNewlines parameter determines whether all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") in the returned string are normalized to '\\n' or whether they are preserved in the original form\n they are encountered in the input. (If stateWhereStringBegins.Line equals the current Line, this method will never normalize any newlines in the returned\n string.)\n

\n
\n
\n

This method trows

\n \n

It may also throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
Note
\n
\n
\n

\n You may only pass CharStreamState values that were\n retrieved from the CharStream instance on which you’re\n calling ReadFrom. Passing a CharStreamState value that was created for another CharStream instance triggers an assert exception in debug builds and will otherwise lead to undefined\n behaviour.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member CreateSubstream<'TSubStreamUserState>:\n    stateWhereSubstreamBegins: CharStreamState<'TUserState>\n -> CharStream<'TSubStreamUserState>\n
\n
\n
\n
\n

\n Creates a new CharStream<'TUserState> instance with the stream chars between the index of the stateWhereSubstreamBegins (inclusive) and the current Index of the stream (exclusive).\n

\n
\n
\n

\n The state of the substream is initialized to stateWhereSubstreamBegin, so that the\n stream and the substream will report the same position (Index, Line, LineBegin and Name) for corresponding chars. However, the beginning and end will\n normally differ between stream and substream, in particular the IndexOfFirstChar and IndexOfLastCharPlus1 values will normally differ between stream and substream.\n

\n
\n
\n

An example:

\n
\n
\n
open FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\nopen FParsec.Error\n\nlet embeddedBlock (beginDelim: string) (endDelim: string) : Parser<_,_> =\n  let expectedEmbeddedBlock = expected \"embedded block\"\n  fun stream ->\n    if stream.Skip(beginDelim) then\n      let stateAtBegin = stream.State\n      let mutable foundString = false\n      let maxChars = System.Int32.MaxValue\n      stream.SkipCharsOrNewlinesUntilString(endDelim, maxChars, &foundString)\n      |> ignore\n      if foundString then\n        // create substream with content between beginDelim and endDelim\n        use substream = stream.CreateSubstream<unit>(stateAtBegin)\n        // here we would normally work with the substream,\n        // in this example we will just extract the string content\n        let str = substream.ReadCharsOrNewlines(System.Int32.MaxValue, true)\n        Reply(str)\n      else\n        Reply(Error, expectedString endDelim)\n    else\n      Reply(Error, expectedEmbeddedBlock)\n
\n
\n
\n
> run (embeddedBlock \"/*\" \"*/\") \"/*substream content*/\";;\nval it : ParserResult<string,unit> = Success: \"substream content\"\n
\n
\n
\n
\n
Note
\n
\n
\n

\n This note does not apply to the Low‐Trust version of FParsec.
\n If you create a substream for a CharStream instance with\n more than one block, the content of the substream needs to be copied. Thus, you can minimize the overhead associated with creating a\n substream by ensuring that the CharStream has only one\n block, either by choosing a sufficiently large blockSize, or by creating the CharStream from a string or char buffer.\n

\n
\n
\n
\n
\n
\n

You may use a stream and its substreams concurrently. However, notice the following warning:

\n
\n
\n
\n
Caution
\n
\n
\n

\n This note does not apply to the Low‐Trust version of FParsec.
\n You may not dispose a stream before all of its substreams are disposed. Disposing a stream before all its substreams are disposed\n triggers an assert exception in debug builds and otherwise lead to undefined behaviour.\n

\n
\n
\n
\n
\n
\n

This method trows

\n \n

It may also throw any of the I/O related exceptions detailed above.

\n
\n
\n
\n
Note
\n
\n
\n

\n You may only pass CharStreamState values that were\n retrieved from the CharStream instance on which you’re\n calling CreateSubstream. Passing a CharStreamState value that was created for another CharStream instance triggers an assert exception in debug builds and will\n otherwise lead to undefined behaviour.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.3 CharStreamIndexToken

\n
\n
\n

An opaque representation of a CharStream char index.

\n
\n
\n
\n
type CharStreamIndexToken = struct\n    member GetIndex: CharStream -> int64\nend\n
\n
\n
\n
\n

\n CharStream methods can handle CharStreamIndexToken values more efficiently than integer char indices.\n

\n
\n
\n

\n You can retrieve CharStreamIndexToken values from the CharStream.IndexToken and CharStreamState<_>.IndexToken properties.\n

\n
\n
\n

\n You can get the char index corresponding to a given CharStreamIndexToken value by calling\n its GetIndex method with the CharStream instance from which the token was retrieved.\n

\n
\n
\n

\n Zero‐initialized CharStreamIndexToken values constructed with the default value type\n constructor are not valid and trying to call a CharStream\n method with such an instance will trigger an exception.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n A CharStreamIndexToken instance may only be used together with the CharSteam instance it was created for.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member GetIndex: CharStream -> int64\n
\n
\n
\n
\n

Returns the stream index represented by the CharStreamIndexToken instance.

\n
\n
\n

\n The CharStream instance passed as the argument must be the\n CharStream instance from which the CharStreamIndexToken was retrieved. Passing a different CharStream instance triggers an assert exception in debug builds and will otherwise lead to undefined\n behaviour.\n

\n
\n
\n

\n An InvalidOperationException is thrown if the CharStreamIndexToken value is zero‐initialized (i.e. constructed with the default value type constructor).\n

\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.4 CharStreamState

\n
\n
\n

\n An immutable value type representation of the state of a CharStream.\n

\n
\n
\n
\n
type CharStreamState<'TUserState> = struct\n    member Tag: int64\n    member IndexToken: CharStreamIndexToken\n    member Line: int64\n    member LineBegin: int64\n    member Name: string\n    member UserState: 'TUserState\n\n    member GetIndex:    CharStream<'TUserState> -> int64\n    member GetPosition: CharStream<'TUserState> -> Position\nend\n
\n
\n
\n
\n

\n You can retrieve CharStreamState values from the CharStream<_>.State property. By passing a CharStreamState value to the BacktrackTo method of a CharStream<_> instance, you can restore the stream to the state\n represented by the CharStreamState value.\n

\n
\n
\n

\n Zero‐initialized CharStreamState values constructed with the default value type constructor\n are not valid and trying to call a CharStream method with\n such an instance will trigger an exception.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n A CharStreamState instance may only be used together with the CharSteam instance it was created for.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
member GetIndex: CharStream<'TUserState> -> int64\n
\n
\n
\n
\n

\n state.GetIndex(stream) is an optimized implementation of state.IndexToken.GetIndex(stream).\n

\n
\n
\n

\n The CharStream<'TUserState> instance passed as the argument must be the CharStream instance from which the CharStreamState was retrieved. Passing a different CharStream instance triggers an assert exception in debug builds and will otherwise lead to undefined\n behaviour.\n

\n
\n
\n

\n An InvalidOperationException is thrown if the CharStreamState instance is zero‐initialized (i.e. constructed with the default value type constructor).\n

\n
\n
\n
\n
\n
\n ⇧\n
member GetPosition: CharStream<'TUserState> -> Position\n
\n
\n
\n
\n

\n state.GetPosition(stream) is an optimized implementation of new Position(state.Name, state.GetIndex(stream), state.Line, state.Column).\n

\n
\n
\n

\n The CharStream<'TUserState> instance passed as the argument must be the CharStream instance from which the CharStreamState was retrieved. Passing a different CharStream instance triggers an assert exception in debug builds and will otherwise lead to undefined\n behaviour.\n

\n
\n
\n

\n An InvalidOperationException is thrown if the CharStreamState instance is zero‐initialized (i.e. constructed with the default value type constructor).\n

\n
\n
\n
\n
\n
\n
\n
\n
\n

6.11.5 TwoChars

\n
\n
\n

An immutable value type representation of two chars:

\n
\n
\n
\n
type TwoChars = struct\n    new: char0: char * char1: char -> TwoChars\n    val Char0: char\n    val Char1: char\nend\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n The detection of invalid byte sequences by the .NET decoders is not entirely reliable. For example, System.Text.UnicodeEncoding (UTF‐16) has an alignment related bug in .NET versions prior to 4.0 that sometimes leads to invalid\n surrogate pairs not being detected. The implementations of more complicated encodings, like GB18030, ISO‐2022 and ISCII, also have several issues with regard to the detection\n of invalid input data.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/error.html", "content": "\n\n\n \n FParsec.Error\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
\n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Members
\n
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.Error\n \n
\n
\n

6.7 FParsec.Error

\n
\n

6.7.1 Interface

\n
\n
\n
\n
// FParsec.dll\n\n[<AutoOpen>] // module is automatically opened when FParsec namespace is opened\nmodule FParsec.Error\n\n// The following type abbreviations and active patterns allow you to\n// treat the ErrorMessage type almost as if it was defined as:\n//\n// [<CustomEquality; NoComparison>]\n// type ErrorMessage =\n//      | Expected           of string\n//      | ExpectedString     of string\n//      | ExpectedStringCI   of string\n//      | Unexpected         of string\n//      | UnexpectedString   of string\n//      | UnexpectedStringCI of string\n//      | Message            of string\n//      | NestedError        of Position * obj * ErrorMessageList\n//      | CompoundError      of string * Position * obj * ErrorMessageList\n//      | OtherErrorMessage  of obj\n\ntype Expected           = ErrorMessage.Expected\ntype ExpectedString     = ErrorMessage.ExpectedString\ntype ExpectedStringCI   = ErrorMessage.ExpectedCaseInsensitiveString\ntype Unexpected         = ErrorMessage.Unexpected\ntype UnexpectedString   = ErrorMessage.UnexpectedString\ntype UnexpectedStringCI = ErrorMessage.UnexpectedCaseInsensitiveString\ntype Message            = ErrorMessage.Message\ntype NestedError        = ErrorMessage.NestedError\ntype CompoundError      = ErrorMessage.CompoundError\ntype OtherErrorMessage  = ErrorMessage.Other\n\n// Unfortunately, F# currently doesn't support active patterns with more\n// than 7 cases, so we have to use partial patterns.\n\nval (|Expected|_|):           ErrorMessage -> string option\nval (|ExpectedString|_|):     ErrorMessage -> string option\nval (|ExpectedStringCI|_|):   ErrorMessage -> string option\nval (|Unexpected|_|):         ErrorMessage -> string option\nval (|UnexpectedString|_|):   ErrorMessage -> string option\nval (|UnexpectedStringCI|_|): ErrorMessage -> string option\nval (|Message|_|):            ErrorMessage -> string option\nval (|NestedError|_|):        ErrorMessage\n                           -> (Position * obj * ErrorMessageList) option\nval (|CompoundError|_|):      ErrorMessage\n                           -> (string * Position * obj * ErrorMessageList) option\nval (|OtherErrorMessage|_|):  ErrorMessage -> obj option\n\n\n// The following literal definition and active pattern allow you to\n// treat the ErrorMessageList type as if it was defined as:\n//\n// [<CompilationRepresentation(CompilationRepresentationFlags.UseNullAsTrueValue);\n//   CustomEquality; NoComparison>]\n// type ErrorMessageList =\n//     | AddErrorMessage of ErrorMessage * ErrorMessageList\n//     | NoErrorMessages\n// with\n//   static member Merge: ErrorMessageList * ErrorMessageList -> ErrorMessageList\n//   static member ToHashSet: ErrorMessageList -> HashSet<ErrorMessage>\n//   static member ToSortedArray: ErrorMessageList -> ErrorMessage[]\n\n[<Literal>]\nval NoErrorMessages: ErrorMessageList = null\n\nval (|ErrorMessageList|NoErrorMessages|):\n    ErrorMessageList -> Choice<ErrorMessage*ErrorMessageList,unit>\n\n\n// Helper functions for creating an ErrorMessageList with a single ErrorMessage\nval expected:                string -> ErrorMessageList\nval expectedStringError:     string -> ErrorMessageList\nval expectedStringCIError:   string -> ErrorMessageList\nval unexpected:              string -> ErrorMessageList\nval unexpectedStringError:   string -> ErrorMessageList\nval unexpectedStringCIError: string -> ErrorMessageList\nval messageError:            string -> ErrorMessageList\nval otherError:              obj    -> ErrorMessageList\nval nestedError:\n              CharStream<_> -> ErrorMessageList -> ErrorMessageList\nval compoundError:\n    string -> CharStream<_> -> ErrorMessageList -> ErrorMessageList\n\n\n// Two convenient helper functions\nval mergeErrors: ErrorMessageList -> ErrorMessageList -> ErrorMessageList\nval isSingleErrorMessageOfType: ErrorMessageType -> ErrorMessageList\n\n\n// A simple container type for holding an ErrorMessageList\n// together with its associated input stream position and user state\n[<Sealed>]\ntype ParserError =\n  new:   position: Position\n       * userState: obj\n       * messages: ErrorMessageList\n      -> ParserError\n\n  member Position:  Position\n  member UserState: obj\n  member Messages:  ErrorMessageList\n\n  override ToString: unit -> string\n  member   ToString: streamWhereErrorOccurred: CharStream -> string\n\n  member WriteTo:\n           textWriter: System.IO.TextWriter\n         * streamWhereErrorOccurred: CharStream\n         * ?tabSize: int\n         * ?columnWidth: int\n         * ?initialIndentation: string * ?indentationIncrement: string\n         -> unit\n\n  member WriteTo:\n           textWriter: System.IO.TextWriter\n         * getStream: (position -> CharStream)\n         * ?tabSize: int\n         * ?columnWidth: int\n         * ?initialIndentation: string * ?indentationIncrement: string\n         -> unit\n\n  member WriteTo:\n           textWriter: System.IO.TextWriter\n         * ?positionPrinter:\n             (System.IO.TextWriter -> Position -> string -> int -> unit)\n         * ?columnWidth: int\n         * ?initialIndentation: string * ?indentationIncrement: string\n         -> unit\n\n  override Equals: obj -> bool\n  override GetHashCode: unit -> int\n
\n
\n
\n
\n
\n
\n

6.7.2 Members

\n
\n
\n
\n
\n
\n ⇧\n
val expected: string -> ErrorMessageList\n
\n
\n
\n
\n

\n expected label creates an ErrorMessageList with a single Expected label message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val expectedStringError: string -> ErrorMessageList\n
\n
\n
\n
\n

\n expectedStringError str creates an ErrorMessageList with a single ExpectedString str message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val expectedStringCIError: string -> ErrorMessageList\n
\n
\n
\n
\n

\n expectedStringCIError str creates an ErrorMessageList with a single ExpectedStringCI str message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val unexpected: string -> ErrorMessageList\n
\n
\n
\n
\n

\n unexpected label creates an ErrorMessageList with a single Unexpected label message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val unexpectedStringError: string -> ErrorMessageList\n
\n
\n
\n
\n

\n unexpectedStringError str creates an ErrorMessageList with a single UnexpectedString str message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val unexpectedStringCIError: string -> ErrorMessageList\n
\n
\n
\n
\n

\n unexpectedStringCIError str creates an ErrorMessageList with a single UnexpectedStringCI str message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val messageError: string -> ErrorMessageList\n
\n
\n
\n
\n

\n messageError msg creates an ErrorMessageList with a single Message msg message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val otherError: obj -> ErrorMessageList\n
\n
\n
\n
\n

\n otherError o creates an ErrorMessageList with a single OtherError o message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val nestedError:\n              CharStream<_> -> ErrorMessageList -> ErrorMessageList\n
\n
\n
\n
\n

\n nestedError stream msgs creates an ErrorMessageList with a single NestedError(stream.Position, stream.UserState, msgs) message, except if msgs is\n already an ErrorMessageList with a single NestedError message, in which case msgs is returned instead.\n

\n
\n
\n
\n
\n
\n ⇧\n
val compoundError:\n    string -> CharStream<_> -> ErrorMessageList -> ErrorMessageList\n
\n
\n
\n
\n

\n compoundError label stream msgs creates an ErrorMessageList with a single CompoundError(label, stream.Position, stream.UserState, msgs) message, except if\n msgs is an ErrorMessageList with a single NestedError(pos2, ustate2, msgs2) message, in which case an\n ErrorMessageList with a single CompoundError(label, pos2, ustate2, msgs2) message is returned instead.\n

\n
\n
\n
\n
\n
\n ⇧\n
val mergeErrors: ErrorMessageList -> ErrorMessageList -> ErrorMessageList\n
\n
\n
\n
\n

\n mergeErrors error1 error2 is an\n abbreviation for ErrorMessageList.Merge(error1, error2).\n

\n
\n
\n
\n
\n
\n ⇧\n
val isSingleErrorMessageOfType: ErrorMessageType -> ErrorMessageList\n
\n
\n
\n
\n

\n isSingleErrorMessageOfType ty msgs\n returns true if and only ifmsgs is\n an ErrorMessageList with a single ErrorMessage with the ErrorMessageType ty.\n

\n
\n
\n
\n
\n
\n ⇧\n
[<Sealed>]\ntype ParserError
\n
\n
\n
\n

\n ParserError is a simple container type for holding an ErrorMessageList together with its associated input stream position and user\n state.\n

\n
\n
\n

The ParserError class has the following members:

\n
\n
\n
\n
\n
\n ⇧\n
new:   position: Position\n     * userState: obj\n     * messages: ErrorMessageList\n    -> ParserError\n
\n
\n
\n
\n

\n Constructs a ParserError from an ErrorMessageList and its associated position.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Position: Position\n
\n
\n
\n
\n

The input stream position of the parser error.

\n
\n
\n
\n
\n
\n ⇧\n
member UserState: obj\n
\n
\n
\n
\n

The user state associated with the parser error.

\n
\n
\n
\n
\n
\n ⇧\n
member Messages: ErrorMessageList\n
\n
\n
\n
\n

The error messages of the parser error.

\n
\n
\n
\n
\n
\n ⇧\n
override ToString: unit -> string\n
\n
\n
\n
\n

Is equivalent to

\n
use sw = new System.IO.StringWriter()\nWriteTo(sw)\nsw.ToString()\n
\n
\n
\n
\n
\n
\n ⇧\n
member ToString: streamWhereErrorOccurred: CharStream -> string\n
\n
\n
\n
\n

Is equivalent to

\n
use sw = new System.IO.StringWriter()\nWriteTo(sw, streamWhereErrorOccurred)\nsw.ToString()\n
\n
\n
\n
\n
\n
\n ⇧\n
member WriteTo:\n         textWriter: System.IO.TextWriter\n       * streamWhereErrorOccurred: CharStream\n       * ?tabSize: int\n       * ?columnWidth: int\n       * ?initialIndentation: string * ?indentationIncrement: string\n       -> unit\n
\n
\n
\n
\n

Is equivalent to

\n
let getStream (pos: Position) =\n    if pos.StreamName = Position.StreamName then streamWhereErrorOccurred\n    else null\n\nWriteTo(textWriter,\n        getStream,\n        ?tabSize = tabSize,\n        ?columWidth = columnWidth,\n        ?initialIndentation = initialIndentation,\n        ?indentationIncrement = indentationIncrement)\n
\n
\n
\n
\n
\n
\n ⇧\n
member WriteTo:\n         textWriter: System.IO.TextWriter\n       * getStream: (position -> CharStream)\n       * ?tabSize: int\n       * ?columnWidth: int\n       * ?initialIndentation: string * ?indentationIncrement: string\n       -> unit\n
\n
\n
\n
\n

\n Writes a string representation of the ParserError to the given TextWriter value.\n

\n
\n
\n

\n For each error getStream is called with the error position. The returned CharStream must be null or contain the content of the CharStream for which the error was generated (at the original indices).\n

\n
\n
\n

\n If getStream returns a non‐null CharStream, the printed error position information is augmented\n with the line of text surrounding the error position, together with a ‘^’‐marker pointing to the exact location of the\n error in the input stream.\n

\n
\n
\n

\n The tabSize parameter (default value: 8) specifies the tab stop distance that this\n method assumes when counting text columns. This parameter only has an effect for error positions where getStream returns a non‐null CharStream.\n

\n
\n
\n

\n The columnWidth parameter (default value: 79) specifies the number of char columns\n that this method should try to fit its output to.\n

\n
\n
\n
\n
\n
\n ⇧\n
member WriteTo:\n         textWriter: System.IO.TextWriter\n       * ?positionPrinter:\n           (System.IO.TextWriter -> Position -> string -> int -> unit)\n       * ?columnWidth: int\n       * ?initialIndentation: string * ?indentationIncrement: string\n       -> unit\n
\n
\n
\n
\n

\n Writes a string representation of the ParserError to the given TextWriter value.\n

\n
\n
\n

\n The format of the position information can be customized by specifying the positionPrinter argument. The given function is expected to print a representation of the passed Position value to the passed TextWriter value. If\n possible, it should indent text lines with the passed string and take into account the maximum column count (including indentation)\n passed as the last argument.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/errormessage.html", "content": "\n\n\n \n FParsec.ErrorMessage\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Remarks
3Members
\n
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.ErrorMessage\n \n
\n
\n

6.8 FParsec.ErrorMessage

\n
\n

6.8.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype ErrorMessageType = Expected                        = 0\n                      | ExpectedString                  = 1\n                      | ExpectedCaseInsensitiveString   = 2\n                      | Unexpected                      = 3\n                      | UnexpectedString                = 4\n                      | UnexpectedCaseInsensitiveString = 5\n                      | Message                         = 6\n                      | NestedError                     = 7\n                      | CompoundError                   = 8\n                      | Other                           = 9\n\ntype ErrorMessage =\n  member Type: ErrorMessageType\n\n  override Equals: obj -> bool\n  override GetHashCode: unit -> int\n  interface System.IEquatable<ErrorMessageList>\n\n// nested types\ntype ErrorMessage.Expected =\n  inherit ErrorMessage\n  new: label: string -> ErrorMessage.Expected\n  member Label: string\n\ntype ErrorMessage.ExpectedString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.ExpectedString\n  member String: string\n\ntype ErrorMessage.ExpectedCaseInsensitiveString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.ExpectedCaseInsensitiveString\n  member String: string\n\ntype ErrorMessage.Unexpected =\n  inherit ErrorMessage\n  new: label: string -> ErrorMessage.Unexpected\n  member Label: string\n\ntype ErrorMessage.UnexpectedString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.UnexpectedString\n  member String: string\n\ntype ErrorMessage.UnexpectedCaseInsensitiveString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.UnexpectedCaseInsensitiveString\n  member String: string\n\ntype ErrorMessage.Message =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.Message\n  member String: string\n\ntype ErrorMessage.NestedError =\n  inherit ErrorMessage\n\n  new:   position: Position * userState: obj * messages: ErrorMessageList\n      -> ErrorMessage.NestedError\n\n  member Position:  Position\n  member UserState: obj\n  member Messages:  ErrorMessageList\n\ntype ErrorMessage.CompoundError =\n  inherit ErrorMessage\n\n  new:   labelOfCompound: string\n       * nestedErrorPosition: Position\n       * nestedErrorUserState: obj\n       * nestedErrorMessages: ErrorMessageList\n      -> ErrorMessage.CompoundError\n\n  member LabelOfCompound: string\n  member NestedErrorPosition: Position\n  member NestedErrorUserState: obj\n  member NestedErrorMessages: ErrorMessageList\n\ntype ErrorMessage.Other =\n  inherit ErrorMessage\n  new: data: obj -> ErrorMessage.Other\n  member Data: obj\n
\n
\n
\n
\n
\n
\n

6.8.2 Remarks

\n
\n
\n

\n ErrorMessage is the abstract base class for FParsec\n error messages. Parser functions return ErrorMessage values within an ErrorMessageList.\n

\n
\n
\n

\n There are several subtypes of ErrorMessage that represent specific kind of error messages. These subtypes are defined as nested classes within\n ErrorMessage.\n

\n
\n
\n

\n The active patterns and type abbreviations in the FParsec.Error module allow you to treat the\n ErrorMessage type almost as if it was defined as an F#\n discriminated union type.\n

\n
\n
\n
\n
\n

6.8.3 Members

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage
\n
\n
\n
\n

ErrorMessage is the abstract base class for FParsec error messages.

\n
\n
\n
type ErrorMessage =\n  member Type: ErrorMessageType\n\n  override Equals: obj -> bool\n  override GetHashCode: unit -> int\n  interface System.IEquatable<ErrorMessageList>\n
\n
\n
\n

Please also see the remarks above.

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.Expected
\n
\n
\n
\n

\n Parsers report this ErrorMessage when the input does\n not match the expected input.\n

\n
\n
\n
type ErrorMessage.Expected =\n  inherit ErrorMessage\n  new: label: string -> ErrorMessage.Expected\n  member Label: string\n
\n
\n
\n

The string label describes the expected input.

\n
\n
\n

\n This error message can be generated with the labeling operator <?>.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.ExpectedString
\n
\n
\n
\n

\n Parsers report this ErrorMessage when the input does\n not match an expected string constant.\n

\n
\n
\n
type ErrorMessage.ExpectedString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.ExpectedString\n  member String: string\n
\n
\n
\n

\n This ErrorMessage is mainly generated by the pstring parser and its variants.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.ExpectedCaseInsensitiveString
\n
\n
\n
\n

\n Parsers report this ErrorMessage when the input does\n not match an expected case‐insensitive string constant.\n

\n
\n
\n
type ErrorMessage.ExpectedCaseInsensitiveString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.ExpectedCaseInsensitiveString\n  member CaseInsensitiveString: string\n
\n
\n
\n

\n This ErrorMessage is mainly generated by the pstringCI parsers and its variants.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.Unexpected
\n
\n
\n
\n

\n Parsers report this ErrorMessage when they encounter\n some unexpected input.\n

\n
\n
\n
type ErrorMessage.Unexpected =\n  inherit ErrorMessage\n  new: label: string -> ErrorMessage.Unexpected\n  member Label: string\n
\n
\n
\n

The string label describes the unexpected input.

\n
\n
\n

\n This ErrorMessage is mainly generated by the notFollowedByL primitive.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.UnexpectedString
\n
\n
\n
\n

\n Parsers report this ErrorMessage when they encounter\n an unexpected string constant.\n

\n
\n
\n
type ErrorMessage.UnexpectedString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.UnexpectedString\n  member String: string\n
\n
\n
\n

\n This ErrorMessage is mainly generated by the notFollowedByString parser.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.UnexpectedCaseInsensitiveString
\n
\n
\n
\n

\n Parsers report this ErrorMessage when they encounter\n an unexpected case‐insensitive string constant.\n

\n
\n
\n
type ErrorMessage.UnexpectedCaseInsensitiveString =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.UnexpectedCaseInsensitiveString\n  member CaseInsensitiveString: string\n
\n
\n
\n

\n This ErrorMessage is mainly generated by the notFollowedByStringCI parser.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.Message
\n
\n
\n
\n

\n Parsers report this ErrorMessage when an the error\n does not fit the other ErrorMessage types.\n

\n
\n
\n
type ErrorMessage.Message =\n  inherit ErrorMessage\n  new: string -> ErrorMessage.Message\n  member String: string\n
\n
\n
\n

\n This error message can be generated with the fail and failFatally primitives.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.NestedError
\n
\n
\n
\n

\n Parsers report this ErrorMessage when they\n backtracked after an error occurred.\n

\n
\n
\n
type ErrorMessage.NestedError =\n  inherit ErrorMessage\n\n  new:   position: Position * userState: obj * messages: ErrorMessageList\n      -> ErrorMessage.NestedError\n\n  member Position:  Position\n  member UserState: obj\n  member Messages:  ErrorMessageList\n
\n
\n
\n

\n The Position property\n describes the stream position where the original error occurred that triggered the backtracking. The UserState property contains the user state value from before the backtracking (upcasted to obj). The Messages property contains the\n error messages of the original error.\n

\n
\n
\n

\n This error message is mainly generated by the attempt, >>? and .>>? primitives.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.CompoundError
\n
\n
\n
\n

\n Parsers report this ErrorMessage when a\n “compound” failed to parse.\n

\n
\n
\n
type ErrorMessage.CompoundError =\n  inherit ErrorMessage\n\n  new:   labelOfCompound: string\n       * nestedErrorPosition: Position\n       * nestedErrorUserState: obj\n       * nestedErrorMessages: ErrorMessageList\n      -> ErrorMessage.CompoundError\n\n  member LabelOfCompound: string\n  member NestedErrorPosition: Position\n  member NestedErrorUserState: obj\n  member NestedErrorMessages: ErrorMessageList\n
\n
\n
\n

\n This error message is mainly generated by the compound‐labelling operator <??>.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ErrorMessage.Other
\n
\n
\n
\n

\n User‐defined parsers can return this ErrorMessage to\n report application‐specific error data.\n

\n
\n
\n
type ErrorMessage.Other =\n  inherit ErrorMessage\n  new: data: obj -> ErrorMessage.Other\n  member Data: obj\n
\n
\n
\n

\n To display OtherError values in error messages, you will have to define your own error\n printer, as ParserError.ToString/WriteTo ignores them.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/errormessagelist.html", "content": "\n\n\n \n FParsec.ErrorMessageList\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Remarks
3Members
\n
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.ErrorMessageList\n \n
\n
\n

6.9 FParsec.ErrorMessageList

\n
\n
\n

Represents a list of error messages.

\n
\n
\n
\n

6.9.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\n[<Sealed; AllowNullLiteral>]\ntype ErrorMessageList =\n  member Head: ErrorMessage\n  member Tail: ErrorMessageList\n\n  new: head: ErrorMessage -> ErrorMessageList\n  new: head: ErrorMessage * tail: ErrorMessageList -> ErrorMessageList\n  new: head: ErrorMessage * tailMessage: ErrorMessage -> ErrorMessageList\n\n  static member Merge: ErrorMessageList * ErrorMessageList -> ErrorMessageList\n  static member ToHashSet: ErrorMessageList -> HashSet<ErrorMessage>\n  static member ToSortedArray: ErrorMessageList -> ErrorMessage[]\n\n  override Equals: obj -> bool\n  override GetHashCode: unit -> int\n  interface System.IEquatable<ErrorMessageList>\n
\n
\n
\n
\n
\n
\n

6.9.2 Remarks

\n
\n
\n

\n The ErrorMessageList represents a list of error messages in which the order of the messages\n carries no meaning and any duplicates and empty messages are ignored. Essentially, an ErrorMessageList is constructed as a singly‐linked list, but used as a set.\n

\n
\n
\n

\n A null value represents an empty ErrorMessageList.\n

\n
\n
\n

\n The ErrorMessage values in an ErrorMessageList are usually all associated with the same input stream position and user state.\n For example, the error messages returned by a parser in a Reply\n value describe an error at the CharStream position\n that is current when the parser returns.\n

\n
\n
\n

\n In order to enforce set semantics in comparison operations, the ErrorMessageList overrides\n the Equals and GetHashCode.\n

\n
\n
\n
\n
\n

6.9.3 Members

\n
\n
\n
\n
\n
\n ⇧\n
member Head: ErrorMessage\n
\n
\n
\n
\n

\n The first ErrorMessage in this\n list. This property is never null.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Tail: ErrorMessageList\n
\n
\n
\n
\n

\n The remaining ErrorMessage values\n in this list after the first ErrorMessage.\n

\n
\n
\n

\n If there are no remaining ErrorMessage values, this property is null.\n

\n
\n
\n
\n
\n
\n ⇧\n
new: head: ErrorMessage -> ErrorMessageList\n
\n
\n
\n
\n

\n Constructs a new ErrorMessageList with a single ErrorMessage value.\n

\n
\n
\n

\n This constructor throws a NullReferenceException if head is null.\n

\n
\n
\n
\n
\n
\n ⇧\n
new: head: ErrorMessage * tail: ErrorMessageList -> ErrorMessageList\n
\n
\n
\n
\n

\n Constructs a new ErrorMessageList with Head set to head and Tail set to tail.\n

\n
\n
\n

\n This constructor throws a NullReferenceException if head is null.\n

\n
\n
\n
\n
\n
\n ⇧\n
new: head: ErrorMessage * tailMessage: ErrorMessage -> ErrorMessageList\n
\n
\n
\n
\n

\n new ErrorMessageList(head, tailmessage) is equivalent to new ErrorMessageList(head, new ErrorMessageList(tailMessage)).\n

\n
\n
\n
\n
\n
\n ⇧\n
static member Merge: ErrorMessageList * ErrorMessageList -> ErrorMessageList\n
\n
\n
\n
\n

\n Creates a new ErrorMessageList that contains the ErrorMessage values from both argument lists.\n

\n
\n
\n

\n The order of the ErrorMessage\n values in the newly created list is an implementation detail that you should not depend on.\n

\n
\n
\n
\n
\n
\n ⇧\n
static member ToHashSet: ErrorMessageList -> HashSet<ErrorMessage>\n
\n
\n
\n
\n

\n Converts the ErrorMessageList to a HashSet<ErrorMessageList>. Duplicate error messages and empty Expected..., Unexpected... and Message messages are filtered out when the list is converted to a\n set.\n

\n
\n
\n
\n
\n
\n ⇧\n
static member ToSortedArray: ErrorMessageList -> ErrorMessage[]\n
\n
\n
\n
\n

\n Converts the ErrorMessageList to a array that is sorted by a total order. Duplicate\n error messages and empty Expected..., Unexpected... and Message\n messages are filtered out when the list is converted to the array.\n

\n
\n
\n

The order of the sorted array is an implementation detail and may change in the future.

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/index.html", "content": "\n\n\n \n Reference\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference\n \n
\n
\n

6 Reference

\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
4FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/operatorprecedenceparser.html", "content": "\n\n\n \n FParsec.OperatorPrecedenceParser\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
4FParsec.OperatorPrecedenceParser
\n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Members
\n
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.OperatorPrecedenceParser\n \n
\n
\n

6.4 FParsec.OperatorPrecedenceParser

\n
\n

6.4.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype Associativity  = None  = 0\n                    | Left  = 1\n                    | Right = 2\n\ntype OperatorType  = Infix   = 0\n                   | Prefix  = 1\n                   | Postfix = 2\n\n\ntype Operator<'TTerm, 'TAfterString, 'TUserState>=\n  member Type: OperatorType\n  member Associativity: Associativity\n  member Precedence: int\n\n  member IsAssociative: bool\n  member IsTernary: bool\n\n  member String: string\n  member TernaryRightString: string // null for non-ternary operators\n\n// the following four types inherit from Operator<_,_,_>\ntype InfixOperator<'TTerm, 'TAfterString, 'TUserState> = // ...\ntype PrefixOperator<'TTerm, 'TAfterString, 'TUserState> = // ...\ntype PostfixOperator<'TTerm, 'TAfterString, 'TUserState> = // ...\ntype TernaryOperator<'TTerm, 'TAfterString, 'TUserState> = // ...\n\n\ntype OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState> =\n  member ExpressionParser: Parser<'TTerm,'TUserState>\n  member TermParser: Parser<'TTerm,'TUserState> with get, set\n\n  member AddOperator: Operator<'TTerm, 'TAfterString, 'TUserState> -> unit\n\n  member RemoveOperator: Operator<'TTerm, 'TAfterString, 'TUserState> -> bool\n  member RemoveInfixOperator: string -> bool\n  member RemovePrefixOperator: string -> bool\n  member RemovePostfixOperator: string -> bool\n  member RemoveTernaryOperator: string * string -> bool\n  member Operators: seq<PrecedenceParserOp<'a,'u>>\n\n  member OperatorConflictErrorFormatter:\n    (   Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n     -> Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n     -> ErrorMessageList)\n    with get, set\n\n  member MissingTernary2ndStringErrorFormatter:\n    (   Position * Position\n      * TernaryOperator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n     -> ErrorMessageList)\n    with get, set\n
\n
\n
\n
\n
\n
\n

6.4.2 Members

\n
\n
\n
\n
\n
\n ⇧\n
type Operator<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The Operator type represents an immutable operator definition for the OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState> (OPP) class.\n

\n
\n
\n
[<ReferenceEquality>]\ntype Operator<'TTerm, 'TAfterString, 'TUserState> =\n  member Type: OperatorType\n  member Associativity: Associativity\n  member Precedence: int\n\n  member IsAssociative: bool\n  member IsTernary: bool\n\n  member String: string\n  member TernaryRightString: string // null for non-ternary operators\n
\n
\n
\n

\n The Operator class is the abstract base class of the InfixOperator, PrefixOperator, PostfixOperator and TernaryOperator classes. With these four concrete classes you can define binary infix (e.g. “1 +\n 1”), unary prefix (e.g. “‒1”), unary postfix (e.g. “1++”) and C‐style ternary operators (e.g. “a ?\n b : c”) for the OperatorPrecedenceParser (OPP) class.\n

\n
\n
\n

\n If you have look at the constructors for the concrete operator classes, you’ll see that operators are constructed from an operator string,\n an “after‐string‐parser”, a precedence level, an associativity value and a mapping function that is applied after the\n expression is parsed.\n

\n
\n
\n

\n Ternary operators are treated as special infix operators and require a string and associated after‐string‐parser parser for each of the two\n operator parts.\n

\n
\n
\n
\n
\n
Associativity and precedence
\n
\n
\n

\n While infix operators can be left‐, right‐ and non‐associative (see the Associativity type), prefix and postfix operators can only be associative (true) or non‐associative (false). See below for details on\n how precedence and associativity influence the operator precedence parser.\n

\n
\n
\n
Textual representation of operators
\n
\n
\n

\n The operator string and the after‐string‐parser determine the textual representation of an operator. Usually, the after‐string‐parser is\n used for parsing the whitespace after an operator string.\n

\n
\n
\n

\n OPP instances have separate “namespaces” for prefix operators on the one hand and infix, postfix or ternary operators on\n the other hand. Hence, you can configure an OPP instance to recognize a prefix operator with the same string as the (first) string of an\n infix, postfix or ternary operator. However, no two prefix operators and no two infix, postfix or ternary operators can have the same\n (first) string. The second string of a ternary operator cannot be used for any other operator at the same time.\n

\n
\n
\n

\n The OPP class parses operator strings greedily. This means, for example, that if you define a prefix operator with the string \"-\" and another prefix operator with the\n string \"--\", then the input -- in a prefix location will always be parsed as a -- operator, never as two successive - operators.\n

\n
\n
\n
How the OPP applies the after‐string‐parser
\n
\n
\n

\n If the OPP encounters the operator string in the input, it will apply the after‐string‐parser directly after the operator string. If the\n after‐string‐parser succeeds, the operator will be accepted. If the after‐string‐parser fails without consuming input (or changing the\n parser state any another way), the OPP will backtrack to before the operator string and will not try to parse any other operator at this\n location. If the after‐string‐parser parser fails after consuming input, the OPP will itself fail with this error.\n

\n
\n
\n

\n This backtracking behaviour can be exploited to conditionally accept an operator depending on the input following the operator string.\n For example, the after‐string‐parser definition in PrefixOperator(\"not\", notFollowedBy letter >>. spaces, 1, true, (* ... *)) will ensure that the \"not\" in \"notAnOperator\" cannot be parsed as an operator.\n

\n
\n
\n
The mapping function argument of the operator constructors
\n
\n
\n

\n When an OPP instance has finished parsing a sub‐expresssion involving an operator, it uses the mapping function supplied as the last\n argument to the operator constructor to map the parsed term(s) to a new term. Usually this mapping function constructs an AST node or\n directly transforms the terminal values.\n

\n
\n
\n

\n The operator classes InfixOperator, PrefixOperator, etc. all support two alternative\n types of mapping functions. The simpler type of mapping function only gets passed the parsed term(s). The other type of mapping function\n also gets passed the result(s) of the after‐string‐parser(s).\n

\n
\n
\n
More uses of the after‐string‐parser
\n
\n
\n

\n The combination of individually configurable after‐string‐parsers and mapping functions make the OPP class quite flexible in addressing\n various practical parsing needs.\n

\n
\n
\n

\n One use of the after‐string‐parser is discussed in the user’s guide section on parsing F# infix operators.\n

\n
\n
\n

\n Another use is demonstrated in the following example. It shows how\n you can use the after‐string‐parser to get hold of the precise text location of the parsed operator (which is often useful for\n diagnostic purposes in your application):\n

\n
\n
\n
open FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nlet opp = new OperatorPrecedenceParser<_,_,_>()\n\nlet ws = spaces\n\ntype Assoc = Associativity\n\nlet adjustPosition offset (pos: Position) =\n    Position(pos.StreamName, pos.Index + int64 offset,\n             pos.Line, pos.Column + int64 offset)\n\n// To simplify infix operator definitions, we define a helper function.\nlet addInfixOperator str prec assoc mapping =\n    let op = InfixOperator(str, getPosition .>> ws, prec, assoc, (),\n                           fun opPos leftTerm rightTerm ->\n                               mapping\n                                   (adjustPosition -str.Length opPos)\n                                   leftTerm rightTerm)\n    opp.AddOperator(op)\n\n// Of course, you can define similar functions for other operator types.\n\n// With the helper function in place, you can define an operator with\n// a mapping function that gets passed the text location of the\n// parsed operator as the first argument.\naddInfixOperator \"+\" 1 Assoc.Left (fun opPos leftTerm rightTerm -> (* ... *))\n
\n
\n
\n
\n
\n
\n
\n

\n
Members of Operator<'TTerm, 'TAfterString, 'TUserState>:\n

\n
\n
\n
\n
\n
\n ⇧\n
member Type: OperatorType\n
\n
\n
\n
\n

\n The operator’s type: Infix, Prefix or Postfix.\n

\n
\n
\n

Ternary operators are treated as special infix operators.

\n
\n
\n
\n
\n
\n ⇧\n
member Associativity: Associativity\n
\n
\n
\n
\n

\n The operator’s associativity: None,\n Left or Right.\n

\n
\n
\n

\n For associative prefix operators this value is Associativity.Right, for associative postfix operators this\n value is Associativity.Left.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Precedence: int\n
\n
\n
\n
\n

\n The operator’s precedence value. The value is always greater than zero. Operators with a numerically higher precedence value take\n precedence over operators with lower precedence values.\n

\n
\n
\n
\n
\n
\n ⇧\n
member IsAssociative: bool\n
\n
\n
\n
\n

\n Is equivalent to Associativity != Associativity.None.\n

\n
\n
\n
\n
\n
\n ⇧\n
member IsTernary: bool\n
\n
\n
\n
\n

\n Indicates whether the operator is a TernaryOperator.\n

\n
\n
\n
\n
\n
\n ⇧\n
member String: string\n
\n
\n
\n
\n

The operator’s string specified during construction.

\n
\n
\n

For ternary operators this property returns the left string.

\n
\n
\n
\n
\n
\n ⇧\n
member TernaryRightString: string
\n
\n
\n
\n

The right string of a TernaryOperator.

\n
\n
\n

For non‐ternary operators this property is null.

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
type InfixOperator<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The InfixOperator<'TTerm, 'TAfterString, 'TUserState> type represents a binary infix operator definition (e.g. the + in 1 + 1) for\n the OperatorPrecedenceParser class.\n

\n
\n
\n
type InfixOperator<'TTerm, 'TAfterString, 'TUserState> =\n  inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * associativity: Associativity\n       * mapping: 'TTerm -> 'TTerm -> 'TTerm\n      -> InfixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * associativity: Associativity\n       * dummy: unit // disambiguates overloads in F#\n       * mapping: 'TAfterString -> 'TTerm -> 'TTerm -> 'TTerm\n      -> InfixOperator<'TTerm, 'TAfterString, 'TUserState>\n
\n
\n
\n

\n The two constructors only differ in the type of the mapping they accept. To help F#’s type inference discern both constructors, the\n second constructor accepts an additional dummy argument.\n

\n
\n
\n

\n Please see the documentation for the Operator base class\n for more information.\n

\n
\n
\n
\n
\n
\n ⇧\n
type PrefixOperator<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The PrefixOperator<'TTerm, 'TAfterString, 'TUserState> type represents a unary prefix operator definition (e.g. the - in -1) for the OperatorPrecedenceParser class.\n

\n
\n
\n
type PrefixOperator<'TTerm, 'TAfterString, 'TUserState> =\n  inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * isAssociative: bool\n       * mapping: 'TTerm -> 'TTerm\n      -> PrefixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * isAssociative: bool\n       * dummy: unit // disambiguates overloads in F#\n       * mapping: 'TAfterString -> 'TTerm -> 'TTerm\n      -> PrefixOperator<'TTerm, 'TAfterString, 'TUserState>\n
\n
\n
\n

\n The two constructors only differ in the type of the mapping they accept. To help F#’s type inference discern both constructors, the\n second constructor accepts an additional dummy argument.\n

\n
\n
\n

\n Please see the documentation for the Operator base class\n for more information.\n

\n
\n
\n
\n
\n
\n ⇧\n
type PostfixOperator<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The PostfixOperator<'TTerm, 'TAfterString, 'TUserState> type represents a unary postfix operator definition (e.g. the ++ in 1++) for the OperatorPrecedenceParser class.\n

\n
\n
\n
type PostfixOperator<'TTerm, 'TAfterString, 'TUserState> =\n  inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * isAssociative: bool\n       * mapping: 'TTerm -> 'TTerm\n      -> PostfixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * isAssociative: bool\n       * dummy: unit // disambiguates overloads in F#\n       * mapping: 'TAfterString -> 'TTerm -> 'TTerm\n      -> PostfixOperator<'TTerm, 'TAfterString, 'TUserState>\n
\n
\n
\n

\n The two constructors only differ in the type of the mapping they accept. To help F#’s type inference discern both constructors, the\n second constructor accepts an additional dummy argument.\n

\n
\n
\n

\n Please see the documentation for the Operator base class\n for more information.\n

\n
\n
\n
\n
\n
\n ⇧\n
type TernaryOperator<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The TernaryOperator<'TTerm, 'TAfterString, 'TUserState> type represents a C‐style ternary operator definition (e.g. the ? : in a ? b : c) for the OperatorPrecedenceParser class.\n

\n
\n
\n
type TernaryOperator<'TTerm, 'TAfterString, 'TUserState> =\n  inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   leftString: string\n       * afterLeftStringParser: Parser<'TAfterString,'TUserState>\n       * rightString: string\n       * afterRightStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * associativity: Associativity\n       * mapping: 'TTerm -> 'TTerm -> 'TTerm -> 'TTerm\n      -> TernaryOperator<'TTerm, 'TAfterString, 'TUserState>\n\n  new:   operatorString: string\n       * afterStringParser: Parser<'TAfterString,'TUserState>\n       * precedence: int\n       * isAssociative: bool\n       * dummy: unit // disambiguates overloads in F#\n       * mapping:   'TAfterString -> 'TAfterString -> 'TTerm -> 'TTerm -> 'TTerm\n                 -> 'TTerm\n      -> TernaryOperator<'TTerm, 'TAfterString, 'TUserState>\n
\n
\n
\n

\n The two constructors only differ in the type of the mapping they accept. To help F#’s type inference discern both constructors, the\n second constructor accepts an additional dummy argument.\n

\n
\n
\n

\n Please see the documentation for the Operator base class\n for more information.\n

\n
\n
\n
\n
\n
\n ⇧\n
type OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState>
\n
\n
\n
\n

\n The OperatorPrecedenceParser class (OPP) represents a dynamically configurable parser\n for parsing expression grammars involving binary infix (e.g. 1 + 1), unary prefix (e.g. -1), unary\n postfix (e.g. 1++) and C‐style ternary operators (e.g. a ? b : c).\n

\n
\n
\n

\n You can configure an OPP instance by adding and removing operator definitions in the form of Operator values. If you add an operator that conflicts with a previous\n operator definition, AddOperator will raise an ArgumentException. The Operators property returns a snapshot of the currently defined set of operators. The RemoveInfixOperator, RemovePrefixOperator, etc. members remove operator definitions based\n only on their text representation. All Remove... members return\n false if no matching operator was previously defined, otherwise true.\n

\n
\n
\n

\n The actual expression parser of the OPP is exposed through the ExpressionParser property. The ExpressionParser value is a constant closure that forwards all work to internal instance methods. This ensures\n that the behaviour of the expression parser always reflects the latest configuration of the OPP instance. You can safely call the ExpressionParser concurrently from multiple threads,\n as long as the configuration of the OPP instance is not changed at the same time.\n

\n
\n
\n

\n Before you can call the ExpressionParser you\n first need to set the TermParser. The\n OPP instance uses the TermParser to parse the terms in\n between the operators. Often the TermParser will not\n just parse terminal values but will also recursively call the ExpressionParser, for example to parse an expression between parentheses. Note that the TermParser also needs to consume any trailing whitespace.\n

\n
\n
\n

This example shows how to define a parser for very simple arithmetic expressions:

\n
open FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nlet ws = spaces\nlet str_ws s = pstring s >>. ws\n\nlet opp = new OperatorPrecedenceParser<float,unit,unit>()\nlet expr = opp.ExpressionParser\nlet term = (pfloat .>> ws) <|> between (str_ws \"(\") (str_ws \")\") expr\nopp.TermParser <- term\n\ntype Assoc = Associativity\n\nopp.AddOperator(InfixOperator(\"+\", ws, 1, Assoc.Left, fun x y -> x + y))\nopp.AddOperator(InfixOperator(\"*\", ws, 2, Assoc.Left, fun x y -> x * y))\n
\n
> run expr \"1 + 2*(3 + 4)\";;\nval it : ParserResult<float,unit> = Success: 15.0\n
\n
\n
\n

\n The following points explain how expressions are parsed depending on\n precedence and associativity of the involved operators:\n

\n
\n
\n
    \n
  • \n Operators with higher precedence bind tighter. For example, if the prefix operator “~” has a lower precedence than the infix\n operator “&” then “~x&y” will be parsed as “~(x&y)”.\n
    • \n
\n
\n
\n
    \n
  • \n Ternary operators are treated as special infix operators. The middle expression (e.g. “expr2” in “expr1 ? expr2 :\n expr3”) is parsed as a “fresh” expression that is not influenced by the precedence of the surrounding operators.\n
    • \n
\n
\n
\n
    \n
  • \n

    Operators with identical precedence are parsed as follows:

    \n
    Here o1,   o2   are two infix operators,\n     pre1, pre2 are two prefix operators,\n     po1,  po2  are two postfix operators\nand all operators have identical precedence.\n\npre1 x o1 y  ==>  (pre1 x) o1 y\nx o1 y po1   ==>  x o1 (y po1)\nx o1 y o2 z  ==>  (x o1 y) o2 z  if o1 and o2 are left-associative\nx o1 y o2 z  ==>  x o1 (y o2 z)  if o1 and o2 are right-associative\npre1 x po1   ==>  (pre1 x) po1   if pre1 or po1  is associative\npre1 pre2 x  ==>  pre1 (pre2 x)  if pre1 or pre2 is associative\nx po1 po2    ==>  (x po1) po2    if po1  or po2  is associative\n
    \n
    • \n
\n
\n
\n
    \n
  • \n
    \n

    \n If the parser encounters conflicting operators, e.g. if a right‐associative\n infix operators follows a left‐associative operator with the same precedence level, the OPP fails and returns with an error generated\n with the help of the OperatorConflictErrorFormatter.\n

    \n
    \n
    \n

    In the following situations the OPP will fail with an operator conflict error:

    \n
    [Same notation as above, all operators have identical precedence.]\n\nx o1 y o2 z  if o1 and o2 have different associativity\n             or o1 and o2 are non-associative\npre1 pre2 x  if pre1 and pre2 are non-associative\npre1 x po1   if pre1 and po1  are non-associative\nx po1 po2    if po1  and po2  are non-associative\n
    \n
    \n
    \n

    \n By giving all operators different precedence levels and making all operators associative, you can exclude any possible operator conflict.\n A practical reason for defining operators that can lead to conflicts in the inputs (e.g. non‐associative operators) is to force the user\n to explicitely parenthesize an expression involving such operators.\n

    \n
    \n
    • \n
\n
\n
\n

\n
Members of OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState>:\n

\n
\n
\n
\n
\n
\n ⇧\n
member ExpressionParser: Parser<'TTerm,'TUserState>\n
\n
\n
\n
\n

\n The expression parser. This is a constant closure that forwards all work to internal instance methods, so that the behaviour of the\n expression parser always reflects the latest configuration of the OPP instance.\n

\n
\n
\n

\n You can safely call the ExpressionParser concurrently from multiple threads, as long\n as the configuration of the OperatorPrecedenceParser instance is not changed at the\n same time.\n

\n
\n
\n
\n
\n
\n ⇧\n
member TermParser: Parser<'TTerm,'TUserState> with get, set\n
\n
\n
\n
\n

\n This parser is called to parse the terms in between the operators. There is no default, so you must set this parser before you can call\n the ExpressionParser. Note that the term\n parser is also expected to parse any whitespace after a term.\n

\n
\n
\n
\n
\n
\n ⇧\n
member AddOperator: Operator<'TTerm, 'TAfterString, 'TUserState> -> unit\n
\n
\n
\n
\n

\n Adds an operator to the grammar. Raises an ArgumentException if\n the operator definition conflicts with a previous definition.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RemoveOperator: Operator<'TTerm, 'TAfterString, 'TUserState> -> bool\n
\n
\n
\n
\n

\n Removes the given Operator instance from the\n grammar. Returns false if the Operator instance was not previously registered, otherwise true.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RemoveInfixOperator: string -> bool\n
\n
\n
\n
\n

\n Removes the InfixOperator with the given string\n from the grammar. Returns false if no infix operator with that string was previously\n registered, otherwise true.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RemovePrefixOperator: string -> bool\n
\n
\n
\n
\n

\n Removes the PrefixOperator with the given\n string from the grammar. Returns false if no prefix operator with that string was\n previously registered, otherwise true.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RemovePostfixOperator: string -> bool\n
\n
\n
\n
\n

\n Removes the PostfixOperator with the given\n string from the grammar. Returns false if no postfix operator with that string was\n previously registered, otherwise true.\n

\n
\n
\n
\n
\n
\n ⇧\n
member RemoveTernaryOperator: string * string -> bool\n
\n
\n
\n
\n

\n Removes the TernaryOperator with the given\n left and right strings from the grammar. Returns false if no ternary operator with\n these strings was previously registered, otherwise true.\n

\n
\n
\n
\n
\n
\n ⇧\n
member Operators: seq<PrecedenceParserOp<'a,'u>>\n
\n
\n
\n
\n

\n Returns a sequence with a snapshot of the operators currently registered with the OperatorPrecedenceParser.\n

\n
\n
\n
\n
\n
\n ⇧\n
member OperatorConflictErrorFormatter:\n  (   Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n   -> Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n   -> ErrorMessageList)\n  with get, set\n
\n
\n
\n
\n

\n The OperatorConflictErrorFormatter function is called by the OPP instance when it\n encounters conflicting operators in the input. The two passed tuples contain the stream\n positions, operator definitions and the after‐string‐parser values for the two conflicting operators. The returned ErrorMessageList will become part of the error\n messages returned by the OPP’s ExpressionParser.\n

\n
\n
\n

\n You can set this formatter to customize the error messages generated when the OPP instance encounters conflicting operators in the\n inputs. Of course, if your operator grammar doesn’t allow for conflicting operators in the input, the OperatorConflictErrorFormatter will never be called and there’s no need to customize it. The user’s guide\n section on parsing F# infix operators contains an example\n with a custom OperatorConflictErrorFormatter.\n

\n
\n
\n
\n
\n
\n ⇧\n
member MissingTernary2ndStringErrorFormatter:\n  (   Position * Position\n    * TernaryOperator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n   -> ErrorMessageList)\n  with get, set\n
\n
\n
\n
\n

\n The MissingTernary2ndStringErrorFormatter function is called by the OPP instance\n when it can’t parse the second operator string of a C‐style ternary operator (e.g. the : in a ? b : c). The passed tuple contains (in order) the position of the first operator string, the position where the\n the second string was expected, the operator definition and the after‐string‐parser value for the left operator part. The returned ErrorMessageList will become part of the error\n messages returned by the OPP’s ExpressionParser.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/parser-overview.html", "content": "\n\n\n \n Parser overview\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > Parser overview\n \n
\n
\n

6.1 Parser overview

\n
\n
\n \n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.1: Parsing single chars\n
ParserDescription
\n pchar c
(variants: skipChar, charReturn)\n 		Parses the char c.
\n anyChar
(variant: skipAnyChar)\n 		Parses any one char.
\n satisfy f
(variants: (skipS|s)atisfy[L])\n 		\n Parses any one char for which the predicate function f returns true.\n
\n anyOf str
(variant: skipAnyOf)\n 		Parses any one char in the string str.
\n noneOf str
(variant: skipNoneOf)\n 		Parses any one char not in the string str.
\n letter
(variants: lower, upper)\n 		\n Parses any one unicode letter char identified by System.Char.IsLetter.\n
\n asciiLetter\n
(variants: asciiLower, asciiUpper)\n 		\n Parses any one char in the range 'a' ‐\n 'z' and 'A' ‐ 'Z'.\n
\n digit
(variants: hex, octal)\n 		\n Parses any one char in the range '0' ‐\n '9'.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.2: Parsing strings\n directly\n
ParserDescription
\n pstring str
(variants: skipString, stringReturn)\n 		Parses the string str.
\n pstringCI str
(variants: skipStringCI, stringCIReturn)\n 		\n Parses any string that case‐insensitively matches the string str.\n
\n anyString n
(variants: skipAnyString)\n 		Parses any sequence of n chars.
\n restOfLine skipNewline
(variant: skipRestOfLine)\n 		\n Parses any chars before the end of the line and, if skipNewline is true, skips to the beginning of the next line (if there is one).\n
\n charsTillString str skipString nMax
(variants:\n charsTillStringCI, skipCharsTillString[CI])\n 		\n Parses all chars before the first occurance of the string str and, if skipString is true, skips over str. Fails if more than nMax chars come\n before str.\n
\n manySatisfy f
(variant: skipManySatisfy)\n 		\n Parses a sequence of zero or more chars that satisfy the predicate function f\n (i.e. chars for which f returns true).\n
\n manySatisfy2 f1 f
(variant: skipManySatisfy2)\n 		\n Parses a sequence of zero or more chars, where the first char must satisfy the predicate function f1 and the remaining chars must satisfy f.\n
\n many1Satisfy f
(variants: (skipM|m)any1Satisfy[2][L])\n 		\n Parses a sequence of one or more chars that satisfy the predicate function f.\n
\n manyMinMaxSatisfy nMin nMax f
(variants: (skipM|m)anyMinMaxSatisfy[2][L])\n 		\n Parses a sequence of nMin or more chars that satisfy the predicate function f, but not more than nMax chars.\n
\n regex pattern\n \n Parses a sequence of one or more chars matched by the .NET regular expression string pattern.\n
\n identifier options\n Parses a Unicode identifier.
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.3: Parsing strings with the\n help of other parsers\n
ParserDescription
\n manyChars cp
(variants: manyChars2)\n 		\n Parses a sequence of zero or more chars with the char parser cp.\n
\n many1Chars cp
(variants: many1Chars2)\n 		\n Parses a sequence of one or more chars with the char parser cp.\n
\n manyCharsTill cp endp
(variants: manyCharsTill[Apply][2])\n 		\n Parses chars with the char parser cp until the parser endp succeeds. Stops after endp.\n
\n manyStrings sp
(variant: many[1]Strings[2])\n 		\n Parses a sequence of zero or more strings with the parser sp. Returns the parsed\n strings in concatenated form.\n
\n stringsSepBy sp sep\n \n Parses a sequence of zero or more occurrences of sp separated by sep. Returns the strings parsed with sp\n and sep in concatenated form.\n
\n skipped p\n \n Applies the parser p. Returns the chars skipped over by p as a string.\n
\n p |> withSkippedString f\n \n Applies the parser p. Returns f str x, where str is the string skipped\n over by p and x is the result\n returned by p.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.4: Parsing numbers\n
ParserDescription
pfloatParses a double‐precision floating‐point number.
\n pint64
(variants: pint(8|16|32))\n 		Parses a 64‐bit signed integer.
\n puint64
(variants: puint(8|16|32))\n 		Parses a 64‐bit unsigned integer.
\n numberLiteral options label\n \n Parses a number literal and returns the result in form of a NumberLiteral value.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.5: Parsing whitespace\n
ParserDescription
\n newline
(variants: skipNewline,\n newlineReturn, unicodeNewline)\n 		\n Parses a newline (\"\\n\", \"\\r\\n\" or \"\\r\"). Returns '\\n'.\n
\n unicodeNewline\n
(variants: skipUnicodeNewline, unicodeNewlineReturn)\n 		\n Parses a Unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\" or \"\\u2029\"). Returns '\\n'.\n
\n spaces
(variant: spaces1)\n 		\n Skips over any sequence of whitespace chars (' ', '\\t' or a newline).\n
\n unicodeSpaces\n
(variant: unicodeSpaces1)\n 		\n Skips over any sequence of Unicode whitespace chars and recognizes (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\" and \"\\u2029\") as newlines.\n
eofOnly succeeds at the end of the input.
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.6: Chaining and piping\n parsers\n
ParserDescription
\n preturn x\n Returns x.
\n p >>% x\n \n Applies the parser p. Returns x.\n
\n p |>> f\n \n Applies the parser p. Returns f x, where x is the result returned by p.\n
\n p1 >>. p2\n \n Applies the parsers p1 and p2 in\n sequence. Returns the result of p2.\n
\n p1 .>> p2\n \n Applies the parsers p1 and p2 in\n sequence. Returns the result of p1.\n
\n p1 .>>. p2\n \n Applies the parsers p1 and p2 in\n sequence. Returns the results in a tuple.\n
\n between pBegin pEnd p\n \n Applies the parsers pBegin, p and\n pEnd in sequence. Returns the result of p.\n
\n pipe2 p1 p2 f
(variants: pipe(3|4|5)\n 		\n Applies the parsers p1 and p2 in\n sequence. Returns f x1 x2, where x1 and x2 are the results returned by p1 and p2.\n
\n p >>= f\n \n First applies the parser p, then applies the function f to the result returned by p and finally applies the parser\n returned by f.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.7: Parsing sequences\n
ParserPEGDescription
\n tuple2 p1 p2
(variants: tuple(3|4|5))\n 		p1 p2\n Applies the parsers p1 and p2 in\n sequence. Returns the results in a tuple.\n
\n parray n p
(variants: skipArray)\n 		\n \n Parses n occurrences of p. Returns\n the results in an array.\n
\n many p
(variant: skipMany)\n 		p*\n Parses zero or more occurrences of p. Returns the results in a list.\n
\n many1 p
(variant: skipMany1)\n 		p+\n Parses one or more occurrences of p. Returns the results in a list.\n
\n sepBy p sep
(variants: sepBy1, skipSepBy[1])\n 		(p (sep p)*)?\n Parses zero or more occurrences of p, separated by sep. Returns the results in a list.\n
\n sepEndBy p sep
(variants: sepEndBy1, skipSepEndBy[1])\n 		(p (sep p)* sep?)?\n Parses zero or more occurrences of p, separated and optionally ended by sep. Returns the results in a list.\n
\n manyTill p endp
(variants: many1Till, skipMany[1]Till)\n 		(!endp p)* endp\n Parses zero or more occurrences of p for as long as endp does not succeed. Stops after endp succeeded. Returns the\n results returned by p in a list.\n
\n chainl1 p op
(variants: chain(l|r)[1])\n 		p (op p)*\n Parses one or more occurrences of p, separated by sep. Returns f_n (... (f_2 (f_1 x_1 x_2) x_3) ...) x_n+1,\n where f_1 to f_n are the functions\n returned by the parser op and x_1 to\n x_n+1 are the values returned by p.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.8: Parsing alternatives and\n recovering from errors\n
ParserDescription
\n p1 <|> p2\n \n Parses p1 or p2. The parser p2 is only tried if p1 fails with a\n non‐fatal error and without changing the parser state. The stream position is part of the parser state, so if p1 fails after consuming input, p2 will not\n be tried.\n
\n choice ps
(variant: choiceL)\n 		\n Is equivalent to p1 <|> p2 <|> ... <|> pn <|> pzero, where p1 … pn are the parsers in the sequence\n ps.\n
\n p <|>% x\n \n Parses p or returns x. Is equivalent\n to p1 <|> preturn x.\n
\n opt p\n
(variant: optional)\n 		\n Parses an optional occurrence of p as an option value. Is equivalent to (p |>> Some) <|>% None\n
\n attempt p\n \n Parses p. If p fails after changing\n the parser state, attempt p will backtrack to the original parser state before reporting a (non‐fatal) error. Thus, attempt p1 <|> p2 will continue to try to\n parse p2 even if p1 fails after\n consuming input.\n
\n p1 >>? p2
(variants: .>>?, .>>.?, >>=?)\n 		\n Behaves like p1 >>. p2, but will backtrack to the beginning if p2 fails with a non‐fatal error and with an unchanged parser state, even if p1 has changed the parser state.\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.9: Conditional parsing and\n looking ahead\n
ParserDescription
\n notEmpty p\n \n Behaves like p, but fails when p\n succeeds without consuming input or changing the parser state in any other way.\n
\n followedBy p
(variant: notFollowedBy)\n 		\n Succeeds without changing the parser state if the parser p succeeds at the current\n position.\n
\n followedByL p label
(variant: notFollowedByL)\n 		\n Behaves like followedBy p, but uses the string label to generate a more descriptive\n error message in case p fails. The string label should describe p.\n
\n notFollowedByEof\n \n Is an optimized version of notFollowedByL eof \"end of input\".\n
\n followedByString str
(variants: (notF|f)ollowedByString[CI])\n 		\n Is an optimized version of followedByL (pstring str) (\"'\" + str + \"'\").\n
\n nextCharSatisfies f
(variants: next2CharsSatisfy, previousCharSatisfies)\n 		\n Is an optimized version of followedBy (satisfy f).\n
\n nextCharSatisfiesNot f
(variants: next2CharsSatisfyNot, previousCharSatisfiesNot)\n 		\n Is an optimized version of notFollowedBy (satisfy f).\n
\n lookAhead p\n Parses p and restores the original parser state afterwards.
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.10: Customizing error\n messages\n
ParserDescription
\n p <?> label\n \n Applies the parser p. If p does not\n change the parser state (usually because p failed), the error messages are replaced with\n expectedError label. The string label should describe p.\n
\n p <??> label\n \n Behaves like p <?> label, but when p fails\n after changing the parser state, a CompoundError message is generated with both the given label\n and the error messages generated by p.\n
\n fail msg\n \n Always fails with a messageError msg. The string msg will be displayed together with other error\n messages generated for the same input position.\n
\n failFatally msg\n \n Always fails with a messageError msg. Returns with a FatalError, so that no error recovery is attempted (except via backtracking constructs).\n
\n
\n
\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n Table 6.1.11: User state handling and\n getting the input stream position\n
ParserDescription
\n getUserState\n Returns the current user state.
\n setUserState u\n Sets the user state to u.
\n updateUserState f\n \n Sets the user state to f u, where u is the current user state.\n
\n userStateSatisfies f\n Succeeds if the current user state satisfies the predicate function f.
\n getPosition\n Returns the current position in the input stream.
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/position.html", "content": "\n\n\n \n FParsec.Position\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.Position\n \n
\n
\n

6.10 FParsec.Position

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype Position =\n  new: streamName: string * index: int64 * line: int64 * column: int64 -> Position\n\n  member StreamName: string\n  member Index: int64\n  member Line: int64\n  member Column: int64\n\n  override ToString: unit -> string\n\n  interface System.IEquatable<Position>\n  interface System.IComparable<Position>\n  interface System.IComparable\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/primitives.html", "content": "\n\n\n \n FParsec.Primitives\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
\n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Members
\n
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.Primitives\n \n
\n
\n

6.2 FParsec.Primitives

\n
\n

6.2.1 Interface

\n
\n
\n
\n
// FParsec.dll\n\n[<AutoOpen>] // module is automatically opened when FParsec namespace is opened\nmodule FParsec.Primitives\n\n[<Literal>] val Ok:         ReplyStatus = ReplyStatus.Ok\n[<Literal>] val Error:      ReplyStatus = ReplyStatus.Error\n[<Literal>] val FatalError: ReplyStatus = ReplyStatus.FatalError\n\n\ntype Parser<'TResult, 'TUserState> = CharStream<'TUserState> -> Reply<'TResult>\n\n\n// Two basic primitives that are only seldomly directly used in user code:\nval preturn: 'a -> Parser<'a,'u>\nval pzero: Parser<'a,'u>\n\n\n// Chaining and piping parsers\n// ==============================\nval (>>=):  Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n\nval (>>%):   Parser<'a,'u> -> 'b -> Parser<'b,'u>\nval (>>.):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\nval (.>>.):  Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\nval between: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'c,'u>\n\nval (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b,'u>\n\nval pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>\n\nval pipe3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n           -> ('a -> 'b -> 'c -> 'd) -> Parser<'d,'u>\n\nval pipe4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n           -> ('a -> 'b -> 'c -> 'd -> 'e) -> Parser<'e,'u>\n\nval pipe5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n           -> Parser<'e,'u>\n           -> ('a -> 'b -> 'c -> 'd -> 'e -> 'f) -> Parser<'f,'u>\n\n\n// Parsing alternatives and recovering from errors\n// ===============================================\nval (<|>):       Parser<'a,'u>  -> Parser<'a,'u> -> Parser<'a,'u>\nval choice:  seq<Parser<'a,'u>> -> Parser<'a,'u>\nval choiceL: seq<Parser<'a,'u>> -> string -> Parser<'a,'u>\n\nval (<|>%):   Parser<'a,'u> -> 'a -> Parser<'a,'u>\nval opt:      Parser<'a,'u> -> Parser<'a option,'u>\nval optional: Parser<'a,'u> -> Parser<unit,'u>\n\nval attempt: Parser<'a,'u> -> Parser<'a,'u>\n\nval (>>=?): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\nval (>>?):  Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\nval (.>>.?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n\n\n// Conditional parsing and looking ahead\n// =====================================\nval notEmpty: Parser<'a,'u> -> Parser<'a,'u>\n\nval followedBy:     Parser<'a,'u> -> Parser<unit,'u>\nval followedByL:    Parser<'a,'u> -> string -> Parser<unit,'u>\nval notFollowedBy:  Parser<'a,'u> -> Parser<unit,'u>\nval notFollowedByL: Parser<'a,'u> -> string -> Parser<unit,'u>\n\nval lookAhead: Parser<'a,'u> -> Parser<'a,'u>\n\n\n// Customizing error messages\n// ==========================\nval (<?>):  Parser<'a,'u> -> string -> Parser<'a,'u>\nval (<??>): Parser<'a,'u> -> string -> Parser<'a,'u>\n\nval fail:        string -> Parser<'a,'u>\nval failFatally: string -> Parser<'a,'u>\n\n\n// Parsing sequences\n// =================\nval tuple2: Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\nval tuple3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n            -> Parser<('a * 'b * 'c),'u>\nval tuple4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n            -> Parser<('a * 'b * 'c * 'd),'u>\nval tuple5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n            -> Parser<'e,'u> -> Parser<('a * 'b * 'c * 'd * 'e),'u>\n\nval parray:    int -> Parser<'a,'u> -> Parser<'a[],'u>\nval skipArray: int -> Parser<'a,'u> -> Parser<unit,'u>\n\nval many:      Parser<'a,'u> -> Parser<'a list,'u>\nval many1:     Parser<'a,'u> -> Parser<'a list,'u>\nval skipMany:  Parser<'a,'u> -> Parser<unit,'u>\nval skipMany1: Parser<'a,'u> -> Parser<unit,'u>\n\nval sepBy:         Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval sepBy1:        Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval skipSepBy:     Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\nval skipSepBy1:    Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n\nval sepEndBy:      Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval sepEndBy1:     Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval skipSepEndBy:  Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\nval skipSepEndBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n\nval manyTill:      Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval many1Till:     Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\nval skipManyTill:  Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\nval skipMany1Till: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n\n[<CompilationRepresentationFlags.Static>]\ntype Inline =\n  static member inline Many: stateFromFirstElement: ('T -> 'State)\n                           * foldState: ('State -> 'T -> 'State)\n                           * resultFromState: ('State -> 'Result)\n                           * elementParser: Parser<'T,'U>\n                           * ?firstElementParser: Parser<'T,'U>\n                           * ?resultForEmptySequence: (unit -> 'Result)\n                          -> Parser<'Result,'U>\n\n  static member inline SepBy: stateFromFirstElement: ('T -> 'State)\n                            * foldState: ('State -> 'Separator -> 'T -> 'State)\n                            * resultFromState: ('State -> 'Result)\n                            * elementParser: Parser<'T,'U>\n                            * separatorParser: Parser<'Separator,'U>\n                            * ?firstElementParser: Parser<'T,'U>\n                            * ?resultForEmptySequence: (unit -> 'Result)\n                            * ?separatorMayEndSequence: bool\n                           -> Parser<'Result,'U>\n\n  static member inline ManyTill: stateFromFirstElement: ('T -> 'State)\n                               * foldState: ('State -> 'T -> 'State)\n                               * resultFromStateAndEnd: ('State -> 'E -> 'Result)\n                               * elementParser: Parser<'T,'U>\n                               * endParser: Parser<'E,'U>\n                               * ?firstElementParser: Parser<'T,'U>\n                               * ?resultForEmptySequence: ('E -> 'Result)\n                              -> Parser<'Result,'U>\n\nval chainl1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u>       -> Parser<'a,'u>\nval chainl:  Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\nval chainr1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u>       -> Parser<'a,'u>\nval chainr:  Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n\n\n// Building parsers using F#'s computation expression syntax\ntype ParserCombinator = // ...\nval parse: ParserCombinator\n\n// Building mutually recursive parser values\nval createParserForwardedToRef: unit -> Parser<'a,'u> * Parser<'a,'u> ref\n
\n
\n
\n
\n
\n
\n

6.2.2 Members

\n
\n
\n
\n
\n
\n ⇧\n
[<Literal>] val Ok: ReplyStatus = ReplyStatus.Ok\n
\n
\n
\n
\n

\n This ReplyStatus value indicates that a\n parser succeeded.\n

\n
\n
\n
\n
\n
\n ⇧\n
[<Literal>] val Error: ReplyStatus = ReplyStatus.Error\n
\n
\n
\n
\n

\n This ReplyStatus value indicates that a\n parser failed.\n

\n
\n
\n
\n
\n
\n ⇧\n
[<Literal>] val FatalError: ReplyStatus = ReplyStatus.FatalError\n
\n
\n
\n
\n

\n This ReplyStatus value indicates that a\n parser failed and no error recovery (except after backtracking) should be tried.\n

\n
\n
\n
\n
\n
\n ⇧\n
type Parser<'TResult, 'TUserState> = CharStream<'TUserState> -> Reply<'TResult>\n
\n
\n
\n
\n

The type of the parser functions supported throughout the FParsec library.

\n
\n
\n
\n
\n
\n ⇧\n
val preturn: 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser preturn x always succeeds with the result x (without changing the parser state).\n

\n
\n
\n

\n preturn x is defined as fun stream -> Reply(x).\n

\n
\n
\n
\n
\n
\n ⇧\n
val pzero: Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser pzero always fails with an empty error message list, i.e. an unspecified\n error.\n

\n
\n
\n

\n pzero x is defined as fun stream -> Reply(Error, NoErrorMessage).\n

\n
\n
\n
\n
\n
\n ⇧\n
val (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p >>= f first applies\n the parser p to the input, then applies the function f to the result returned by p and finally applies the parser\n returned by f to the input.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n Please see the user’s guide chapter Applying parsers in sequence for an\n in‐depth discussion of the behaviour of this and other sequencing combinators.\n

\n
\n
\n
\n
\n
\n

\n The >>= combinator is the conceptual foundation for all combinators that\n consecutively apply multiple parsers to the input. In order to precisely define its behaviour we give an equivalent definition:\n

\n
\n
\n
let (>>=) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) =\n    fun stream ->\n        let reply1 = p stream\n        if reply1.Status = Ok then\n            let p2 = f reply1.Result\n            let stateTag = stream.StateTag\n            let mutable reply2 = p2 stream\n            if stateTag = stream.StateTag then\n                reply2.Error <- mergeErrors reply1.Error reply2.Error\n            reply2\n        else\n            Reply(reply1.Status, reply1.Error)\n
\n
\n
\n
\n
\n
\n ⇧\n
val (>>%): Parser<'a,'u> -> 'b -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p >>% x applies the\n parser p and returns the result x.\n

\n
\n
\n

\n p >>% x is an optimized\n implementation of p >>= fun _ -> preturn x.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p1 >>. p2 applies the\n parsers p1 and p2 in sequence and\n returns the result of p2.\n

\n
\n
\n

\n p1 >>. p2 is an optimized\n implementation of p1 >>= fun _ -> p2.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (.>>): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p1 .>> p2 applies the\n parsers p1 and p2 in sequence and\n returns the result of p1.\n

\n
\n
\n

\n p1 .>> p2 is an optimized\n implementation of p1 >>= fun x -> p2 >>% x.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (.>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n
\n
\n
\n
\n

\n The parser p1 .>>. p2 applies the\n parsers p1 and p2 in sequence and\n returns the results in a tuple.\n

\n
\n
\n

\n p1 .>>. p2 is an optimized\n implementation of\n

\n
p1 >>= fun a ->\np2 >>= fun b -> preturn (a, b)\n
\n
\n
\n
\n
\n
\n ⇧\n
val between: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'c,'u>\n
\n
\n
\n
\n

\n The parser between popen pclose p applies the parsers pOpen, p and pEnd in sequence. It returns the result of p.\n

\n
\n
\n

\n between popen pclose p is an optimized implementation of popen >>. p .>> pclose.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p |>> f applies the\n parser p and returns the result of the function application f x, where x is the result returned by\n p.\n

\n
\n
\n

\n p |>> f is an optimized\n implementation of p >>= fun x -> preturn (f x).\n

\n
\n
\n
\n
\n
\n ⇧\n
val pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>\n
\n
\n
\n
\n

\n The parser pipe2 p1 p2 f applies the parsers p1 and p2 in sequence. It returns the result of the function application f a b, where a\n and b are the results returned by p1\n and p2.\n

\n
\n
\n

\n pipe2 p1 p2 f is\n an optimized implementation of\n

\n
p1 >>= fun a ->\np2 >>= fun b -> preturn (f a b)\n
\n
\n
\n
\n
\n
\n ⇧\n
val pipe3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n           -> ('a -> 'b -> 'c -> 'd) -> Parser<'d,'u>\n
\n
\n
\n
\n

\n The parser pipe3 p1 p2 p3 f applies the parsers p1, p2 and p3 in sequence. It returns the\n result of the function application f a b c, where a, b and c are the results returned by p1, p2 and p3.\n

\n
\n
\n

\n pipe3 p1 p2 p3 f is an optimized implementation of\n

\n
p1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c -> preturn (f a b c)\n
\n
\n
\n
\n
\n
\n ⇧\n
val pipe4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n           -> ('a -> 'b -> 'c -> 'd -> 'e) -> Parser<'e,'u>\n
\n
\n
\n
\n

\n The parser pipe4 p1 p2 p3 p4 f applies the parsers p1, p2, p3\n and p4 in sequence. It returns the result of the function application f a b c d, where a, b, c and d\n are the results returned by p1, p2,\n p3 and p4.\n

\n
\n
\n

\n pipe4 p1 p2 p3 p4 f is an optimized implementation of\n

\n
p1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c ->\np4 >>= fun d -> preturn (f a b c d)\n
\n
\n
\n
\n
\n
\n ⇧\n
val pipe5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n           -> Parser<'e,'u>\n           -> ('a -> 'b -> 'c -> 'd -> 'e -> 'f) -> Parser<'f,'u>\n
\n
\n
\n
\n

\n The parser pipe5 p1 p2 p3 p4 p5 f applies the parsers p1, p2, p3, p4 and p5 in sequence. It returns the result of the function application f a b c d e, where a, b, c, d and\n e are the results returned by p1,\n p2, p3, p4 and p5.\n

\n
\n
\n

\n pipe5 p1 p2 p3 p4 p5 f is an optimized implementation of\n

\n
p1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c ->\np4 >>= fun d ->\np5 >>= fun e -> preturn (f a b c d e)\n
\n
\n
\n
\n
\n
\n ⇧\n
val (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p1 <|> p2 first applies\n the parser p1. If p1 succeeds, the\n result of p1 is returned. If p1\n fails with a non‐fatal error and without changing the parser state, the parser p2 is applied. Note: The stream position is part of the parser state, so if p1 fails after consuming input, p2 will not be applied.\n

\n
\n
\n

\n The choice combinator is a generalization of <|> to more than two parsers.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n Please see the user’s guide chapter on Parsing alternatives for an in‐depth\n discussion of the behaviour of this combinator.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n ⇧\n
val choice: seq<Parser<'a,'u>> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser choice ps is an optimized implementation of p1 <|> p2 <|> ... <|> pn , where p1 … pn are the parsers in the sequence ps.\n

\n
\n
\n

\n choice [p] is\n equivalent to p.
choice [] is equivalent to pzero.\n

\n
\n
\n
\n
\n
\n ⇧\n
val choiceL: seq<Parser<'a,'u>> -> string -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser choiceL ps label is an\n optimized implementation of choice ps <?> label.\n

\n
\n
\n

\n choiceL is slightly faster than choice, because it doesn’t have to aggregate error messages.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (<|>%): Parser<'a,'u> -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p <|>% x is an\n optimized implementation of p <|> preturn x.\n

\n
\n
\n
\n
\n
\n ⇧\n
val opt: Parser<'a,'u> -> Parser<'a option,'u>\n
\n
\n
\n
\n

\n The parser opt p parses an optional occurrence of p as an option value.\n

\n
\n
\n

\n opt p is an optimized implementation of (p |>> Some) <|>% None.\n

\n
\n
\n
\n
\n
\n ⇧\n
val optional: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser optional p skips over an optional occurrence of p.\n

\n
\n
\n

\n optional p is an optimized implementation of (p >>% ()) <|>% ().\n

\n
\n
\n
\n
\n
\n ⇧\n
val attempt: Parser<'a,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser attempt p applies the parser p. If p fails after changing the parser\n state or with a fatal error, attempt p will backtrack to the\n original parser state and report a non‐fatal error.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (>>=?): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p >>=? f behaves like\n p >>= f, except that it will backtrack to the beginning if the parser returned by f fails with a non‐fatal error and without changing the parser state, even if p1 has changed the parser state.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n
\n
\n
\n
\n

\n The parser p1 >>? p2 behaves like\n p1 >>. p2, except that it will backtrack to the beginning if p2 fails\n with a non‐fatal error and without changing the parser state, even if p1 has changed the\n parser state.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (.>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p1 .>>? p2 behaves like\n p1 .>> p2, except that it will backtrack to the beginning if p2 fails\n with a non‐fatal error and without changing the parser state, even if p1 has changed the\n parser state.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (.>>.?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n
\n
\n
\n
\n

\n The parser p1 .>>.? p2 behaves\n like p1 .>>. p2, except that it will backtrack to the beginning if p2 fails\n with a non‐fatal error and without changing the parser state, even if p1 has changed the\n parser state.\n

\n
\n
\n
\n
\n
\n ⇧\n
val notEmpty: Parser<'a,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser notEmpty p behaves like p, except that it fails when p succeeds without consuming input\n or changing the parser state in any other way.\n

\n
\n
\n

\n notEmpty is useful for forcing sequence parsers to consume input. For example, notEmpty (manySatisfy f) behaves like many1Satisfy f.\n

\n
\n
\n
\n
\n
\n ⇧\n
val followedBy: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser followedBy p succeeds if the parser p succeeds at the current position. Otherwise it fails with a non‐fatal error. This parser\n never changes the parser state.\n

\n
\n
\n

\n If the parser followedBy p fails, it returns no descriptive\n error message. Hence it should only be used together with other parsers that take care of a potential error. Alternatively, followedByL p label can be used to ensure a more descriptive error message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val followedByL: Parser<'a,'u> -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser followedByL p behaves like followedBy p, except that it returns an Expected label error\n message when the parser p fails.\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedBy: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser notFollowedBy p succeeds if the parser p fails to parse at the current position. Otherwise it fails with a non‐fatal error. This\n parser never changes the parser state.\n

\n
\n
\n

\n If the parser notFollowedBy p fails, it returns no descriptive\n error message. Hence it should only be used together with other parsers that take care of a potential error. Alternatively, notFollowedByL p label can be used to ensure a more descriptive error message.\n

\n
\n
\n
\n
\n
\n ⇧\n
val notFollowedByL: Parser<'a,'u> -> string -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser notFollowedByL p behaves like notFollowedBy p, except that it returns an Unexpected label error\n message when the parser p fails.\n

\n
\n
\n
\n
\n
\n ⇧\n
val lookAhead: Parser<'a,'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser lookAhead p parses p and restores the original parser state afterwards. If p fails\n after changing the parser state, the error messages are wrapped in a NestedError. If it succeeds, any error messages are discarded. Fatal errors are turned into normal errors.\n

\n
\n
\n
\n
\n
\n ⇧\n
val (<?>): Parser<'a,'u> -> string -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p <?> label applies the\n parser p. If p does not change the\n parser state (usually because p failed), the error messages are replaced with expected label.\n

\n
\n
\n

Please also see the user’s guide chapter on customizing error messages.

\n
\n
\n
\n
\n
\n ⇧\n
val (<??>): Parser<'a,'u> -> string -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser p <??> label behaves\n like p <?> label, except that when p fails after changing the parser state\n (for example, because p consumes input before it fails), a CompoundError message is generated with both the given string\n label and the error messages generated by p.\n

\n
\n
\n

Please also see the user’s guide chapter on customizing error messages.

\n
\n
\n
\n
\n
\n ⇧\n
val fail: string -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser fail msg always fails with a messageError msg. The string msg will be displayed together with other error messages generated for the same input\n position.\n

\n
\n
\n

\n fail msg is equivalent to fun stream -> Reply(Error, messageError msg).\n

\n
\n
\n
\n
\n
\n ⇧\n
val failFatally: string -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser failFatally msg always fails with a messageError msg. It\n returns with a FatalError, so that no error recovery\n is attempted (except via backtracking constructs).\n

\n
\n
\n

\n failFatally msg is equivalent to fun stream -> Reply(FatalError, messageError msg).\n

\n
\n
\n
\n
\n
\n ⇧\n
val tuple2: Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n
\n
\n
\n
\n

\n The parser tuple2 p1 p2 applies the\n parsers p1 and p2 in sequence and\n returns the results in a tuple.\n

\n
\n
\n

\n tuple2 p1 p2 is defined as p1 .>>. p2 and is equivalent to pipe2 p1 p2 (fun a b -> (a, b)).\n

\n
\n
\n
\n
\n
\n ⇧\n
val tuple3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n            -> Parser<('a * 'b * 'c),'u>\n
\n
\n
\n
\n

\n The parser tuple3 p1 p2 p3 applies the parsers p1, p2 and p3 in sequence and returns the results in a tuple.\n

\n
\n
\n

\n tuple3 p1 p2 p3\n is equivalent to pipe3 p1 p2 p3 (fun a b c -> (a, b, c)).\n

\n
\n
\n
\n
\n
\n ⇧\n
val tuple4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n            -> Parser<('a * 'b * 'c * 'd),'u>\n
\n
\n
\n
\n

\n The parser tuple4 p1 p2 p3 p4 applies the parsers p1, p2, p3 and p4 in sequence and returns the results in a tuple.\n

\n
\n
\n

\n tuple4 p1 p2 p3 p4 is equivalent to pipe4 p1 p2 p3 p4 (fun a b c d -> (a, b, c, d)).\n

\n
\n
\n
\n
\n
\n ⇧\n
val tuple5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n            -> Parser<'e,'u> -> Parser<('a * 'b * 'c * 'd * 'e),'u>\n
\n
\n
\n
\n

\n The parser tuple5 p1 p2 p3 p4 p5 applies the parsers p1, p2, p3,\n p4 and p5 in sequence and returns\n the results in a tuple.\n

\n
\n
\n

\n tuple5 p1 p2 p3 p4 p5 is equivalent to pipe5 p1 p2 p3 p4 p5 (fun a b c d e -> (a, b, c, d, e)).\n

\n
\n
\n
\n
\n
\n ⇧\n
val parray: int -> Parser<'a,'u> -> Parser<'a[],'u>\n
\n
\n
\n
\n

\n The parser parray n p parses n occurrences of p and returns the results\n in an array.\n

\n
\n
\n

\n For example, parray 3 p is equivalent to\n pipe3 p p p (fun a b c -> [|a;b;c|]).\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipArray: int -> Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipArray n p is an optimized\n implementation of parray n p |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many: Parser<'a,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser many p repeatedly applies the parser p until p fails. It returns a list of the\n results returned by p. At the end of the sequence p must fail without changing the parser state and without signalling a FatalError, otherwise many p will fail with the error reported by p.\n

\n
\n
\n

\n many p tries to guard against an infinite loop by raising an\n exception if p succeeds without changing the parser state.\n

\n
\n
\n

\n Ignoring efficiency issues, stack space usage and the infinite recursion case, many\n could be defined as follows:\n

\n
let rec many p =\n    (p >>= fun hd ->\n               many p\n               |>> fun tl -> hd::tl)\n    <|>% []\n
\n
\n
\n
\n
\n
\n ⇧\n
val many1: Parser<'a,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser many1 p behaves like many p, except that it requires p to succeed at least one time.\n

\n
\n
\n

\n many1 p is equivalent to pipe2 p (many p) (fun hd tl -> hd::tl).\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipMany p is an optimized implementation of many p |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipMany1 p is an optimized implementation of many1 p |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val sepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser sepBy p sep parses\n zero or more occurrences of p separated by sep (in EBNF: (p (sep p)*)?). It returns a list of the results returned by p.\n

\n
\n
\n

\n sepBy p sep is almost equivalent to\n pipe2 p (many (sep >>. p)) (fun hd tl -> hd::tl) <|>% [], except with regard to a case\n rarely encountered in practice: If sep succeeds without changing the parser state and\n p then fails without changing the state, then sepBy p sep fails too, while the parser given by the almost\n equivalent definition would succeed.\n

\n
\n
\n
\n
\n
\n ⇧\n
val sepBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser sepBy1 p sep parses\n one or more occurrences of p separated by sep (in EBNF: p (sep p)*).\n

\n
\n
\n

\n The parser sepBy1 p behaves like sepBy p, except that it requires p to succeed at least one time. Hence, if sepBy1 succeeds, the\n returned list always contains at least one value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipSepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipSepBy p sep is an\n optimized implementation of sepBy p sep |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipSepBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipSepBy1 p sep is an\n optimized implementation of sepBy1 p sep |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val sepEndBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser sepEndBy p sep parses\n zero or more occurrences of p separated and optionally ended by sep (in EBNF: (p (sep p)* sep?)?). It returns a list of the results\n returned by p.\n

\n
\n
\n

\n sepEndBy p sep tries to guard against an\n infinite loop by raising an exception if p and sep succeed without changing the parser state.\n

\n
\n
\n

\n Ignoring efficiency issues, stack space usage and the infinite recursion case, sepEndBy\n could be defined as follows:\n

\n
let rec sepEndBy p sep =\n    (p >>= fun hd ->\n               sep >>. sepEndBy p sep <|>% []\n               |>> fun tl -> hd::tl)\n    <|>% []\n
\n
\n
\n
\n
\n
\n ⇧\n
val sepEndBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser sepEndBy1 p sep parses\n one or more occurrences of p separated and optionally ended by sep (in EBNF: p (sep p)* sep?). It returns a list of the results\n returned by p.\n

\n
\n
\n

\n The parser sepEndBy1 p behaves like sepEndBy p, except that it requires p to succeed at least one time. Hence, if sepEndBy1 succeeds, the returned list always contains at least one value.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipSepEndBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipSepEndBy p sep is an\n optimized implementation of sepEndBy p sep |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipSepEndBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipSepEndBy1 p sep is an\n optimized implementation of sepEndBy1 p sep |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val manyTill: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser manyTill p endp repeatedly\n applies the parser p for as long as endp fails (without changing the parser state). It returns a list of the results returned by p.\n

\n
\n
\n

\n manyTill p endp is an optimized variant\n of many (notFollowedBy endp >>. p) .>> endp that doesn’t have to apply endp twice at the end of the sequence and that fails with the error reported by endp if endp fails after changing the parser state.\n

\n
\n
\n
\n
\n
\n ⇧\n
val many1Till: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n
\n
\n
\n
\n

\n The parser many1Till p endp behaves like\n manyTill p endp, except that it requires p to succeed at least one time.\n

\n
\n
\n

\n many1Till p endp is an optimized\n implementation of pipe2 p (manyTill p endp) (fun hd tl -> hd::tl).\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipManyTill: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipManyTill p endp is an\n optimized implementation of manyTill p endp |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
val skipMany1Till: Parser<'a,'u> -> Parser<'b,'u> -> Parser<unit,'u>\n
\n
\n
\n
\n

\n The parser skipMany1Till p endp is an\n optimized implementation of many1Till p endp |>> ignore.\n

\n
\n
\n
\n
\n
\n ⇧\n
[<CompilationRepresentationFlags.Static>]\ntype Inline =\n
\n
\n
\n
\n

\n Inline is a static class that contains the following inline helper methods for defining\n optimized sequence parsers:\n

\n \n
\n
\n
\n
\n
\n ⇧\n
static member inline Many: stateFromFirstElement: ('T -> 'State)\n                         * foldState: ('State -> 'T -> 'State)\n                         * resultFromState: ('State -> 'Result)\n                         * elementParser: Parser<'T,'U>\n                         * ?firstElementParser: Parser<'T,'U>\n                         * ?resultForEmptySequence: (unit -> 'Result)\n                        -> Parser<'Result,'U>\n
\n
\n
\n
\n

\n Inline.Many is an inline helper method\n for defining optimized sequence parsers.\n

\n
\n
\n

\n Inline.Many(stateFromFirstElement, foldState, resultFromState, elementParser) expands to\n an optimized implementation of\n

\n
many1 elementParser // requires at least 1 element\n|>> function hd::tl ->\n    resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n
\n
\n
\n
\n
Note
\n
\n
\n

\n The 'State argument to the foldState function is completely independent of FParsec’s usual parser state. The term\n “accumulator” would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.\n

\n
\n
\n
\n
\n
\n

\n If you pass a value for the optional argument resultForEmptySequence,\n the parser expands to an optimized implementation of\n

\n
many elementParser // accepts empty sequence\n|>> function\n    | [] -> resultForEmptySequence()\n    | hd::tl -> resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n
\n
\n
\n

\n If you pass a value for the optional argument firstElementParser, the\n first element of the sequence will be parsed with firstElementParser instead of elementParser.\n

\n
\n
\n

\n The following example shows how you can use Inline.Many to define an optimized parser that behaves like many1 p |>> List.reduce f but avoids the temporary\n allocation of a list:\n

\n
\n
\n
let many1Reduce p f =\n    Inline.Many(elementParser = p,\n                stateFromFirstElement = (fun x0 -> x0),\n                foldState = (fun acc y -> f acc y),\n                resultFromState = (fun acc -> acc))\n
\n
\n
\n

A simple test run:

\n
> run (many1Reduce (pint32 .>> spaces) (+)) \"1 2 3\";;\nval it : ParserResult<int32,unit> = Success: 6\n
\n
\n
\n

\n The following example shows how you can use Inline.Many to create an optimized sequence parser that returns an array instead of a list:\n

\n
\n
\n
let manyA2 p1 p =\n    Inline.Many(firstElementParser = p1,\n                elementParser = p,\n                stateFromFirstElement = (fun x0 ->\n                                             let ra = ResizeArray<_>()\n                                             ra.Add(x0)\n                                             ra),\n                foldState = (fun ra x -> ra.Add(x); ra),\n                resultFromState = (fun ra -> ra.ToArray()),\n                resultForEmptySequence = (fun () -> [||]))\n\nlet manyA p = manyA2 p p\n
\n
\n
\n

Two simple test runs:

\n
> run (manyA (pint32 .>> spaces)) \"\";;\nval it : ParserResult<int32 [],unit> = Success: [||]\n> run (manyA (pint32 .>> spaces)) \"1 2 3\";;\nval it : ParserResult<int32 [],unit> = Success: [|1; 2; 3|]\n
\n
\n
\n
\n
\n
\n ⇧\n
static member inline SepBy: stateFromFirstElement: ('T -> 'State)\n                          * foldState: ('State -> 'Separator -> 'T -> 'State)\n                          * resultFromState: ('State -> 'Result)\n                          * elementParser: Parser<'T,'U>\n                          * separatorParser: Parser<'Separator,'U>\n                          * ?firstElementParser: Parser<'T,'U>\n                          * ?resultForEmptySequence: (unit -> 'Result)\n                          * ?separatorMayEndSequence: bool\n                         -> Parser<'Result,'U>\n
\n
\n
\n
\n

\n Inline.SepBy is an inline helper method\n for defining optimized sequence parsers. By default, parsers defined with Inline.SepBy parse sequences of the form (in EBNF): element (separator element)*\n

\n
\n
\n

\n Inline.SepBy(stateFromFirstElement, foldState, resultFromState, elementParser, separatorParser) expands to an optimized implementation of\n

\n
pipe2 elementParser (many (separatorParser .>>. elementParser))\n      (fun elem0 sepsAndElems ->\n          sepsAndElems\n          |> List.fold (fun acc (sep, e) -> foldState acc sep e)\n                       (stateFromFirstElement elem0)\n          |> resultFromState)\n
\n
\n
\n

\n For most practical purposes the behaviour of the expanded Inline.SepBy parser and the above definition based on many can be considered equivalent, but there is a fringe case where the behaviour\n differs: If separatorParser succeeds without changing the parser state and elementParser then fails without changing the parser state, then the Inline.SepBy parser fails too, while the\n parser given by the definition based on many would\n succeed.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The 'State argument to the foldState function is completely independent of FParsec’s usual parser state. The term\n “accumulator” would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.\n

\n
\n
\n
\n
\n
\n

\n If you pass true as the value for the optional argument separatorMayEndSequence, a separator may also end the sequence, i.e. the parser will accept sequences of the\n following form (in EBNF):\n

\n
element (separator element)* separator?
\n

Note that foldState is not called with the value of an ending separator.

\n
\n
\n

\n If you pass a value for the optional argument resultForEmptySequence, the parser\n returned by Inline.SepBy will call resultForEmptySequence to create the parser result when it encounters an empty sequence. If\n you don’t pass a resultForEmptySequence function, the parser will fail for an empty\n sequence.\n

\n
\n
\n

\n If you pass a value for the optional argument firstElementParser, the first element of a\n sequence will be parsed with firstElementParser instead of elementParser.\n

\n
\n
\n

\n The following example shows how you can use Inline.SepBy to define an optimized parser that behaves like sepBy1 p sep |>> List.reduce f but avoids the temporary allocation of a list:\n

\n
\n
\n
let sepBy1Reduce p sep f =\n    Inline.SepBy(elementParser = p,\n                 separatorParser = sep,\n                 stateFromFirstElement = (fun x0 -> x0),\n                 foldState = (fun acc _ y -> f acc y),\n                 resultFromState = (fun acc -> acc))\n
\n
\n
\n

A simple test run:

\n
> run (sepBy1Reduce pint32 (pstring \",\" >>. spaces) (+)) \"1, 2, 3\";;\nval it : ParserResult<int32,unit> = Success: 6\n
\n
\n
\n

\n The following example shows how one could define CharParsers.stringsSepBy using Inline.SepBy:\n

\n
\n
\n
let stringsSepBy p sep =\n    Inline.SepBy(elementParser = p,\n                 separatorParser = sep,\n                 stateFromFirstElement =\n                     (fun str -> let sb = System.Text.StringBuilder()\n                                 sb.Append(str : string)),\n                                 // sb.Append returns sb\n                 foldState =\n                     (fun sb sep str -> sb.Append(sep : string)\n                                          .Append(str : string)),\n                 resultFromState = (fun sb -> sb.ToString()))\n\nlet testParser : Parser<string,unit> =\n    stringsSepBy (manySatisfy isLetter) (pstring @\"\\\\\" >>% @\"\\\")\n
\n
\n
\n
> run testParser \"\";;\nval it : ParserResult<string,unit> = Success: \"\"\n> run testParser @\"abc\\\\def\\\\\\\\\";;\nval it : ParserResult<string,unit> = Success: \"abc\\def\\\\\"\n
\n
\n
\n
\n
\n
\n ⇧\n
static member inline ManyTill: stateFromFirstElement: ('T -> 'State)\n                             * foldState: ('State -> 'T -> 'State)\n                             * resultFromStateAndEnd: ('State -> 'E -> 'Result)\n                             * elementParser: Parser<'T,'U>\n                             * endParser: Parser<'E,'U>\n                             * ?firstElementParser: Parser<'T,'U>\n                             * ?resultForEmptySequence: ('E -> 'Result)\n                            -> Parser<'Result,'U>\n
\n
\n
\n
\n

\n Inline.ManyTill is an inline helper\n method for defining optimized sequence parsers.\n

\n
\n
\n

\n Inline.ManyTill(stateFromFirstElement, foldState, resultFromState, elementParser, endParser) expands to an optimized implementation of\n

\n
many1Till elementParser endParser // requires at least 1 element\n|>> function hd::tl ->\n    resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n
\n
\n
\n
\n
Note
\n
\n
\n

\n The 'State argument to the foldState function is completely independent of FParsec’s usual parser state. The term\n “accumulator” would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.\n

\n
\n
\n
\n
\n
\n

\n If you pass a value for the optional argument resultForEmptySequence,\n the parser expands to an optimized implementation of\n

\n
manyTill elementParser endParser // accepts empty sequence\n|>> function\n    | [] -> resultForEmptySequence()\n    | hd::tl -> resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n
\n
\n
\n

\n If you pass a value for the optional argument firstElementParser, the\n first element of the sequence will be parsed with firstElementParser instead of elementParser.\n

\n
\n
\n

\n The following example shows how one could define CharParsers.manyCharsTill2 using Inline.ManyTill:\n

\n
\n
\n
let myManyCharsTillApply2 cp1 cp endp f =\n    Inline.ManyTill(firstElementParser = cp1,\n                    elementParser = cp,\n                    endParser = endp,\n                    stateFromFirstElement =\n                        (fun c -> let sb = System.Text.StringBuilder()\n                                  sb.Append(c : char)),\n                                  // sb.Append returns sb\n                    foldState = (fun sb c -> sb.Append(c : char)),\n                    resultFromStateAndEnd = (fun sb e -> f (sb.ToString()) e),\n                    resultForEmptySequence = (fun e -> f \"\" e))\n\nlet myManyCharsTillApply cp endp f = myManyCharsTillApply2 cp cp endp f\n\nlet myRestOfLine : Parser<string,unit> =\n    myManyCharsTillApply anyChar ((newline >>% \"\\\\n\") <|> (eof >>% \"\"))\n                         (fun str nl -> str + nl)\n
\n
\n
\n
> run myRestOfLine \"\";;\nval it : ParserResult<string,unit> = Success: \"\"\n> run myRestOfLine \"abc\\r\\ndef\";;\nval it : ParserResult<string,unit> = Success: \"abc\\n\"\n
\n
\n
\n
\n
\n
\n ⇧\n
val chainl1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser chainl1 p op parses one or\n more occurrences of p separated by op (in EBNF: p (op p)*). It returns the value obtained by\n left associative application of all functions returned by op to the results\n returned by p, i.e. f_n (...(f_2 (f_1 x_1 x_2) x_3) ...) x_n+1, where f_1 to f_n are the functions returned by theparser op and x_1 to x_n+1 are the values returned by p. If only a single occurance of\n p and no occurance of op is parsed,\n the result of p is returned directly.\n

\n
\n
\n

The chainl1 implementation uses constant stack space.

\n
\n
\n
\n
\n
\n ⇧\n
val chainl: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser chainl p op defVal is equivalent to chainl1 p op <|>% defVal.\n

\n
\n
\n
\n
\n
\n ⇧\n
val chainr1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser chainr1 p op parses one or\n more occurrences of p separated by op (in EBNF: p (op p)*). It returns the value obtained by\n right associative application of all functions returned by op to the results\n returned by p, i.e. f1 x_1 (f_2 x_2 (... (f_n x_n x_n+1) ...)), where f_1 to f_n are the functions returned by the parser op and x_1 to x_n+1 are the values returned by p. If only a single occurance of\n p and no occurance of op is parsed,\n the result of p is returned directly.\n

\n
\n
\n

The chainr1 implementation uses constant stack space.

\n
\n
\n
\n
\n
\n ⇧\n
val chainr: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n
\n
\n
\n
\n

\n The parser chainr p op defVal is equivalent to chainr1 p op <|>% defVal.\n

\n
\n
\n
\n
\n
\n ⇧\n
type ParserCombinator =\n
\n
\n
\n
\n

This class is defined as

\n
\n
\n
[<Sealed>]\ntype ParserCombinator() =\n  member t.Delay(f)   = fun state -> (f ()) state\n  member t.Return(x)  = preturn x\n  member t.Bind(p, f) = p >>= f\n  member t.Zero()     = pzero\n  member t.ReturnFrom(p) = p\n  member t.TryWith(p, cf) = fun state -> try p state\n                                         with e -> (cf e) state\n  member t.TryFinally(p, ff) = fun state -> try p state\n                                            finally ff ()\n
\n
\n
\n

\n Instances of this class can be used to build parsers using F#’s computation\n expression syntax. The default instance for this purpose is parse.\n

\n
\n
\n

\n Please see the user’s guide chapter “Where is the monad?” for an\n introduction to the parse {...} syntax.\n

\n
\n
\n

\n Some constructs supported by parse and their translations\n are\n

\n
let! pat = expr in pexpr   ==>   expr >>= (fun pat -> pexpr)\n\nlet pat = expr in pexpr    ==>   let pat = expr in pexpr\n\ndo! expr in pexpr          ==>   expr >>= (fun  () -> pexpr)\n\ndo expr in pexpr           ==>   expr; pexpr\n\nif expr then pexpr1        ==>   if expr then pexpr1\nelse pexpr2                      else pexpr2\n\nif expr then pexpr         ==>   if expr then pexpr1 else pzero\n\nreturn exp                 ==>   preturn rexpr\n\nreturn! expr               ==>   expr\n
\n

\n where expr is any F# expression and pexpr is an expression of type Parser<_,_>. You need to use the !‐constructs whenever you have a\n right hand side expression that evaluates to a parser.\n

\n
\n
\n
\n
\n
\n ⇧\n
val parse: ParserCombinator\n
\n
\n
\n
\n

\n A builder object of type ParserCombinator for\n building parsers using F#’s computation expression syntax.\n

\n
\n
\n
\n
\n
\n ⇧\n
val createParserForwardedToRef: unit -> Parser<'a,'u> * Parser<'a,'u> ref\n
\n
\n
\n
\n

\n let p, pRef = createParserForwardedToRef() creates a parser p that forwards all calls to the parser in the reference cell pRef. Initially, pRef holds a reference to a dummy parser that\n raises an exception on any invocation.\n

\n
\n
\n

\n The JSON parser example in the tutorial shows how you can use\n createParserForwardedToRef to define a parser for a recursive grammar.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/reply.html", "content": "\n\n\n \n FParsec.Reply\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Remarks
3Members
\n
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.Reply\n \n
\n
\n

6.6 FParsec.Reply

\n
\n
\n

\n Represents the return value of a Parser\n function.\n

\n
\n
\n
\n

6.6.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype ReplyStatus = Ok         =  1\n                 | Error      =  0\n                 | FatalError = -1\n\n\ntype Reply<'TResult> = struct\n  new: 'TResult -> Reply<'TResult>\n  new: ReplyStatus * ErrorMessageList -> Reply<'TResult>\n  new: ReplyStatus * 'TResult * ErrorMessageList -> Reply<'TResult>\n\n  val mutable Status: ReplyStatus\n  /// If Status <> Ok then the Result value is undefined and may be null.\n  val mutable Result: 'TResult\n  val mutable Error: ErrorMessageList\n\n  override Equals: obj -> bool\n  override GetHashCode: unit -> int\n  interface System.IEquatable<Reply<'TResult>>\nend\n
\n
\n
\n
\n
\n
\n

6.6.2 Remarks

\n
\n
\n

\n The Reply type is the return type of Parser functions. Similar to a tuple, a Reply value can be viewed as a simple aggregate of its three fields Status, Result and Error.\n

\n
\n
\n

\n The value of the Status field indicates whether the parser\n returning the reply succeeded (ReplyStatus.Ok) or failed (ReplyStatus.Error or ReplyStatus.FatalError). If the value of the Status field is Ok,\n the Result field contains a parser result value; otherwise,\n its value is undefined.\n

\n
\n
\n

\n The Equals override ignores the Result value when it compares two Reply values with Status <> Ok.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The Reply fields are mutable because that allows us to implement library primitives with\n more compact code, for which the .NET JIT produces faster machine code.\n

\n
\n
\n

\n Of course, if you object to mutable structs on religious\n grounds or if you’re not familiar with the somewhat subtle behaviour of mutable structs in certain sitations, you can always treat the\n Reply type as if it was immutable.\n

\n
\n
\n
\n
\n
\n
\n
\n

6.6.3 Members

\n
\n
\n
\n
\n
\n ⇧\n
new: 'TResult -> Reply<'TResult>\n
\n
\n
\n
\n

\n Constructs a Reply instance with the Status field set to Ok, the Result field set to the argument value and the Error field set to null.\n

\n
\n
\n
\n
\n
\n ⇧\n
new: ReplyStatus * ErrorMessageList -> Reply<'TResult>\n
\n
\n
\n
\n

\n Constructs a Reply instance with the Status and Error fields set to the respective argument values and the Result field initialized to Unchecked.defaultof<'TResult>.\n

\n
\n
\n

\n This constructor is usually used for constructing an error reply, like in Reply(Error, expected \"something\").\n

\n
\n
\n
\n
\n
\n ⇧\n
new: ReplyStatus * 'TResult * ErrorMessageList -> Reply<'TResult>\n
\n
\n
\n
\n

\n Constructs a Reply instance with the Status, Result and Error fields set to\n the respective argument values.\n

\n
\n
\n
\n
\n
\n ⇧\n
val mutable Status: ReplyStatus\n
\n
\n
\n
\n

\n The Status field contains a ReplyStatus enum value indicating whether a parser succeeded (Ok) or failed (Error or FatalError). By returning a FatalError instead of an Error a parser can signal that no error recovery should be tried\n (except through backtracking mechanisms).\n

\n
\n
\n
\n
\n
\n ⇧\n
val mutable Result: 'TResult\n
\n
\n
\n
\n

\n If the value of the Status field is Ok, the Result field contains a parser result value; otherwise, its value is undefined and may be equal to Unchecked.defaultof<'TResult>. (The result value in a Reply returned by an unsuccessful parser is generally an implementation detail of the parser that you should not\n depend on.)\n

\n
\n
\n
\n
\n
\n ⇧\n
val mutable Error: ErrorMessageList\n
\n
\n
\n
\n

\n The Error field holds a list of error messages in the form of an ErrorMessageList value. An empty ErrorMessageList is represented as a null value.\n

\n
\n
\n

\n The error messages returned by a parser in a Reply value implicitly refer to the state\n of the CharStream as it is when the parser\n returns. Since the ErrorMessage\n values stored in the ErrorMessageList do not\n themselves contain an error position, they can only be interpreted together with the position of the CharStream as it is when the parser returns.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/staticmapping.html", "content": "\n\n\n \n FParsec.StaticMapping\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Remarks
3Members
\n
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.StaticMapping\n \n
\n
\n

6.5 FParsec.StaticMapping

\n
\n
\n

This module defines functions for creating optimized mapping functions between keys and values.

\n
\n
\n

This module is not available in the Low‐Trust version of FParsec.

\n
\n
\n
\n

6.5.1 Interface

\n
\n
\n
\n
// FParsec.dll\n\nnamespace FParsec\n\ntype Range = struct\n  new: min: int * max: int -> Range\n\n  val Min: int\n  val Max: int\nend\n\nmodule StaticMapping =\n\n  val createStaticCharIndicatorFunction:\n          invert: bool -> charsInSet: seq<char>   -> (char -> bool)\n\n  val createStaticCharRangeIndicatorFunction:\n          invert: bool -> rangesInSet: seq<Range> -> (char -> bool)\n\n  val createStaticIntIndicatorFunction:\n          invert: bool -> valuesInSet: seq<int>   -> (int -> bool)\n\n  val createStaticIntRangeIndicatorFunction:\n          invert: bool -> rangesInSet: seq<Range> -> (int -> bool)\n\n\n  val createStaticIntMapping:\n          defaultValue: 'T -> keyValues: seq<int*'T>   -> (int -> 'T)\n\n  val createStaticIntRangeMapping:\n          defaultValue: 'T -> keyValues: seq<Range*'T> -> (int -> 'T)\n\n\n  val createStaticStringMapping:\n          defaultValue: 'T -> keyValues: seq<string*'T> -> (string -> 'T)\n\n
\n
\n
\n
\n
\n
\n

6.5.2 Remarks

\n
\n
\n

\n The functions in the StaticMapping module use runtime code generation via System.Reflection.Emit to create optimized mapping functions between keys and values.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n Runtime code generation is relatively expensive, so the functions in this module should only be used for optimizing static mappings that are\n potentially called a (very) large number of times.\n

\n
\n
\n
\n
\n
\n
\n
Note
\n
\n
\n

The code generated for the optimized mapping functions will occupy memory until the associated AppDomain is unloaded.

\n
\n
\n
\n
\n
\n

\n The performance of the generated functions depends a lot on the individual key‐value mapping and the application‐specific call pattern. Ignoring\n the overhead of the function call, the generated mapping functions should generally be as fast as an equivalent statically compiled\n switch‐statement in C# or F#. In certain cases they will even be faster.\n

\n
\n
\n

\n The code size of the generated functions increases about linearly with the number of key ranges (i.e. continuous sequences of keys with the same\n value). Hence, you should only use the StaticMapping module for small mappings. If you try\n to turn arbitrarily large key‐value mappings into static mapping functions, you’ll likely hit upon certain implementation limitations (of this\n module’s code, of Reflection.Emit or of the\n CLR’s JIT).\n

\n
\n
\n

\n If the conditional compilation symbol DEBUG_STATIC_MAPPING is defined when compiling FParsec, the generated\n mapping functions will compute each result with two different methods and check the results against each other. Of course, this means that they\n will take more than twice the time than without the DEBUG_STATIC_MAPPING symbol and will also consume more memory. In Debug builds of FParsec DEBUG_STATIC_MAPPING is switched on by default, since the StaticMapping module is still relatively new.\n

\n
\n
\n
\n
Note
\n
\n
\n

Measuring and comparing the performance of the generated mapping functions only makes sense in Release builds.

\n
\n
\n
\n
\n
\n
\n
\n

6.5.3 Members

\n
\n
\n
\n
\n
\n ⇧\n
type Range
\n
\n
\n
\n

\n Represents an immutable range between the integer values Min and Max (inclusive).\n

\n
\n
\n
type Range = struct\n  new: min: int * max: int -> Range\n\n  val Min: int\n  val Max: int\nend\n
\n
\n
\n

\n The Min value must not be larger than the Max value. In a Debug build this condition is checked by an assert‐check in the Range constructor.\n

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticCharIndicatorFunction:\n        invert: bool -> charsInSet: seq<char> -> (char -> bool)\n
\n
\n
\n
\n

\n Creates an optimized indicator function for the chars specified by the charsInSet\n sequence.\n

\n
\n
\n

\n If invert is false (true), the returned indicator function will return true (false) if and only if it is called with a char contained\n in charsInSet.\n

\n
\n
\n

charsInSet may contain duplicate char values.

\n
\n
\n

\n Internally, this function collects continuous ranges of chars into Range values and then uses the same compilation strategy as createStaticCharRangeIndicatorFunction.\n

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticCharRangeIndicatorFunction:\n        invert: bool -> rangesInSet: seq<Range> -> (char -> bool)\n
\n
\n
\n
\n

\n Creates an optimized indicator function for the chars in the ranges specified by the rangesInSet sequence.\n

\n
\n
\n

\n If invert is false (true), the returned indicator function will return true (false) if and only if it is called with a char contained\n in at least one of the ranges of rangesInSet.\n

\n
\n
\n

\n rangesInSet may contain overlapping or duplicate ranges. However, the ranges must not\n contain values less than 0 or greater than 0xffff (the minimum and maximum UTF‐16 char values), otherwise an ArgumentException is\n thrown.\n

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticIntIndicatorFunction:\n        invert: bool -> valuesInSet: seq<int> -> (int -> bool)\n
\n
\n
\n
\n

\n Creates an optimized indicator function for the integers specified by the valuesInSet\n sequence.\n

\n
\n
\n

\n If invert is false (true), the returned indicator function will return true (false) if and only if it is called with an integer\n contained in valuesInSet.\n

\n
\n
\n

valuesInSet may contain duplicate integer values.

\n
\n
\n

\n Internally, this function collects continues ranges of integer into Range values and then uses the same compilation strategy as createStaticIntRangeIndicatorFunction.\n

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticIntRangeIndicatorFunction:\n        invert: bool -> rangesInSet: seq<Range> -> (int -> bool)\n
\n
\n
\n
\n

\n Creates an optimized indicator function for the integers in the ranges specified by the rangesInSet sequence.\n

\n
\n
\n

\n If invert is false (true), the returned indicator function will return true (false) if and only if it is called with an int contained in at least one of the ranges of rangesInSet.\n

\n
\n
\n

rangesInSet may contain overlapping or duplicate ranges.

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticIntMapping:\n        defaultValue: 'T -> keyValues: seq<int*'T> -> (int -> 'T)\n
\n
\n
\n
\n

Creates an optimized mapping function that maps integer keys to values.

\n
\n
\n

\n The keyValues sequence specifies the key‐value pairs for the mapping. All keys not\n specified in keyValues are mapped to defaultValue.\n

\n
\n
\n

\n This function throws an ArgumentException if keyValues contains a duplicate key.\n

\n
\n
\n

\n Internally, this function collects continues ranges of integer keys with equal values[1] into Range values and\n then uses the same compilation strategy as createStaticIntRangeMapping.\n

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticIntRangeMapping:\n        defaultValue: 'T -> keyValues: seq<Range*'T> -> (int -> 'T)\n
\n
\n
\n
\n

Creates an optimized mapping function that maps integer key ranges to values.

\n
\n
\n

\n The keyValues sequence specifies the range‐value pairs for the mapping. All keys not\n contained in one of the ranges in keyValues are mapped to defaultValue.\n

\n
\n
\n

\n This function throws an ArgumentException if keyValues contains an overlapping or\n duplicate key range.\n

\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n ⇧\n
val createStaticStringMapping:\n        defaultValue: 'T -> keyValues: seq<string*'T> -> (string -> 'T)\n
\n
\n
\n
\n

Creates an optimized mapping function that maps string keys to values.

\n
\n
\n

\n The keyValues sequence specifies the key‐value pairs for the mapping. All keys not\n specified in keyValues are mapped to defaultValue. A null key is not supported.\n

\n
\n
\n

\n createStaticStringMapping throws an ArgumentException if keyValues contains a duplicate key or a null key. If the generated mapping function is called with a null string, it throws a NullReferenceException.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The compilation strategy employed by createStaticStringMapping does not handle all\n mappings equally well. It is optimized for mapping a relatively small set of string symbols to constants. If you want to use createStaticStringMapping to optimize a frequently used mapping in your program, you should\n test how well createStaticStringMapping handles your situation (in a Release build!)\n and see whether the performance is worth the compilation costs and the additional code dependency.\n

\n
\n
\n
\n
\n
\n

Please also see the remarks at the beginning of this section.

\n
\n
\n
\n
\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n In the case of a reference type the values are only compared for reference‐equality. In the case of a value type the values are only compared if\n the type implements System.IEquality<_> or is an int enum type.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/reference/text.html", "content": "\n\n\n \n FParsec.Text\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
6Reference
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser overview
2FParsec.Primitives
3FParsec.CharParsers
\n 4\n FParsec.OperatorPrecedenceParser
5FParsec.StaticMapping
6FParsec.Reply
7FParsec.Error
8FParsec.ErrorMessage
9FParsec.ErrorMessageList
10FParsec.Position
11FParsec.CharStream
12FParsec.Text
\n \n \n \n \n \n \n \n \n \n \n \n
1Interface
2Members
\n
\n
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Reference > FParsec.Text\n \n
\n
\n

6.12 FParsec.Text

\n
\n

6.12.1 Interface

\n
\n
\n
\n
// FParsecCS.dll\n\nnamespace FParsec\n\ntype Text =\n    static member CountTextElements: string -> int\n    static member FoldCase: char -> char\n    static member FoldCase: string -> string\n    static member IsWhitespace: char -> bool\n    static member NormalizeNewlines: string -> string\n\n
\n
\n
\n
\n
\n
\n

6.12.2 Members

\n
\n
\n
\n
\n
\n ⇧\n
static member CountTextElements: string -> int\n
\n
\n
\n
\n

\n FParsec.Text.CountTextElements(str) is equivalent to\n System.Globalization.StringInfo(str).LengthInTextElements.\n

\n
\n
\n
\n
\n
\n ⇧\n
static member FoldCase: char -> char\n
\n
\n
\n
\n

\n FParsec.Text.FoldCase(chr)\n is an optimized implementation of FParsec.Text.FoldCase(string chr).\n

\n
\n
\n
\n
\n
\n ⇧\n
static member FoldCase: string -> string\n
\n
\n
\n
\n

\n Returns a case‐folded copy of the string argument. All chars are mapped using the (non‐Turkic) 1‐to‐1 case folding mappings (v. 8.0.0) for Unicode code points in the Basic\n Multilingual Plane, i.e. code points below 0x10000. If the case‐folded string equals the argument string, the original argument is returned\n (to preserve its reference identity). If the argument is null, null is returned.\n

\n
\n
\n
\n
\n
\n ⇧\n
static member IsWhitespace: char -> bool\n
\n
\n
\n
\n

\n FParsec.Text.IsWhitespace(chr) is a faster\n implementation of System.Char.IsWhiteSpace(chr).\n

\n
\n
\n

\n In the LOW_TRUST‐version of FParsec this method simply forwards all calls to System.Char.IsWhiteSpace(chr).\n

\n
\n
\n
\n
\n
\n ⇧\n
static member NormalizeNewlines: string -> string\n
\n
\n
\n
\n

\n Returns the passed string with all occurrences of \"\\r\\n\" and \"\\r\" replaced by \"\\n\". If the normalized string equals the\n argument string, the original argument is returned (to preserve its reference identity). If the argument is null, null is returned.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/tutorial.html", "content": "\n\n\n \n Tutorial\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Preliminaries
\n 2\n Parsing a single float
\n 3\n Parsing a float between brackets
4Abstracting parsers
\n 5\n Parsing a list of floats
6Handling whitespace
7Parsing string data
\n 8\n Sequentially applying parsers
9Parsing alternatives
10F#’s value restriction
11Parsing JSON
12What now?
\n
5User’s Guide
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > Tutorial\n \n
\n
\n

4 Tutorial

\n
\n
\n

\n This tutorial introduces you to the basic concepts of FParsec. Our goal is to give you an intuition for how you can build parser applications\n using the FParsec library. We will only cover the basic ideas and only cursorily explore FParsec’s API, but hopefully we will cover enough ground\n to enable you to further explore FParsec with the help of the user’s guide, the API reference and the sample parsers in the Samples folder.\n

\n
\n
\n

\n A Japanese translation of this tutorial by Gab_km is available here.
A Russian translation of this tutorial by Dmitry Vlasov is\n available here.\n

\n
\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Preliminaries
2Parsing a single float
\n 3\n Parsing a float between brackets
4Abstracting parsers
5Parsing a list of floats
6Handling whitespace
7Parsing string data
\n 8\n Sequentially applying parsers
9Parsing alternatives
10F#’s value restriction
11Parsing JSON
12What now?
\n
\n
\n
\n
\n

4.1 Preliminaries

\n
\n
\n

\n FParsec is built as two DLLs: FParsec.dll and FParsecCS.dll. To use FParsec in your project, you\n can either let NuGet install one of the NuGet\n packages, or you can build the two FParsec DLLs from source. The easiest way to build FParsec from source is using the Visual Studio\n solution files in the Build/VS11 folder of the source code package. Any project that uses FParsec has to reference\n both DLLs. See Download and Installation for more details.\n

\n
\n
\n

\n All FParsec types and modules are declared in the FParsec namespace. This namespace contains\n some basic classes (such as CharStream\n and Reply) and four F# modules, namely\n

\n
    \n
  • \n Primitives, containing basic type definitions and\n parser combinators,\n
    • \n
  • \n CharParsers, containing parsers for chars, strings\n and numbers, and functions for applying parsers to input streams,\n
    • \n
  • \n Error, containing types and helper\n functions for creating, processing and formatting parser error messages,\n
    • \n
  • \n StaticMapping, containing functions for\n compiling static key‐value mappings into optimized functions.\n
    • \n
\n
\n
\n

All code snippets in this tutorial assume that you’ve opened the FParsec namespace:

\n
open FParsec\n
\n

\n Opening the FParsec namespace also automatically opens the Primitives, CharParsers and Error\n modules.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n All code snippets in this tutorial are contained in the Samples/Tutorial project. Having this project open while\n reading the tutorial can be quite helpful. For example, you can hover the mouse over an identifier to get an Intellisense popup with the\n inferred type. And if you’re curious how a library function is implemented, you can click the Go to definition context\n menu option to view its source code.\n

\n
\n
\n
\n
\n
\n
\n
\n

4.2 Parsing a single float

\n
\n
\n

Parsing input with FParsec involves two steps:

\n
    \n
  1. building a parser and
    2. \n
  2. applying the parser to the input.
    3. \n
\n
\n
\n

Let’s start with a simple example: parsing a single floating‐point number in a string.

\n
\n
\n

\n In this case the first step, building the parser, is trivial, because the CharParsers module already comes with a built‐in float parser:\n

\n
val pfloat: Parser<float,'u>
\n
\n
\n

\n The generic type Parser<'Result,'UserState> is the type of all parsers in FParsec. If you follow the hyperlink into the reference, you’ll see that Parser is a type abbreviation for a\n function type. However, at this point we don’t need to go into the details of the Parser type. It’s enough to note that the first type argument\n represents the type of the parser result. Thus, in the case of pfloat the type tells us that if the parser succeeds it returns a floating‐point number of type float. We won’t use a “user state” in this tutorial, so you can just ignore the\n second type argument for the time being.\n

\n
\n
\n

\n To apply the pfloat parser to a\n string, we can use the run function\n from the CharParsers module:\n

\n
val run: Parser<'Result, unit> -> string -> ParserResult<'Result,unit>
\n
\n
\n

\n run is the simplest function out of several provided by the CharParsers module for running parsers on input. Other functions allow you,\n for example, to run parsers directly on the contents of a file or a System.IO.Stream.\n

\n
\n
\n

\n run applies the parser passed as the\n first argument to the string passed as the second argument and returns the return value of the parser in form of a ParserResult value. The ParserResult type is a discriminated union type with\n the two cases: Success and Failure. In case the parser succeeds, the\n ParserResult value contains\n the result value, otherwise it contains an error message.\n

\n
\n
\n

To simplify testing we write a little helper function that prints the result value or error message:

\n
let test p str =\n    match run p str with\n    | Success(result, _, _)   -> printfn \"Success: %A\" result\n    | Failure(errorMsg, _, _) -> printfn \"Failure: %s\" errorMsg\n
\n
\n
\n

\n With this helper function in place, we can test pfloat by executing\n

\n
test pfloat \"1.25\"
\n

which produces the output

\n
Success: 1.25
\n
\n
\n

\n Testing pfloat with a number literal\n that has an invalid exponent\n

\n
test pfloat \"1.25E 3\"
\n

yields the error message

\n
Failure: Error in Ln: 1 Col: 6\n1.25E 3\n     ^\nExpecting: decimal digit\n
\n
\n
\n
\n
\n

4.3 Parsing a float between brackets

\n
\n
\n

\n Implementing parsers with FParsec typically means combining higher‐level parsers from lower‐level ones. You start with the parser primitives\n provided by the library and then successively combine these into higher‐level parsers until you finally have a single parser for the complete\n input.\n

\n
\n
\n

\n In the following sections we will illustrate this approach by discussing various sample parsers that build on each other. In this section we\n will begin with a very simple parser for a floating‐point number between brackets:\n

\n
let str s = pstring s\nlet floatBetweenBrackets = str \"[\" >>. pfloat .>> str \"]\"\n
\n
\n
\n
\n
Note
\n
\n
\n

\n If you’re trying to compile this or another code snippet and you get a compiler error mentioning F#’s “value restriction”,\n please see section 4.10.\n

\n
\n
\n
\n
\n
\n

\n The definition of str and floatBetweenBrackets involves three library functions that we haven’t yet introduced: pstring, >>. and .>>.\n

\n
\n
\n

The function

\n
val pstring: string -> Parser<string,'u>
\n

\n takes a string as the argument and returns a parser for that string. When this parser is applied to an input stream it checks whether the\n following chars in the input stream match the given string. If the chars match the complete string, the parser consumes them, i.e. skips over\n them. Otherwise it fails without consuming any input. When the parser succeeds, it also returns the given string as the parser result, but since\n the string is a constant, you’ll rarely make use of the result.\n

\n
\n
\n

\n The pstring function isn’t named\n string because otherwise it would hide the built‐in F# function string. In general, parser names in FParsec that would otherwise conflict with built‐in F# function names are prefixed\n by a single p char. pfloat is\n another example for this naming convention.\n

\n
\n
\n

\n To save a few keystrokes we abbreviate pstring as str. So, for instance, str \"[\" is a parser that skips over the char\n '['.\n

\n
\n
\n

\n The binary operators >>.\n and .>> have the following\n types:\n

\n
val (>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n
\n
\n
\n

\n As you can see from these signatures, both operators are parser combinators that construct a new parser from the two argument parsers. The\n parser p1 >>. p2 parses p1 and p2 in sequence and returns the result of p2.\n The parser p1 .>> p2 also parses p1 and p2 in sequence, but it returns the result of p1 instead of p2. In each case the point points to the side of the\n parser whose result is returned. By combining both operators in p1 >>. p2 .>> p3 we obtain a parser\n that parses p1, p2 and p3 in sequence and returns the result from p2.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n With the somewhat imprecise wording “parses p1 and p2 in sequence” we actually mean: The parser p1 is\n applied to the input and if p1 succeeds then p2 is applied to the remaining input; in case any of the two element parsers fails, the aggregate parser immediately\n propagates the error message.\n

\n
\n
\n

\n In the documentation for FParsec we often use expressions such as “parses p” or “parses an occurrence of p” instead\n of the technically more accurate “applies the parser p to the remaining input and\n if p succeeds …”, hoping that the exact meaning is obvious from the context.\n

\n
\n
\n
\n
\n
\n

\n The following tests show that floatBetweenBrackets parses valid input as expected and\n produces informative error messages when it encounters invalid input:\n

\n
\n
\n
> test floatBetweenBrackets \"[1.0]\";;\nSuccess: 1.0\n\n> test floatBetweenBrackets \"[]\";;\nFailure: Error in Ln: 1 Col: 2\n[]\n ^\nExpecting: floating-point number\n\n> test floatBetweenBrackets \"[1.0\";;\nFailure: Error in Ln: 1 Col: 5\n[1.0\n    ^\nNote: The error occurred at the end of the input stream.\nExpecting: ']'\n
\n
\n
\n
\n
\n

4.4 Abstracting parsers

\n
\n
\n

One of FParsec’s greatest strengths is the ease with which you can define your own parser abstractions.

\n
\n
\n

\n Take for instance the floatBetweenBrackets from the previous section. If you intend to also\n parse other elements between strings, you could define your own specialized combinator for this purpose:\n

\n
let betweenStrings s1 s2 p = str s1 >>. p .>> str s2\n
\n
\n
\n

\n You could then define floatInBrackets and other parsers with the help of this combinator:\n

\n
let floatBetweenBrackets = pfloat |> betweenStrings \"[\" \"]\"\nlet floatBetweenDoubleBrackets = pfloat |> betweenStrings \"[[\" \"]]\"\n
\n
\n
\n
\n
Note
\n
\n
\n

\n In case you’re new to F#:
pfloat |> betweenStrings \"[\" \"]\"\n is just another way to write betweenStrings \"[\" \"]\" pfloat.\n

\n
\n
\n
\n
\n
\n

\n Once you notice that you frequently need to apply a parser between two others, you could go a step further and factor betweenStrings as follows:\n

\n
let between pBegin pEnd p  = pBegin >>. p .>> pEnd\nlet betweenStrings s1 s2 p = p |> between (str s1) (str s2)\n
\n
\n
\n

\n Actually, you don’t need to define between, because this is already a built‐in FParsec combinator.\n

\n
\n
\n

\n These are all trivial examples, of course. But since FParsec is merely an F# library and not some external parser generator tool, there are no\n limits to the abstractions you can define. You can write functions that take whatever input you need, do some arbitrarily complex computations\n on the input and then return a special purpose parser or parser combinator.\n

\n
\n
\n

\n For example, you could write a function that takes a regular‐expression pattern as the input and returns a Parser for parsing input conforming to that pattern. This\n function could use another parser to parse the pattern into an AST and then compile this AST into a special‐purpose parser function.\n Alternatively, it could construct a .NET regular expression from the pattern and then return a parser function that uses FParsec’s CharStream API to directly apply the regex to\n the input stream (which is what the built‐in regex parser actually does).\n

\n
\n
\n

\n Another example are extensible parser applications. By storing parser functions in dictionaries or other data structures and defining an\n appropriate extension protocol, you could allow plugins to dynamically register new parsers or modify existing ones.\n

\n
\n
\n

\n The possibilities are really endless. But before you can fully exploit these possibilities, you first need to be familiar with the fundamentals\n of FParsec.\n

\n
\n
\n
\n
\n

4.5 Parsing a list of floats

\n
\n
\n

\n We’ve already spent three sections on discussing how to parse a single floating‐point number, so it’s about time we try something more\n ambitious: parsing a list of floating‐point numbers.\n

\n
\n
\n

\n Let us first assume that we need to parse a sequence of floating‐point numbers in brackets, i.e. text in the following EBNF format: (\"[\" float \"]\")*. Valid input strings in this format are for example: \"\", \"[1.0]\", \"[2][3][4]\".\n

\n
\n
\n

\n Since we already have a parser for a float between brackets, we only need a way to repeatedly apply this parser to parse a sequence. This is\n what the many combinator is for:\n

\n
val many: Parser<'a,'u> -> Parser<'a list,'u>
\n
\n
\n

\n The parser many p repeatedly applies the parser p until p fails, i.e. it “greedily” parses as many occurrences of p as possible. The results of p are returned as a list in the order\n of occurrence.\n

\n
\n
\n

\n Some simple tests show that many floatInBrackets works as expected:\n

\n
> test (many floatBetweenBrackets) \"\";;\nSuccess: []\n> test (many floatBetweenBrackets) \"[1.0]\";;\nSuccess: [1.0]\n> test (many floatBetweenBrackets) \"[2][3][4]\";;\nSuccess: [2.0; 3.0; 4.0]\n
\n
\n
\n

\n If floatBetweenBrackets fails after consuming input, then the combined parser fails\n too:\n

\n
> test (many floatBetweenBrackets) \"[1][2.0E]\";;\nFailure: Error in Ln: 1 Col: 9\n[1][2.0E]\n        ^\nExpecting: decimal digit\n
\n
\n
\n

\n Note that many also succeeds for an\n empty sequence. If you want to require at least one element, you can use many1 instead:\n

\n
> test (many1 floatBetweenBrackets) \"(1)\";;\nFailure: Error in Ln: 1 Col: 1\n(1)\n^\nExpecting: '['\n
\n
\n
\n
\n
Tip
\n
\n
\n

\n If you’d prefer the last error message to be worded in terms of the higher level floatBetweenBrackets parser instead of the lower level str \"[\" parser, you could use the <?> operator as in the following example:\n

\n
\n
\n
> test (many1 (floatBetweenBrackets <?> \"float between brackets\")) \"(1)\";;\nFailure: Error in Ln: 1 Col: 1\n(1)\n^\nExpecting: float between brackets\n
\n
\n
\n

\n Please see section 5.8 of the user’s guide to learn more about customizing error\n messages.\n

\n
\n
\n
\n
\n
\n

\n If you just want to skip over a sequence and don’t need the list of parser results, you could use the optimized combinators skipMany or skipMany1 instead of many and many1.\n

\n
\n
\n

\n Another frequently used combinator for parsing sequences is sepBy:\n

\n
val sepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list, 'u>
\n
\n
\n

\n sepBy takes an “element”\n parser and a “separator” parser as the arguments and returns a parser for a list of elements separated by separators. In EBNF\n notation sepBy p pSep could be written as (p (pSep p)*)?. Similar to many, there are several variants of sepBy.\n

\n
\n
\n

\n With the help of sepBy we can parse a\n more readable list format, where floating‐point numbers are separated by a comma:\n

\n
floatList: \"[\" (float (\",\" float)*)? \"]\"
\n
\n
\n

\n Valid input strings in this format are for example: \"[]\", \"[1.0]\", \"[2,3,4]\".\n

\n
\n
\n

The straightforward implementation of this format is

\n
let floatList = str \"[\" >>. sepBy pfloat (str \",\") .>> str \"]\"\n
\n
\n
\n

Testing floatList with valid test strings yields the expected result:

\n
> test floatList \"[]\";;\nSuccess: []\n> test floatList \"[1.0]\";;\nSuccess: [1.0]\n> test floatList \"[4,5,6]\";;\nSuccess: [4.0; 5.0; 6.0]\n
\n
\n
\n

Testing with invalid input shows that floatList produces helpful error messages:

\n
> test floatList \"[1.0,]\";;\nFailure: Error in Ln: 1 Col: 6\n[1.0,]\n     ^\nExpecting: floating-point number\n\n> test floatList \"[1.0,2.0\";;\nFailure: Error in Ln: 1 Col: 9\n[1.0,2.0\n        ^\nNote: The error occurred at the end of the input stream.\nExpecting: ',' or ']'\n
\n
\n
\n
\n
\n

4.6 Handling whitespace

\n
\n
\n

\n FParsec treats whitespace (spaces, tabs, newlines, etc) just as any other input, so our floatList parser can’t yet deal with whitespace:\n

\n
> test floatBetweenBrackets \"[1.0, 2.0]\";;\nFailure: Error in Ln: 1 Col: 5\n[1.0, 2.0]\n    ^\nExpecting: ']'\n
\n
\n
\n

If we want the parser to ignore whitespace, we need to make this explicit in the parser definition.

\n
\n
\n

\n First, we need to define what we want to accept as whitespace. For simplicity we will just use the built‐in spaces parser, which skips over any (possibly empty)\n sequence of ' ', '\\t', '\\r' or '\\n' chars.\n

\n
let ws = spaces\n
\n
\n
\n

\n Next, we need to insert the ws parser at every point where we want to ignore whitespace. In\n general it’s best to skip whitespace after one parses elements, i.e. skip trailing instead of leading whitespace, because that reduces\n the need for backtracking (which will be explained below). Hence, we insert ws at two places\n to skip over any whitespace after brackets or numbers:\n

\n
\n
\n
let str_ws s = pstring s .>> ws\nlet float_ws = pfloat .>> ws\nlet numberList = str_ws \"[\" >>. sepBy float_ws (str_ws \",\") .>> str_ws \"]\"\n
\n
\n
\n

A simple test shows that numberList ignores whitespace:

\n
> test numberList @\"[ 1 ,\n                          2 ] \";;\nSuccess: [1.0; 2.0]\n
\n
\n
\n

\n If we introduce an error on the second line, we see that FParsec automatically keeps track\n of the line count:\n

\n
> test numberList @\"[ 1,\n                         2; 3]\";;\n\nFailure: Error in Ln: 2 Col: 27\n                         2; 3]\n                          ^\nExpecting: ',' or ']'\n
\n
\n
\n

\n Our numberList parser still doesn’t skip leading whitespace, because that’s not necessary\n when we put it together with other parsers that skip all trailing whitespace. If we wanted to parse a whole input stream with only a list of\n floating‐point numbers, we could use the following parser:\n

\n
let numberListFile = ws >>. numberList .>> eof\n
\n
\n
\n

\n The end‐of‐file parser eof will\n generate an error if the end of the stream hasn’t been reached. This is useful for making sure that the complete input gets consumed. Without\n the eof parser the following test\n wouldn’t produce an error:\n

\n
> test numberListFile \" [1, 2, 3] [4]\";;\nFailure: Error in Ln: 1 Col: 12\n [1, 2, 3] [4]\n           ^\nExpecting: end of input\n
\n
\n
\n
\n
\n

4.7 Parsing string data

\n
\n
\n

\n FParsec contains various built‐in parsers for chars, strings, numbers and whitespace. In this section we will introduce a few of the char and\n string parsers. For an overview of all available parsers please refer to the parser overview in the\n reference.\n

\n
\n
\n

\n You’ve already seen several applications of the pstring parser (abbreviated as str), which simply skips over a\n constant string in the input. When the pstring parser succeeds, it also returns the skipped string as the parser result. The following example\n demonstrates this:\n

\n
\n
\n
> test (many (str \"a\" <|> str \"b\")) \"abba\";;\nSuccess: [\"a\"; \"b\"; \"b\"; \"a\"]\n
\n
\n
\n

\n In this example we also used the <|> combinator to combine two alternative parsers. We’ll discuss this combinator in more detail below.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n We refer to both pstring and\n pstring \"a\" as “parsers”. Strictly speaking, pstring is function taking a string argument and\n returning a Parser, but it’s more\n convenient to just refer to it as a (parametric) parser.\n

\n
\n
\n
\n
\n
\n

\n When you don’t need the result of the pstring parser, you can alternatively use the skipString\n parser, which returns the unit value ()\n instead of the argument string. In this case it doesn’t make any difference to performance whether you use pstring or skipString, since the returned string is a constant. However, for most other built‐in parsers and combinators you\n should prefer the variants with the “skip” name prefix when you don’t need the parser result values, because these will generally\n be faster. If you look at the parser overview, you’ll see “skip” variants for many of\n the built‐in parsers and combinators.\n

\n
\n
\n

\n If you want to parse a case insensitive string constant you can use pstringCI and skipStringCI. For example:\n

\n
> test (skipStringCI \"<float>\" >>. pfloat) \"<FLOAT>1.0\";;\nSuccess: 1.0\n
\n
\n
\n

\n Frequently one needs to parse string variables whose chars have to satisfy certain criteria. For instance, identifiers in programming languages\n often need to start with a letter or underscore and then need to continue with letters, digits or underscores. To parse such an identifier you\n could use the following parser:\n

\n
\n
\n
let identifier =\n    let isIdentifierFirstChar c = isLetter c || c = '_'\n    let isIdentifierChar c = isLetter c || isDigit c || c = '_'\n\n    many1Satisfy2L isIdentifierFirstChar isIdentifierChar \"identifier\"\n    .>> ws // skips trailing whitespace\n
\n
\n
\n

\n Here we have used the many1Satisfy2L string parser, which is one of several primitives for parsing strings based on char predicates (i.e.\n functions that take a char as input and return a boolean value). It parses any sequence of one or more chars (hence the “many1” in\n the name) whose first char satisfies the first predicate function and whose remaining chars satisfy the second predicate (hence the\n “Satisfy2”). The string label given as the third argument (hence the “L”) is used in error message to describe the\n expected input.\n

\n
\n
\n

The following tests show how this parser works:

\n
> test identifier \"_\";;\nSuccess: \"_\"\n> test identifier \"_test1=\";;\nSuccess: \"_test1\"\n> test identifier \"1\";;\nFailure: Error in Ln: 1 Col: 1\n1\n^\nExpecting: identifier\n
\n
\n
\n
\n
Tip
\n
\n
\n

\n If you want to parse identifiers based on the Unicode XID syntax, consider using the built‐in identifier parser.\n

\n
\n
\n
\n
\n
\n

\n Many string formats are complicated enough that you need to combine several char and string parser primitives. For example, consider the\n following string literal format:\n

\n
  stringLiteral: '\"' (normalChar|escapedChar)* '\"'\n  normalChar:    any char except '\\' and '\"'\n  escapedChar:   '\\\\' ('\\\\'|'\"'|'n'|'r'|'t')\n
\n
\n
\n

A straightforward translation of this grammar to FParsec looks like:

\n
let stringLiteral =\n    let normalChar = satisfy (fun c -> c <> '\\\\' && c <> '\"')\n    let unescape c = match c with\n                     | 'n' -> '\\n'\n                     | 'r' -> '\\r'\n                     | 't' -> '\\t'\n                     | c   -> c\n    let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> unescape)\n    between (pstring \"\\\"\") (pstring \"\\\"\")\n            (manyChars (normalChar <|> escapedChar))\n
\n
\n
\n

In this example we use several library functions that we haven’t yet introduced:

\n
    \n
  • \n satisfy parses any char that\n satisfies the given predicate function.\n
    • \n
  • \n anyOf parses any char contained in\n the argument string.\n
    • \n
  • \n The pipeline combinator |>> applies the function on the right side (unescape) to\n the result of the parser on the left side (anyOf \"\\\\nrt\\\"\").\n
    • \n
  • \n The choice combinator <|> applies the parser on the right side if the parser on the left side fails, so that normalChar <|> escapedChar can parse both normal and escaped chars. (We will discuss this\n operator in more detail two sections below.)\n
    • \n
  • \n manyChars parses a sequence of\n chars with the given char parser and returns it as a string.\n
    • \n
\n
\n
\n

Let’s test the stringLiteral parser with a few test inputs:

\n
> test stringLiteral \"\\\"abc\\\"\";;\nSuccess: \"abc\"\n> test stringLiteral \"\\\"abc\\\\\\\"def\\\\\\\\ghi\\\"\";;\nSuccess: \"abc\"def\\ghi\"\n> test stringLiteral \"\\\"abc\\\\def\\\"\";;\nFailure: Error in Ln: 1 Col: 6\n\"abc\\def\"\n     ^\nExpecting: any char in ‘\\nrt\"’\n
\n
\n
\n

Instead of parsing the string literal char‐by‐char we could also parse it “snippet‐by‐snippet”:

\n
let stringLiteral2 =\n    let normalCharSnippet = many1Satisfy (fun c -> c <> '\\\\' && c <> '\"')\n    let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> function\n                                                            | 'n' -> \"\\n\"\n                                                            | 'r' -> \"\\r\"\n                                                            | 't' -> \"\\t\"\n                                                            | c   -> string c)\n    between (pstring \"\\\"\") (pstring \"\\\"\")\n            (manyStrings (normalCharSnippet <|> escapedChar))\n
\n
\n
\n

\n Here we have used the manyStrings combinator, which parses a sequence of strings with the given string parser and returns the strings in\n concatenated form.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n We have to require normalCharSnippet to consume at least one char, i.e. use many1Satisfy instead of manySatisfy. Otherwise normalCharSnippet would succeed even if doesn’t consume input, escapedChar would never be called and manyStrings would eventually throw an exception to\n prevent an infinite loop.\n

\n
\n
\n
\n
\n
\n

\n Parsing a string chunk‐wise using an optimized parser like many1Satisfy is usually a bit faster than parsing it char‐wise using manyChars and satisfy. In this case we can optimize our parser even a bit\n further – once we realize that two normal char snippets must be separated by at least one escaped char:\n

\n
let stringLiteral3 =\n    let normalCharSnippet = manySatisfy (fun c -> c <> '\\\\' && c <> '\"')\n    let escapedChar = (* like in stringLiteral2 *)\n    between (pstring \"\\\"\") (pstring \"\\\"\")\n            (stringsSepBy normalCharSnippet escapedChar)\n\n
\n

\n The stringsSepBy combinator\n parses a sequence of strings (with the first argument parser) separated by other strings (parsed with the second argument parser). It returns\n all parsed strings, including the separator strings, as a single, concatenated string.\n

\n
\n
\n

\n Note that stringLiteral3 uses manySatisfy instead of many1Satisfy in its normalCharSnippet definition, so that it can parse escaped chars that are not separated by normal chars. This can’t\n lead to an infinite loop because escapedChar can’t succeed without consuming input.\n

\n
\n
\n
\n
\n

4.8 Sequentially applying parsers

\n
\n
\n

\n Whenever you need to apply multiple parsers in sequence and only need the result of one of them, a suitable combination of >>. and .>> operators will do the job. However, these\n combinators won’t suffice if you need the result of more than one of the involved parsers. In that case you can use the pipe2, …, pipe5 combinators, which apply multiple parsers in sequence\n and pass all the individual results to a function that computes the aggregate result.\n

\n
\n
\n

\n For instance, with the pipe2\n combinator\n

\n
val pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> b -> 'c) -> Parser<'c,'u>
\n

\n you can construct a parser pipe2 p1 p2 f that sequentially applies the two parsers p1 and p2 and then returns the result of the\n function application f x1 x2, where x1 and x2 are the results returned by p1 and p2.\n

\n
\n
\n

\n In the following example we use pipe2\n to parse a product of two numbers:\n

\n
let product = pipe2 float_ws (str_ws \"*\" >>. float_ws)\n                    (fun x y -> x * y)\n
\n
> test product \"3 * 5\";;\nSuccess: 15.0\n
\n
\n
\n

\n The pipe2-5 combinators are particularly useful for constructing AST objects. In the following example we use pipe3 to parse a string constant definition\n into a StringConstant object:\n

\n
type StringConstant = StringConstant of string * string\n\nlet stringConstant = pipe3 identifier (str_ws \"=\") stringLiteral\n                           (fun id _ str -> StringConstant(id, str))\n
\n
> test stringConstant \"myString = \\\"stringValue\\\"\";;\nSuccess: StringConstant (\"myString\",\"stringValue\")\n
\n
\n
\n

\n If you just want to return the parsed values as a tuple, you can use the predefined tuple2-5\n parsers. For instance, tuple2 p1 p2 is equivalent to pipe2 p1 p2 (fun x1 x2 -> (x1, x2)).\n

\n
\n
\n

\n The tuple2 parser is also available\n under the operator name .>>., so that you can write p1 .>>. p2 instead of tuple2 p1 p2. In the following example we parse a pair of comma separated numbers with this operator:\n

\n
> test (float_ws .>>. (str_ws \",\" >>. float_ws)) \"123, 456\";;\nSuccess: (123.0, 456.0)\n
\n
\n
\n

\n Hopefully you find the >>‐with‐1‐or‐2‐dots‐notation intuitive\n by now.\n

\n
\n
\n

\n If you need a pipe or tuple parser with\n more than 5 arguments, you can easily construct one using the existing ones. For example, do you have an idea how to define a pipe7 parser? This footnote gives a possible solution: [1]\n

\n
\n
\n
\n
\n

4.9 Parsing alternatives

\n
\n
\n

\n In the section on Parsing string data we\n already shortly introduced the choice combinator <|>:\n

\n
val (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,u>\n
\n
\n
\n

\n This combinator allows you to support multiple alternative input formats at a given input position. For example, in the above section we used\n <|> to combine a parser\n for unescaped chars and a parser for escaped chars into a parser that supports both: normalChar <|> escapedChar.\n

\n
\n
\n

\n Another example that shows how <|> works is the following parser for boolean variables:\n

\n
let boolean =     (stringReturn \"true\"  true)\n              <|> (stringReturn \"false\" false)\n
\n
\n
\n

\n Here we have also used the stringReturn parser, which skips the string constant given as the first argument and, if successful, returns the\n value given as the second argument.\n

\n
\n
\n

Testing the boolean parser with some inputs yields:

\n
> test boolean \"false\";;\nSuccess: false\n> test boolean \"true\";;\nSuccess: true\n> test boolean \"tru\";;\nFailure: Error in Ln: 1 Col: 1\ntru\n^\nExpecting: 'false' or 'true'\n
\n
\n
\n

\n The behaviour of the <|>\n combinator has two important characteristics:\n

\n
    \n
  • \n <|> only tries the\n parser on the right side if the parser on the left side fails. It does not implement a longest match rule.\n
    • \n
  • However, it only tries the right parser if the left parser fails without consuming input.
    • \n
\n
\n
\n

\n A consequence of the second point is that the following test fails because the parser on the left side of <|> consumes whitespace before it fails:\n

\n
> test ((ws >>. str \"a\") <|> (ws >>. str \"b\")) \" b\";;\nFailure: Error in Ln: 1 Col: 2\n b\n ^\nExpecting: 'a'\n
\n
\n
\n

Fortunately, we can easily fix this parser by factoring out ws:

\n
> test (ws >>. (str \"a\" <|> str \"b\")) \" b\";;\nSuccess: \"b\"\n
\n
\n
\n

\n If you’re curious why <|> behaves this way and how you can handle situations where you need <|> to try the alternative parser even if the\n first parser fails after consuming input, please see section 5.6 and section 5.7 in the user’s guide.\n

\n
\n
\n

\n If you want to try more than two alternative parsers, you can chain multiple <|> operators, like in p1 <|> p2 <|> p3 <|> ..., or you can use the choice combinator, which accepts a sequence of parsers as the argument, like in choice [p1; p2; p3; ...].\n

\n
\n
\n
\n
\n

4.10 F#’s value restriction

\n
\n
\n

\n When you start writing your own parsers with FParsec or try to compile some individual code snippets from above, you’ll come across a compiler\n issue that often causes some head‐scratching among new users of F# and FParsec: the value restriction. In this section\n we’ll explain the value restriction and how you can handle it in your FParsec programs.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n If you find the discussion in this section too technical for the moment, just skip to the next section and come back later when you actually\n see a compiler message mentioning “value restriction” for the first time.\n

\n
\n
\n
\n
\n
\n

F#’s value restriction is the reason that the following code snippet does not compile

\n
open FParsec\nlet p = pstring \"test\"\n
\n

\n even though the following snippet compiles without a problem[2]:\n

\n
open FParsec\nlet p = pstring \"test\"\nrun p \"input\"\n
\n
\n
\n

The compiler error generated for the first sample is the following:

\n
error FS0030: Value restriction.\nThe value 'p' has been inferred to have generic type\n    val p : Parser<string,'_a>\nEither make the arguments to 'p' explicit or,\nif you do not intend for it to be generic, add a type annotation.\n
\n
\n
\n

\n When you work with FParsec you’ll sooner or later see this or similar error messages, in particular if you work with the interactive console\n prompt. Fortunately, this kind of error is usually easy to workaround.\n

\n
\n
\n

\n The problem with the first snippet above is that the F# compiler infers the p value to have\n an unresolved generic type, although F# doesn’t permit a generic value in this situation. The return type of the pstring function is Parser<string,'u>, where the type parameter 'u represents the type of the CharStream user state. Since there is nothing in the first\n snippet that constrains this type parameter, the compiler infers the type Parser<string,'_a> for the parser value p, with '_a representing an unresolved type\n parameter.\n

\n
\n
\n

\n In the second snippet this problem doesn’t occur, because the use of p as the first argument\n to the run function constrains the user\n state type. Since run only accepts\n parsers of type Parser<'t,unit>, the\n compiler infers the non‐generic type Parser<string,unit> for p.\n

\n
\n
\n

This example suggests two ways to handle the value restriction in FParsec programs:

\n
    \n
  • \n Either make sure that the type of a parser value is constrained to a non‐generic type by subsequent uses of this parser value in the same\n compilation unit,\n
    • \n
  • \n or provide an explicit type annotation to manually constrain the type of the parser value (usually, a few type annotations in key locations are\n enough for a whole parser module).\n
    • \n
\n
\n
\n

Often it is convenient to define some type abbreviations like the following

\n
type UserState = unit // doesn't have to be unit, of course\ntype Parser<'t> = Parser<'t, UserState>\n
\n
\n
\n

With such an abbreviation in place, type annotations become as simple as

\n
let p : Parser<_> = pstring \"test\"\n
\n
\n
\n

\n Of course, constraining the type of a parser value to a non‐generic type is only a solution if you don’t actually need a generic type. If you do\n need a generic value, you’ll have to apply other techniques, as they are for example explained in the F# reference or in a blog entry by Dmitry Lomov. However, FParsec Parser values (not parametric parser\n functions) are usually only used in the context of a specific parser application with a fixed user state type. In that situation constraining\n the type is indeed the appropriate measure to avoid a value restriction error.\n

\n
\n
\n
\n
\n

4.11 Parsing JSON

\n
\n
\n

Now that we have discussed the basics of FParsec we are well prepared to work through a real world parser example: a JSON parser.

\n
\n
\n

\n JSON (JavaScript Object Notation) is a text‐based data interchange format with a simple and lightweight syntax. You can find descriptions of the\n syntax on json.org and in RFC 4626.\n

\n
\n
\n

\n In many applications one only has to deal with JSON files describing one particular kind of object. In such a context it sometimes can be\n appropriate to write a specialized parser just for that specific kind of JSON file. In this tutorial, however, we will follow a more general\n approach. We will implement a parser that can parse any general JSON file into an AST, i.e. an intermediate data structure describing the\n contents of the file. Applications can then conveniently query this data structure and extract the information they need. This is an approach\n comparable to that of XML parsers which build a data structure describing the document tree of an XML document. The great advantage of this\n approach is that the JSON parser itself becomes reusable and the document specific parsing logic can be expressed in the form of simple\n functions processing the AST of the JSON document.\n

\n
\n
\n

\n The natural way to implement an AST in F# is with the help of a discriminated union type. If you look at the JSON\n specification, you can see that a JSON value can be a string, a number, a boolean, null, a comma‐separated list of values in square\n brackets, or an object with a sequence of key‐value pairs in curly brackets.\n

\n
\n
\n

In our parser we will use the following union type to represent JSON values:

\n
type Json = JString of string\n          | JNumber of float\n          | JBool   of bool\n          | JNull\n          | JList   of Json list\n          | JObject of Map<string, Json>\n
\n
\n
\n

\n Here we’ve chosen the F# list type to represent a sequence of values and the Map type to represent a sequence of key‐value pairs, because these types are particularly\n convenient to process in F#.[3] Note that the Json type is recursive, since both JList and\n JObject values can themselves contain Json values. Our parser will have to reflect this recursive structure.\n

\n
\n
\n
\n
Tip
\n
\n
\n

\n If you’re new to FParsec and have a little time, it would be a good exercise to try to implement the JSON parser on your own (with the help\n of the reference documentation). This tutorial already covered almost everything you need and the JSON grammar is simple enough that this\n shouldn’t take too much time. Of course, you can always peek at the implementation below if you get stuck.\n

\n
\n
\n
\n
\n
\n

\n We start the actual parser implementation by covering the simple null and boolean cases:\n

\n
let jnull  = stringReturn \"null\" JNull\nlet jtrue  = stringReturn \"true\"  (JBool true)\nlet jfalse = stringReturn \"false\" (JBool false)\n
\n
\n
\n

\n Handling the number case is just as simple, because the JSON number format is based on the typical floating‐point number format used in many\n programming languages and hence can be parsed with FParsec’s built‐in pfloat parser:\n

\n
let jnumber = pfloat |>> JNumber\n
\n

\n (Note that F# allows us to pass the object constructor JNumber as a function argument.)\n

\n
\n
\n

\n If you compare the precise number format supported by pfloat with that in the JSON spec, you’ll see that pfloat supports a superset of the JSON format. In contrast\n to the JSON format the pfloat parser\n also recognizes NaN and Infinity values,\n accepts a leading plus sign, accepts leading zeros and even supports the hexadecimal float format of Java and C99. Depending on the context this\n behaviour can be considered a feature or a limitation of the parser. For most applications it probably doesn’t matter, and the JSON RFC clearly\n states that a JSON parser may support a superset of the JSON syntax. However, if you’d rather only support the exact JSON number format, you can\n implement such a float parser rather easily based on the configurable numberLiteral parser (just have a look at how this is\n currently done in the pfloat\n source).\n

\n
\n
\n

\n The JSON string format takes a little more effort to implement, but we’ve already parsed a similar format with the stringLiteral parsers in section 4.7, so we can just adapt one of those parsers for\n our purpose:\n

\n
\n
\n
let str s = pstring s\n\nlet stringLiteral =\n    let escape =  anyOf \"\\\"\\\\/bfnrt\"\n                  |>> function\n                      | 'b' -> \"\\b\"\n                      | 'f' -> \"\\u000C\"\n                      | 'n' -> \"\\n\"\n                      | 'r' -> \"\\r\"\n                      | 't' -> \"\\t\"\n                      | c   -> string c // every other char is mapped to itself\n\n    let unicodeEscape =\n    \t/// converts a hex char ([0-9a-fA-F]) to its integer number (0-15)\n        let hex2int c = (int c &&& 15) + (int c >>> 6)*9\n\n        str \"u\" >>. pipe4 hex hex hex hex (fun h3 h2 h1 h0 ->\n            (hex2int h3)*4096 + (hex2int h2)*256 + (hex2int h1)*16 + hex2int h0\n            |> char |> string\n        )\n\n    let escapedCharSnippet = str \"\\\\\" >>. (escape <|> unicodeEscape)\n    let normalCharSnippet  = manySatisfy (fun c -> c <> '\"' && c <> '\\\\')\n\n    between (str \"\\\"\") (str \"\\\"\")\n            (stringsSepBy normalCharSnippet escapedCharSnippet)\n\nlet jstring = stringLiteral |>> JString\n
\n
\n
\n

\n stringLiteral parses string literals as a sequence of normal char snippets separated by\n escaped char snippets. A normal char snippet is any sequence of chars that does not contain the chars '\"' and '\\\\'. An escaped char snippet consists of a backslash followed\n by any of the chars '\\\\', '\\\"', '/',\n 'b', 'f', 'n', 'r', 't',\n or an Unicode escape. An Unicode escape consists of an 'u' followed by four hex chars representing an UTF‐16 code point.\n

\n
\n
\n

\n The grammar rules for JSON lists and objects are recursive, because\n any list or object can contain itself any kind of JSON value. Hence, in order to write parsers for the list and object grammar rules, we need a\n way to refer to the parser for any kind of JSON value, even though we haven’t yet constructed this parser. Like it is so often in computing, we\n can solve this problem by introducing an extra indirection:\n

\n
let jvalue, jvalueRef = createParserForwardedToRef<Json, unit>()\n
\n
\n
\n

\n As you might have guessed from the name, createParserForwardedToRef creates a parser (jvalue) that\n forwards all invocations to the parser in a reference cell (jvalueRef). Initially, the\n reference cell holds a dummy parser, but since the reference cell is mutable, we can later replace the dummy parser with the actual value\n parser, once we have finished constructing it.\n

\n
\n
\n

\n The JSON RFC sensibly only permits spaces, (horizontal) tabs, line feeds and carriage returns as whitespace characters, which allows us to use\n the built‐in spaces parser for\n parsing whitespace:\n

\n
let ws = spaces\n
\n
\n
\n

\n Both JSON lists and objects are syntactically represented as a comma‐separated lists of “elements” between brackets, where\n whitespace is allowed before and after any bracket, comma and list element. We can conveniently parse such lists with the following helper\n function:\n

\n
let listBetweenStrings sOpen sClose pElement f =\n    between (str sOpen) (str sClose)\n            (ws >>. sepBy (pElement .>> ws) (str \",\" >>. ws) |>> f)\n
\n
\n
\n

\n This function takes four arguments: an opening string, a closing string, an element parser and a function that is applied to the parsed list of\n elements.\n

\n
\n
\n

With the help of this function we can define the parser for a JSON list as follows:

\n
let jlist   = listBetweenStrings \"[\" \"]\" jvalue JList\n
\n
\n
\n

JSON objects are lists of key‐value pairs, so we need a parser for a key‐value pair:

\n
let keyValue = stringLiteral .>>. (ws >>. str \":\" >>. ws >>. jvalue)\n
\n

\n (Remember, the points on both sides of .>>. indicate that the results of the two parsers on both sides are returned as a tuple.)\n

\n
\n
\n

\n By passing the keyValue parser to listBetweenStrings we obtain a parser for JSON objects:\n

\n
let jobject = listBetweenStrings \"{\" \"}\" keyValue (Map.ofList >> JObject)\n
\n
\n
\n

\n Having defined parsers for all the possible kind of JSON values, we can combine the\n different cases with a choice parser\n to obtain the finished parser for JSON values:\n

\n
\n
\n
do jvalueRef := choice [jobject\n                        jlist\n                        jstring\n                        jnumber\n                        jtrue\n                        jfalse\n                        jnull]\n
\n
\n
\n

\n The jvalue parser doesn’t accept leading or trailing whitespace, so we need to define our\n parser for complete JSON documents as follows:\n

\n
let json = ws >>. jvalue .>> ws .>> eof\n
\n
\n
\n

\n This parser will try to consume a complete JSON input stream and, if successful, will return a Json AST of the input as the parser result\n

\n
\n
\n

\n And that’s it, we’re finished with our JSON parser. If you want to try this parser out on some sample input, please take a look at the JSON\n project in the Samples folder.\n

\n
\n
\n
\n
\n

4.12 What now?

\n
\n
\n

\n If this tutorial has whet your appetite for a more in‐depth introduction to FParsec, just head over to the user’s guide. If you can’t wait to write your own parser, then bookmark the parser overview page, maybe take a short look at the example parsers in the Samples folder and just start hacking. You can always consult the user’s guide at a later point should you get stuck\n somewhere.\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n [1]\n \n
let pipe7 p1 p2 p3 p4 p5 p6 p7 f =\n    pipe4 p1 p2 p3 (tuple4 p4 p5 p6 p7)\n          (fun x1 x2 x3 (x4, x5, x6, x7) -> f x1 x2 x3 x4 x5 x6 x7)\n
\n
[2]Assuming you referenced the two FParsec DLLs.
[3]\n If you need to parse huge sequences and objects, it might be more appropriate to use an array and dictionary for JList and JObject respectively.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/applying-parsers-in-sequence.html", "content": "\n\n\n \n Applying parsers in sequence\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
4Applying parsers in sequence
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1\n The definition of >>.\n
\n 2\n Merging error messages
3The StateTag
4Generalizing >>.
5\n The >>= combinator\n
\n
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Applying parsers in sequence\n \n
\n
\n

5.4 Applying parsers in sequence

\n
\n
\n

\n Now that we have discussed how Parser functions work, we can start explaining how FParsec’s parser combinators work.\n

\n
\n
\n

\n In this chapter we will discuss combinators that allow you to apply multiple parsers in sequence, i.e. parse the beginning of the input with the\n first parser, then parse the following input with the second parser, and so on.\n

\n
\n
\n
\n

\n 5.4.1 The definition of >>.\n

\n
\n
\n

The simplest combinators for sequentially applying two parsers are

\n
val (>>.):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>):   Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n
\n
\n
\n

\n Both operators take two parsers as arguments and return a combined parser that applies the two parsers in sequence. As you can infer from the\n type signatures, p1 >>. p2 returns the result of p2\n and p1 .>> p2 the result of p1. In each\n case the point points to the parser whose result is returned.\n

\n
\n
\n

\n In order to explain exactly what it means to apply two parser in sequence, we give a full definition of the >>. operator:\n

\n
let (>>.) (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) =\n    fun stream ->\n        let reply1 = p1 stream\n        if reply1.Status = Ok then\n            let stateTag = stream.StateTag\n            let mutable reply2 = p2 stream\n            if stateTag = stream.StateTag then                        // (1)\n                reply2.Error <- mergeErrors reply1.Error reply2.Error // (2)\n            reply2\n        else // reconstruct error reply with new result type\n            Reply(reply1.Status, reply1.Error)\n
\n
\n
\n

\n The implementation of p1 >>. p2 should be quite self‐explanatory: First p1 is applied to the input stream. If p1 succeeds, i.e. if the status of reply1 is Ok, p2 is applied to stream and the reply of p2 then becomes the reply\n of the combined parser. However, if p1 fails, its reply is immediately propagated as the\n reply of the combined parser. Since reply1 has type Reply<'a,'u> but p1 >>. p2 needs to return a\n Reply<'b,'u>, the error reply needs to be\n reconstructed with a new result type before it can be returned.\n

\n
\n
\n
\n
\n

5.4.2 Merging error messages

\n
\n
\n

\n We mentioned earlier that the error messages returned in the Reply.Error\n field implicitly refer to the state of the CharStream at the time the parser returns. In particular the error messages refer to the then current stream\n position. Since the messages do not contain themselves a separate record of the error position they can only be interpreted together with\n the CharStream state.\n

\n
\n
\n

\n When p2 does not change the parser state, the error messages from both replies refer to the\n state of the CharStream as it is when\n p1 >>. p2 returns. Thus, the combinator needs to merge the (immutable) ErrorMessageLists from both replies, so that the\n returned list contains all the relevant error messages (see the line marked with (2)).\n

\n
\n
\n

\n In order to check whether the CharStream state has changed, the combinator does not compare the full states from before and after p2 is invoked. Instead it only compares the StateTag values (see line (1)). This improves performance and — for most\n practical purpose — is almost equivalent to comparing the full state, as we will discuss below.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The way (>>.) handles errors and\n merges error messages is a template for all combinators in FParsec that perform multiple sequential parser invocations.\n

\n
\n
\n
\n
\n
\n

\n You may wonder why the error messages get merged even though p1 succeeded. The somewhat\n counterintuitive reason is that parsers can return nonempty error message lists even when they don’t fail. For example, a parser that skips over\n the optional string \"str\" will\n return Reply(Ok, (), expectedString \"str\") if it doesn’t find the string in the input. In this case the error message describes what further input the parser\n could have parsed at the current stream position. If subsequently a parser fails at the same position, all error messages for the same position\n can be aggregated to give the user as much information as possible about what went wrong and what alternative inputs could have been parsed at\n the given position.\n

\n
\n
\n

The following sample demonstrates the helpful effect of this error handling behaviour:

\n
let str s = pstring s\n\nlet oneOrTwoInts =\n    str \"(\" >>. tuple2 pint32 (opt (str \",\" >>. spaces >>. pint32)) .>> str \")\"\n
\n
> run oneOrTwoInts \"(1 2)\";;\nval it : ParserResult<(int32 * int32 option),unit> = Failure:\nError in Ln: 1 Col: 3\n(1 2)\n  ^\nExpecting: ')' or ','\n
\n

\n This error message wouldn’t mention the possibility of a missing comma if the .>> combinator did not merge error messages for\n the same position when the left‐hand side parser succeeds.\n

\n
\n
\n
\n
\n

\n 5.4.3 The StateTag\n

\n
\n
\n

\n Parser combinators often need to check whether a parser has changed the CharStream state. In a typical FParsec application these\n checks are performed so frequently that an efficient implementation is important for the overall parser performance. Since a straightforward\n comparison of the complete CharStream\n states can be quite expensive, the CharStream class provides a shortcut for this purpose: the StateTag.\n

\n
\n
\n

\n The StateTag is a\n simple integer counter that is incremented every time a CharStream method changes the state. Thus, if the StateTag hasn’t changed, you can safely infer\n that the state hasn’t changed either.[1] Except for some\n special cases, the opposite is also true: if the StateTag has changed, the state has changed too.\n

\n
\n
\n

\n In the following special cases checking whether the StateTag has changed is not equivalent to\n checking whether the CharStream state\n has changed, because the tag may change even though the state doesn’t:\n

\n
    \n
  • \n A parser calls the basic Skip or Read methods with a 0 offset or an empty argument string.\n
    • \n
  • \n A parser seeks the CharStream to the\n current position or replaces the user state with the current value.\n
    • \n
  • \n A parser makes several calls to CharStream methods and in later calls undoes the changes it made in earlier calls.\n
    • \n
\n
\n
\n

\n The first and second cases only have practical relevance for generic or parameterized parsers and can be simply avoided by checking the\n arguments before calling the respective CharStream methods. The third case only arises in the context of backtracking and it too can be easily avoided,\n either by using the BacktrackTo method for backtracking or by manually restoring the StateTag after the backtracking.\n

\n
\n
\n

\n In practice these special cases are extremely rare, usually without consequences for the parser behaviour and always easily avoidable. Hence,\n FParsec combinators make free use of the StateTag to check whether a parser has changed the CharStream state.\n

\n
\n
\n
\n
\n

\n 5.4.4 Generalizing >>.\n

\n
\n
\n

\n The parsers p1 .>> p2 and p1 >>. p2 only return the\n results of p1 and p2 respectively. If\n you want to combine the results from both p1 and p2, you could use the pipe2 combinator instead:\n

\n
val pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>
\n
\n
\n

\n The parser pipe2 p1 p2 f will apply p1 and p2 in sequence, exactly like >>., but instead of returning one of the result\n values of p1 and p2 it will return the\n result of the function application f x1 x2,\n where x1 and x2 are the results returned\n by p1 and p2.\n

\n
\n
\n

\n There are also pipe3, pipe4 and pipe5 combinators, in case you need more than two\n arguments. Often these combinators are used to pass arguments to object constructors, like in the following example of a parser for a\n comma‐separated list of XYZ coordinates:\n

\n
type Data = Point of float*float*float\n\nlet ws = spaces\nlet str s = pstring s .>> ws\nlet number = pfloat .>> ws\nlet point = pipe3 number (str \",\" >>. number) (str \",\" >>. number)\n                  (fun x y z -> Point(x, y, z))\n
\n
> run point \"1, 2, 3\";;\nval it : ParserResult<Data,unit> = Success: Point (1.0,2.0,3.0)\n
\n
\n
\n

\n If you just want to return the parsed values as a tuple, you can use the predefined tuple2-5\n parsers. For example, tuple2 p1 p2 is equivalent to pipe2 p1 p2 (fun x1 x2 -> (x1, x2).\n

\n
\n
\n

\n tuple2 is also available under the\n operator name .>>., so\n that you can write p1 .>>. p2 instead of tuple2 p1 p2.\n

\n
\n
\n

There is no pipe1 combinator, but there is an operator for the same purpose:

\n
val (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b, 'u>\n
\n
\n
\n

This operator is used similarly to the F#’s ubiquitous pipeline operator |>:

\n
type Expression = Number of int\n                | Identifier of string\n\nlet number = pint32 |>> Number\n
\n
> run number \"123\";;\nval it : ParserResult<Expression,unit> = Success: Number 123\n
\n
\n
\n
\n
\n

\n 5.4.5 The >>= combinator\n

\n
\n
\n

\n All the sequencing and piping combinators we have discussed so far could be implemented with the help of the “bind” combinator:\n

\n
val (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>
\n
\n
\n

\n Instead of two parsers this combinator takes a parser and a function producing a parser as arguments. The combined parser p >>= f first applies the parser p to the input, then it applies the\n function f to the result returned by p\n and finally it applies the parser returned by f to the input. If we knew in advance that\n p returns x then p >>= f would be equivalent to p >>. (f x).\n

\n
\n
\n

\n The >>= combinator is\n quite versatile. For example, the following code implements five of the previously discussed combinators in terms of >>= and the trivial preturn primitive:\n

\n
\n
\n
let preturn x = fun stream -> Reply(x)\n\nlet (|>>) p  f    = p  >>= fun x -> preturn (f x)\nlet (.>>) p1 p2   = p1 >>= fun x -> p2 >>= fun _ -> preturn x\nlet (>>.) p1 p2   = p1 >>= fun _ -> p2 >>= fun y -> preturn y\nlet (.>>.) p1 p2  = p1 >>= fun x -> p2 >>= fun y -> preturn (x, y)\nlet pipe2 p1 p2 f = p1 >>= fun x -> p2 >>= fun y -> preturn (f x y)\n
\n
\n
\n

\n In typical FParsec code >>= is only seldomly used, because in many situations where >>= could in principle be used one of the\n other specialized operators is more convenient to use and faster. However, on a conceptual level this combinator is important, because its\n generality allows us to define and test many combinators through their equivalence with a parser defined in terms of >>=. This combinator is also significant for\n the role it plays in the monadic parser construction syntax, see section 5.10.\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n Of course, this doesn’t apply if you manually set back the StateTag to the old value. There is also the\n purely theoretical possibility that the StateTag has overflown and was incremented exactly 264 times (or 232 if you define the SMALL_STATETAG conditional compiler symbol).\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/customizing-error-messages.html", "content": "\n\n\n \n Customizing error messages\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
8Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Customizing error messages\n \n
\n
\n

5.8 Customizing error messages

\n
\n
\n

\n Generating relevant and informative parser error messages is one of FParsec’s greatest strengths. The top‐down approach of recursive‐descent\n parsing guarantees that there is always enough context to describe the exact cause of a parser error and how it could be avoided. FParsec\n exploits this context to automatically generate descriptive error messages whenever possible. This chapter explains how you can ensure with\n minimal efforts that your parser always produces understandable error messages.\n

\n
\n
\n

\n As we already described in detail in section 5.4.2, error reporting in\n FParsec is based on the following two principles:\n

\n
    \n
  • \n Parsers that fail or could have consumed more input return as part of their Reply an ErrorMessageList describing the input they expected or the reason they failed.\n
    • \n
  • \n Parser combinators aggregate all error messages that apply to the same input position and then propagate these error messages as appropriate.\n
    • \n
\n
\n
\n

\n The various error messages in the previous chapters demonstrate that the built‐in error reporting usually works quite well even without any\n intervention by the parser author. However, sometimes FParsec lacks the information necessary to produce an informative error message by itself.\n

\n
\n
\n

\n Consider for example the many1Satisfy f parser, which parses a string consisting of one or more chars satisfying the\n predicate function f. If this parser fails to parse at least one char, the generated error is\n not very helpful:\n

\n
> run (many1Satisfy isLetter) \"123\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\n123\n^\nUnknown Error(s)\n
\n
\n
\n

\n The problem here is that many1Satisfy can’t describe what chars the function predicate accepts. Hence, when you don’t use many1Satisfy as part of a combined\n parser that takes care of a potential error, you better replace it with many1SatisfyL, which allows you to describe the\n accepted input with a label (hence the “L”):\n

\n
\n
\n
> run (many1SatisfyL isLetter \"identifier\") \"123\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: identifier\n
\n
\n
\n

\n There are also labelled variants of other parsers and combinators, for example choiceL and notFollowedByL.\n

\n
\n
\n

If there is no labelled parser variant or you want to replace a predefined error message, you can always use the labelling operator

\n
val (<?>): Parser<'a,'u> -> string -> Parser<'a,'u>\n
\n
\n
\n

\n The parser p <?> label behaves like p, except\n that the error messages are replaced with expectedError label if\n p does not change the parser state (usually because p failed).\n

\n
\n
\n

\n For example, if FParsec didn’t provide many1SatisfyL, you could define it yourself as\n

\n
let many1SatisfyL f label = many1Satisfy f <?> label\n
\n
\n
\n

\n The labelling operator is particularly useful for producing error messages in terms of higher‐level grammar productions instead of error messages\n in terms of lower‐level component parsers. Suppose you want to parse a string literal with the following parser\n

\n
let literal_ = between (pstring \"\\\"\") (pstring \"\\\"\")\n                       (manySatisfy ((<>) '\"'))\n
\n
\n
\n

\n If this parser encounters input that doesn’t start with a double quote it will fail with the error message produced by the parser for the opening\n quote:\n

\n
> run literal_ \"123\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: '\"'\n
\n
\n
\n

In situations like these an error message that mentions the aggregate thing you’re trying to parse will often be more helpful:

\n
let literal = literal_ <?> \"string literal in double quotes\"\n
\n
> run literal \"123\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: string literal in double quotes\n
\n
\n
\n

\n Note that <?> only\n replaces the error message if the parser doesn’t consume input. For example, our literal\n parser won’t mention that we’re trying to parse a string literal if it fails after the initial double quote:\n

\n
> run literal \"\\\"abc def\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 9\n\"abc def\n        ^\nNote: The error occurred at the end of the input stream.\nExpecting: '\"'\n
\n
\n
\n

\n With the compound labelling operator <??> you can make sure that the compound gets mentioned even if the parser fails after consuming input:\n

\n
let literal = literal_ <??> \"string literal in double quotes\"\n
\n
> run literal \"\\\"abc def\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\n\"abc def\n^\nExpecting: string literal in double quotes\n\nstring literal in double quotes could not be parsed because:\n  Error in Ln: 1 Col: 9\n  \"abc def\n          ^\n  Note: The error occurred at the end of the input stream.\n  Expecting: '\"'\n
\n
\n
\n
\n
Tip
\n
\n
\n

\n If you don’t like the formatting of these error messages, you can write a custom formatter for your application. The data structure in which\n error messages are stored is easy to query and process. See the reference for the Error module.\n

\n
\n
\n
\n
\n
\n

\n The parsers we discussed so far in this chapter only generated Expected error messages, but FParsec also supports other type of error messages. For example, the notFollowedByL parser generates\n Unexpected error messages:\n

\n
> run (notFollowedByL spaces \"whitespace\") \" \";;\nval it : ParserResult<unit,unit> = Failure:\nError in Ln: 1 Col: 1\n\n^\nUnexpected: whitespace\n
\n
\n
\n

\n Error messages that don’t fit into the Expected and Unexpected categories can be produced with the fail and failFatally primitives:\n

\n
\n
\n
let theory =\n    charsTillString \"3) \" true System.Int32.MaxValue\n     >>. (pstring \"profit\" <|> fail \"So much about that theory ... ;-)\")\n\nlet practice = \"1) Write open source library 2) ??? 3) lot's of unpaid work\"\n\n
\n
> run theory practice;;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 40\n1) Write open source library 2) ??? 3) lot's of unpaid work\n                                       ^\nExpecting: 'profit'\nOther error messages:\n  So much about that theory... ;-)\n
\n
\n
\n

\n If you can’t get the built‐in operators and parsers to produce the error message you need, you can always drop down one API level and write a\n special‐purpose parser combinator.\n

\n
\n
\n

\n The following example shows how you can define a custom between combinator that includes the position of the opening delimiter as part of the error message that gets\n generated when the closing delimiter cannot be parsed.\n

\n
\n
\n
let betweenL (popen: Parser<_,_>) (pclose: Parser<_,_>) (p: Parser<_,_>) label =\n  let expectedLabel = expected label\n  let notClosedError (pos: Position) =\n     messageError (sprintf \"The %s opened at %s was not closed.\"\n                           label (pos.ToString()))\n  fun (stream: CharStream<_>) ->\n    // The following code might look a bit complicated, but that's mainly\n    // because we manually apply three parsers in sequence and have to merge\n    // the errors when they refer to the same parser state.\n    let state0 = stream.State\n    let reply1 = popen stream\n    if reply1.Status = Ok then\n      let stateTag1 = stream.StateTag\n      let reply2 = p stream\n      let error2 = if stateTag1 <> stream.StateTag then reply2.Error\n                   else mergeErrors reply1.Error reply2.Error\n      if reply2.Status = Ok then\n        let stateTag2 = stream.StateTag\n        let reply3 = pclose stream\n        let error3 = if stateTag2 <> stream.StateTag then reply3.Error\n                     else mergeErrors error2 reply3.Error\n        if reply3.Status = Ok then\n          Reply(Ok, reply2.Result, error3)\n        else\n          Reply(reply3.Status,\n                mergeErrors error3 (notClosedError (state0.GetPosition(stream))))\n      else\n        Reply(reply2.Status, reply2.Error)\n    else\n      let error = if state0.Tag <> stream.StateTag then reply1.Error\n                  else expectedLabel\n      Reply(reply1.Status, error)\n
\n
\n
\n

\n The behaviour of the betweenL combinator differs from that of the standard between combinator in two ways:\n

\n
    \n
  • \n If popen fails without changing the parser state, betweenL popen p pclose label fails with expected label, just like between popen p pclose <?> label would have.\n
    • \n
  • \n If pclose fails without changing the parser state, betweenL additionally prints the opening position of the compound.\n
    • \n
\n
\n
\n

The following tests demonstrate this behaviour:

\n
\n
\n
let stringLiteral = betweenL (str \"\\\"\") (str \"\\\"\")\n                             (manySatisfy ((<>) '\"'))\n                             \"string literal in double quotes\"\n
\n
> run stringLiteral \"\\\"test\\\"\";;\nval it : ParserResult<string,unit> = Success: \"test\"\n\n> run stringLiteral \"\\\"test\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 6\n\"test\n     ^\nNote: The error occurred at the end of the input stream.\nExpecting: '\"'\nOther messages:\n  The string literal in double quotes opened at (Ln: 1, Col: 1) was not closed.\n\n> run stringLiteral \"test\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\ntest\n^\nExpecting: string literal in double quotes\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/debugging-a-parser.html", "content": "\n\n\n \n Debugging a parser\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
11Debugging a parser
\n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n Setting a breakpoint
2Tracing a parser
\n
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Debugging a parser\n \n
\n
\n

5.11 Debugging a parser

\n
\n
\n

\n Debugging a parser implemented with the help of a combinator library has its special challenges. In particular, setting a breakpoint and stepping\n through the code is not as straightforward as in a regular recursive descent parser. Furthermore, stack traces can be difficult to decipher\n because of the ubiquitous use of anonymous functions.[1] However, with the help of\n the techniques we explain in this chapter, working around these issues should be easy.\n

\n
\n
\n
\n

5.11.1 Setting a breakpoint

\n
\n
\n

Suppose you have a combined parser like

\n
let buggyParser = pipe2 parserA parserB (fun a b -> ...)
\n

\n and you would like to break into the debugger whenever buggyParser calls parserB. One thing you could try is to set a breakpoint at the beginning of parserB. However, that’s only possible if parserB is not itself a combined parser, and even then you still have the problem that your breakpoint is also\n triggered whenever parserB is called from any other place in your source. Similarly, a\n breakpoint you set in pipe2 will\n probably be triggered by many other parsers besides buggyParser.\n

\n
\n
\n

Fortunately there’s a simple workaround if you can modify and recompile the code. Just define a wrapper function like the following

\n
let BP (p: Parser<_,_>) stream =\n    p stream // set a breakpoint here\n
\n
\n
\n

Then redefine the buggy parser as

\n
let buggyParser = pipe2 parserA (BP parserB) (fun a b -> ...)\n
\n
\n
\n

\n If you now set a breakpoint at the body of the BP function, it will be triggered whenever parserB is called from buggyParser.\n

\n
\n
\n

\n With such a wrapper it’s also easy define a precise conditional breakpoint. For example, if you only want to break once the parser has reached\n line 100 of the input file, you could use the breakpoint condition stream.Line >= 100.\n

\n
\n
\n

By the way, you don’t need to set the breakpoint in the debugger. You can also write it directly into the code:

\n
let BP (p: Parser<_,_>) (stream: CharStream<_>) =\n    // this will execute much faster than a\n    // conditional breakpoint set in the debugger\n    if stream.Line >= 100L then\n        System.Diagnostics.Debugger.Break()\n    p stream\n
\n
\n
\n
\n
Note
\n
\n
\n

\n There are some issues with setting breakpoints in or stepping into anonymous or curried F# functions in Visual Studio 2008. In Visual Studio\n 2010 many of these issues have been fixed.\n

\n
\n
\n

\n If you’re using Visual Studio, don’t forget to switch on the “Suppress JIT optimization on module load” option in the Tools –\n Options – Debugging – General dialog. And, when possible, use a debug build (of FParsec) for debugging.\n

\n
\n
\n
\n
\n
\n
\n
\n

5.11.2 Tracing a parser

\n
\n
\n

\n Occasionally you have a parser that doesn’t work as expected and playing around with the input or staring at the code long enough just isn’t\n enough for figuring out what’s wrong. In such cases the best way to proceed usually is to trace the execution of the parser. Unfortunately,\n stepping through the parser under a debugger can be quite tedious, because it involves stepping through long sequences of nested invocations of\n parser combinators. A more convenient approach often is to output tracing information to the console or a logging service.\n

\n
\n
\n

A simple helper function for printing trace information to the console could like the following example:

\n
let (<!>) (p: Parser<_,_>) label : Parser<_,_> =\n    fun stream ->\n        printfn \"%A: Entering %s\" stream.Position label\n        let reply = p stream\n        printfn \"%A: Leaving %s (%A)\" stream.Position label reply.Status\n        reply\n
\n
\n
\n

To demonstrate how you could use such a tracing operator, let’s try to debug the following buggy (and completely silly) parser:

\n
\n
\n
let number = many1Satisfy isDigit\n\nlet emptyElement = pstring \"[]\" : Parser<_,unit>\nlet numberElement = pstring \"[\" >>. number .>> pstring \"]\"\nlet nanElement = pstring \"[NaN]\"\n\nlet element = choice [emptyElement\n                      numberElement\n                      nanElement] .>> spaces\n\nlet elements : Parser<_,unit> = many element\n
\n
\n
\n

The following test run shows that the above parser is indeed buggy:

\n
> run elements \"[] [123] [NaN]\";;\nval it : ParserResult<string list,unit> = Failure:\nError in Ln: 1 Col: 11\n[] [123] [NaN]\n          ^\nUnknown Error(s)\n
\n
\n
\n

\n You probably don’t need trace information to figure out why the \"NaN\" bit of the string doesn’t get parsed, but let’s pretend you do. Obviously, there’s something wrong with the\n element parser. To find out what’s wrong, let’s decorate the element parser and all subparsers with the <!> operator and\n an appropriate label:\n

\n
\n
\n
let number = many1Satisfy isDigit <!> \"number\"\n\nlet emptyElement  = pstring \"[]\"                           <!> \"emptyElement\"\nlet numberElement = pstring \"[\" >>. number .>> pstring \"]\" <!> \"numberElement\"\nlet nanElement    = pstring \"[NaN]\"                        <!> \"nanElement\"\n\nlet element = choice [emptyElement\n                      numberElement\n                      nanElement] .>> spaces <!> \"element\"\n\nlet elements  : Parser<_,unit> = many element\n
\n
\n
\n

If you now run the parser on the same input as before, you get the following output:

\n
> run elements \"[] [123] [NaN]\";;\n(Ln: 1, Col: 1): Entering element\n(Ln: 1, Col: 1): Entering emptyElement\n(Ln: 1, Col: 3): Leaving emptyElement (Ok)\n(Ln: 1, Col: 4): Leaving element (Ok)\n(Ln: 1, Col: 4): Entering element\n(Ln: 1, Col: 4): Entering emptyElement\n(Ln: 1, Col: 4): Leaving emptyElement (Error)\n(Ln: 1, Col: 4): Entering numberElement\n(Ln: 1, Col: 5): Entering number\n(Ln: 1, Col: 8): Leaving number (Ok)\n(Ln: 1, Col: 9): Leaving numberElement (Ok)\n(Ln: 1, Col: 10): Leaving element (Ok)\n(Ln: 1, Col: 10): Entering element\n(Ln: 1, Col: 10): Entering emptyElement\n(Ln: 1, Col: 10): Leaving emptyElement (Error)\n(Ln: 1, Col: 10): Entering numberElement\n(Ln: 1, Col: 11): Entering number\n(Ln: 1, Col: 11): Leaving number (Error)\n(Ln: 1, Col: 11): Leaving numberElement (Error)\n(Ln: 1, Col: 11): Leaving element (Error)\nval it : ParserResult<string list,unit> = Failure:\nError in Ln: 1 Col: 11\n[] [123] [NaN]\n          ^\nUnknown Error(s)\n
\n
\n
\n

\n This trace log clearly reveals that the element parser failed because the numberElement parser failed after consuming the left bracket and thus the choice parser never got to try the the nanElement parser. Of course, this issue could be easily avoided by factoring out the bracket\n parsers from the emptyElement, numberElement and nanElement parsers. Also, if we had used many1SatisfyL instead of manySatisfy for the number parser, we would have gotten an error message more descriptive than “Unknown\n error(s)” (see the chapter on customizing error messages).\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n Although, debugging a parser written with a combinator library is often still easier than debugging one generated by an opaque parser generator\n tool.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/index.html", "content": "\n\n\n \n User’s Guide\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide\n \n
\n
\n

5 User’s Guide

\n
\n
\n

\n
\n
\n

\n This user’s guide is an in‐depth introduction to parsing with FParsec. It explains how Parser functions work, covers the most important parser\n combinators in detail, explains how you can customize error messages, and discusses some important practical aspects of parser writing, such as\n debugging and performance optimizations.\n

\n
\n
\n

\n The aim of this user’s guide is to prepare you for writing “real world” parser applications with FParsec. It doesn’t try to cover\n every feature of the library, but focuses on covering the core concepts such that you can gain a deep understanding of the library design.\n

\n
\n
\n

\n Although there is some overlap between the tutorial and this user’s guide, it’s probably a good idea to read the\n tutorial first, since it will give you a quick overview of the library that will later help you put things into perspective. You might also want\n to experiment with some small parsers before you start reading the user’s guide, or maybe in parallel to reading it, so that it becomes easier\n for you to relate the dry technical discussions to exciting practical applications ☺\n

\n
\n
\n

\n The first seven chapters of this user’s guide build on each other. The remaining chapters are rather independent and can be read in any order.\n

\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
2Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
6Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
9Parsing with user state
10Where is the monad?
11Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/internals-of-a-simple-parser-function.html", "content": "\n\n\n \n Internals of a simple Parser function\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
3\n Internals of a simple Parser function\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1The code
2The Reply type
\n 3\n \n The parser state and the line and column count\n
\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Internals of a simple Parser function\n \n
\n
\n

\n 5.3 Internals of a simple Parser function\n

\n
\n
\n

\n In the beginning of this user’s guide we noted that asciiLower “parses” a lower case ASCII char and that skipString “skips” over a string, but we haven’t yet explained what it actually means for a Parser function to “parse”\n a letter or “skip” a string. That’s what we will do in this chapter. To explain how Parser functions work, we will discuss the implementation\n of a simple string parser. This also gives us the opportunity to explain some important details about the Reply and CharStream types.\n

\n
\n
\n
\n

5.3.1 The code

\n
\n
\n

The parser whose implementation we will discuss in this chapter is

\n
val stringReturn: string -> 'a -> Parser<'a,'u>\n
\n
\n
\n

\n Like skipString str the parser stringReturn str result skips over the string str, but it returns result as part of its reply value, instead of the unit value () that skipString str returns. This makes stringReturn a bit more general than skipString. Indeed, the two library parsers pstring and skipString are actually implemented with the help of stringReturn. For example, skipString str is defined as stringReturn str ().\n

\n
\n
\n

\n A simplified version[1] of the actual implementation of stringReturn in the library is\n

\n
let stringReturn str result : Parser<_,_> =                                // 1\n    checkStringContainsNoNewlineChar str \"pstring/skipString/stringReturn\" // 2\n    let error = expectedString str                                         // 3\n    fun stream ->                                                          // 4\n        if stream.Skip(str) then                                           // 5\n            Reply(result)                                                  // 6\n        else                                                               // 7\n            Reply(Error, error)                                            // 8\n
\n
\n
\n

\n Let’s start with the general structure of this implementation: We define a function stringReturn with two parameters that returns a function closure. The type annotation : Parser<_,_> on line 1 fixes\n the type of the returned function closure to Parser<'a,'u> and in particular constrains the type of its argument to CharStream<'u>. Remember, the type Parser<'a,'u> is simply an abbreviation for CharStream<'u> -> Reply<'a>, where 'a represents the result type and 'u the\n user state type.\n

\n
\n
\n

\n Implementing our parameterized parser as a function returning a parser closure allows us to factor out common setup work that only needs to be\n done once for every parser.[2] In this case we only need to\n check once (line 2) whether the string contains a newline char, i.e. '\\r' or '\\n', (we’ll explain below why this is necessary) and in line 3 we preconstruct the error message that is later used whenever the parser is applied and doesn’t find str in the input (we’ll write more about error messages in later chapters).\n

\n
\n
\n

\n The actual parsing logic is completely straightforward: On line 5 the parser calls the CharStream’s Skip method with the argument\n str. If the next chars in the stream match str, Skip advances the stream’s position by the length of the passed string and returns true; otherwise, it doesn’t change the position of the stream and returns false. Thus, if the string is skipped, the parser returns with a Reply value containing the result (line 6). Otherwise,\n it returns a Reply with the preconstructed error\n message (line 8).\n

\n
\n
\n
\n
\n

\n 5.3.2 The Reply type\n

\n
\n
\n

\n This is a good time to discuss the Reply type in a\n little more detail.\n

\n
\n
\n
type Reply<'TResult> = struct\n  new: 'TResult -> Reply<'TResult>\n  new: ReplyStatus * ErrorMessageList -> Reply<'TResult>\n  new: ReplyStatus * 'TResult * ErrorMessageList -> Reply<'TResult>\n\n  val mutable Status: ReplyStatus\n  /// If Status <> Ok then the Result value is undefined and may be null.\n  val mutable Result: 'TResult\n  val mutable Error: ErrorMessageList\nend\n
\n
\n
\n

\n Similar to a tuple, a Reply can be seen as an\n aggregate of it three fields: Status,\n Result and Error. The Status field contains a ReplyStatus enum value indicating whether the parser succeeded (Ok) or failed (Error or FatalError). By returning a FatalError instead of an Error a parser can signal that no error recovery should be\n tried (except through backtracking mechanisms, which we explain later). If the Status is Ok, the Result field contains the parser result; otherwise, its value\n is undefined (and null). The Error field holds a list of error messages in the form of\n an ErrorMessageList value. An empty ErrorMessageList is represented as a null value.\n

\n
\n
\n

\n The 1‐argument constructor we use in line 6 sets the status to Ok and the result value to result. The 2‐argument constructor we use in line 8 sets the status to Error and the error message to error. The Reply type also\n defines a 3‐argument constructor, which simply sets the fields to the respective argument values. The default valuetype constructor with 0\n arguments initializes the Reply value to Reply(Error, null).\n

\n
\n
\n

\n The error messages returned in the Reply value\n implicitly refer to the current stream position. Since the ErrorMessage values stored in the ErrorMessageList do not themselves contain an\n error position, they can only be interpreted together with the position of the CharStream as it is when the parser returns.\n

\n
\n
\n
\n
\n

5.3.3 The parser state and the line and column count

\n
\n
\n

\n Usually one CharStream<'u> instance is created per input file and all parser functions\n involved in parsing elements of the same file are passed the same CharStream instance. Since calling the methods of a CharStream may change its state, parser functions have to be\n careful about when and how they change the CharStream state, because it obviously may affect all parsers subsequently called.\n

\n
\n
\n

\n In the example above, stringReturn only advances the stream position when it succeeds. This makes it an atomic string parser,\n because it does not consume input if only the beginning of the argument string matches the input. Whether or not a parser consumes input before\n it fails has important implications for the error handling, as we will discuss later in this user’s guide.\n

\n
\n
\n

\n Except for the freely customizable UserState, all the mutable state information in the CharStream<'u> instance pertains to the location of the next char in the text stream. The most\n important element of the state is the char Index, which uniquely identifies the UTF‐16 char in the stream. In addition to the index of the next char, the\n CharStream also keeps track of char’s\n Line number and the index\n of the first char in the line, the LineBegin. By combining the Index and LineBegin we can calculate a Column. The CharStream’s Name serves as a description or identifier for\n the stream.\n

\n
\n
\n

\n Only the char index is strictly necessary for the core stream functionality. We also store the other pieces of state information in a CharStream<'u> instance because having all parser state information in one place reduces complexity\n and allows us to expose a more convenient API to Parser functions.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n The CharStream<'u>.State property returns a snapshot of all\n the mutable state components in the form of a CharStreamState<'u> value.\n

\n
\n
\n

\n The state information that is exposed through the CharStream<'u>.State property is\n all the state that is tracked by FParsec parsers, which is why we also refer to\n it as the parser state.[3]\n

\n
\n
\n
\n
\n
\n

\n Ideally, the CharStream class would\n keep track of the column and line count in a completely automated fashion. Ideally, the CharStream class would give the user a way to freely specify\n the recognized set of newline character sequences and all CharStream methods then would automatically detect such newlines in the input. Unfortunately, such a configuration\n option would be difficult to implement efficiently and would likely have a severe impact on performance (at least in comparison to the\n hard‐coded alternative, and with the current language and compiler support).\n

\n
\n
\n

\n Since the CharStream can’t provide\n automatic support for all possible notions of a newline, it exposes two sets of methods in its interface. One set provides the basic stream\n operations, such as skipping a certain number of UTF‐16 chars or matching a string with the stream content. These methods come without any\n automatic newline detection, but they offer optimal performance and give the user complete freedom to manually register any kind of newline. The\n other set of methods provides some frequently needed higher‐level text operations, such as skipping over a sequence of whitespace chars or\n reading a sequence of chars satisfying a given predicate function. These other methods automatically detect any of the 3 standard newline char\n sequences \"\\n\",\n \"\\r\\n\" and \"\\r\", because that’s the notion of a newline used by most text applications. In combination both sets of methods\n cover the needs of a majority of text parsers in a convenient and efficient manner.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n Maybe you wonder why we don’t just leave the line and column count completely to the user instead of complicating the CharStream API. The reason we keep track of a line count in\n the CharStream class is that most\n non‐trivial text‐parsing applications require a line count for error reporting purposes. Implementing it at a relatively low API level brings\n significant performance advantages and relieves higher‐level API users from constantly having to code around the special case of newline\n chars.\n

\n
\n
\n
\n
\n
\n

\n If you have a look at the reference documentation for CharStream, you’ll see that the CharStream methods that automatically detect newlines are easily discernible by their name. The Skip method we used in the\n above example does not belong to these methods, which is why we have to make sure in line 2 that the string\n doen’t contain any newlines. In practice one hardly ever uses a parser like stringReturn with a string containing a newline,\n hence lifting this restriction wouldn’t be worth the effort, especially since simple workarounds are available.[4]\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
[1]\n The library version is a bit more complicated because it contains optimized paths for argument strings with only 1 or 2 chars.\n
[2]\n Even parsers without a parameter, like e.g. asciiLower, are actually compiled as properties returning a new function object every time they are called. This is\n because the user state type variable makes asciiLower generic, while function values can only have a non‐generic type.\n
\n [3]\n \n Strictly speaking, a CharStream<'a> instance has a little more publically observable mutable\n state than the one that is also exposed through the State property. For example, the MinRegexSpace configuration parameter is\n not tracked in the State parameter. Another example is the value of the IndexOfLastCharPlus1 property\n which changes once the last char of the stream is detected. However, there shouldn’t be a reason that a parser needs to restore the old values\n of these properties upon backtracking, so we just treat these properties as constant and ignore them when we discuss the mutable CharStream state.\n
\n [4]\n \n For example, stringReturn \"str1\\nstr2\" result\n can be replaced with attempt (skipString \"str1\" >>. newline >>. stringReturn \"str2\" result).\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/looking-ahead-and-backtracking.html", "content": "\n\n\n \n Looking ahead and backtracking\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
7Looking ahead and backtracking
\n \n \n \n \n \n \n \n \n \n \n \n
1Backtracking
2Parser predicates
\n
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Looking ahead and backtracking\n \n
\n
\n

5.7 Looking ahead and backtracking

\n
\n

5.7.1 Backtracking

\n
\n
\n

\n Sometimes you need more than the default one token look‐ahead of <|>, either because it really can’t be\n avoided or because avoiding it would be too inconvenient. In those instances you can use one of the combinators attempt, >>?, .>>? or >>=? to force a parser to backtrack after\n an error.\n

\n
\n
\n

The attempt combinator

\n
val attempt: Parser<'a,'u> -> Parser<'a,'u>\n
\n

\n takes a parser as the argument and returns a wrapped parser that behaves exactly like the argument, except that if the argument parser fails\n with an output state different from the input state or with a fatal error, the wrapped parser will backtrack to the original input state and\n report a non‐fatal error.\n

\n
\n
\n

\n You can observe the effect of the attempt combinator in the following error message:\n

\n
> run (attempt (pstring \"a\" >>. pstring \"b\")) \"ac\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 1\nac\n^\n\nThe parser backtracked after:\n  Error in Ln: 1 Col: 2\n  ac\n   ^\n  Expecting: 'b'\n
\n
\n
\n

\n The next example demonstrates the effect of attempt on the choice combinator.\n

\n
\n
\n
let str s = pstring s\nlet ab = str \"a\" .>>. str \"b\"\nlet ac = str \"a\" .>>. str \"c\"\n
\n
\n
\n

\n Without attempt the following\n test produces an error:\n

\n
> run (ab <|> ac) \"ac\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 2\nac\n ^\nExpecting: 'b'\n
\n
\n
\n

\n By introducing attempt we allow\n the <|> combinator to\n recover from the error in the first branch:\n

\n
> run ((attempt ab) <|> ac) \"ac\";;\nval it : ParserResult<(string * string),unit> = Success: (\"a\", \"c\")\n
\n
\n
\n

\n Sometimes it can be a disadvantage that attempt will trigger backtracking after any error returned by the argument parser, no matter how much content the\n parser has consumed. Consider for example a parser like prefix >>. expr, where expr is a parser for a potentially large and deeply nested expression. If you wrap this parser\n with attempt then the wrapped\n parser will not only backtrack if an error occurs within the prefix or directly after the prefix, but also if it occurs anywhere in the\n expression. However, in most cases you only want the parser to backtrack if the error occurs directly after the prefix, not if the error occurs\n deeply inside the expression parser. For situations like this FParsec defines the >>?, .>>?, .>>.? and >>=? operators.\n

\n
\n
\n

\n The >>? combinator\n

\n
val (>>?):  Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n
\n

\n behaves like the >>.\n operator, except that p1 >>? p2 will backtrack to the beginning if p2 fails with a non‐fatal error and without changing the parser state, even if p1 has changed the parser state. Similarly, .>>?, .>>.? and >>=? behave like .>>, .>>. and >>=, except that they will backtrack to the\n beginning if the second parser fails with a non‐fatal error and without changing the parser state\n

\n
\n
\n

\n The following tests illustrate the differences between backtracking implemented via attempt and .>>.?.\n

\n
\n
\n
let bInBrackets = str \"[\" >>. str \"b\" .>> str \"]\"\n
\n
\n
\n

\n A test with attempt on the left\n side of <|>:\n

\n
> run ((attempt (str \"a\" .>>. bInBrackets)) <|> ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 2\na[B]\n ^\nExpecting: 'c'\n
\n
\n
\n

\n A test with attempt on both sides\n of <|>:\n

\n
> run ((attempt (str \"a\" .>>. bInBrackets)) <|> attempt ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 1\na[B]\n^\n\nThe parser backtracked after:\n  Error in Ln: 1 Col: 2\n  a[B]\n   ^\n  Expecting: 'c'\n\nThe parser backtracked after:\n  Error in Ln: 1 Col: 3\n  a[B]\n    ^\n  Expecting: 'b'\n
\n
\n
\n

\n A test with .>>.?\n instead of attempt on the left\n side of <|>:\n

\n
> run (str \"a\" .>>.? bInBrackets <|> ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 3\na[B]\n  ^\nExpecting: 'b'\n
\n
\n
\n

\n You can of course chain multiple of the >>? and .>>? operators to backtrack longer distances, like in prefix1 >>? prefix2 >>? p .>>? postfix.\n

\n
\n
\n
\n
Note
\n
\n
\n

\n When implementing backtracking parsers you should generally prefer the >>?, .>>? and .>>.? combinators to the attempt combinator, because the\n former combinators offer finer control over the exact backtracking behaviour and hence will often lead to better error reporting. Note\n however that neither can completely replace the other.\n

\n
\n
\n
\n
\n
\n

\n Backtracking combinators can also be useful when parsing sequences. In the chapter “Parsing sequences” we briefly discussed the\n following example:\n

\n
\n
\n
let ws = spaces\nlet str s = pstring s\nlet numberInBrackets = str \"[\" >>. pint32 .>> str \"]\" .>> ws\n
\n
> run (many numberInBrackets >>. str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 10\n[1] [2] [c]\n         ^\nExpecting: integer number (32-bit, signed)\n
\n
\n
\n

\n The problem here is that the argument parser to many fails after consuming input if it encounters a bracket that is not followed by a digit. If we decided that\n this is a defect of the parser as opposed to the grammar, we could fix it by simply replacing a >>. with >>?.\n

\n
\n
\n
let numberInBrackets = str \"[\" >>? pint32 .>> str \"]\" .>> ws\n
\n
> run (many numberInBrackets .>> str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2]\n
\n
\n
\n

\n A similar example is the sepEndBy1 combinator for parsing a sequence of one or more elements separated and optionally ended by a separator.\n If FParsec didn’t provide this combinator, you could define it yourself using many and >>?:\n

\n
let sepEndBy1_ p sep =\n    pipe2 p (many (sep >>? p)) (fun hd tl -> hd::tl) .>> opt sep\n
\n
\n
\n

\n The following tests show that our sepEndBy1 replacement works as expected:\n

\n
> run (sepEndBy1_ pint32 (str \";\")) \"1;2;3\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3]\n> run (sepEndBy1_ pint32 (str \";\")) \"1;2;3;\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3]\n
\n
\n
\n

\n Note however that in contrast to sepEndBy1_ the version of sepEndBy1 provided by FParsec doesn’t need to parse the\n separator twice when it terminates a sequence.\n

\n
\n
\n
\n
\n

5.7.2 Parser predicates

\n
\n
\n

\n The backtracking combinators allow you to “look ahead” by tentatively parsing input and then backtracking if an error occurs.\n However, they don’t allow you to conditionally parse the input with one parser depending on the success or failure of another parser. This is\n what the following two combinators are for:\n

\n
\n
\n
val followedBy:    Parser<'a,'u> -> Parser<unit,'u>\nval notFollowedBy: Parser<'a,'u> -> Parser<unit,'u>\n
\n
\n
\n

\n The parser followedBy p (notFollowedBy p) succeeds without changing the parser state if p succeeds (fails) when applied at the current position.\n

\n
\n
\n

For example, both the following parser definitions only parse positive integer literals without a leading zero:

\n
let p1 = followedBy    (satisfy ((<>) '0')) >>. pint32\nlet p2 = notFollowedBy (pstring \"0\")        >>. pint32\n
\n
\n
\n

\n Both definitions will correctly parse \"123\" and fail to parse \"01\":\n

\n
> run p1 \"123\";;\nval it : ParserResult<int32,unit> = Success: 123\n> run p1 \"01\";;\nval it : ParserResult<int32,unit> =  Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnknown Error(s)\n> run p2 \"123\";;\nval it : ParserResult<int32,unit> = Success: 123\n> run p2 \"01\";;\nval it : ParserResult<int32,unit> = Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnknown Error(s)\n
\n
\n
\n

\n While both parsers work as expected, the generated error messages aren’t very helpful. The problem is that followedBy and notFollowedBy can’t generate better error messages,\n because they don’t know what kind of input their argument parsers accept.[1] To improve the error messages you can either use the “labeled” combinator variants\n followedByL and notFollowedByL or you could use\n the labelling operator <?> that we will discuss in the next chapter.\n

\n
\n
\n

For example:

\n
\n
\n
> run (followedByL (satisfy ((<>) '0')) \"positive int w/o leading 0\" >>. pint32)\n      \"01\";;\nval it : ParserResult<int32,unit> =  Failure:\nError in Ln: 1 Col: 1\n01\n^\nExpecting: positive int w/o leading 0\n\n> run (followedBy (satisfy ((<>) '0')) >>. pint32 <?> \"positive int w/o leading 0\")\n      \"01\";;\nval it : ParserResult<int32,unit> = Failure:\nError in Ln: 1 Col: 1\n01\n^\nExpecting: positive int w/o leading 0\n\n> run (notFollowedByL (pstring \"0\") \"'0'\" >>. pint32) \"01\";;\nval it : ParserResult<int32,unit> = Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnexpected: '0'\n
\n
\n
\n

\n The parser notFollowedByL (pstring \"0\") \"'0'\" from the last example could actually be simplified to notFollowedByString \"0\", which uses the specialized parser predicate notFollowedByString. In table 6.1.9 you’ll find an overview of all available parser\n predicates.\n

\n
\n
\n

\n A frequent application for the notFollowedBy predicate are sequence parsers similar to many (notFollowedBy pEnd >>. p) .>> pEnd. If you are writing such a parser, you should check whether you can replace it with an application of one of the\n manyTill parsers.\n Please consult the reference for more details.\n

\n
\n
\n

\n Before we conclude this chapter we want to emphasize that you’re not limited to the built‐in (backtracking) combinators of FParsec. A great\n advantage of FParsec is the simplicity with which you can write custom combinators using the low‐level API.\n

\n
\n
\n

For example, you could define a combinator that backtracks if the result of the argument parser doesn’t satisfy a predicate function:

\n
\n
\n
let resultSatisfies predicate msg (p: Parser<_,_>) : Parser<_,_> =\n    let error = messageError msg\n    fun stream ->\n      let state = stream.State\n      let reply = p stream\n      if reply.Status <> Ok || predicate reply.Result then reply\n      else\n          stream.BacktrackTo(state) // backtrack to beginning\n          Reply(Error, error)\n
\n
\n
\n

With this combinator you could conveniently define a parser for positive ints:

\n
\n
\n
let positiveInt = pint32 |> resultSatisfies (fun x -> x > 0)\n                                            \"The integer must be positive.\"\n
\n
> run positiveInt \"1\";;\nval it : ParserResult<int32,unit> = Success: 1\n> run positiveInt \"-1\";;\nError in Ln: 1 Col: 1\n-1\n^\nThe integer must be positive.\n
\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n In the case of notFollowedBy p the problem is clear: notFollowedBy p fails if p succeeds and\n when p succeeds, p doesn’t generate an\n error message that notFollowedBy could reuse. In the case of followedBy p the situation is\n different: followedBy p fails if p fails, so followedBy could try to reuse the error messages\n generated by p. However, the error messages generated by the argument parser will in\n practice often not suffice to explain what kind of input is expected. So, for reasons of consistency and performance, followedBy doesn’t even try to reuse the error\n messages generated by the argument parser.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/parser-functions.html", "content": "\n\n\n \n Parser functions\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Parser functions\n \n
\n
\n

5.1 Parser functions

\n
\n
\n

\n An FParsec parser is a function that reads input from a text stream. When it succeeds, it returns a result value (e.g. a parsed number or an AST node); when it fails, it returns error messages describing what went wrong.\n

\n
\n
\n

\n The following type abbreviation from the Primitives module defines the basic type of parser function supported throughout the FParsec library:\n

\n
type Parser<'Result,'UserState> = CharStream<'UserState> -> Reply<'Result>
\n
\n
\n

\n As you can see from this definition, parser functions only accept a single argument: a CharStream<'UserState> instance. The CharStream class is FParsec’s specialized stream type for\n “text” streams, i.e. streams of Unicode chars. A CharStream can either be created directly from a string or it can be created from a file path or System.IO.Stream. In the latter cases the CharStream will take care of decoding the\n binary input into UTF‐16 chars, similar to what a System.IO.StreamReader does. What separates CharStream from the StreamReader and similar classes is that it comes with some advanced features that make it especially suitable for\n backtracking parser applications.\n

\n
\n
\n

\n We will discuss the purpose of the 'UserState type in more detail in later chapters. For now\n it’s enough to note that the user state is a user‐definable component of the CharStream state. If you don’t need a user state, you will\n normally define 'UserState to be unit.\n To save some key strokes and screen real estate, we usually abbreviate 'UserState as 'u.\n

\n
\n
\n

\n The Reply<'Result> value returned from a parser function is a a simple value type container for the\n parser result and possible error messages. It contains a status field indicating whether the parser succeeded or not, a field for the result\n value (of type 'Result) and a field with a possibly empty list of error messages. We will\n explain these fields in more details in section 5.3.\n

\n
\n
\n

\n A very basic example of a parser is the asciiLower parser from the CharParsers module:\n

\n
val asciiLower: Parser<char,'u>
\n

\n It parses any lower case ASCII char, i.e. any char in the range 'a' ‐ 'z',\n and, if successful, returns the parsed char as part of its reply.\n

\n
\n
\n

\n Many predefined parsers expect one or more parameter values as arguments. Take for instance the skipString function:\n

\n
val skipString: string -> Parser<unit,'u>
\n

It takes a string as an argument and returns a parser that skips over this (and only this) string in the input.

\n
\n
\n
\n
Note
\n
\n
\n

\n Implementing parser grammars with FParsec usually means composing parsers for higher‐level grammar rules from parsers for lower‐level rules.\n You start with simple parsers for the leaf nodes of your grammar and then work your way up step‐by‐step until you eventually obtain a parser\n for the complete grammar. The simple representation of parsers as functions makes this composition particularly easy and allows for a\n straightforward and intuitive implementation of the library primitives.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/parsing-alternatives.html", "content": "\n\n\n \n Parsing alternatives\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
6Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Parsing alternatives\n \n
\n
\n

5.6 Parsing alternatives

\n
\n
\n

FParsec’s main operator for trying to parse input with alternative parsers is

\n
val (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>\n
\n
\n
\n

\n This operator implements a form of prioritized choice: it only tries to parse input with the second parser if the first parser fails.\n

\n
\n
\n

The following example illustrates this behaviour:

\n
type Char = AsciiChar of char\n          | Char of char\n\nlet asciiLetter = asciiLetter |>> AsciiChar\nlet letter = letter |>> Char\n
\n
> run (asciiLetter <|> letter) \"a\";;\nval it : ParserResult<Char,unit> = Success: AsciiChar 'a'\n> run (letter <|> asciiLetter) \"a\";;\nval it : ParserResult<Char,unit> = Success: Char 'a'\n> run (asciiLetter <|> letter) \"ä\";;\nval it : ParserResult<Char,unit> = Success: Char 'ä'\n
\n
\n
\n

The prioritized choice also implies that FParsec doesn’t enforce a longest‐match rule like in regular expressions:

\n
> run (pstring \"a\" <|> pstring \"ab\") \"ab\";;\nval it : ParserResult<string,unit> = Success: \"a\"\n
\n
\n
\n

\n If you want to accept more than two alternatives, you can either chain multiple <|> operators, like in p1 <|> p2 <|> p3, or you can use the choice combinator, which accepts a sequence of parsers as\n the argument, like in choice [p1; p2; p3]. In both cases the argument parsers are tried from left to right until a parser succeeds.\n

\n
\n
\n

\n A good understanding of the <|> operator is important for productively working with FParsec, so let’s have a look at its implementation:\n

\n
\n
\n
let (<|>) (p1: Parser<'a,'u>) (p2: Parser<'a,'u>) : Parser<'a,'u> =\n    fun stream ->\n        let stateTag = stream.StateTag\n        let mutable reply = p1 stream\n        if reply.Status = Error && stateTag = stream.StateTag then\n            let error1 = reply.Error\n            reply <- p2 stream\n            if stateTag = stream.StateTag then\n                reply.Error <- mergeErrors reply.Error error1\n        reply\n
\n
\n
\n

\n As you can see, the parser p1 <|> p2 works as follows: First, it applies the parser p1 to the input stream. If p1 succeeds, the reply of p1 is returned. If p1 fails with a non‐fatal\n error (i.e. with the status Error,\n not FatalError) and without\n changing the parser state, the parser p2 is applied. If p2 does not change the parser state, the error messages from both parsers are merged. (We compare the StateTag values instead of\n the actual parser states for optimization reasons, see section 5.4.3.)\n

\n
\n
\n

\n The most important point to note here is that p1 <|> p2 will always\n return with the reply of p1 if p1 changes\n the parser state, even if p1 eventually fails. Remember that the stream position is part of\n the parser state, so if p1 fails after consuming input, p2 will not be applied. Since a parser usually consumes input as soon as it can accept at least one atomic token from\n the input, this means that p1 <|> p2 by default implements backtracking with only a “one token\n look‐ahead”.\n

\n
\n
\n

Consider the following example:

\n
let parserA = spaces >>. pstring \"a\"\nlet parserB = spaces >>. pstring \"b\"\nrun (parserA <|> parserB) \" b\";;\n
\n
> run (parserA <|> parserB) \" b\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 2\n b\n ^\nExpecting: 'a'\n
\n
\n
\n

\n The combined parser fails because parserA fails after consuming the whitespace, so that parserB never gets tried.\n

\n
\n
\n

Of course, this simple parser could be easily fixed by factoring out the common prefix:

\n
> run (spaces >>. (pstring \"a\" <|> pstring \"b\")) \" b\";;\nval it : ParserResult<string,unit> = Success: \"b\"\n
\n
\n
\n

\n The restriction of the look‐ahead in p1 <|> p2 may strike you\n as odd at first, but it has two big advantages:\n

\n
    \n
  1. \n The error reporting is simplified and error messages are easier to understand because terminal errors can only occur at one position at a time.\n
    2. \n
  2. \n Parser developers are guided towards more efficient grammar implementations because parsers requiring more than a one token look‐ahead need to\n be explicitly annotated with the attempt or >>? combinators (see the next chapter).[1]\n
    3. \n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n In case you’re wondering: No, we’re not trying to sell a design limitation as a feature here. In Parsec, the Haskell library on which FParsec’s\n design was originally based, the limited look‐ahead is essential for the library design, because it allows Parsec to exploit Haskell’s laziness\n in order to ensure space efficiency. FParsec has a different implementation in which the limited look‐ahead has no effect on space efficiency. We stick to the limited look‐ahead because\n we think it’s the appropriate default behaviour for a parser combinator library like FParsec. Now, admittedly, if FParsec could automatically\n optimize the implementation of a parser in a way that minimized backtracking, e.g. by automatically left‐factoring grammars, then backtracking\n would be less of a problem and a different default behaviour might become more attractive.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/parsing-sequences.html", "content": "\n\n\n \n Parsing sequences\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1The many parser
\n 2\n \n sepBy and sepEndBy\n
\n 3\n Parsing a sequence without creating a list
\n
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Parsing sequences\n \n
\n
\n

5.5 Parsing sequences

\n
\n
\n

\n In the previous chapter we discussed various ways to sequentially apply two or more parsers. In this section we will explain how to repeatedly\n apply the same parser in order to parse a sequence with an arbitrary number of elements.\n

\n
\n
\n
\n

\n 5.5.1 The many parser\n

\n
\n
\n

\n In regular expressions and many grammar formalisms a Kleene Star marks a parser rule\n that can be repeatedly applied. For example, number* could represent\n a sequence of zero or more numbers.\n

\n
\n
\n

\n In FParsec the many combinator takes\n the place of the Kleene Star:\n

\n
val many: Parser<'a,'u> -> Parser<'a list, 'u>
\n
\n
\n

\n With many the number example could\n be translated into the following FParsec code:\n

\n
let ws = spaces\nlet number = pint32 .>> ws\n
\n
> run (many number) \"1 2 3 4\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3; 4]\n
\n
\n
\n

\n The parser many p repeatedly applies the parser p until p fails, i.e. it “greedily” parses as many occurrences of p as possible. The results of p are returned as a list in the order\n of occurrence.\n

\n
\n
\n

\n At the end of a sequence parsed with many p the argument parser p must fail\n without consuming input (or changing the parser state in any other way). When p fails after\n consuming input, many p fails with the error returned by p.\n

\n
\n
\n

The following example illustrates this behaviour:

\n
\n
\n
let ws = spaces\nlet str s = pstring s\nlet numberInBrackets = str \"[\" >>. pint32 .>> str \"]\" .>> ws\n
\n
\n
\n

\n The many numberInBrackets parser successfully parses the first two numbers in this test run:\n

\n
> run (many numberInBrackets .>> str \"(c)\") \"[1] [2] (c)\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2]\n
\n
\n
\n

However, the same parser fails while trying to parse the 3rd number in this test run:

\n
> run (many numberInBrackets >>. str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult<string,unit> = Failure:\nError in Ln: 1 Col: 10\n[1] [2] [c]\n         ^\nExpecting: integer number (32-bit, signed)\n
\n
\n
\n

\n The many parser failed here because\n the numberInBrackets parser failed after consuming input. In the chapter on looking ahead and backtracking we’ll come back to this example and discuss how you can modify the\n numberInBrackets parser such that it fails without consuming input if an opening bracket is\n not followed by a number.[1]\n

\n
\n
\n

\n Since many p\n continues until p fails, you have to be a little careful not to supply an argument parser\n p that can succeed without consuming input. The following example shows what happens if you\n accidentally supply such an argument parser:\n

\n
\n
\n
> run (many (many digit .>> ws)) \"123 456\";;\nSystem.InvalidOperationException: (Ln: 1, Col: 8): The combinator 'many' was\napplied to a parser that succeeds without consuming input and without changing\nthe parser state in any other way. (If no exception had been raised, the\ncombinator likely would have entered an infinite loop.)\n   (... stack trace ...)\nStopped due to error\n
\n
\n
\n

\n The problem here is that many digit .>> ws will succeed\n without changing the parser state if it can’t parse any digits or trailing whitespace. Thus, if the combined parser hadn’t have thrown an\n exception, it would have entered an infinite loop at the end of the input.\n

\n
\n
\n

\n We can easily avoid the error in the last example by requiring the inner parser to consume at least one digit. Instead of many digit, which succeeds with an empty list if can’t parse\n any digits, we can use many1 digit, which fails if it can’t parse at least one digit:\n

\n
> run (many (many1 digit .>> ws)) \"123 456\";;\nval it : ParserResult<char list list,unit> =\n  Success: [['1'; '2'; '3']; ['4'; '5'; '6']]\n
\n
\n
\n

\n Before we continue, we should point out that an example like many1 digit is somewhat\n artificial, because you hardly ever want to parse digit chars into a list. If you want to parse numbers, one of the number parsers is usually the best way forward. If you actually need the individual\n chars, you normally need them as a string, not as a list.\n

\n
\n
\n
\n
Tip
\n
\n
\n

\n If you want to parse a sequence of chars, you should generally prefer one of the specialized string parsers.\n

\n
\n
\n
\n
\n
\n

\n If you just want to skip over a sequence and don’t need the list of parser results, you can use the optimized combinators skipMany or skipMany1.\n

\n
\n
\n
\n
\n

\n 5.5.2 sepBy and sepEndBy\n

\n
\n
\n

\n Often the elements of a sequence are separated by some separator. A convenient way to parse such sequences are the sepBy and sepEndBy combinators.\n

\n
\n
\n

\n sepBy p sep parses a sequence of p separated by sep and returns the results in a list. sepEndBy parses a sequence of p separated and optionally ended by sep.\n

\n
\n
\n

With these combinators you could for example define the following two parsers for a semicolon‐separated list of numbers in brackets:

\n
let str s = pstring s\nlet sepList    = between (str \"[\") (str \"]\") (sepBy    pint32 (str \";\"))\nlet sepEndList = between (str \"[\") (str \"]\") (sepEndBy pint32 (str \";\"))\n
\n
\n
\n

\n The sepList parser only accepts lists where the semicolons only occur between two numbers:\n

\n
> run sepList \"[]\";;\nval it : ParserResult<int32 list,unit> = Success: []\n> run sepList \"[1;2;3]\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3]\n> run sepList \"[1;2;3;]\";;\nval it : ParserResult<int32 list,unit> = Failure:\nError in Ln: 1 Col: 8\n[1;2;3;]\n       ^\nExpecting: integer number (32-bit, signed)\n
\n
\n
\n

The sepEndList parser also accepts a terminating semicolon:

\n
> run sepEndList \"[1;2;3]\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3]\n> run sepEndList \"[1;2;3;]\";;\nval it : ParserResult<int32 list,unit> = Success: [1; 2; 3]\n
\n
\n
\n

\n Like for the many combinator, there\n are also variants of the sepBy and\n sepEndBy parsers that require at\n least one element in the sequence and/or skip over a sequence without returning the results. Have a look at the parser overview.\n

\n
\n
\n
\n
\n

5.5.3 Parsing a sequence without creating a list

\n
\n
\n

\n If you want to parse a sequence and you don’t need the results as an F# list, you can avoid the allocation of a temporary list by defing a\n custom sequence parser using the inline helper methods Inline.Many and Inline.SepBy.\n

\n
\n
\n

\n For example, if you wanted to define a variant of many that parses the elements directly into a ResizeArray, i.e.\n a System.Collections.Generic.List, you could use the following definition:\n

\n
\n
\n
let manyRA p =\n  // the compiler expands the call to Inline.Many to an optimized sequence parser\n  Inline.Many(elementParser = p,\n              stateFromFirstElement = (fun x0 ->\n                                         let ra = ResizeArray<_>()\n                                         ra.Add(x0)\n                                         ra),\n              foldState = (fun ra x -> ra.Add(x); ra),\n              resultFromState = (fun ra -> ra),\n              resultForEmptySequence = (fun () -> ResizeArray<_>()))\n
\n
\n
\n

A test run:

\n
> run (manyRA (pint32 .>> spaces)) \"1 2 3\";;\nval it : ParserResult<System.Collections.Generic.List<int32>,unit> =\n  Success: seq [1; 2; 3]\n
\n
\n
\n

\n The reference documentation for the Inline class contains some more examples.\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n

\n many doesn’t automatically\n backtrack when the argument parser fails after changing the parser state for two reasons:\n

\n
    \n
  • \n In most situations automatic backtracking would only obscure error messages, because the reported input error was indeed severe and\n backtracking would only trigger secondary error messages that detract from the main error.\n
    • \n
  • \n In the few instances where you rely on backtracking behaviour you can easily introduce it using the combinators detailed in section 5.7. Marking the occasions where you rely on backtracking with these combinators makes\n your parser implementations easier to debug and optimize.\n
    • \n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/parsing-with-user-state.html", "content": "\n\n\n \n Parsing with user state\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
9Parsing with user state
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Overview
\n 2\n \n Recursive grammars with nesting restrictions\n
\n 3\n \n Parameterizing a parser through the user state\n
\n
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Parsing with user state\n \n
\n
\n

5.9 Parsing with user state

\n
\n
\n

\n Each CharStream<'u> holds a value of the freely definable user state type 'u. In previous chapters we just ignored the user state and always assumed 'u to be unit. In this section we finally get to discuss the\n purpose of the user state and how you can use it in your parsers.\n

\n
\n
\n
\n

5.9.1 Overview

\n
\n
\n

\n The user state allows you to introduce additional variables into the state tracked by FParsec parsers. It has the following two important\n properties:\n

\n
    \n
  • \n The user state is stored in the CharStream<'u> instance and hence\n associated with the input. It is not shared globally and not associated with particular parser instances. The same parser instances can be\n concurrently applied to different CharStream<'u> instances with\n different user state instances.\n
    • \n
  • \n The user state is tracked by FParsec parsers together with the input stream position. This means in particular that a parser restores the\n previous user state value when it backtracks.\n
    • \n
\n
\n
\n
\n
Important
\n
\n
\n

\n If you want changes to the user state to be undone during backtracking, you must change the user state by assigning a new value to the user\n state, not by mutating an existing user state value.\n

\n
\n
\n
\n
\n
\n

\n With the help of the user state you can implement context sensitive parsers, i.e. parsers whose behaviour not only depends on the immediate\n input but also on the context of the input. In general this works as follows:\n

\n
    \n
  1. You establish a context by defining variables in the user state.
    2. \n
  2. You update the context depending on the input by letting parsers update the user state.
    3. \n
  3. You parse input depending on the context by making the parser behaviour dependent on the user state variables.
    4. \n
\n
\n
\n

\n The user state is exposed through the UserState property of the CharStream<'u>. You can implement parsers using the low‐level API that directly access this property, or you can use the\n following parser primitives from the CharParsers\n module:\n

\n \n
\n
\n

\n The next section contains an example employing updateUserState to change the user state and userStateSatisfies to check for parser\n preconditions.\n

\n
\n
\n
\n
\n

5.9.2 Recursive grammars with nesting restrictions

\n
\n
\n

\n An important area of application for context sensitive parsers are recursive grammars where certain grammar elements cannot nest within others\n or where grammar elements need to be parsed differently depending on the nesting context.\n

\n
\n
\n

\n Consider for example a textual markup languages like HTML. Many such markup languages support various “inline tags” to annotate\n text in a paragraph. Usually these inline tags can nest arbitrarily, except for a few tags with special restrictions. One of these restrictions\n often is that hyperlinks must not contain hyperlinks, even though they can contain any other inline content. Other restrictions may apply to\n elements allowed in superscript text or footnotes. A convenient way to enforce such restrictions during parsing is to introduce variables into\n the user state that keep track of the nesting context. The following example demonstrates this approach.[1]\n

\n
\n
\n

The following parser for a tiny markup‐language employs the user state

\n
    \n
  1. to ensure that nested hyperlinks are not accepted and
    2. \n
  2. \n to parse potentially nested quotations between matching pairs of '\\'' or '\\\"' chars.\n
    3. \n
\n
\n
\n
open FParsec\n\ntype Element = Text of string\n             | Bold of Element list\n             | Italic of Element list\n             | Url of string * Element list\n             | Quote of char * Element list\n\ntype UserState =\n    {InLink: bool\n     QuoteStack: char list}\n    with\n       static member Default = {InLink = false; QuoteStack = []}\n\nlet ws    = spaces\nlet ws1   = spaces1\nlet str s = pstring s\n\nlet elements, elementsR = createParserForwardedToRef()\n\nlet text = many1Satisfy (isNoneOf \"<>'\\\"\\\\\") |>> Text\nlet escape = str \"\\\\\" >>. (anyChar |>> (string >> Text))\n\nlet quote (q: char) =\n  let pq = str (string q)\n\n  let pushQuote =\n      updateUserState (fun us -> {us with QuoteStack = q::us.QuoteStack})\n\n  let popQuote =\n      updateUserState (fun us -> {us with QuoteStack = List.tail us.QuoteStack})\n\n  let isNotInQuote =\n      userStateSatisfies (fun us -> match us.QuoteStack with\n                                    | c::_ when c = q -> false\n                                    | _ -> true)\n\n  isNotInQuote >>. between pq pq\n                           (between pushQuote popQuote\n                                    (elements |>> fun ps -> Quote(q, ps)))\n\n// helper functions for defining tags\n\nlet tagOpenBegin tag =\n    str (\"<\" + tag)\n    >>? nextCharSatisfiesNot isLetter // make sure tag name is complete\n    <?> \"<\" + tag + \"> tag\"\n\nlet tagOpen tag = tagOpenBegin tag >>. str \">\"\nlet tagClose tag = str (\"</\" + tag + \">\")\n\nlet tag t p f =\n    between (tagOpen t) (tagClose t)\n            (p |>> f)\n\nlet attributeValue =\n    ws >>. str \"=\" >>. ws\n    >>. between (str \"\\\"\") (str \"\\\"\")\n                (manySatisfy (isNoneOf \"\\n\\\"\"))\n\nlet attribute s = str s >>. attributeValue\n\nlet nonNestedTag tag pAttributesAndClose pBody f\n                 isInTag setInTag setNotInTag =\n    tagOpenBegin tag\n    >>. ((fun stream ->\n            if not (isInTag stream.UserState) then\n                stream.UserState <- setInTag stream.UserState\n                Reply(())\n            else // generate error at start of tag\n                stream.Skip(-tag.Length - 1)\n                Reply(FatalError,\n                      messageError (\"Nested <\" + tag + \"> tags are not allowed.\")))\n         >>. pipe2 pAttributesAndClose pBody f\n             .>> (tagClose tag >>. updateUserState setNotInTag))\n\n// the tags\n\nlet bold   = tag \"b\" elements Bold\nlet italic = tag \"i\" elements Italic\n\nlet url = nonNestedTag \"a\" (ws >>. attribute \"href\" .>> (ws >>. str \">\"))\n                       elements\n                       (fun url phrases -> Url(url, phrases))\n                       (fun us -> us.InLink)\n                       (fun us -> {us with InLink = true})\n                       (fun us -> {us with InLink = false})\n\n\nlet element = choice [text\n                      escape\n                      quote '\\''\n                      quote '\\\"'\n                      bold\n                      italic\n                      url]\n\ndo elementsR:= many element\n\nlet document = elements .>> eof\n
\n
\n> runParserOnString document UserState.Default \"\"\n    \"A \\\"'text' with 'nested \\\"<b>quotes</b>\\\"'.\\\"\";;\nval it : ParserResult<Element list,UserState> = Success:\n[Text \"A \";\n Quote\n   ('\"',\n    [Quote ('\\'',[Text \"text\"]); Text \" with \";\n     Quote ('\\'',[Text \"nested \"; Quote ('\"',[Bold [Text \"quotes\"]])]); Text \".\"])]\n\n> runParserOnString document UserState.Default \"\"\n    @\"<b>Text <i></i>with</b> <a href=\"\"url\"\">'link' but no \\<blink\\></a>\";;\nval it : ParserResult<Element list,UserState> = Success:\n[Bold [Text \"Text \"; Italic []; Text \"with\"]; Text \" \";\n Url(\"url\",\n     [Quote ('\\'',[Text \"link\"]); Text \" but no \"; Text \"<\"; Text \"blink\";\n      Text \">\"])]\n\n> runParserOnString document UserState.Default \"\"\n    \"<a href=\\\"url\\\"><a href=\\\"nested\\\">test</a></a>\";;\nval it : ParserResult<Element list,UserState> = Failure:\nError in Ln: 1 Col: 15\n<a href=\"url\"><a href=\"nested\">test</a></a>\n              ^\nNested <a> tags are not allowed.\n
\n
\n
\n
\n
\n

5.9.3 Parameterizing a parser through the user state

\n
\n
\n

\n The user state is also a good place to store parser configuration data that is specific to a “parser job”. For example, a compiler\n that processes multiple compilation units could put configuration data that is specific to the compilation unit, e.g. include paths, into the\n user state and then parse different compilation units with the same Parser instance, like in the following code:\n

\n
\n
\n
type CompilationUnitAST = (* ... *)\n\ntype UserState = {\n    IncludePaths = string list\n    (* ... *)\n}\n\nlet parser : Parser<CompilationUnitAST, UserState> = (* ... *)\n\nlet parseCompilationUnit file encoding includePaths (* ... *) =\n    let initialUserState = {IncludePaths = includePaths; (* ... *)}\n    runParserOnFile parser initialUserState file encoding\n
\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
\n [1]\n \n An alternative way to handle such restrictions at the parser level would be to define separate instances of the parser for each possible\n combination of restrictions, e.g. separate parsers for inline elements at the top level, for inline elements within hyperlinks, for elements\n within hyperlinks within superscript text and so on. However, with an increasing number of restrictions this approach quickly falls victim to\n the combinatorial explosion caused by the recursive nature of the involved parsers.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/performance-optimizations.html", "content": "\n\n\n \n Performance optimizations\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
12Performance optimizations
\n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n Performance guidelines
\n 2\n Low‐level parser implementations
\n
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Performance optimizations\n \n
\n
\n

5.12 Performance optimizations

\n
\n
\n

\n In the past, the relatively poor performance of parser combinator libraries has often been cited as the primary impediment to their more\n widespread adoption. For this reason optimal performance stood front and center as a design goal during the development of FParsec and a lot of\n effort has been spent on optimizing parsing speed. As a result, FParsec has become so fast that parsers implemented with FParsec often\n significantly outperform parsers created by parser generator tools like fslex & fsyacc.\n

\n
\n
\n

\n In general, a parser implemented in FParsec can get close to the performance of a hand‐optimized recursive‐descent parser written in C#. Due to\n the multi‐layered architecture of the FParsec API, you always have the option to fall back to the lower‐level API should a particular parser\n component implemented with the high‐level API turn out to be too slow. Hence, if you choose FParsec for implementing your parsers, you don’t have\n to worry that performance will become a reason for switching away from FParsec.\n

\n
\n
\n
\n

5.12.1 Performance guidelines

\n
\n
\n

If you strive for optimal performance in your parser applications, try to adhere to the following guidelines:

\n
\n
\n
Avoid backtracking
\n
\n
\n

\n Try to avoid backtracking where possible. Sometimes it’s already enough to factor out a common prefix from a parser expression to avoid\n backtracking, e.g. by transforming (prefix >>? p1) <|> (prefix >>? p2) to prefix >>. (p1 <|> p2). Some simple backtracking can also be avoided by\n parsing whitespace as trailing whitespace instead of leading whitespace.\n

\n
\n
\n

\n If you’re designing a programming or markup language, you should try to minimize the need for backtracking, both to simplify parsing and to\n avoid exponential worst‐case behaviour.\n

\n
\n
\n
Prefer specialized parsers
\n
\n
\n

\n FParsec provides a number of specialized parsers and combinators for various purposes. Using more specialized primitives instead of\n reimplementing them with generic combinators will often safe you time and improve parsing speed.\n

\n
\n
\n

In particular:

\n
    \n
  • \n Prefer the skip... variants of parsers and combinators if you\n don’t need the parser results.\n
    • \n
  • Parse whitespace with the built‐in whitespace parsers.
    • \n
  • Parse numbers with the built‐in number parsers.
    • \n
  • \n Prefer to parse strings with the many[1]Satisfy[2][L] parsers.\n
    • \n
  • \n Consider parsing unicode identifiers with the identifier parser.\n
    • \n
\n
\n
\n
Construct parsers once
\n
\n
\n

\n Constructing a parser can be relatively expensive in comparison to a single invocation of the parser. Hence, if you repeatedly apply the\n same parser, you should make sure that you construct the parser only once, either by preconstructing it at the beginning or by lazily\n constructing the parser and then caching it.\n

\n
\n
\n

Usually the place where parsers get inadvertently constructed more than once is inside closures.

\n
\n
\n

For example, if you have a local function like

\n
fun stream ->\n    let reply = (parser1 >>. parser2) stream\n    if reply.Status = Ok then // ...\n    else // ...\n
\n

\n you should avoid the repeated construction of parser1 >>. parser2 every\n time the closure is called by moving the construction outside of the closure, as in\n

\n
let parser = parser1 >>. parser2\nfun stream ->\n    let reply = parser stream\n    if reply.Status = Ok then //...\n    else // ...\n
\n
\n
\n

\n Also, you shouldn’t wrap a parser expression inside a function just to avoid F#’s value restriction if you can achieve the same goal with a\n type annotation. For example, you should not try to fix the compiler error in the first example of the tutorial chapter on F#’s value restriction by replacing\n

\n
let p = pstring \"test\"
\n

with

\n
let p stream = pstring \"test\" stream
\n
\n
\n
\n Avoid parse {...} expressions\n
\n
\n
\n

See Why the monadic syntax is slow.

\n
\n
\n
\n Avoid regex parsers\n
\n
\n
\n

\n The regex parser parses a\n string by applying a .NET regular expression to the input. Since .NET regular expressions are relatively slow, you should reserve the use of\n the regex parser for patterns\n that you can’t easily express with other FParsec parsers and combinators.\n

\n
\n
\n
\n Consider optimizing large choice\n parsers\n
\n
\n
\n

\n Formal grammars for programming languages or DSLs often have one or two grammar rules at their core that essentially just enumerate a long\n list of possible ways to form a statement or expression in that language. A straightforward FParsec implementation of such a grammar rule\n typically uses the choice\n combinator to combine a list of parsers for all the alternatives.\n

\n
\n
\n

\n Usually such an implementation with a large choice‐based parser will do just fine. However, if parsing performance is critical for your application,\n replacing a large choice\n parser with a custom‐made combinator can be an optimization with a high benefit‐cost ratio. The next section explains this optimization in\n more detail.\n

\n
\n
\n
\n
\n
\n
\n
\n
\n

5.12.2 Low‐level parser implementations

\n
\n
\n

\n FParsec’s high‐level API consists of its built‐in parsers and combinators in the Primitives and CharParsers module. The high‐level API allows you to easily construct\n parsers in a concise and rather declarative way. Usually you will author most of your parsers using the high‐level API, because that’s the most\n productive way to do it.\n

\n
\n
\n

\n However, sometimes you might find that a specific piece of parser functionality is a bit inconvenient to express through the high‐level API or\n that the high‐level implementation isn’t as fast as you had hoped for. In those situations it’s a great advantage that FParsec allows you to\n drop down to the low‐level API, so that you can implement your own special‐purpose parser and combinator primitives.\n

\n
\n
\n

\n We have already covered the basics of the low‐level API in the chapters on the internals of\n a simple parser function and applying parsers in sequence. In this section we will discuss\n some examples that demonstrate how you can use low‐level parser implementations for optimization purposes.\n

\n
\n
\n

\n One example of a parser implemented using the low‐level API is contained in the samples folder of the FParsec distribution in samples/FSharpParsingSample/FParsecVersion/parser.fs. It is a parser for an identifier string that is not identical with a\n keyword.\n

\n
\n
\n

\n The low‐level implementation uses another parser, identifierString, to parse an identifier\n string and then backtracks when the parsed string is a keyword:\n

\n
let identifier : Parser<string, unit> =\n    let expectedIdentifier = expected \"identifier\"\n    fun stream ->\n        let state = stream.State\n        let reply = identifierString stream\n        if reply.Status <> Ok || not (isKeyword reply.Result) then reply\n        else // result is keyword, so backtrack to before the string\n            stream.BacktrackTo(state)\n            Reply(Error, expectedIdentifier)\n
\n
\n
\n

The same parser could also be implemented with the high‐level API:

\n
let identifier =\n    attempt (identifierString\n             >>= fun str ->\n                     if not (isKeyword str) then preturn str\n                     else pzero) <?> \"identifier\"\n\n
\n
\n
\n

\n The high‐level version is a bit more concise, but whether it is also easier to understand is debatable. The low‐level version seems at least a\n bit more self‐explanatory and hence is probably more accessible to new FParsec users. Since the low‐level implementation is also significantly\n faster than the high‐level one, this is a good example for a parser that can be improved through a low‐level implementation.\n

\n
\n
\n

\n If you wanted to optimize the performance of the identifier parser even more, you could replace the identifierString parser invocation with direct calls to CharStream methods. However, whether the potential performance\n gain would be worth the loss in code modularity and maintainability is questionable. A more promising optimization often is to integrate the\n identifier parser into a higher‐level choice‐based parser, like it is done below in the last example of this section.\n

\n
\n
\n

\n choice parsers with long list of\n argument parsers are performance‐wise one of the weakest spots of FParsec’s high‐level API. As we noted in the previous section, formal grammars\n for programming languages or DSLs often have one or two grammar rules at their core that essentially just enumerate a long list of possible ways\n to form a statement or expression in that language. A straightforward implementation of such a grammar rule using the choice combinator yields only sub‐optimal performance,\n since the choice parser has no\n knowledge about its argument parsers and has to try one parser after another.\n

\n
\n
\n

\n This makes large choice‐based\n parsers an excellent optimization opportunity. With your knowledge about the parser grammar you can often narrow down the set of possible\n parsers just by peeking at the following one or two chars in the input. Having identified the set of possible parsers (often only consisting of\n one parser), you can then considerably speed up the dispatch to the right subparser.\n

\n
\n
\n

For example, take a look at the JSON‐value parser from the tutorial:

\n
choice [jobject\n        jlist\n        jstring\n        jnumber\n        jtrue\n        jfalse\n        jnull]\n
\n
\n
\n

\n If you look at the definitions for the argument parsers, you’ll see that in almost all cases one can decide which parser should handle the input\n just based on the next char in the input. Hence, we could replace the choice‐based parser with the following low‐level\n implementation:\n

\n
\n
\n
let error = expected \"JSON value\"\nfun (stream: CharStream<_>) ->\n    match stream.Peek() with\n    | '{' -> jobject stream\n    | '[' -> jlist stream\n    | '\"' -> jstring stream\n    | 't' when stream.Skip(\"true\")  -> Reply(JBool true)\n    | 'f' when stream.Skip(\"false\") -> Reply(JBool false)\n    | 'n' when stream.Skip(\"null\")  -> Reply(JNull)\n    | _ ->\n        let stateTag = stream.StateTag\n        let mutable reply = jnumber stream\n        if reply.Status = Error && stateTag = stream.StateTag then\n           reply.Error <- error\n        reply\n
\n
\n
\n

\n A drawback of such a low‐level implementation is that you have to be a bit careful not to overlook any of the possible grammar cases. This is\n why we applied the jnumber parser in the “catch‐all” case, so that we don’t\n depend on the precise grammar rules for numbers.\n

\n
\n
\n

\n You also need to consider how the low‐level implementation affects error messages. When a choice parser fails, it will generate an error message\n with the error messages from all the argument parsers it tried. This gives a human reader usually enough context to understand the error. For a\n low‐level implementation it can take a little more effort to ensure that the error messages for every case contain enough information about the\n grammar context. For example, in our implementation above we had to replace the default error message by jnumber with a custom one, so that the error message generated by the catch‐all case doesn’t create the impression that\n a JSON value can only be a number.\n

\n
\n
\n

\n By now it is probably obvious that a low‐level parser implementation can actually be quite simple to write, but that it also comes at a certain\n cost in terms of code modularity and maintainability. Having the option of a low‐level implementation can certainly be what saves a project in\n certain situations and should give you some peace of mind with regard to parser performance, but generally you should only consider it as a\n backup option for those cases where you really need it.\n

\n
\n
\n

\n The following example shows again how you can replace a choice‐based parser with a low‐level implementation, this time with a grammar that is a bit more representative of\n a typical programming language:\n

\n
\n
\n
type Expr = Number float\n          | LetBinding ...\n          | IfThenElse ...\n          | ...\n\ntype UserState = //...\n\ntype Parser<'result> = Parser<'result, UserState>\n\ntype Keyword = None = 0\n             | If   = 1\n             | Let  = 2\n             // ...\n\nlet stringToKeyword = createStaticStringMapping\n                          Keyword.None\n                          [\"if\", Keyword.If\n                           \"let\", Keyword.Let\n                           // ...\n                          ]\n\nlet str s = pstring s\n\nlet identifierString : Parser<string> = // ...\n\nlet identifierRest (id: string) : Parser<Expr> = ...\n\nlet number : Parser<Expr> = // ... (parser for floating-point number)\n\nlet ifThenElseRest   : Parser<Expr> = // ...\nlet letBindingRest   : Parser<Expr> = // ...\nlet exprInParensRest : Parser<Expr> = // ...\n\n// The parser after this comment is a replacement for\n//     let identifierStringButNoKeyword =\n//         (* implementation like identifier parser in the first example above *)\n//\n//     let identifier   : Parser<Expr> = identifierStringButNoKeyword\n//                                       >>= identifierRest\n//\n//     let ifThenElse   : Parser<Expr> = str \"if\"  >>. ifThenElseRest\n//     let letBinding   : Parser<Expr> = str \"let\" >>. letBindingRest\n//     let exprInParens : Parser<Expr> = str \"(\"   >>. exprInParensRest\n//\n//     let expr = choice [identifierStringNoKeyword\n//                        number\n//                        ifThenElse\n//                        exprInParens\n//                        // ...\n//                       ]\n//\nlet expr : Parser<Expr> =\n  fun stream ->\n    let stateTag = stream.StateTag\n    let reply = identifierString stream\n    if reply.Status = Ok then\n      match stringToKeyword reply.Result with\n      | Keyword.None -> identifierRest reply.Result stream\n      | Keyword.If   -> ifThenElseRest stream\n      | Keyword.Let  -> letBindingRest stream\n      // ...\n    elif reply.Status = Error && stateTag = stream.StateTag then // no identifier\n      match stream.Peek() with\n      | '(' -> stream.Skip(); exprInParensRest stream\n      | c when isDigit c -> number stream\n      // ...\n    else // error within identifier string\n      Reply(reply.Status, reply.Error)\n\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/running-parsers-on-input.html", "content": "\n\n\n \n Running parsers on input\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
2Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Running parsers on input\n \n
\n
\n

5.2 Running parsers on input

\n
\n
\n

\n While it is not difficult to construct a CharStream instance yourself, then apply a parser function to the CharStream, then interpret the returned Reply value and finally dispose the CharStream again, it takes less effort to instead use one of\n the several runParser... functions from the CharParsers module.\n

\n
\n
\n

\n Among the runParser... functions run is the most convenient for simple testing purposes:\n

\n
val run: Parser<'a, unit> -> string -> ParserResult<'a,unit>
\n
\n
\n

\n run applies the parser given as the\n first argument to a CharStream\n constructed from the string argument and then captures the return value as ParserResult value. The ParserResult type is a simple discriminated union\n that is a bit more convenient to interpret than the Reply values returned by Parser functions.\n

\n
\n
\n

For example:

\n
> run pint32 \"0xff\";;\nval it : ParserResult<int32,unit> = Success: 255\n\n> run pint32 \"0xgf\";;\nval it : ParserResult<int32,unit> = Failure:\nError in Ln: 1 Col: 3\n0xgf\n  ^\nExpecting: hexadecimal digit\n
\n
\n
\n

\n The text messages displayed in these examples after the = signs are the default string\n representations of the returned ParserResult values, just like they are printed in the F# Interactive console. The reference documentation describes the two union\n cases Success and Failure of the ParserResult type in more detail.\n

\n
\n
\n

\n run only supports parser functions\n with no user state, i.e. with a unit user state. If you want to test parsers that depend on a\n user state, you will need to use one of the other runParser...\n functions, e.g. runParserOnString. Please see the reference for more details on the runParser... functions.\n

\n
\n
\n

\n Note that the runParser... functions are primarily meant for the\n “end‐users” of parsers, i.e. those users that apply an aggregate parser on the content of a complete input stream. This is a\n situation different from the one where you implement a Parser function yourself. In the latter case you typically work directly with the input CharStream and output Reply values.\n

\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/tips-and-tricks.html", "content": "\n\n\n \n Tips and tricks\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
\n 10\n Where is the monad?
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parallel parsing
\n 2\n Dispatching parsers through a dictionary
3Memoizing parsers
\n 4\n Parsing F# infix operators
\n
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Tips and tricks\n \n
\n
\n

5.13 Tips and tricks

\n
\n
\n
\n
\n Contents\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parallel parsing
\n 2\n Dispatching parsers through a dictionary
3Memoizing parsers
4Parsing F# infix operators
\n
\n
\n
\n
\n

5.13.1 Parallel parsing

\n
\n
\n

\n If your parser grammar is suitable for parallel parsing, parallelizing the parser has the potential to dramatically accelerate parsing on\n multi‐core machines. In the following we will shortly discuss requirements and strategies for parallelizing an FParsec parser.\n

\n
\n
\n

For a parser grammar to be well suited for parallel parsing, the grammar and the typical input must satisfy the following two criteria:

\n
    \n
  • Parts of the input must be independently parseable, i.e. parts must be parseable without knowlege about the other parts.
    • \n
  • These parts must be large enough and easily enough identifiable within the total input.
    • \n
\n
\n
\n

\n Often, the easiest and most beneficial way to parallelize the parsing stage of an application is to parse multiple input files in parallel. In\n the simplest case you have multiple independent “compilation units” that can be parsed in parallel. This works even for C/C++,\n where a badly designed preprocesser generally makes efficient parsing quite hard. In many programming languages and markup languages you can\n also parse in parallel files that are “included”, “opened” or “imported” within source files. However,\n this usually only works if the language allows such includes only at well‐defined points in the grammar. In languages like C/C++, where the\n unstructured text content of other files can be included at essentially arbitrary positions in the source, parsing the included files in\n parallel is generally quite hard. (In C/C++ it’s even hard to avoid parsing the same file multiple times when it is included multiple times).\n

\n
\n
\n

\n If you’re dealing with large input files or very slow parsers, it might also be worth trying to parse multiple sections within a single file in\n parallel. For this to be efficient there must be a fast way to find the start and end points of such sections. For example, if you are parsing a\n large serialized data structure, the format might allow you to easily skip over segments within the file, so that you can chop up the input into\n multiple independent parts that can be parsed in parallel. Another example could be a programming languages whose grammar makes it easy to skip\n over a complete class or function definition, e.g. by finding the closing brace or by interpreting the indentation. In this case it\n might be worth not to parse the definitions directly when they are encountered, but instead to skip over them, push their text content\n into a queue and then to process that queue in parallel.\n

\n
\n
\n

Here are some tips for parallel parsing with FParsec:

\n
    \n
  • \n All FParsec parsers are thread‐safe and can be safely applied concurrently to different CharStream instances, as long as you don’t introduce mutable\n shared state yourself.\n
    • \n
  • \n CharStream instances are not\n thread‐safe and a single instance must not be accessed concurrently.\n
    • \n
  • \n However, you can call the CreateSubstream method to create a substream for a CharStream. A CharStream and its substreams can be safely accessed\n concurrently.\n
    • \n
  • \n If you want to parse multiple files in parallel, you should also create the CharStream instances in parallel, because the CharStream constructors that accept file\n paths or binary streams perform I/O operations that benefit from parallelization.\n
    • \n
  • \n If you parallelize your parser, consider introducing an option for switching off parallel execution, since debugging a multi‐threaded parser is\n harder than debugging a single‐threaded one.\n
    • \n
\n
\n
\n
\n
\n

5.13.2 Dispatching parsers through a dictionary

\n
\n
\n

\n A technique that is often useful for making a parser modular and easily extensible is to store Parser functions in dictionaries and then to delegate\n parsing to one of the Parser\n functions in the dictionary based on the input.\n

\n
\n
\n

\n For example, a parser for a markup language could be implemented by defining a generic tag parser that delegates the parsing of the tagged\n content to a specific parser for the respective tag name. The following code shows how this could be done:\n

\n
\n
\n
open FParsec\nopen System.Collections.Generic\n\n// For simplicity we don't define a full-blown markup language here,\n// just a parser for two simple non-recursive \"tags\" in square brackets.\n// The chapter on \"parsing with user state\" contains a slightly more developed\n// sample for a markup language, though without a dictionary-based tag parser.\n\ntype Tag = Bold of string\n         | Url of string * string\n\n// We store the tag parser dictionary in the user state, so that we can\n// concurrently parse multiple input streams with the same parser instance\n// but differerent tag dictionaries.\n\ntype TagParserMap = Dictionary<string,Parser<Tag,UserState>>\n\nand UserState = {\n        TagParsers: TagParserMap\n     }\n\nlet defaultTagParsers = TagParserMap()\n\nlet isTagNameChar1 = fun c -> isLetter c || c = '_'\nlet isTagNameChar = fun c -> isTagNameChar1 c || isDigit c\nlet expectedTag = expected \"tag starting with '['\"\n\nlet tag : Parser<Tag, UserState> =\n  fun stream ->\n    if stream.Skip('[') then\n        let name = stream.ReadCharsOrNewlinesWhile(isTagNameChar1, isTagNameChar,\n                                                   false)\n        if name.Length <> 0 then\n            let mutable p = Unchecked.defaultof<_>\n            if stream.UserState.TagParsers.TryGetValue(name, &p) then p stream\n            else\n                stream.Skip(-name.Length)\n                Reply(Error, messageError (\"unknown tag name '\" + name + \"'\"))\n        else Reply(Error, expected \"tag name\")\n    else Reply(Error, expectedTag)\n\nlet str s = pstring s\nlet ws = spaces\nlet text = manySatisfy (function '['|']' -> false | _ -> true)\n\ndefaultTagParsers.Add(\"b\", str \"]\" >>. text .>> str \"[/b]\" |>> Bold)\n\ndefaultTagParsers.Add(\"url\",      (str \"=\" >>. manySatisfy ((<>)']') .>> str \"]\")\n                             .>>. (text .>> str \"[/url]\")\n                             |>> Url)\n\nlet parseTagString str =\n    runParserOnString tag {TagParsers = TagParserMap(defaultTagParsers)} \"\" str\n\n
\n
> parseTagString \"[b]bold text[/b]\";;\nval it : ParserResult<Tag,UserState> = Success: Bold \"bold text\"\n\n> parseTagString \"[url=http://tryfsharp.org]try F#[/url]\";;\nval it : ParserResult<Tag,UserState> =\n  Success: Url (\"http://tryfsharp.org\",\"try F#\")\n\n> parseTagString \"[bold]test[/bold]\";;\nval it : ParserResult<Tag,UserState> = Failure:\nError in Ln: 1 Col: 2\n[bold]test[/bold]\n ^\nunknown tag name 'bold'\n
\n
\n
\n
\n
\n

5.13.3 Memoizing parsers

\n
\n
\n

\n If your parser implementation backtracks a lot when parsing typical inputs and as a result repeatedly applies some Parser functions at the same input position, it can be\n beneficial to memoize these Parser\n functions, i.e. cache their results for each input position.\n

\n
\n
\n

\n In the extreme case, memoization can mean the difference between linear and exponential\n execution times. In practice, FParsec is typically used for formal grammars that hardly require any extensive backtracking, so that memoization\n would usually only have a negative affect on performance.\n

\n
\n
\n

\n In situation where you really do need to memoize parsers, you can work with a generic memoize combinator like the one in the following example:\n

\n
\n
\n
open FParsec\nopen System.Collections.Generic\n\n// We need a place to store the cached parser results. Since we want parser\n// instances to be able to concurrently access different caches for different\n// input streams, we will use a user state variable for this purpose. Since we\n// don't want the backtracking to undo changes to the cache,  we will use a\n// mutable dictionary for this purpose.\n\ntype UserState = {\n        MemoCache: Dictionary<MemoKey, obj>\n        // ...\n    }\n\n// An entry in the MemoCache must be uniquely identified by its MemoKey. In this\n// example the MemoKey includes the stream index value and a reference to the\n// memoized parser instance. Should the result of a memoized Parser function in\n// your implementation also depend on the UserState value, you will have to\n// extend the MemoKey with a UserState member. Similarly, if you want to cache\n// results for more than one stream in the MemoCache, you'll have to extend the\n// MemoKey with an identifier for the stream.\n\nand [<CustomEquality; NoComparison>]\n    MemoKey = struct\n        new (parser: obj, stream: CharStream) =\n            {Parser = parser; Index = stream.Index}\n\n        val Parser: obj\n        val Index: int64\n\n        interface System.IEquatable<MemoKey> with\n            member t.Equals(other: MemoKey) =\n                t.Index = other.Index && t.Parser = other.Parser\n\n        override t.Equals(otherObj: obj) =\n            match otherObj with\n            | :? MemoKey as other ->\n                t.Index = other.Index && t.Parser = other.Parser\n            | _ -> false\n\n        override t.GetHashCode() = int32 t.Index\nend\n\n/// Returns a memoized version of the argument parser\nlet memoize (p: Parser<'a,UserState>) : Parser<'a,UserState> =\n    fun stream ->\n        let key = MemoKey(p, stream)\n        let memoCache = stream.UserState.MemoCache\n        let mutable boxedReply = null\n        if memoCache.TryGetValue(key, &boxedReply) then\n            boxedReply :?> Reply<'a>\n        else\n            let reply = p stream\n            memoCache.Add(key, box reply)\n            reply\n
\n
\n
\n
\n
\n

5.13.4 Parsing F# infix operators

\n
\n
\n

\n F# supports user‐definable infix operators whose precedence and associativity depend on the first chars of the operator name. For example, the\n F# spec states that operators that start with * are left‐associative, while operators that start with ** are right associative and have a higher precedence, so that 1*2*.3**4**.5 is parsed as ((1*2)*.(3**(4 **.5))).\n

\n
\n
\n

\n Since the precedence and associativity rules are fixed, you can parse F# expressions with a static operator precedence grammar, i.e. without\n having to reconfigure the parser when a new operator is defined in the parsed source code. However, it’s probably not immediately obvious how to\n do this with FParsec’s OperatorPrecedenceParser class (OPP), since the OPP normally expects all possible operators to be (individually)\n specified before they are used.\n

\n
\n
\n

\n The trick to supporting whole classes of operator names without having to reconfigure the OPP at run‐time is to shift part of the operator\n parsing to the after‐string‐parser, like in the following example:\n

\n
\n
\n
open FParsec\n\ntype Expr = InfixOpExpr of string * Expr * Expr\n          | Number of int\n\nlet ws  = spaces  // whitespace parser\n\nlet isSymbolicOperatorChar = isAnyOf \"!%&*+-./<=>@^|~?\"\nlet remainingOpChars_ws = manySatisfy isSymbolicOperatorChar .>> ws\n\nlet opp = new OperatorPrecedenceParser<Expr, string, unit>()\nopp.TermParser <- pint32 .>> ws |>> Number\n\n// a helper function for adding infix operators to opp\nlet addSymbolicInfixOperators prefix precedence associativity =\n    let op = InfixOperator(prefix, remainingOpChars_ws,\n                           precedence, associativity, (),\n                           fun remOpChars expr1 expr2 ->\n                               InfixOpExpr(prefix + remOpChars, expr1, expr2))\n    opp.AddOperator(op)\n\n// the operator definitions:\naddSymbolicInfixOperators \"*\"  10 Associativity.Left\naddSymbolicInfixOperators \"**\" 20 Associativity.Right\n// ...\n\n
\n
> run opp.ExpressionParser \"1*2*.3**4**.5\";;\nval it : ParserResult<Expr,unit> = Success\nInfixOpExpr\n  (\"*.\", InfixOpExpr (\"*\", Number 1, Number 2),\n         InfixOpExpr (\"**\", Number 3, InfixOpExpr (\"**.\", Number 4, Number 5)))\n
\n
\n
\n

\n If you use the after‐string‐parser in this manner for operators that can lead to operator conflicts in the input, e.g. non‐associative\n operators, then you also need to replace the default OperatorConflictErrorFormatter, since otherwise the default formatter may print truncated operator names:\n

\n
\n
\n
addSymbolicInfixOperators \"<\"  1 Associativity.None\n
\n
> run opp.ExpressionParser \"1 <= 2 <=. 3\";;\nval it : ParserResult<Expr,unit> = Failure:\nError in Ln: 1 Col: 9\n1 <= 2 <=. 3\n        ^\nThe infix operator '<' (precedence: 1, non-associative) conflicts with the\ninfix operator '<' (precedence: 1, non-associative) on the same line at column 3.\n
\n
\n
\n

An error formatter that prints the full operator names could look like the following:

\n
\n
\n
opp.OperatorConflictErrorFormatter <-\n  fun (pos1, op1, afterString1) (pos2, op2, afterString2) ->\n    let msg = sprintf \"The operator '%s' conflicts with the previous operator '%s' at %A.\"\n                       (op2.String + afterString2)\n                       (op1.String + afterString1) pos1\n    messageError msg\n
\n
> run opp.ExpressionParser \"1 <= 2 <=. 3\";;\nval it : ParserResult<Expr,unit> = Failure:\nError in Ln: 1 Col: 9\n1 <= 2 <=. 3\n        ^\nThe operator '<=.' conflicts with the previous operator '<=' at (Ln: 1, Col: 3).\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/html/users-guide/where-is-the-monad.html", "content": "\n\n\n \n Where is the monad?\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n \n \n \n \n \n \n \n \n \n \n \n
FParsec Documentation
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1About FParsec
2License
\n 3\n Download and installation
4Tutorial
5User’s Guide
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
1Parser functions
\n 2\n Running parsers on input
\n 3\n \n Internals of a simple Parser\n function\n
\n 4\n Applying parsers in sequence
5Parsing sequences
\n 6\n Parsing alternatives
\n 7\n Looking ahead and backtracking
\n 8\n Customizing error messages
\n 9\n Parsing with user state
10Where is the monad?
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n 1\n An example using the monadic syntax
\n 2\n How the monadic syntax works
3The term “monad”
\n 4\n Why the monadic syntax is slow
\n
\n 11\n Debugging a parser
\n 12\n Performance optimizations
13Tips and tricks
\n
6Reference
\n
\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n FParsec Documentation > User’s Guide > Where is the monad?\n \n
\n
\n

5.10 Where is the monad?

\n
\n
\n

\n If you have previously used Haskell’s Parsec library or an early version of FParsec you’re probably wondering by now where the “monadic\n syntax” has gone. There’s also a chance that you’ve stumbled upon FParsec while searching for a “monadic parser library” for\n F#/.Net and you’re now wondering whether FParsec actually is one.\n

\n
\n
\n

\n To answer these questions right away: FParsec supports a monadic parser construction syntax, but this syntax is only an optional feature, not the\n foundation of the library design. FParsec doesn’t use the monadic syntax internally and we no longer recommend using it for new parser projects\n when performance is a concern.\n

\n
\n
\n
\n

5.10.1 An example using the monadic syntax

\n
\n
\n

With the monadic syntax you can, for example, write a parser for a pair of floating‐point numbers as follows:

\n
\n
\n
open FParsec\n\nlet ws = spaces // whitespace parser\n\nlet str_ws str = parse {do! skipString str\n                        do! ws\n                        return ()}\n\nlet number_ws = parse {let! number = pfloat\n                       do! ws\n                       return number}\n\nlet pairOfNumbers = parse {do! str_ws \"(\"\n                           let! number1 = number_ws\n                           let! number2 = number_ws\n                           do! str_ws \")\"\n                           return (number1, number2)}\n
\n
\n
\n

\n We’ll explain how the F# compiler handles the parse {...} expressions in the next\n section. For now, just compare the previous implementation with the following one using the usual FParsec combinators:\n

\n
\n
\n
open FParsec\n\nlet ws = spaces // whitespace parser\n\nlet str_ws str = skipString str >>. ws\n\nlet number_ws = pfloat .>> ws\n\nlet pairOfNumbers = between (str_ws \"(\") (str_ws \")\")\n                            (tuple2 number_ws number_ws)\n
\n
\n
\n

\n The latter implementation is obviously more concise, but – at least for users without prior exposure to FParsec – the first implementation is\n probably a bit more intuitive and self‐explanatory. What makes the first implementation so intuitive is that the syntax of the parse {...} expressions is a) very close to what developers are used to from their normal work with F#\n and b) expressive enough that it obviates the need for many of FParsec’s basic combinators. Unfortunately, the intuitiveness of the monadic\n syntax comes at the price of a large performance penalty.\n

\n
\n
\n
\n
\n

5.10.2 How the monadic syntax works

\n
\n
\n

\n To explain how the monadic syntax works, we need to take a look at how the F# compiler translates the parse {...} expressions.\n

\n
\n
\n

\n The foundation for the monadic syntax is the >>= combinator introduced in section 5.4.5:\n

\n
val (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b>
\n
\n
\n

\n This operator takes a parser and a function returning a parser as arguments. The combined parser p >>= f first applies the parser p to the input, then it applies the\n function f to the result returned by p\n and finally it applies the parser returned by f to the input. As we exlained in section 5.4.5, this way to combine parsers is powerful enough that we can express\n many other sequencing combinators in terms of >>= and preturn.\n

\n
\n
\n

\n For example, we could implement the pipe3 combinator for sequentially applying three parsers as follows:\n

\n
\n
\n
let pipe3 p1 p2 p3 f =\n    p1 >>= fun x1 ->\n             p2 >>= fun x2 ->\n                      p3 >>= fun x3 ->\n                               preturn (f x1 x2 x3)\n
\n
\n
\n

\n Directly using the >>=\n and preturn combinators obviously\n leads to somewhat unwieldy and unreadable expressions. Fortunately, F#’s computation expressions allow us to rewrite this expression in a more\n intuitive way:\n

\n
\n
\n
let pipe3 p1 p2 p3 f =\n    parse {let! x1 = p1\n           let! x2 = p2\n           let! x3 = p3\n           return f x1 x2 x3}\n
\n
\n
\n

\n The parse object that we reference\n in this and other code snippets of this chapter is a so‐called “builder” object for computation expressions. It is defined in\n FParsec’s Primitives module. Using the methods of\n this object, the F# compiler translates the computation expression in the curly braces to the following equivalent expression:\n

\n
\n
\n
let pipe3 p1 p2 p3 f =\n    parse.Delay(fun () ->\n      parse.Bind(p1, fun x1 ->\n        parse.Bind(p2, fun x2 ->\n          parse.Bind(p3, fun x3 ->\n            parse.Return(f (x1 x2 x3))))))\n
\n
\n
\n

\n When we replace the parse object\n method calls with the respective method bodies, we will see that this definition is equivalent to our original definition using >>= and preturn.\n

\n
\n
\n

\n The Bind, Return and Delay methods of the parse object are defined as:\n

\n
    member t.Bind(p, f) = p >>= f\n    member t.Return(x) = preturn x\n    member t.Delay(f:(unit -> Parser<'a,'u>)) = fun stream -> (f ()) stream\n
\n
\n
\n

\n Substituting these method bodies into the previous expression yields an expression that is very similar to the original one (except for the\n additional indirection introduced by the Delay method[1]):\n

\n
\n
\n
let pipe3 p1 p2 p3 f =\n    fun stream ->\n      (p1 >>= fun x1 ->\n                p2 >>= fun x2 ->\n                         p3 >>= fun x3 ->\n                                  preturn (f x1 x2 x3)) stream\n
\n
\n
\n

\n In summary, the parse {...} syntax is syntactic sugar for defining parsers with the >>= operator. The\n expressiveness of this syntax stems from the power of the >>= operator.\n

\n
\n
\n
\n
\n

5.10.3 The term “monad”

\n
\n
\n

\n A function with a signature like the one of the >>= operator is often called “bind”. The above examples make it obvious why: the >>= combinator binds the\n result of the parser on the left‐hand side to the function argument on the right‐hand side.\n

\n
\n
\n

\n The Parser type together with the\n >>= and preturn operations constitute a monad, which is an abstraction in type theory that denotes this kind\n of combination of a generic type with associated bind and return operations.\n

\n
\n
\n

\n Discussing the theoretical background of monads would be outside the scope of this user’s guide. For our purposes it is enough to note that the\n monad abstraction is so useful for certain applications that F# comes with built‐in syntax support for monadic expressions. FParsec utilizes\n this language feature (computation expressions) to enable parse {...} expressions.\n

\n
\n
\n

\n Be assured that you don’t need to know anything about monads in general in order to use FParsec’s parse {...} expressions. To fully understand this feature all you need to know to is how the F# compiler translates parse {...} expressions into normal code.\n

\n
\n
\n

\n Besides let!, do! and return there are some more language constructs that are supported inside parse {...} expressions. Please refer to the reference documentation for\n more information.\n

\n
\n
\n
\n
\n

5.10.4 Why the monadic syntax is slow

\n
\n
\n

\n Compared to parsers implemented with only the usual FParsec operators and functions, parsers implemented with parse {...} expressions can be up to several times slower.\n

\n
\n
\n

\n The relatively bad performance can be directly attributed to the way parse {...} expressions are compiled. As you have seen above, a parse {...} expression is simply translated into a series of nested closures that are chained through calls to the >>= operator. With the\n current compiler technology and the current implementation of FParsec this introduces some significant overhead.\n

\n
\n
\n

\n Every time a Parser\n function constructed with the parse {...} syntax is called:\n

\n
    \n
  • \n Two function closures get newly instantiated for each invocation of the >>= operator: the closure that is passed as\n the second argument to >>= and the closure that is returned by >>=.\n
    • \n
  • \n Any parser created inside a parse {...} expression gets (re‐)created every time execution reaches that\n point in the expression.\n
    • \n
\n
\n
\n

\n In principle, you can avoid the overhead described in the second point by moving the construction of parser functions out of the parse {...} expression.\n

\n
\n
\n

For example, you can avoid the repeated construction of the skipString parsers in

\n
let numberInParens = parse {do! skipString \"(\"\n                            let! number = pfloat\n                            do! skipString \")\"\n                            return number}\n
\n

by rewriting the code as

\n
let parenOpen = skipString \"(\"\nlet parenClose = skipString \")\"\nlet numberInParens = parse {do! parenOpen\n                            let! number = pfloat\n                            do! parenClose\n                            return number}\n
\n
\n
\n

\n However, if you wanted to factor out any parser construction from a parse {...} expression, you’d also have to factor out any use of parser combinators, which would take away a lot from the\n attractiveness of the syntax.\n

\n
\n
\n

\n If performance is not that important for your application, you can just ignore that a parser like skipString \"(\" is repeatedly constructed,\n since its construction is relatively cheap. But if you do the same for parsers based on regex or anyOf, where the construction potentially involves some\n relatively expensive compilation or runtime code generation, you might be surprised just how slow your parsers can become.\n

\n
\n
\n

\n Because of the described performance issues, we recommend not to use parse {...} expressions and instead work with FParsec’s rich set of operators and other combinators. Not only does the\n operator‐based notation (which is used everywhere else in FParsec’s documentation) lead to faster parsers, it also allows for more concise\n parser code with a higher signal‐to‐noise ratio.\n

\n
\n
\n
\n
\n
Footnotes:
\n \n \n \n \n \n
[1]\n The computation expression specification does not require a Delay method. So, we could avoid\n the overhead associated with the additional indirection by removing the Delay method from\n the ParserCombinator\n class. However, this would make the behaviour of parse expressions somewhat counter‐intuitive, as the behaviour would differ from the behaviour of F#’s seq and async expressions.\n
\n
\n
\n
\n
\n
\n\n\n\n" }, { "path": "Doc/misc/removed-many-variants.fs", "content": "open FParsec\n\n// the following optimized variants of the many combinator were removed in the update to 0.9 of FParsec\n\nlet manyRev p = Inline.Many((fun x -> [x]), (fun xs x -> x::xs), (fun xs -> xs), p, resultForEmptySequence = fun () -> [])\nlet manyFold acc0 f p = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.Many((fun x -> optF.Invoke(acc0, x)), (fun acc x -> optF.Invoke(acc, x)), (fun acc -> acc), p, resultForEmptySequence = fun () -> acc0)\nlet manyReduce f altX p = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.Many((fun x0 -> x0), (fun x0 x -> optF.Invoke(x0, x)), (fun x0 -> x0), p, resultForEmptySequence = fun () -> altX)\n\nlet many1Rev p = Inline.Many((fun x -> [x]), (fun xs x -> x::xs), (fun xs -> xs), p)\nlet many1Fold acc0 f p = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.Many((fun x -> optF.Invoke(acc0, x)), (fun acc x -> optF.Invoke(acc, x)), (fun x -> x), p)\nlet many1Reduce f p = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.Many((fun x0 -> x0), (fun x0 x -> optF.Invoke(x0, x)), (fun x0 -> x0), p)\n\nlet sepByRev p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), (fun xs -> xs), p, sep, resultForEmptySequence = fun () -> [])\nlet sepByFold acc0 f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x -> optF.Invoke(acc0, x)), (fun acc _ x -> optF.Invoke(acc, x)), (fun acc -> acc), p, sep, resultForEmptySequence = fun () -> acc0)\nlet sepByReduce f altX p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x0 -> x0), (fun x0 _ x -> optF.Invoke(x0, x)), (fun x0 -> x0), p, sep, resultForEmptySequence = fun () -> altX)\n\nlet sepBy1Rev p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), (fun xs -> xs), p, sep)\nlet sepBy1Fold acc0 f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x -> optF.Invoke(acc0, x)), (fun acc _ x -> optF.Invoke(acc, x)), (fun acc -> acc), p, sep)\nlet sepBy1Reduce f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x0 -> x0), (fun x0 _ x -> optF.Invoke(x0, x)), (fun x0 -> x0), p, sep)\n\nlet sepEndByRev p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), (fun xs -> xs), p, sep, separatorMayEndSequence = true, resultForEmptySequence = fun () -> [])\nlet sepEndByFold acc0 f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x -> optF.Invoke(acc0, x)), (fun acc _ x -> optF.Invoke(acc, x)), (fun acc -> acc), p, sep, separatorMayEndSequence = true, resultForEmptySequence = fun () -> acc0)\nlet sepEndByReduce f altX p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x0 -> x0), (fun x0 _ x -> optF.Invoke(x0, x)), (fun x0 -> x0), p, sep, separatorMayEndSequence = true, resultForEmptySequence = fun () -> altX)\n\nlet sepEndBy1Rev p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), (fun xs -> xs), p, sep, separatorMayEndSequence = true)\nlet sepEndBy1Fold acc0 f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x -> optF.Invoke(acc0, x)), (fun acc _ x -> optF.Invoke(acc, x)), (fun acc -> acc), p, sep, separatorMayEndSequence = true)\nlet sepEndBy1Reduce f p sep = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.SepBy((fun x0 -> x0), (fun x0 _ x -> optF.Invoke(x0, x)), (fun x0 -> x0), p, sep, separatorMayEndSequence = true)\n\nlet manyTillRev p endp = Inline.ManyTill((fun x -> [x]), (fun xs x -> x::xs), (fun xs _ -> xs), p, endp, resultForEmptySequence = fun _ -> [])\nlet manyTillFold acc0 f p endp = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.ManyTill((fun x -> optF.Invoke(acc0, x)), (fun acc x -> optF.Invoke(acc, x)), (fun acc _ -> acc), p, endp, resultForEmptySequence = fun _ -> acc0)\nlet manyTillReduce f altX p endp = let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n Inline.ManyTill((fun x0 -> x0), (fun x0 x -> optF.Invoke(x0, x)), (fun x0 _ -> x0), p, endp, resultForEmptySequence = fun _ -> altX)\n" }, { "path": "Doc/src/changelog.txt", "content": "[section Changelog]\n[no-subsection-numbers]\n\n[section#v2_0 Version 2.0, 2022-11-01]\n- Dropped support for .NET Framework 4.5 and PCL and switched to using .NET 6.\n- Changed NuGet build to always enable code signing. [url \"https://github.com/stephan-tolksdorf/fparsec/pull/55\" Contributed] by Radek Krahl -- thanks Radek!\n- Norman Krämer [url \"https://github.com/stephan-tolksdorf/fparsec/pull/54\" fixed] an error in the `CharStream.Skip` documentation -- thanks Norman! \n- Nathan Adams [url \"https://github.com/stephan-tolksdorf/fparsec/pull/56\" fixed] a typo in the User's Guide -- thanks Nathan!\n- Theodore Tsirpanis [url \"https://github.com/stephan-tolksdorf/fparsec/pull/92\" optimized and cleaned-up] the codebase, taking advantage of new framework and langauge features and removing PCL support -- thanks Theodore!\n\n[/section]\n\n[section#v1_1_1 Version 1.1.1, 2020-02-01]\n- Fixed NuGet build to target the AnyCPU platform instead of the default platform of the build machine. Vadim Slynko and tpisciotta reported this issue -- thanks Vadim and tpisciotta! \n[/section]\n\n[section#v1_1 Version 1.1.0, 2020-01-05]\n- **Behaviour change**: `pfloat` now parses out-of-range finite values as plus or minus infinity instead of returning an error. This unifies the `pfloat` behaviour on all platforms after [url \"https://docs.microsoft.com/en-us/dotnet/core/compatibility/2.2-3.0#floating-point-parsing-operations-no-longer-fail-or-throw-an-overflowexception\" the behaviour change of `System.Double.Parse` on .NET Core 3]. \n- Enrico Sada [url \"https://github.com/stephan-tolksdorf/fparsec/pull/38\" modernized] the F# and C# project files and the build script for the NuGet packages -- thanks Enrico!\n- Added SourceLink support, which was prepared and championed by Cameron Taggart -- thanks Cameron!\n- Maxime Didier [url \"https://github.com/stephan-tolksdorf/fparsec/pull/20\" fixed] a bug in the Low-Trust version of the `CharStream` constructors that accept a file path argument: The stream's `Name` property wasn't initialized. Thanks Maxime!\n- Fixed missing parser definitions in the @Parsing JSON@ section of the tutorial spotted by Josh Quintus -- thanks Josh!\n- Andre Wesseling [url \"https://github.com/stephan-tolksdorf/fparsec/pull/18\" fixed] a parser definition in the @Where is the monad@ section of the User's Guide -- thanks Andre!\n- Frederik K. [url \"https://github.com/stephan-tolksdorf/fparsec/pull/28\" fixed] an error in the `ErrorMessage` documentation -- thanks Frederik!\n- Jonathan Roeber [url \"https://github.com/stephan-tolksdorf/fparsec/pull/40\" fixed] an error in the `previousCharSatisfiesNot` -- thanks Jonathan!\n- Vegard Løkken [url \"https://github.com/stephan-tolksdorf/fparsec/pull/42\" fixed] an error in the `unicodeSpaces` documentation -- thanks Vegard!\n\n[/section]\n\n[section#v1_0_3 Version 1.0.3, 2017-08-20]\n- Modern solution and project files for .NET Core and VS 2017 were added to the source folders. Huge thanks to [url \"https://github.com/neoeinstein\" Marcus Griep] for spearheading the effort to make FParsec .NET Standard compatible and contributing the new project and solution files!\n- The old build script for the NuGet packages was replaced by a PowerShell script that uses the new project files.\n- The FParsec NuGet package now contains assemblies for .NET Standard 1.6.\n- The non-netstandard assemblies of FParsec now reference the FSharp.Core 4.0.0.1 NuGet package, which should maximize compatibility when binding redirects aren't available.\n- A [url \"https://github.com/stephan-tolksdorf/fparsec/blob/master/.vscode/tasks.json\" [= .vscode/tasks.json]] file with some task definitions for Visual Studio Code was added.\n- The source repository was moved to GitHub.\n- Added a `stringsSepBy1` parser ([url \"https://github.com/stephan-tolksdorf/fparsec/pull/4\" contributed by Robin Munn] -- thanks Robin!).\n- Added a link to the [url \"http://dmitriyvlasov.ru/publication/fparsec-tutorial/\" Russian translation of the tutorial] by Dmitry Vlasov -- thanks Dmitry!\n- Fixed documentation typos. One was spotted by Brandon Dimperio, another by ZelteHonor -- thanks Brandon and ZelteHonor!\n- Renamed `CLR45` to `AGGRESSIVE_INLINING` to better match its purpose.\n[/section]\n\n[section#v1_0_2 Version 1.0.2, 2015-09-27]\n- replaced all uses of `[url \"https://msdn.microsoft.com/en-us/library/hz49h034.aspx\" Char.GetUnicodeCategory]` with `[url \"https://msdn.microsoft.com/en-us/library/h6sx68ke.aspx\" CharCodeInfo.GetUnicodeCategory]`, since the former may or may not track the current Unicode standard and the latter is the only one supported by the PCL API subset\n- updated the case folding, whitespace and XID property data tables to Unicode 8.0.0\n- added a PCL Profile 259 version to the FParsec NuGet package\n- removed the Silverlight, VS9 and VS10 solution files and the Mono Makefile\n- updated the Lex & Yacc version of the =FSharpParsingSample= to use the [url \"http://fsprojects.github.io/FsLexYacc/\" `FsLexYacc`] NuGet packages\n- fixed documentation typos (two were spotted by Francois Nardon and Patrick McDonald -- thanks Francois and Patrick!)\n[/section]\n\n[section#v1_0_1 Version 1.0.1, 2013-06-25]\n- The maintainership of the FParsec NuGet package(s) was handed over from Ryan Riley,\n Huw Simpson, Cameron Taggart and Khan Thompson to Stephan Tolksdorf. Thanks\n Ryan, Huw, Cameron and Khan for creating and maintaining the previous versions of the NuGet package!\n- FParsec now has two @NuGet packages@, built with a new fsx script\n- fixed a [url \"https://bitbucket.org/fparsec/main/pull-request/3/bug-fix-use-the-leaveopen-parameter-passed/diff\" bug in one of the `CharStream` constructors] (reported and patched by Andrew Smith -- thanks Andrew!)\n- added `USE_STATIC_MAPPING_FOR_IS_ANY_OF` and `UNALIGNED_READS` as default compilation options in the Visual Studio projects (the default options now match the ones used by the \"Big Data edition\" NuGet package)\n- some minor code tweaking / micro-optimizations\n- fixed some minor documentation issues\n[/section]\n\n[section#v1_0 Version 1.0.0, 2012-07-19]\n- disabled code generation in `isAnyOf`, `isNoneOf`, `anyOf`, `skipAnyOf`, `noneOf` and `skipNoneOf` by default (you can reenable it using the new `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option)\n- annotated some `CharStream` methods with the .NET 4.5 `AggressiveInlining` option (see the new `CLR45` compilation option)\n- updated case folding and XID property tables to Unicode 6.1.0\n- fixed two documentation typos (spotted by Rasmus Meldgaard and Kurt Schelfthout -- thanks Rasmus and Kurt!)\n[/section]\n\n[section#v0_9_2 Version 0.9.2, 2012-03-09]\n- fixed compilation in Visual Studio 11 Beta\n- added missing `ReturnFrom` member to `parse` builder object (reported by Kurt Schelfthout and Tomas Petricek -- thanks Kurt and Tomas!)\n- added workaround for .NET `ConsoleStream` [url \"https://bitbucket.org/fparsec/main/issue/23/reading-from-systemio__consolestream-hangs\" issue] (reported by Alexander Kahl -- thanks Alexander!)\n- set `AllowPartiallyTrustedCallers` and `SecurityTransparent` assembly attributes in LOW_TRUST NET4 build (as suggested by hammett -- thanks hammett!)\n- changed encoding of [= FParsecCS/Strings.cs] to UTF-8 (with signature) to fix Visual Studio build on machines with Japanese locale (the encoding issue was reported on [url \"http://d.hatena.ne.jp/ZOETROPE\"] -- thank you!)\n- fixed some documentation issues (incorporating feedback from Alexander Gelkin, Antoine Latter and Stephen Swensen -- thanks Alexander, Antoine and Stephen!)\n- add link to the [url \"http://blog.livedoor.jp/gab_km/archives/1437534.html\" Japanese translation of the tutorial] by Gab_km (thanks Gab_km!)\n[/section]\n\n[section#v0_9_1 Version 0.9.1, 2011-05-22]\n- added [= /nooptimizationdata] compiler flag as a workaround for an F# comiler [url \"https://bitbucket.org/fparsec/main/issue/16/fparsec-needs-nooptimizationdata-to-avoid\" issue] (reported by Michael Giagnocavo -- thanks Michael!)\n- fixed an [url \"https://bitbucket.org/fparsec/main/issue/17/json-parser-fails-to-build\" issue] in the JSON sample (reported by Ryan Riley -- thanks Ryan!)\n- fixed the error message formatting when an error line contains unaccounted newlines or ends with a combining character sequence\n- added warning to [^building-fparsec-with-mono installation notes] that the `regex` parser doesn't work on Mono (reported by Laurent Le Brun -- thanks Laurent!)\n- fixed some documentation issues (one of which was reported by Michael Giagnocavo -- thanks Michael!)\n[/section]\n\n[section#v0_9 Version 0.9.0, 2011-04-26]\n\n- @Highlights@\n- @Changes to high-level API@\n- [@ Removed variants of `many`, `sepBy`, `sepEndBy` and `manyTill`]\n- [@ Details on changes to `manyChars`, `manyCharsTill` and their variants]\n- @Changes to low-level API@\n- @Background on low-level API changes@\n\n[dl\n[ #Highlights#]\n[\n- a new @tutorial@ and @user's guide@\n- 2x performance improvements due to a refactored low-level API\n- new `identifier` parser for parsing identifiers based on Unicode XID syntax\n- new `StaticMapping` module for compiling static key to value\n mappings into optimized functions (supports `char`, `int` and `string`\n as key types)\n]\n\n[[# Changes to high-level API]]\n[\n- the modules `FParsed.Primitives`, `FParsec.CharParsers` and `FParsec.Error` are now automatically opened when the `FParsec` namespace is opened\n- new combinators `.>>.`, `.>>.?`, `notEmpty`, `stringsSepBy`\n- new parsers `identifier`, `[^unicodeSpaces-parsers unicodeSpaces[1]]`, `notFollowedByEof`\n- `whitespace` and `unicodeWhitespace` has been removed\n- `unicodeNewline` no longer recognizes the form feed char `'\\f'` (`'\\u000C'`) as a newline character\n- some variants of `many`, `sepBy`, `sepEndBy`and `manyTill` [^removed-variants-of-many-sepby-sependby-and-manytill have been removed]\n- the `...FoldApply` inline variants of `many`, `sepBy`, `sepEndBy` and `manyTill`\n have been consolidated in the `[^reference.Primitives.members.Inline FParsec.Primitives.Inline]` helper class\n- sequence parsers now throw a `System.InvalidOperationException` instead of a `System.Exception` to prevent an infinite loop\n- `anyOf`, `noneOf`, `isAnyOf` and `isNoneOf` now use runtime code generation (except in the [^low-trust-version Low-Trust version]). *If you run into performance issues after upgrading to version 0.9*, make sure that you don't unnecessarily recreate `anyOf` or `noneOf` parsers, see [^construct-parsers-once here] and [^why-the-monadic-syntax-is-slow here].\n- `pstring`, `notFollowedByString` and similar parsers now have optimized code paths\n for argument strings with only 1 char\n- the behaviour of `manyChars` and `manyCharsTill` and their variants [^details-on-changes-to-manychars-manycharstill-and-their-variants has slightly changed]\n- the skip variants of `manyChars` and `manyCharsTill` [^removed-skip-variants-of-manyChars have been removed]\n- Some renamings and function signature changes:\n [table#renamings\n [[Old] [New]]\n [[`[no-auto-link restOfLine]`] [`restOfLine true`]]\n [[`[no-auto-link skipRestOfLine]`] [`skipRestOfLine true`]]\n [[`skipToEndOfLine`] [`skipRestOfLine false`]]\n [[`skipToString[CI] str n`] [`skipCharsTillString[CI] str false n`]]\n [[`[no-auto-link charsTillString][CI] str n`] [`charsTillString[CI] str true n`]]\n [[`[no-auto-link skipCharsTillString][CI] str n`] [`skipCharsTillString[CI] str true n`]]\n [[`followedByChar chr`]\n[``\nif chr = '\\r' || chr = '\\n' then followedByNewline\nelse followedByString (string chr)\n``]]\n\n [[`notFollowedByChar chr`]\n[``\nif chr = '\\r' || chr = '\\n' then notFollowedByNewline\nelse notFollowedByString (string chr)\n``]]\n [[`currentCharSatisfies f`] [`nextCharSatisfies f`]]\n [[`[no-auto-link nextCharSatisfies] f`] [`next2CharsSatisfy (fun _ c1 -> f c1)`]]\n ]\n\n- `OperatorPrecedenceParser` has changed:\n - all types have been moved from the `[no-auto-link FParsec.OperatorPrecedenceParser]` module into the main `FParsec` namespace\n - the operator types `InfixOp`, `PrefixOp`, ... classes have been renamed to `InfixOperator`, `PrefixOperator`, ...\n - `Assoc` has been renamed to `Associativity`\n - the \"whitespace-parser\" argument of the operator types has been [^Operator generalized into an \"after-string-parser\" argument]\n - if you previously used the `InfixOp'`, `PrefixOp'`, ... constructors to supply a mapping that read the text position of the parsed operator from the passed `[no-auto-link State]` instance, read [^get-position-with-after-string-parser this]\n - the `AddOperators` method has been removed, call `AddOperator` instead\n - the `OperatorConflictErrorFormatter` replaced the `OperatorConflictHandler`\n]\n\n\n\n[[# Removed variants of `many`, `sepBy`, `sepEndBy` and `manyTill`]]\n[\nThe `...Rev`, `...Fold` and `...Reduce` variants of `many`, `sepBy`, `sepEndBy`and `manyTill` have been removed.\n\nIf you previously used these variants, you can easily define them in your own code using the `|>>` combinator, as documented in the reference documentation for the previous version. For example:\n``\nlet manyRev p = many p |>> List.rev\nlet manyFold acc0 f p = many p |>> List.fold f acc0\nlet manyReduce f defVal p = (many1 p |>> List.reduce f) <|>% defVal.\n``\n\nIf you need *optimized* implementations, you can define them using the new `Inline` helper class.\nThe file [url \"https://github.com/stephan-tolksdorf/fparsec/blob/master/Doc/misc/removed-many-variants.fs\" [= Doc/misc/removed-many-variants.fs]] contains optimized definitions for all removed variants.\n]\n\n\n[[# Details on changes to `manyChars`, `manyCharsTill` and their variants]]\n[\nThe behaviour of all variants of `manyChars` and `manyCharsTill` has slightly changed. Now\n`manyChars cp` is equivalent to `many cp`, except that it returns a string instead of char list. Previously, `manyChars cp` behaved like `many (attempt cp)`, i.e. it automatically backtracked if the char parser had failed after consuming input. The same change has been made to the behaviour of all other variants of `manyChars` and `manyCharsTill`. The new behaviour is more consistent with the rest of the libary and allows a faster implementation with the new low-level API.\n\nThere probably aren't many parsers that relied on the old behaviour.\n\nThe behaviour change made the `skip` variants of `manyChar` and `manyCharsTill` obsolete, since e.g. `skipManyChars` would do exactly the same as `skipMany cp`. [#removed-skip-variants-of-manyChars Hence, the `skip` variants have been removed].\n]\n\n[[# Changes to low-level API]]\n[\n- The old `[no-auto-link CharStream]` and `[no-auto-link State]` classes have been merged into a single `CharStream` class with a mutable interface.\n- `Parser` functions now take a `[^CharStream_1 CharStream<'u>\\ ]` instance as the input argument.\n- The `Reply` type has been moved to the main `FParsec` namespace and no longer has a `[no-auto-link State]` member.\n- Parser state comparisons are now done with the help of the `CharStream`'s `StateTag`.\n- Various methods from the old `[no-auto-link CharStream.Iterator]` and `[no-auto-link State]` types have been renamed in the new `CharStream` class and have new signatures:\n - When you adapt old code, the following changes **require particular attention**:\n - the old `[no-auto-link Iterator.Read]` methods *did not* advance the input stream position, but the new `CharStream.Read` now *do* (as is the expected behaviour in traditional stream classes)\n - the old `[no-auto-link Read]()` is equivalent to the new `Peek()`\n - the old `[no-auto-link Read](int)` is equivalent to the new `PeekString(int)`\n - the old `[no-auto-link Peek]()` is equivalent to the new (and old) `[^Peek_int Peek](1)`\n - More renamings:\n - `Next`, `Advance` ? `[^Skip-members Skip]`\n - `_Increment`, `_Decrement` ? `[^SkipAndPeek-members SkipAndPeek]`\n - `[no-auto-link SkipRestOfLine]` with string output ? `ReadRestOfLine`\n - `SkipCharOrNewlines` with string output ? `ReadCharsOrNewlines`\n - `SkipToString` ? `[^SkipCharsOrNewlinesUntilString-members SkipCharsOrNewlinesUntilString]`\n - `SkipToStringCI` ? `[^SkipCharsOrNewlinesUntilCaseFoldedString-members SkipCharsOrNewlinesUntilCaseFoldedString]`\n - `ReadUntil` ? `ReadFrom`\n - `[no-auto-link CharStream.FoldCase]` ? `Text.FoldCase`\n - `[no-auto-link CharStream.NormalizeNewlines]` ? `Text.NormalizeNewlines`\n- New `CharStream` methods:\n - `[^Skip-members Skip]` methods with char and string arguments\n - `SkipUnicodeWhitespace`\n - `SkipNewlineThenWhitespace`\n- The `ErrorMessage` and `ErrorMessageList` types are now defined in the C# library part. This allows us to implement full parsers in C#.\n The `FParsec.Error` module contains type abbreviations and active patters that provide the familiar interface to F# clients.\n- All error messages used by built-in FParsec parsers are now defined in the C# classes `FParsec.Strings` and `FParsec.Errors`. This should simplify customization and internationalization efforts.\n\n]\n\n[[# Background on low-level API changes]]\n[\nPreviously parsers were implemented as functions operating on an immutable parser state in the form of a `State` instance. A parser function received a `State` instance as the input and returned a `State` instance as part of its return value. Since `State` instances were immutable, a parser function had to create a new `State` instance to advance the input stream, e.g. by calling `state.Advance(2)`.\n\nThis architecture was motivated by the desire to provide an API as \"functional\" as possible, an API that shields users from the underlying imperative/mutable nature of input streams. When FParsec originally started as a relatively close port of Haskell's Parsec library, this design felt like a natural fit for a functional parser library. However, later, when FParsec moved away from its Parsec roots (to improve performance and provide more features), it became increasingly clear that the immutable `CharStream`-`State`-design was the main obstacle preventing FParsec from reaching the performance of hand-optimized recursive-descent parsers.\n\nInitial tests with some quick prototypes revealed that the allocation and garbage collection of temporary `State` instances took up to 50% or more of the run time of typical parsers -- even though the `State` class was already heavily optimized. These tests also indicated that consolidating the stream and state classes into a classical imperative stream class simplified the overall library implementation and made the library source code more accessible to new users.\n\nThe main drawback of the API change is that it requires modifications to practically all low-level parser code. Another drawback is that backtracking is slightly less convenient with the new low-level API (as the parser state has to be explicitly saved and restored, while previously one could just continue with an old state instance).\n\nSince FParsec's high-level API is only minimally affected by the change, the advantages seem to outweigh the costs.\n]\n\n\n]\n\n[/section]\n[no-auto-link]\n[section#v0_8_x Version 0.8.x, no release]\n[dl\n[New features/ improvements]\n[\n- case-insensitive matching with `pstringCI`, `charsTillStringCI`, etc.\n (using the Unicode 1-to-1 case folding mappings for chars in the BMP)\n- various new parsers and combinators, including `restOfLine`,\n `skipToString`, `manySatisfyMinMax`, `manyStrings`, `withSkippedString`\n- new functions `runParserOnSubstring` and `runParserOnSubstream`\n- various performance improvements\n- Silverlight support\n- F# 1.9.6.16 compatibility\n]\n\n[Design changes]\n[\n- standardized on a single input stream type (`FParsec.CharStream`)\n and a single concrete parser state type (`FParsec.State`)\n- refactored the `Reply<_,_>`, `ErrorMessage` and `ParserError` types:\n * error replies now also contain a complete `State`\n * whether a parser has changed the state is now determined by\n checking the input and the output state for equality, instead of testing\n the `Consumed` flag\n * replaced the `Reply<_,_>.Flags` with a `Status` field\n * replaced the various helper functions for constructing a `Reply` with\n three overloaded `Reply<_,_>` constructors (with different arities)\n- all char parsers are now \"newline aware\", i.e. they normalize\n any of the three standard newline representations (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`)\n to \"\\n\" and they properly increment the line count whenever they parse\n a newline; **this means that the behaviour of almost all char parsers has\n changed with regard to how newline chars are handled**\n]\n[Bug fixes]\n[\n- The `CharStream` class now uses the serialization API to persist the decoder\n state for backtracking purposes. Previously it relied on the decoder loosing\n its state at block boundaries after a certain sequence of method calls. The\n previous approach works in practice for the .NET decoders of the standard\n unicode encodings and for simple stateless encodings like ASCII and ANSI,\n but it relies on undocumented behaviour and it does not work reliably for\n encodings like GB18030, ISO-2022 or ISCII.\n- In previous FParsec versions the `CharStream` file path/System.IO.Stream\n constructors failed with an `IndexOutOfRange` exception when the file/stream\n was empty and encoding detection was not turned off\n (reported by Vesa Karvonen - thanks Vesa!).\n- In previous FParsec versions the `NumberLiteral.String` returned by the\n `numberLiteral` parser included parsed suffix chars despite the\n documentation claiming the opposite. (The testing code was buggy too.)\n Applications that rely on this behaviour can now use the new\n `NumberLiteralOptions.IncludeSuffixCharsInString` to force the\n `numberLiteral` parser to include any suffix chars in the returned string.\n- Fixed behaviour of `>>=?`, `>>?` and `.>>?` when second parser fails with fatal error without changing the parser state.\n- Fixed behaviour of `nextCharSatisfies[Not]` when current \"char\" is a `\"\\r\\n\"` newline.\n]\n\n[Other breaking changes]\n[\n- renamed the module `CharParser` to `CharParsers`\n- moved `CharParser.OperatorPrecedenceParser` into separate module\n\n- `FParsec.Primitives`:\n * **subtle change:** renamed `message` to `fail` and `fail` to `failFatally`\n * renamed `pair`, `triple` and `quad` to `tuple2`, `tuple3` and `tuple4`\n * renamed `manyFoldLeft` to `manyFold` and changed the\n argument order of the accumulator and function argument\n * removed `manyFoldRight`\n * renamed `count` to `parray` and changed the return type,\n renamed `skipCount` to `skipArray`\n * renamed `followedBy` and `notFollowedBy` to `followedByL` and\n `notFollowedByL` and introduced `followedBy` and `notFollowedBy` functions\n that take no second argument\n * moved `ParserResult<_>` to `CharParsers` and changed constructor arguments\n * removed applyParser\n * removed `|>>=`, now `>>=` automatically uses an optimized branch for\n uncurried functions\n * removed `endBy` and `endBy1` (`endBy p sep` can be replaced with\n `many (p .>> sep)` and `endBy1 p sep` with `many1 (p .>> sep)`)\n\n- `FParsec.CharParsers`:\n * renamed `manyTillString` to `charsTillString`\n * removed `applyParser` from the public interface\n * removed `getIndex`, `skip`, `registerNL`, `extract`, `regexp`\n (these low-level operations should be done directly through the\n `State<_>`/`CharStream.Iterator` interface)\n * removed `anyCharOrNL` (no longer needed, see design changes above)\n * removed `nSatisfy` (can be replaced with `manySatisfyMinMax`)\n * removed `unicodeDigit` and `unicodeNumber` (can be replaced with\n `satisfy System.Char.IsDigit` and `satisfy System.Char.IsNumber`)\n * moved the helper functions `expectedError`, `unexpectedError` etc. into\n the `Error` module\n\n- `FParsec.CharStream`:\n * string constructor takes more arguments\n * `Iterator.Peek(i)` now returns the `EndOfStreamChar` char instead\n of throwing an exception if the char peeked at lies before the beginning\n of the stream\n]\n]\n[/section]\n\n[section#v0_7_3_1 Version 0.7.3.1, 2009-02-26]\n- Fixed a bug in `CharParser.normalizeNewlines`/`CharStream.NormalizeNewlines`.\n This bug also affected the `skipped` and `manyTillString` parsers, which\n internaly call `normalizeNewlines` to normalize the returned string.\n\n The bug was reported by Greg Chapman - thanks Greg!\n\n When given a multi-line string in which the lines are delimited by `\"\\r\\n\"` but\n the last line does not end in a newline, the buggy `normalizeNewlines`\n replaced the chars on the last line with `'\\n'` chars.\n\n- Changed the signature of `Helper.SkipOverWhitespace`.\n[/section]\n\n[section#v0_7_3 Version 0.7.3, 2008-12-08]\nBreaking changes (all of which should have little or no impact\non existing code bases):\n- `CharStream.Iterator` instances now compare equal if and only if they belong\n to the same `CharStream` and point to the same index (previously they\n compared only equal if their internal representations were identical)\n- the constructor argument of `Error.otherError` is now expected to be\n comparable with F#'s structural comparison function `compare`,\n see http://research.microsoft.com/fsharp/manual/spec2.aspx#_Toc207785725\n- the signature of the second `ParserError.ToString` overload has changed\n- `CharParser.errorToString` and `printErrorLine` have been deprecated\n\nNew features:\n- reimplemented the error formatting code in `FParsec.Error`\n- added new `State<_>.AdvanceTo` and `CharStream.Iterator.Advance` overloads\n- slightly modified the error reporting in `Primitives.sepEndBy`\n- some documentation fixes\n[/section]\n\n[section#v0_7_2 Version 0.7.2, 2008-11-17]\n- added `CharParser.OperatorPrecedenceParser`\n- changed the overflow checking in `pint32` such that it will\n not be affected by an expected future change in F#'s\n `int32 -> uint64` conversion behaviour\n- added `CharParser.pint16`, `puint16`, `pint8`, `puint8`\n- changed the signatures in CharParser.fsi to use the `Parser<_,_>` type\n abbreviation\n- fixed outdated documentation of `CharParser.expectedError`\n- some minor optimizations\n[/section]\n\n[section#v0_7_1 Version 0.7.1, 2008-09-29]\nBreaking changes:\n- renamed `Primitives.Reply._tag` member to `Flags` and gave\n it a proper enumeration type\n- `CharParser.State` is now a reference type\n- Removed `CharParser.State.Flags` member\n- deprecated `Primitives.reconstructError`\n[/section]\n\n[section#v0_7_0_1 Version 0.7.0.1, 2008-09-23]\nBreaking change:\n- changed the case of the `FParsec.Error.Pos` members\n (This wasn't already done in 0.7 because of an oversight.)\n[/section]\n\n[section#v0_7 Version 0.7.0, 2008-09-13]\nBugfixes:\n- made `FParsec.Error.Pos` IComparable to prevent `ParserError.ToString`\n from throwing an exception under rare circumstances\n- corrected the argument checking for some `CharStream.Iterator` methods\n for very large arguments\n\nNew features:\n- compatibility with the F# CTP release\n- a configurable parser for number literals: `CharParser.numberLiteral`\n- `CharParser.pfloat` now also parses `NaN`, `Infinity` and\n hexadecimal floating point literals as supported by IEEE754r, C99 and\n Java (but different from the hex representation supported by F#)\n- new helper functions `CharParser.floatToHexString`, `floatOfHexString`,\n `float32ToHexString` and `float32OfHexString`\n- integer parsers: `Charparser.pint32`, `puint64`, `puint32`, `puint64`\n- new sample: a JSON parser\n- various optimizations and some code cleanup\n- new `CharStream.Iterator` members `ReadUntil`, `Increment` and `Decrement`\n- new `State` member `AdvanceTo`\n- new function `Primitives.createParserForwardedToRef`\n- new combinator `|>>=` in `Primitives`\n\nBreaking changes:\n- renamed the parsers `char` and `string` to `pchar` and `pstring`\n (This is in deference to the built-in F# functions `char`\n and `string`, which weren't yet around when the first version\n of FParsec was released.)\n- changed the case of the properties of the `Reply` and `State` types\n (This reflects the emerging consensus in the F# community that all\n public members of types should be named in PascalCase.)\n- deprecated `State.AdvanceNL` (use the 3 parameter Advance overload instead)\n- deprecated the `Primitives` helper functions `isOk`, `isEmpty`, ...\n (the `Reply` properties `IsOk`, `IsEmpty`,... should be used instead)\n- deprecated the `CharParser` helper functions `matchChar`, `readChar`, ...\n (the `State.Iter` methods `Match`, `Read`, ... should be used instead)\n- deprecated `Primitives.option`, `<|>$` should be used instead\n- made `CharParser.CharList` internal (If you need this helper class for\n your code, just copy the implementation to your source.)\n- `State.Flags()` now has more bits (and less bits are reset on a position change)\n[/section]\n\n[section#v0_6 Version 0.6.0, 2008-05-20]\n- fixed a bug in `manyTillString` (the code keeping track of newlines was\n buggy)\n- fixed a bug in `CharParser.` (the error reporting was inconsistent with\n `Primitives.` in the rare case where `` is applied inside an\n `attempt (...) label` clause to a parser that returns an `EmptyOk` reply)\n- various changes for F# 1.9.4.15\n- added `skipped` parser to `CharParser`\n- added `nextCharSatifiesNot`, `prevCharSatifiesNot`,\n `currCharSatisfies`, `currCharSatisfiesNot` to `CharParser` module;\n the behaviours of the existing `nextCharSatisfies` and `prevCharSatisfies`\n were slightly changed (see =fparsec.html= for more details)\n- added `TryWith` and `TryFinally` members to `Primitivs.ParserCombinator`\n- added `triple` and `quad` parsers to `Primitives` module\n- set `CompilationRepresentationFlags.PermitNull` for `Error.ParserError`\n- various optimizations\n- some documentation fixes, including corrections for the docs of\n the `CharParser` error generation helper functions (`expectedError` etc.)\n[/section]\n\n[section#v0_5_1 Version 0.5.1, 2008-01-20]\n- added `pipe2`, `pipe3` and `pipe4` primitives\n- replaced `count` and `skipCount` primitives with optimized versions\n- minor optimizations in `spaces` and `spaces1`\n- added `pfloat` char parser\n- minor documentation fixes\n[/section]\n\n[section#v0_5 Version 0.5.0, 2008-01-15]\n- Major design change: all lazy computations were removed and the types\n `Output` and `Reply` unified. The new implementation is considerably simpler\n and also compiles with F# 1.9.3.7.\n- Fixed a bug in =build.bat= (reported by Santosh Zachariah - thanks Santosh!)\n[/section]\n\n[section#v0_4_4 Version 0.4.4, 2008-01-13]\n- fixed a minor issue in CharParser.attempt\n- added `.>>!` and `>>.!` primitives\n- added `skipManySatisfy` and `skipMany1Satisfy` char parsers\n[/section]\n\n[section#v0_4_3 Version 0.4.3, 2008-01-12]\n- fixed bugs in the CharParser versions of `` and `attempt`.\n- added `>>?` primitive\n- added `skipSatisfy` and `skipSatisfyL` char parsers\n- minor documentation fixes\n[/section]\n\n[section#v0_4_2 Version 0.4.2, 2008-01-04]\n- performance improvements in `CharStream.Iterator`\n- minor documentation fixes\n[/section]\n\n[section#v0_4_1 Version 0.4.1, 2008-01-02]\n- documentation fixes\n- new sample application: a parser for Parsing Expression Grammars\n- `newline` and `unicodeNewline` now return `'\\n'`, instead of 1 or 2\n- added `whitespace` parser and changed `unicodeWhitespace`\n- added `spaces` parser (equivalent to `skipManyChars whitespace`)\n- removed `newlineRepl` parameter from `manyTillString`\n- added `skipManyTill` and `skipManyCharsTill`\n- generalized types of skipManyChars and skipManyChars1\n[/section]\n\n[section#v0_4 Version 0.4.0, 2007-12-30]\nInitial public release\n[/section]\n\n[/no-auto-link]\n[/section]\n" }, { "path": "Doc/src/contact.txt", "content": "[section Contact]\n\n[section Contact]\nFParsec currently doesn't have its own discussion forum or mailing list.[br] (Please let me know if you'd like that to be changed.)\n\nCurrently the best place to get a quick answer to any FParsec-related question is: [url \"https://stackoverflow.com/search?q=fparsec\" StackOverflow.com].\n\nYou can also email me (Stephan) directly at: [url \"mailto:fparsec [at] quanttec.com\" fparsec \\[at\\] quanttec.com]. Please don't hesitate to contact me with any feedback or question regarding FParsec. I'm always happy to hear from FParsec users.\n\n[/section]\n\n[section Impressum]\n*Author:* Stephan Tolksdorf\n\n*Address:*[br]\nGeschwister-Scholl-Allee 253[br]\n25524 Itzehoe[br]\nGermany[br]\n\n[url \"mailto:fparsec [at] quanttec.com\" fparsec \\[at\\] quanttec.com]\n[/section]\n\n[/section]\n\n" }, { "path": "Doc/src/documentation.txt", "content": "[section FParsec Documentation]\n\n[html-template \"template.html\"]\n[(* [link-to-dir-for-index] *)]\n\n[default-code-language \"f#\"]\n[split-section]\n\nFParsec is a [url \"https://en.wikipedia.org/wiki/Parser_combinator\" parser combinator] library for [url \"http://fsharp.org\" F#].\n\nWith FParsec you can implement [url \"https://en.wikipedia.org/wiki/Recursive_descent_parser\" recursive-descent] text parsers for [url \"https://en.wikipedia.org/wiki/Formal_grammar\" formal grammars].\n\nFParsec's features include:\n- support for context-sensitive, infinite look-ahead grammars,\n- automatically generated, highly readable error messages,\n- Unicode support,\n- efficient support for very large files,\n- an embeddable, runtime-configurable [url \"https://en.wikipedia.org/wiki/Operator-precedence_parser\" operator-precedence parser] component,\n- a simple, efficient and easily extensible API,\n- an implementation thoroughly optimized for performance,\n- comprehensive documentation,\n- a permissive open source @license@.\n\nFParsec is an F# adaptation of [url \"https://www.haskell.org/haskellwiki/Parsec\" Parsec], the popular parser combinator library for Haskell by [url \"https://www.microsoft.com/en-us/research/people/daan\" Daan Leijen]. While the implementations of Parsec and FParsec are completely different, they share a similar top-level API.\n\n[/ **Latest release:**] FParsec 2.0.0, 2022-11-01, [url \"https://github.com/stephan-tolksdorf/fparsec/archive/master.zip\" Download], [^download-and-installation.nuget-packages NuGet packages], [^about.changelog Changes]\n\n[auto-link{\n do-not-pick-up-as-link-targets = [ \"ToString\"],\n only-link-if-used-as-member = [\"UserState\", \"Tag\"],\n only-link-if-not-used-as-member = [\"Error\"],\n only-link-if-reference = [\"minRegexSpace\", \"normalizeNewlines\", \"skipString\", \"skipNewline\", \"String\"],\n links = [\"Error\" : reference.Primitives.members.Error,\n \"FParsec.Primitives\" : reference.Primitives,\n \"FParsec.CharParsers\" : reference.CharParsers,\n \"FParsec.Error\" : reference.Error ,\n \"CharParsers.normalizeNewlines\" : reference.CharParsers.members.normalizeNewlines,\n \"Text.FoldCase\" : reference.Text.members.FoldCase,\n \"FParsec.Text.FoldCase\" : reference.Text.members.FoldCase,\n \"Text.NormalizeNewlines\" : reference.Text.members.NormalizeNewlines,\n \"FParsec.Text.NormalizeNewlines\" : reference.Text.members.NormalizeNewlines,\n \"RError\" : reference.Reply.members.Error,\n \"case-insensitively\" : reference.CharStream.CharStream.remarks.case-insensitive-matching,\n \"CharStream\" : reference.CharStream.CharStream_1,\n \"CharStream<_>\" : reference.CharStream.CharStream_1,\n \"ErrorMessageList.Merge\" : reference.ErrorMessageList.members.Merge,\n \"api-reference\": reference\n ],\n urls = [\"FileStream\" : \"https://msdn.microsoft.com/en-us/library/system.io.filestream.aspx\",\n \"System.IO.Stream\" : \"https://msdn.microsoft.com/en-us/library/system.io.stream.aspx\",\n \"Decoder\" : \"https://msdn.microsoft.com/en-us/library/system.text.decoder.aspx\",\n \"System.Text.Encoding\" : \"https://msdn.microsoft.com/en-us/library/system.text.encoding.aspx\",\n \"Encoding\" : \"https://msdn.microsoft.com/en-us/library/system.text.encoding.aspx\",\n \"System.IEquatable\" : \"https://msdn.microsoft.com/en-us/library/ms131187.aspx\",\n \"System.IComparable\" : \"https://msdn.microsoft.com/en-us/library/system.icomparable.aspx\",\n \"System.Char.IsWhiteSpace\" : \"https://msdn.microsoft.com/en-us/library/t809ektx.aspx\",\n \"System.Collections.IStructuralEquatable\" : \"https://msdn.microsoft.com/en-us/library/system.collections.istructuralequatable(VS.100).aspx\",\n \"HashSet\" : \"https://msdn.microsoft.com/en-us/library/bb359438.aspx\",\n \"System.Globalization.StringInfo\" : \"https://msdn.microsoft.com/en-us/library/system.globalization.stringinfo.aspx\",\n \"LengthInTextElements\" : \"https://msdn.microsoft.com/en-us/library/system.globalization.stringinfo.lengthintextelements.aspx\",\n \"System.Text.RegularExpressions\" : \"https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.aspx\",\n \"System.Text.RegularExpressions.Regex\" : \"https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.regex.aspx\",\n \"System.Text.NormalizationForm\" : \"https://msdn.microsoft.com/en-us/library/system.text.normalizationform.aspx\",\n \"Regex\" : \"https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.regex.aspx\",\n \"System.Text.RegularExpressions.Match\" : \"https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.match.aspx\",\n \"StringWriter\" : \"https://msdn.microsoft.com/en-us/library/system.io.stringwriter.aspx\",\n \"System.IO.StringWriter\" : \"https://msdn.microsoft.com/en-us/library/system.io.stringwriter.aspx\",\n \"TextWriter\" : \"https://msdn.microsoft.com/en-us/library/system.io.textwriter.aspx\",\n \"System.IO.TextWriter\" : \"https://msdn.microsoft.com/en-us/library/system.io.textwriter.aspx\",\n \"ArgumentException\" : \"https://msdn.microsoft.com/en-us/library/system.argumentexception.aspx\",\n \"ArgumentOutOfRangeException\" : \"https://msdn.microsoft.com/en-us/library/system.argumentoutofrangeexception.aspx\",\n \"NotSupportedException\" : \"https://msdn.microsoft.com/en-us/library/system.notsupportedexception.aspx\",\n \"NullReferenceException\" : \"https://msdn.microsoft.com/en-us/library/system.nullreferenceexception.aspx\",\n \"IOException\" : \"https://msdn.microsoft.com/en-us/library/system.io.ioexception.aspx\",\n \"DecoderFallbackException\" : \"https://msdn.microsoft.com/en-us/library/system.text.decoderfallbackexception.aspx\",\n \"computation-expressions\": \"https://msdn.microsoft.com/en-us/library/dd233182.aspx\",\n \"computation-expression\": \"https://msdn.microsoft.com/en-us/library/dd233182.aspx\",\n \"monad\": \"https://en.wikipedia.org/wiki/Monad_%28functional_programming%29\",\n \"f-interactive\": \"https://msdn.microsoft.com/en-us/library/dd233175.aspx\",\n \"MethodImplOptions\": \"https://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.methodimploptions(v=vs.110).aspx\"\n ] }]\n\n[section#about About FParsec]\n[split-section]\n[output-in-subdirectory]\n\n[include \"fparsec-vs-alternatives.txt\"]\n[include \"status-and-roadmap.txt\"]\n[include \"changelog.txt\"]\n[include \"contact.txt\"]\n\n[/section]\n\n[include \"license.txt\"]\n[include \"download-and-installation.txt\"]\n\n[include \"tutorial.txt\"]\n[include \"users-guide.txt\"]\n[include \"reference.txt\"]\n\n[/auto-link]\n\n[/section]" }, { "path": "Doc/src/download-and-installation.txt", "content": "[section Download and installation]\n\nFParsec is distributed in source code form and as NuGet packages.\n\nIf you're new to FParsec, I'd recommend to start by downloading the [url \"https://github.com/stephan-tolksdorf/fparsec/archive/master.zip\" source code package] and experimenting a bit with the included sample projects. With the project and solution files building the library and the samples is as easy as clicking a button.\n\nThe source package also includes a complete copy of the HTML documentation for offline viewing.\n\n[toc]\n\n[section NuGet packages]\n\nThere are two NuGet packages of FParsec, which are built with different configuration options.\n\nThe [url \"https://nuget.org/packages/FParsec\" basic package] uses the @Low-Trust version@ of FParsec, which uses no [url \"https://msdn.microsoft.com/en-us/library/t2yzs44b.aspx\" unverifiable code] and is optimized for maximum portability. The main limitation of this version is that any input stream is completely read into a string before parsing, which limits the maximum practical input size. This package also contains assemblies for .NET Standard 2.0.\n\nThe [url \"https://nuget.org/packages/FParsec-Big-Data-Edition/\" \"Big Data edition\" package] uses the non-Low-Trust version of FParsec that is optimized for maximum performance and supports extremely large input streams. Since this configuration is also the default configuration of the solution files included with the source code, it is sometimes referred to as the \"normal\" version of FParsec. This version of FParsec does use \"unsafe\" (i.e. unverifiable) code involving unmanaged pointers. It also uses code generation in the implementation of `isAnyOf`, `isNoneOf`, `anyOf`, `skipAnyOf`, `noneOf` and `skipNoneOf`. [* Unfortunately, this version is currently not compatible with .NET Standard/.NET Core.]\n\nShould you measure a significant performance /degradation/ when switching to the Big Data edition, you're probably inadvertently recreating the same `isAnyOf`- or `isNoneOf`-based parsers again and again, as explained [^construct-parsers-once here] and [^why-the-monadic-syntax-is-slow here].\n\nThe .NET Framework assemblies in the NuGet packages are strongly signed. Their assembly version numbers will only be incremented for breaking changes. The .NET Standard assembly in the `FParsec` package is not signed.\n\nThe NuGet packages include PDBs and SourceLink support, which should allow you to step through FParsec code in the debugger of your IDE.\n\n[/section]\n\n[section Getting the source]\n\nFParsec's source code repository is hosted on GitHub at: [url \"https://github.com/stephan-tolksdorf/fparsec\" github.com/stephan-tolksdorf/fparsec]\n\nYou can clone the source code using Git or you can [url \"https://github.com/stephan-tolksdorf/fparsec/archive/master.zip\" download it as a zip-file].\n\nIt's an FParsec project policy to check only stable and tested code into the master branch of the GitHub repository, so you can normally just work with the master version of FParsec.\n\n[tip\n[url \"https://git-fork.com/\" Fork] is a great free GUI for Git for Windows and MacOS.\n]\n\n[/section]\n\n[section FParsec is built as two DLLs]\n\nFParsec's source code is written in both C# and F#. Since neither the C# nor the F# compiler directly support the other language, the respective components need to be built separately.\n\nHence, FParsec is built as two DLLs. The C# bits are compiled into the =FParsecCS.dll= and the F# bits (which depend on the C# bits) are compiled into =FParsec.dll=.\n\n*Projects that use FParsec thus have to reference both DLLs.*\n\nIf you reference the DLLs in the [@ F# Interactive] console, you need to reference `FParsecCS.dll` before you reference `FParsec.dll`.\n\n[note If you don't want to distribute the FParsec DLLs together with the assembly of your project, you can use the [url \"https://msdn.microsoft.com/en-us/library/dd233171.aspx\" =staticlink= command-line option] of the F# compiler to merge the FParsec DLLs into your assembly.\n\nUnfortunately, the same option cannot be used to merge =FParsecCS.dll= into the =FParsec.dll=, as the public definitions in =FParsecCS.dll= wouldn't be reexported by =FParsec.dll=. For similar reasons it also doesn't seem to be possible to use tools like [url \"http://research.microsoft.com/en-us/people/mbarnett/ILMerge.aspx\" ILMerge] or [url \"http://code.google.com/p/il-repack/\" il-repack] to obtain a merged =FParsec.dll= that can be properly consumed by F# programs.\n]\n\n[/section]\n\n[section Building FParsec from source]\n\nThe solution file [= FParsec.sln] in the root source folder and the associated project files in the subfolders can be used to build FParsec from the command line or with IDEs such as Visual Studio 2019 or JetBrains Rider. \n\nTo build the Low-Trust version of FParsec, you have to specifiy either `Debug-LowTrust` or `Release-LowTrust` as the configuration. The `Debug` and `Release` configurations build the non-Low-Trust version of FParsec, which currently is not compatible with the .NET Core runtime. \n\n[note In contrast to JetBrains Rider, Visual Studio 2019 currently does not support setting the supported target frameworks depending on the configuration. Due to this issue one currently has to use the separate `FParsec-LowTrust.sln` solution for building the Low-Trust version of FParsec in VS 2019.]\n\nThe =Test= project in the solution files contains the unit tests for FParsec.\n\nThe file [url \"https://github.com/stephan-tolksdorf/fparsec/blob/master/.vscode/tasks.json\" [= .vscode/tasks.json]] contains some convenient task definitions for Visual Studio Code.\n\nThe NuGet packages are built with the [url \"https://github.com/stephan-tolksdorf/fparsec/blob/master/pack.ps1\" [= pack.ps1]] PowerShell script.\n\n[/section]\n\n[section#low-trust-version The Low-Trust version of FParsec]\n\nFor optimization reasons the normal implementation (the \"Big Data edition\") of FParsec involves [url \"https://msdn.microsoft.com/en-us/library/t2yzs44b.aspx\" unverifiable code] using unmanaged pointers and runtime code generation. \n\nIf you compile FParsec with the `LOW_TRUST` conditional compiler symbol, the unverifiable code is replaced with a \"safe\" alternative. This allows FParsec to be run in environments with \"reduced trust\", such as medium trust ASP.NET applications, and it also allows FParsec to be compiled against reduced subsets of the .NET API.\n\nIn the `Debug-LowTrust` and `Release-LowTrust` configurations of the [= FParsec.sln] solution file in the root source folder, `LOW_TRUST` is automatically defined as `true`.\n\nThe Low-Trust version of FParsec has the following two major limitations:\n- A `CharStream` that is constructed from a `System.IO.Stream` or a file path reads the complete file into a single string during construction. *This severely limits the maximum practical input stream size.*\n- The `StaticMapping` module is not supported.\n\n[/section]\n\n\n[section Configuration options]\n\nYou can configure FParsec's source code with a number of conditional compilation symbols (a.k.a. preprocessor defines). Besides the [^low-trust-version Low-Trust option], these symbols mostly serve tuning purposes.\n\n[dl Options for =FParsecCS.dll=\n\n[`LOW_TRUST`]\n[See [^low-trust-version above].]\n\n[`#AGGRESSIVE_INLINING#`]\n[\nRequires a version of NET ≥ 4.5.\n\nAnnotates some functions with the `MethodImplOptions.AggressiveInlining` attribute.\n]\n\n[`PCL`]\n[Compile for a PCL subset of the .NET API. Removed in version 2.0.0.]\n\n[`#SMALL_STATETAG#`]\n[\nUse a 32-bit `StateTag` in the `CharStream` class instead of the default 64-bit one.\n\nThis is an optimization for 32-bit runtimes. You can find more information about the state tag in [^ users-guide.applying-parsers-in-sequence.the-statetag] of the user's guide.\n]\n\n[`#UNALIGNED_READS#`]\n[\n[small [/ This option does not affect the @Low-Trust version@ of FParsec.]][br]\n\nOptimize for CPUs that support fast unaligned memory reads, i.e. any modern x86-based CPU.\n\nThis option only makes a noticeable difference is some specific situations.\n]\n\n]\n\n[dl Options for =FParsec.dll=\n\n[`LOW_TRUST`]\n[See [^low-trust-version above].]\n\n[`[no-auto-link UNALIGNED_READS]`]\n[\nSee [^UNALIGNED_READS above].\n]\n\n[`NOINLINE`]\n[\nDo not force inlining of certain parser combinators.\n\nThis option enables you to step through the respective combinators during debugging.\n]\n\n[`#USE_STATIC_MAPPING_FOR_IS_ANY_OF#`]\n[\n[small [/ This option does not affect the @Low-Trust version@ of FParsec.]][br]\n\nUse `StaticMapping.createStaticCharIndicatorFunction` for the implementation of `isAnyOf`, `isNoneOf`, `anyOf`, `skipAnyOf`, `noneOf` and `skipNoneOf` for generating optimized char predicate functions using runtime code generation.\n\nRuntime code generation is a relatively expensive operation, so this optimization is primarily meant for parsers that are applied to large (or lots of) input streams. Please see the remarks for the `StaticMapping` module for more information.\n\nIf you run into noticeable performance problems or memory leaks when enabling this option, you're probably inadvertently recreating the same `isAnyOf`- or `isNoneOf`-based parser again and again, as explained [^construct-parsers-once here] and [^why-the-monadic-syntax-is-slow here].\n]\n\n[`DEBUG_STATIC_MAPPING` ]\n[\n[small [/ This option does not affect the @Low-Trust version@ of FParsec.]][br]\n\nSee [^DEBUG_STATIC_MAPPING `StaticMapping` documentation].\n]\n\n]\n\n[/section]\n\n[/section]\n" }, { "path": "Doc/src/fparsec-vs-alternatives.txt", "content": "[section FParsec vs alternatives]\n\nThe following tables contain a bullet-point comparison between FParsec and the two main alternatives for parsing with F#: parser generator tools (e.g. fslex & fsyacc) and \"hand-written\" recursive descent parsers.\n\n[table Relative advantages\n\n[[Parser-generator tools] [FParsec] [Hand-written[br] recursive-descent parser]]\n[\n[\n- Declarative and easy-to-read syntax\n- Ensures adherence to grammar formalism\n- Can check for certain kinds of ambiguity in grammar\n- You don't have to think about performance. Either the generated parser is fast enough, or not. There's not much you can do about it.\n]\n[\n- Implemented as F# library, so no extra tools or build steps\n- Parsers are first-class values within the language\n- Succinct and expressive syntax\n- Modular and easily extensible\n- Extensive set of predefined parsers and combinators\n- Semi-automatically generated, highly readable error messages\n- Supports arbitrary lookahead and backtracking\n- Runtime-configurable operator-precedence parser component\n- Does not require a pre-parsing tokenization phase\n- Comprehensive documentation\n- Extensively unit-tested\n]\n[\n- No extra tools or build steps\n- Most amenable to individual requirements\n- Potentially as fast as technically possible\n- Parsers are relatively portable if you stick to simple language features and keep library dependencies to a minimum\n]\n]\n]\n\n[table Relative disadvantages\n\n[[Parser-generator tools] [FParsec] [Hand-written[br] recursive-descent parser]]\n[\n[\n- Restricted to features of grammar formalism\n- Extra tools and compilation steps\n- Reliance on opaque generator tool, that is often hard to debug, optimize or extend\n- Static grammar that can't be changed at runtime\n- Often hard to generate good error messages\n- Many tools generate comparatively slow parsers\n- Some tools have only limited Unicode support\n- Portability problems\n]\n[\n- Tradeoff between declarativeness and performance\n- Syntax less readable than PEG or Regular Expression syntax\n- [url \"https://en.wikipedia.org/wiki/Left_recursion\" Left-recursive] grammar rules have to be rewritten\n- Does not support a pre-parsing tokenization phase\n- You have to learn the API\n- Limited to F#\n- Code-dependence on FParsec\n- Aggressive performance optimizations add complexity to parts of the lower-level FParsec source code\n]\n[\n- You have to write everything yourself, which can take a lot of effort\n- Implementing (fast) parsers requires some experience\n- Expression (sub)grammars with infix operators can be ugly and inefficient to parse with\n a pure recursive-descent parser, so you might also have to write some\n kind of embedded operator precedence parser\n]\n]\n]\n\n[/section]" }, { "path": "Doc/src/license.txt", "content": "[section License]\n\nExcept where noted otherwise, the FParsec library in source and binary form is distributed under the @Simplified BSD License@. The Simplified BSD License (a.k.a. \"2-clause BSD License\") is a simple, permissive license that is [url \"http://www.opensource.org/licenses/bsd-license.php\" OSI-compliant].\n\nFParsec incorporates data derived from the [url \"http://unicode.org/ucd/\" Unicode Character Database] v. 8.0.0, Copyright (c) 1991-2015 Unicode, Inc., which is distributed under the following terms:[br][url \"http://www.unicode.org/terms_of_use.html#Exhibit1\"]\n\nThe documentation in the =Doc= folder is licensed under the [@ Creative Commons Attribution-NonCommercial 3.0 Unported License]. This Creative Commons license does not allow you to use the documentation for commercial purposes without permission. This means, for example, that you cannot sell the documentation in book form for profit or put it on a web content farm in order to earn money with ads. However, *you can of course use the documentation in a commercial context* (e.g. put it on the intranet of a commercial corporation), as long as you're not trying to directly earn money from the text of the documentation.\n\n[section Simplified BSD License]\n\nCopyright (c) 2007-2022, Stephan Tolksdorf. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\n- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\n\n[* This software is provided by the copyright holders \"as is\" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.]\n[/section]\n\n[section Creative Commons Attribution-NonCommercial 3.0 Unported License]\n\nSummary: [url \"https://creativecommons.org/licenses/by-nc/3.0/\"]\n\nThe full license text: [url \"https://creativecommons.org/licenses/by-nc/3.0/legalcode\"]\n\n[/section]\n\n[/section]\n" }, { "path": "Doc/src/reference-charparsers.txt", "content": "\n[section#CharParsers FParsec.CharParsers]\n\n[interface-reference]\n[section Interface]\n[$$interface]\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsec.dll\n\n[] // module is automatically opened when FParsec namespace is opened\nmodule [no-auto-link FParsec.CharParsers]\n\nopen FParsec.Error\nopen FParsec.Primitives\n``]\n\n[``\n\n// Running parsers on input\n// ========================\n``]\n[``\ntype @ParserResult@<'Result,'UserState>``]\n[\nValues of this union type are returned by the @runParser functions@ (not by `Parser<_,_>` functions).\n\n[interface-members]\n[``=\n``]\n\n[``\n | @Success@ of 'Result * 'UserState * Position\n``]\n[`Success(result, userState, endPos)` holds the result and the user state returned by a successful parser, together with the position where the parser stopped.]\n\n[``\n | @Failure@ of string * ParserError * 'UserState\n``]\n[\n`Failure(errorAsString, error, userState)` holds the parser error and the user state returned by a failing parser, together with the string representation of the parser error. The `ParserError` value `error` contains an `ErrorMessageList` and the position and user state value associated with the error.\n]\n[/interface-members]\n]\n\n[``\n[#runparser-functions]\n``]\n\n[``\nval @runParserOnString@:\n Parser<'a,'u> -> 'u -> streamName: string -> string\n -> ParserResult<'a,'u>\n``]\n[\n`runParserOnString p ustate streamName str` runs the parser `p` on the content of the string `str`, starting with the initial user state `ustate`. The `streamName` is used in error messages to describe the source of the input (e.g. a file path) and may be empty. The parser's `Reply` is captured and returned as a `ParserResult` value.\n]\n\n[``\n\nval @runParserOnSubstring@:\n Parser<'a,'u> -> 'u -> streamName: string -> string -> int -> int\n -> ParserResult<'a,'u>\n``]\n[\n`runParserOnSubstring p ustate streamName str index count` runs the parser `p` directly on the content of the string\n`str` between the indices `index` (inclusive) and `index + count` (exclusive), starting with the initial user state `ustate`. The `streamName` is used in error messages to describe the source of the input (e.g. a file path) and may be empty. The parser's `Reply` is captured and returned as a `ParserResult` value.\n]\n\n[``\n\nval @runParserOnStream@:\n Parser<'a,'u> -> 'u -> streamName: string\n -> System.IO.Stream -> System.Text.Encoding\n -> ParserResult<'a,'u>\n``]\n[\n`runParserOnStream p ustate streamName stream encoding` runs the parser `p` on the content of the `System.IO.Stream` `stream`, starting with the initial user state `ustate`. The `streamName` is used in error messages to describe the source of the input (e.g. a file path) and may be empty. In case no Unicode byte order mark is found, the stream data is assumed to be encoded with the given `encoding`. The parser's `Reply` is captured and returned as a `ParserResult` value.\n]\n\n[``\n\nval @runParserOnFile@:\n Parser<'a,'u> -> 'u -> path: string -> System.Text.Encoding\n -> ParserResult<'a,'u>\n``]\n[\n`runParserOnFile p ustate path encoding` runs the parser `p` on the content of the file at the given `path`, starting with the initial user state `ustate`. In case no Unicode byte order mark is found, the file data is assumed to be encoded with the given `encoding`. The parser's `Reply` is captured and returned as a `ParserResult` value.\n]\n\n[``\n\nval @run@: Parser<'a, unit> -> string -> ParserResult<'a,unit>\n``]\n[\n`run parser str` is a convenient abbreviation for `runParserOnString parser () \"\" str`.\n]\n\n[``\n\n// Reading the input stream position and handling the user state\n// =============================================================\n``]\n[``\nval @getPosition@: Parser\n``]\n[\nThe parser `getPosition` returns the current position in the input stream.\n\n`getPosition` is defined as `fun stream -> Reply(stream.Position)`.\n]\n\n[``\n\nval @getUserState@: Parser<'u,'u>\n``]\n[\nThe parser `getUserState` returns the current user state.\n\n`getUserState` is defined as `fun stream -> Reply(stream.UserState)`.\n]\n\n[``val @setUserState@: 'u -> Parser\n``]\n[\nThe parser `setUserState u` sets the user state to `u`.\n\n`setUserState u` is defined as\n``\nfun stream ->\n stream.UserState <- u\n Reply(())\n``\n]\n\n[``\nval @updateUserState@: ('u -> 'u) -> Parser\n``]\n[\n`updateUserState f` is defined as\n``\nfun stream ->\n stream.UserState <- f stream.UserState\n Reply(())\n``\n]\n\n[``\n\nval @userStateSatisfies@: ('u -> bool) -> Parser\n``]\n[\nThe parser `userStateSatisfies f` succeeds if the predicate function `f` returns `true` when applied to the current `UserState`, otherwise it fails.\n\n[note\nIf the parser `userStateSatisfies f` fails, it returns no descriptive error message; hence it should only be used together with other parsers that take care of a potential error.\n]\n]\n\n\n[``\n\n// Parsing single chars\n// ====================\n``]\n\n[``\nval @pchar@: char -> Parser\n``] [`pchar c` parses the char `c` and returns `c`. If `c = '\\r'` or `c = '\\n'` then `pchar c` will parse any one newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) and return `c`.\n]\n\n[``\nval @skipChar@: char -> Parser\n``]\n[`skipChar c` is an optimized implementation of `pchar c |>> ignore`.]\n\n[``\nval @charReturn@: char -> 'a -> Parser<'a,'u>\n``]\n[`charReturn c result` is an optimized implementation of `pchar c >>% result`.]\n\n[``\n\nval @anyChar@: Parser\n``]\n[`anyChar` parses any single char or newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`). Returns the parsed char, or `'\\n'` in case a newline was parsed.\n]\n\n[``\nval @skipAnyChar@: Parser\n``]\n[`skipAnyChar` is an optimized implementation of `anyChar |>> ignore`.\n]\n\n[``\n[#satisfy-parsers]\nval @satisfy@: (char -> bool) -> Parser\n``]\n[`satisfy f` parses any one char or newline for which the predicate function `f` returns `true`. It returns the parsed char. Any newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) is converted to the single char `'\\n'`. Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called with `'\\r'` and `satisfy f` will never return the result `'\\r'`.\n\nFor example, `satisfy (fun c -> '0' <= c && c <= '9')` parses any decimal digit.\n\n[note\nIf the parser `satisfy f` fails, it returns no descriptive error message (because it does not know what chars `f` accepts); hence it should only be used together with other parsers that take care of a potential error. Alternatively, `satisfyL f label` can be used to ensure a more descriptive error message.\n]\n]\n\n[``\nval @skipSatisfy@: (char -> bool) -> Parser\n``]\n[`skipSatisfy f` is an optimized implementation of `satisfy f |>> ignore`.]\n\n[``\nval @satisfyL@: (char -> bool) -> string -> Parser\n``]\n[`satisfy f label` is an optimized implementation of `satisfy f label`.]\n\n[``\nval @skipSatisfyL@: (char -> bool) -> string -> Parser\n``]\n[`skipSatisfyL f label` is an optimized implementation of `skipSatisfy f label`.]\n\n[``\n\nval @anyOf@: seq -> Parser\n``]\n[`anyOf str` parses any char contained in the char sequence `chars`. It returns the parsed char. If `chars` contains the char `'\\n'`, `anyOf chars` parses any newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) and returns it as `'\\n'`. (Note that it does not make a difference whether or not `chars` contains `'\\r'` and that `anyOf chars` will never return `'\\r'`.)\n\nFor example, `anyOf \". \\t\\n\"` will parse any of the chars `'.'`, `' '`, `'\\t'` or any newline.\n\n`anyOf chars` is defined as `satisfy (isAnyOf chars)`.\n\nFor performance critical parsers it might be worth replacing instances of `anyOf` in loops with a `manySatisfy`-based parser. For example, `manyChars (anyOf \". \\t\\n\")` could be replaced with `manySatisfy (function '.'|' '|'\\t'|'\\n' -> true | _ -> false)`.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\nval @skipAnyOf@: seq -> Parser\n``]\n[\n`skipAnyOf chars` is an optimized implementation of `anyOf chars |>> ignore`.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\nval @noneOf@: seq -> Parser\n``]\n[`noneOf chars` parses any char not contained in the char sequence `chars`. It returns the parsed char. If `chars` does not contain the char `'\\n'`, `noneOf chars` parses any newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) and returns it as as `'\\n'`. (Note that it does not make a difference whether or not `chars` contains `'\\r'` and that `noneOf chars` will never return `'\\r'`.)\n\nFor example, `noneOf \". \\t\\n\"` will parse any char other than `'.'`, `' '`, `'\\t'`, `'\\r'` or `'\\n'`.\n\n`noneOf chars` is defined as `satisfy (isNoneOf chars)`.\n\nFor performance critical parsers it might be worth replacing instances of `noneOf` in loops with a `manySatisfy`-based parser. For example, `manyChars (noneOf \". \\t\\n\")` could be replaced with `manySatisfy (function '.'|' '|'\\t'|'\\n' -> false | _ -> true)`.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\nval @skipNoneOf@: seq -> Parser\n``]\n[\n`skipNoneOf chars` is an optimized implementation of `noneOf chars |>> ignore`.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\n\nval @asciiLower@: Parser\n``]\n[Parses any char in the range `'a'` - `'z'`. Returns the parsed char.]\n[``\nval @asciiUpper@: Parser\n``]\n[Parses any char in the range `'A'` - `'Z'`. Returns the parsed char.]\n[``\nval @asciiLetter@: Parser\n``]\n[Parses any char in the range `'a'` - `'z'` and `'A'` - `'Z'`. Returns the parsed char.]\n\n[``\n\nval @lower@: Parser\n``]\n[Parses any UTF-16 lowercase letter char identified by `System.Char.IsLower`. Returns the parsed char.]\n[``\nval @upper@: Parser\n``]\n[Parses any UTF-16 uppercase letter char identified by `System.Char.IsUpper`. Returns the parsed char.]\n\n[``\nval @letter@: Parser\n``]\n[Parses any UTF-16 letter char identified by `System.Char.IsLetter`. Returns the parsed char.]\n\n[``\n\nval @digit@: Parser``]\n[Parses any char in the range `'0'` - `'9'`. Returns the parsed char.]\n[`` // parses '0'-'9'\n``]\n[``\nval @hex@: Parser``]\n[Parses any char in the range `'0'` - `'9'`, `'a'` - `'f'` and `'A'` - `'F'`. Returns the parsed char.]\n[`` // parses '0'-'9', 'a'-'f', 'A'-'F'\n``]\n[``\nval @octal@: Parser``]\n[Parses any char in the range `'0'` - `'7'`. Returns the parsed char.]\n[`` // parses '0'-'7'\n``]\n\n[``\n\n// predicate functions corresponding to the above parsers\n``]\n\n[``\nval @isAnyOf@: seq -> (char -> bool)\n``]\n[`isAnyOf chars` returns a predicate function. When this predicate function is applied to a char, it returns `true` if and only if the char is contained in the char sequence `chars`.\n\nFor example, the function `isAnyOf \".,;\"` returns `true` when applied to the chars `'.'`, `','` or `';'`, and `false` for all other chars.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\nval @isNoneOf@: seq -> (char -> bool)\n``]\n[`isNoneOf chars` returns a predicate function. When this predicate function is applied to a char, it returns `true` if and only if the char is not contained in char sequence `chars`.\n\nFor example, the function `isNoneOf \".,;\"` returns `false` when applied to the chars `'.'`, `','` or `';'`, and `true` for all other chars.\n\nThis function is affected by the `USE_STATIC_MAPPING_FOR_IS_ANY_OF` compilation option.\n]\n\n[``\nval inline @isAsciiUpper@: char -> bool\n``]\n[Returns `true` for any char in the range `'A'` - `'Z'` and `false` for all other chars.]\n\n\n[``\nval inline @isAsciiLower@: char -> bool\n``]\n[Returns `true` for any char in the range `'a'` - `'z'` and `false` for all other chars.]\n\n[``\nval inline @isAsciiLetter@: char -> bool\n``]\n[Returns `true` for any char in the range `'a'` - `'z'`, `'A'` - `'Z'` and `false` for all other chars.]\n\n\n\n[``\nval inline @isUpper@: char -> bool\n``]\n[`isUpper` is equivalent to `System.Char.IsUpper`.]\n\n[``\nval inline @isLower@: char -> bool\n``]\n[`isLower` is equivalent to `System.Char.IsLower`.]\n\n[``\nval inline @isLetter@: char -> bool\n``]\n[`isLetter` is equivalent to `System.Char.IsLetter`.]\n\n[``\nval inline @isDigit@: char -> bool\n``]\n[Returns `true` for any char in the range `'0'` - `'9'` and `false` for all other chars.]\n\n[``\nval inline @isHex@: char -> bool\n``]\n[Returns `true` for any char in the range `'0'` - `'9'`, `'a'` - `'f'`, `'A'` - `'F'` and `false` for all other chars.]\n\n[``\nval inline @isOctal@: char -> bool\n``]\n[Returns `true` for any char in the range `'0'` - `'7'` and `false` for all other chars.]\n\n[``\n\n// Parsing whitespace\n// ==================\n``]\n\n[``\nval @tab@: Parser\n``]\n[Parses the tab char `'\\t'` and returns `'\\t'`.\n\n[note\nA tab char is treated like any other non-newline char: the column number is incremented by (only) 1.]\n]\n\n[``\n\nval @newline@: Parser\n``]\n[Parses a newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`). Returns `'\\n'`. Is equivalent to `pchar '\\n'`.]\n\n[``\nval @skipNewline@: Parser\n``]\n[`skipNewline` is an optimized implementation of `newline |>> ignore`.]\n\n[``\nval @newlineReturn@: 'a -> Parser<'a,'u>\n``]\n[`newlineReturn result` is an optimized implementation of `newline >>% result`.]\n\n[``\n\nval @unicodeNewline@: Parser\n``]\n[\nParses a Unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"`, or `\"\\u2029\"`). Returns `'\\n'`. In contrast to all other parsers in FParsec except `unicodeWhitespace` this parser also increments the internal line count for Unicode newline characters other than `'\\n'` and `'\\r'`.\n\n[note This method does not recognize the form feed char `'\\f'` (`'\\u000C'`) as a newline character.]\n\n[note\nThis parser is included only for the sake of completeness. If you design your own parser grammar, we recommend not to accept any character sequence other than `\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"` for a newline. The three usual newline representations already make text parsing complicated enough.\n]\n]\n[``\nval @skipUnicodeNewline@: Parser\n``]\n[`skipUnicodeNewline` is an optimized implementation of `newline |>> ignore`.]\n\n[``\nval @unicodeNewlineReturn@: 'a -> Parser<'a,'u>\n``]\n[`unicodeNewlineReturn result` is an optimized implementation of `newline >>% result`.]\n\n[``\n\nval @spaces@: Parser\n``]\n[Skips over any sequence of *zero* or more whitespaces (space (`' '`), tab (`'\\t'`) or newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`)).]\n\n[``\nval @spaces1@: Parser\n``]\n[Skips over any sequence of *one* or more whitespaces (space (`' '`), tab(`'\\t'`) or newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`)).]\n\n\n[``\n[#unicodeSpaces-parsers]\nval @unicodeSpaces@: Parser\n``]\n[\nSkips over any sequence of *zero* or more Unicode whitespace chars and registers any Unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"`or `\"\\u2029\"`) as a newline.\n\n[note This method does not recognize the form feed char `'\\f'` (`'\\u000C'`) as a newline character.]\n\n[note\nThis parser is included only for the sake of completeness. If you design your own parser grammar, we recommend not to accept any whitespace character other than `' '`, `'\\t'`, `'\\r'` and `'\\n'`. There is no need to make whitespace parsing unnecessary complicated and slow.\n]\n]\n\n[``\nval @unicodeSpaces1@: Parser\n``]\n[\nSkips over any sequence of *one* or more Unicode whitespace char and registers any Unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"`or `\"\\u2029\"`) as a newline.\n\nSee also the notes above for `unicodeSpaces`.\n]\n\n[``\n\nval @eof@: Parser\n``]\n[The parser `eof` only succeeds at the end of the input. It never consumes input.]\n\n\n[``\n\n// Parsing strings directly\n// ========================\n``]\n[``\nval @pstring@: string -> Parser\n``]\n[`pstring str` parses the string `str` and returns `str`. It is an atomic parser: either it succeeds or it fails without consuming any input.\n\n`str` may not contain newline chars (`'\\n'` or `'\\r'`), otherwise `pstring str` raises an `ArgumentException`.\n]\n\n[``\nval @skipString@: string -> Parser\n``]\n[`skipString str` is an optimized implementation of `pstring str |>> ignore`.]\n\n[``\nval @stringReturn@: string -> 'a -> Parser<'a,'u>\n``]\n[`stringReturn str result` is an optimized implementation of `pstring str >>% result`.]\n\n[``\n\nval @pstringCI@: string -> Parser\n``]\n[`pstringCI str` parses any string that case-insensitively matches the string `str`. It returns the *parsed* string.\n `pstringCI str` is an atomic parser: either it succeeds or it fails without consuming any input.\n\n`str` may not contain newline chars (`'\\n'` or `'\\r'`), otherwise `pstringCI str` raises an `ArgumentException`.\n]\n\n[``\nval @skipStringCI@: string -> Parser\n``]\n[`skipStringCI str` is an optimized implementation of `pstringCI str |>> ignore`.]\n\n[``\nval @stringCIReturn@: string -> 'a -> Parser<'a,'u>\n``]\n[`stringCIReturn str result` is an optimized implementation of `pstringCI str >>% result`.]\n\n[``\n\nval @anyString@: int32 -> Parser\n``]\n[`anyString n` parses any sequence of `n` chars or newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`). It returns the parsed string. In the returned string all newlines are normalized to `\"\\n\"`. `anyString n` is an atomic parser: either it succeeds or it fails without consuming any input.\n]\n\n[``\nval @skipAnyString@: int32 -> Parser\n``]\n[`skipAnyString n` is an optimized implementation of `anyString n |>> ignore`.]\n\n[``\n\nval @restOfLine@: skipNewline: bool -> Parser\n``]\n[`restOfLine skipNewline` parses any chars before the end of the line and,\nif `skipNewline` is `true`, skips to the beginning of the next line (if there is one).\nIt returns the parsed chars before the end of the line as a string (without a newline).\nA line is terminated by a newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) or the end of the input stream.\n\nFor example, `sepBy (restOfLine false) newline` will parse an input file and split it into lines:\n\n``{fsi}\n> run (sepBy (restOfLine false) newline) \"line1\\nline2\\n\";;\nval it : ParserResult = Success: [\"line1\"; \"line2\"; \"\"]\n``\n\nNote that you could not use `many (restOfLine true)` in this example, because at the end of the input `restOfLine` succeeds without consuming input, which would cause `many` to throw an exception.\n]\n\n[``\nval @skipRestOfLine@: skipNewline: bool -> Parser\n``]\n[`skipRestOfLine skipNewline` is an optimized implementation of `restOfLine skipNewline |>> ignore`.]\n\n\n[``\n[#charsTillString-parsers]\nval @charsTillString@:\n string -> skipString: bool -> maxCount: int -> Parser\n``]\n[`charsTillString skipString maxCount` parses all chars before the first occurance of the string `str` and, if `skipString` is `true`, skips over `str`. It returns the parsed chars before the string. If more than `maxCount` chars come before the first occurance of `str`, the parser *fails after consuming* `maxCount` chars.\n\nNewlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) are counted as single chars and in the returned string all newlines are normalized to `\"\\n\"`, but `str` may not contain any newline.\n\n`charsTillString str maxCount` raises\n- an `ArgumentException`, if `str` contains a newline char (`'\\n'` or `'\\r'`),\n- an `ArgumentOutOfRangeException`, if `maxCount` is negative.\n]\n\n[``\nval @skipCharsTillString@:\n string -> skipString: bool -> maxCount: int -> Parser\n``]\n[`skipCharsTillString str maxCount` is an optimized implementation of `charsTillString str maxCount |>> ignore`.]\n\n\n[``\n\nval @charsTillStringCI@:\n string -> skipString: bool -> maxCount: int -> Parser\n``]\n[\n`charsTillStringCI str maxCount` parses all chars before the first case-insensitive occurance of the string `str` and, if `skipString` is `true`, skips over it. It returns the parsed chars before the string. If more than `maxCount` chars come before the first case-insensitive occurance of `str` the parser *fails after consuming* `maxCount` chars.\n\nNewlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) are counted as single chars, but `str` may not contain any newline.\n\n`charsTillStringCI str maxCount` raises\n- an `ArgumentException`, if `str` contains a newline char (`'\\n'` or `'\\r'`),\n- an `ArgumentOutOfRangeException`, if `maxCount` is negative.\n]\n\n[``\nval @skipCharsTillStringCI@:\n string -> skipString: bool -> maxCount: int -> Parser\n``]\n[`skipCharsTillStringCI str maxCount` is an optimized implementation of `charsTillStringCI str maxCount |>> ignore`.]\n\n[``\n\nval @manySatisfy@: (char -> bool) -> Parser\n``]\n[\n`manySatisfy f` parses a sequence of *zero* or more chars that satisfy the predicate function `f` (i.e. chars for which `f` returns `true`). It returns the parsed chars as a string.\n\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) is converted to the single char `'\\n'`. Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called with `'\\r'` and the string returned by `manySatisfy f` will never contain an `'\\r'`.\n\nFor example, `manySatisfy (function ' '|'\\t'|'\\n' -> true | _ -> false)` parses zero or more whitespaces and returns them as a string.\n\n[caution\nThe function predicate `f` must not access the currently used `CharStream` itself, because `manySatisfy` relies on `f` not having any side-effect on the internal state of the stream.\n]\n]\n\n[``\nval @manySatisfy2@: (char -> bool) -> (char -> bool) -> Parser\n``]\n[`manySatisfy2 f1 f` behaves like `manySatisfy f`, except that the first char of the parsed string must satisfy `f1` instead of `f`.\n\nFor example, `manySatisfy ((=) '.') isDigit` will parse a dot followed by zero or more decimal digits. If there is no dot, the parser succeeds with an empty string.\n]\n\n[``\nval @skipManySatisfy@: (char -> bool) -> Parser\n``]\n[`skipManySatisfy f` is an optimized implementation of `manySatisfy f |>> ignore`.]\n\n[``\nval @skipManySatisfy2@: (char -> bool) -> (char -> bool) -> Parser\n``]\n[`skipManySatisfy2 f1 f` is an optimized implementation of `manySatisfy2 f1 f |>> ignore`.]\n\n[``\n[#many1Satisfy-parsers]\nval @many1Satisfy@: (char -> bool) -> Parser\n``]\n[`many1Satisfy f` parses a sequence of *one* or more chars that satisfy the predicate function `f` (i.e. chars for which `f` returns `true`). It returns the parsed chars as a string. If the first char does not satisfy `f`, this parser fails without consuming input.\n\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) is converted to the single char `'\\n'`. Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called with `'\\r'` and the string returned by `many1Satisfy f` will never contain an `'\\r'`.\n\nFor example, `many1Satisfy isDigit` parses a number consisting of one or more decimal digits and returns it as a string.\n\n[caution\nThe function predicate `f` must not access the currently used `CharStream` itself, because `many1Satisfy` relies on `f` not having any side-effect on the internal state of the stream.\n]\n\n[note\nIf the parser `many1Satisfy f` fails, it returns no descriptive error message (because it does not know what chars `f` accepts); hence it should only be used together with other parsers that take care of a potential error. Alternatively, `many1SatisfyL f label` can be used to ensure a more descriptive error message.\n]\n]\n\n[``\nval @many1Satisfy2@: (char -> bool) -> (char -> bool) -> Parser\n``]\n[`many1Satisfy2 f1 f` behaves like `many1Satisfy f`, except that the first char of the parsed string must satisfy `f1` instead of `f`.\n\nFor example, `many1Satisfy2 isLetter (fun c -> isLetter c || isDigit c)` will parse any string consisting of one letter followed by zero or more letters or digits.\n]\n\n[``\nval @skipMany1Satisfy@: (char -> bool) -> Parser\n``]\n[`skipMany1Satisfy f` is an optimized implementation of `many1Satisfy f |>> ignore`.]\n\n[``\nval @skipMany1Satisfy2@: (char -> bool) -> (char -> bool) -> Parser\n``]\n[`skipMany1Satisfy2 f1 f` is an optimized implementation of `many1Satisfy2 f1 f |>> ignore`.]\n\n[``\n\nval @many1SatisfyL@:\n (char -> bool) -> string -> Parser\n``]\n[`many1SatisfyL f label` is an optimized implementation of `many1Satisfy f label`.]\n\n[``\nval @many1Satisfy2L@:\n (char -> bool) -> (char -> bool) -> string -> Parser\n``]\n[`many1Satisfy2L f1 f label` is an optimized implementation of `many1Satisfy2 f1 f label`.]\n\n[``\nval @skipMany1SatisfyL@:\n (char -> bool) -> string -> Parser\n``]\n[`skipMany1SatisfyL f label` is an optimized implementation of `skipMany1Satisfy f label`.]\n\n[``\nval @skipMany1Satisfy2L@:\n (char -> bool) -> (char -> bool) -> string -> Parser\n``]\n[`skipMany1Satisfy2L f1 f label` is an optimized implementation of `skipMany1Satisfy2 f1 f label`.]\n\n[``\n[#manyMinMaxSatisfy-parsers]\nval @manyMinMaxSatisfy@:\n int -> int -> (char -> bool) -> Parser\n``]\n[`manyMinMaxSatisfy minCount maxCount f` parses a sequence of `minCount` or more chars that satisfy the predicate function `f` (i.e. chars for which `f` returns `true`), but not more than `maxCount` chars. It returns the parsed chars as a string. This parser is atomic, i.e. if the first `minCount` chars do not all satisfy `f`, the parser fails without consuming any input.\n\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) is converted to the single char `'\\n'`. Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called with `'\\r'` and the string returned by `manyMinMaxSatisfy minCount maxCount f` will never contain an `'\\r'`.\n\n`manyMinMaxSatisfy minCount maxCount f` raises an `ArgumentOutOfRangeException` if `maxCount` is negative.\n\nFor example, `manyMinMaxSatisfy 4 8 isHex` parses a string that consists of at least 4 hexadecimal digits. If there are 8 or more hex chars, this parser stops after the 8th.\n\n[caution\nThe function predicate `f` must not access the currently used `CharStream` itself, because `manyMinMaxSatisfy` relies on `f` not having any side-effect on the internal state of the stream.\n]\n\n[note\nIf the parser `manyMinMaxSatisfy minCount maxCount f` fails, it returns no descriptive error message (because it does not know what chars `f` accepts); hence it should only be used together with other parsers that take care of a potential error. Alternatively, `manyMinMaxSatisfyL f label` can be used to ensure a more descriptive error message.\n]\n\n]\n\n[``\nval @manyMinMaxSatisfy2@:\n int -> int -> (char -> bool) -> (char -> bool) -> Parser\n``]\n[`manyMinMaxSatisfy2 minCount maxCount f1 f` behaves like `manyMinMaxSatisfy minCount maxCount f`, except that the first char of the parsed string must satisfy `f1` instead of `f`.\n\nFor example, `manyMinMaxSatisfy2 3 5 ((=) '.') isDigit` parses a dot followed by 2-4 decimal digits.\n]\n\n[``\nval @skipManyMinMaxSatisfy@:\n int -> int -> (char -> bool) -> Parser\n``]\n[`skipManyMinMaxSatisfy minCount maxCount f` is an optimized implementation of `manyMinMaxSatisfy minCount maxCount f |>> ignore`.]\n\n[``\nval @skipManyMinMaxSatisfy2@:\n int -> int -> (char -> bool) -> (char -> bool) -> Parser\n``]\n[`skipManyMinMaxSatisfy2 minCount maxCount f1 f` is an optimized implementation of `manyMinMaxSatisfy2 minCount maxCount f1 f |>> ignore`.]\n\n[``\nval @manyMinMaxSatisfyL@:\n int -> int -> (char -> bool) -> string -> Parser\n``]\n[`manyMinMaxSatisfyL minCount maxCount f label` is an optimized implementation of `manyMinMaxSatisfy minCount maxCount f label`.]\n\n[``\nval @manyMinMaxSatisfy2L@:\n int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser\n``]\n[`manyMinMaxSatisfy2L minCount maxCount f1 f label` is an optimized implementation of `manyMinMaxSatisfy2 minCount maxCount f1 f label`.]\n\n[``\nval @skipManyMinMaxSatisfyL@:\n int -> int -> (char -> bool) -> string -> Parser\n``]\n[`skipManyMinMaxSatisfyL minCount maxCount f label` is an optimized implementation of `skipManyMinMaxSatisfy minCount maxCount f label`.]\n\n[``\nval @skipManyMinMaxSatisfy2L@:\n int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser\n``]\n[`skipManyMinMaxSatisfy2L minCount maxCount f1 f label` is an optimized implementation of `skipManyMinMaxSatisfy2 minCount maxCount f1 f label`.]\n\n[``\n\nval @regex@: string -> Parser\n``]\n[\n`regex pattern` matches the .NET [url \"https://msdn.microsoft.com/en-us/library/az24scfc.aspx\" regular expression] given by the string `pattern` on the chars beginning at the current index in the input stream. If the regular expression matches, the parser skips the matched chars and returns them as a string. If the regular expression does not match, the parser fails without consuming input.\n\nThe `System.Text.RegularExpressions.Regex` object that is internally used to match the pattern is constructed with the `[url \"https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.regexoptions.aspx\" RegexOptions]` `MultiLine` and `ExplicitCapture`. In order to ensure that the regular expression can only match at the beginning of a string, `\"\\\\A\"` is automatically prepended to the pattern. You should avoid the use of greedy expressions like `\".*\"`, because these might trigger a scan of the complete input every time the regex is matched.\n\nNewline chars (`'\\r'` and `'\\n'`) in the pattern are interpreted literally. For example, an `'\\n'` char in the pattern will only match `\"\\n\"`, not `\"\\r\"` or `\"\\r\\n\"`. However, in the returned string all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) are normalized to `\"\\n\"`.\n\nFor large files the regular expression is *not* applied to a string containing *all* the remaining chars in the stream. The number of chars that are guaranteed to be visible to the regular expression is specified during construction of the `CharStream`. If one of the @runParser functions@ is used to run the parser, this number is 43690.\n]\n\n[``\n\ntype @IdentifierOptions@ =\n new: ?isAsciiIdStart: (char -> bool) *\n ?isAsciiIdContinue: (char -> bool) *\n ?normalization: System.Text.NormalizationForm *\n ?normalizeBeforeValidation: bool *\n ?allowJoinControlChars: bool *\n ?preCheckStart: (char -> bool) *\n ?preCheckContinue: (char -> bool) *\n ?allowAllNonAsciiCharsInPreCheck: bool *\n ?label: string *\n ?invalidCharMessage: string -> IdentifierOptions\n``]\n[\n\nThe configuration options for the `identifier` parser.\n\n[dl\n[`isAsciiIdStart`]\n[\nSpecifies the ASCII characters that are valid as the first character of an identifier. This predicate function is called once for each char in the range `'\\u0001'`--`'\\u007f'` during construction of the `IdentifierOptions` object. By default, the ASCII chars `'A'`--`'Z'` and `'a'`--`'z'` can start an identifier.\n]\n\n[`isAsciiIdContinue`]\n[\nSpecifies the ASCII characters that are valid as non-first characters of an identifier. This predicate function is called once for each char in the range `'\\u0001'`--`'\\u007f'` during construction of the `IdentifierOptions` object. Normally the chars for which `isAsciiIdContinue` returns `true` should include all chars for which `isAsciiIdStart` returns `true`. By default, the ASCII chars `'A'`--`'Z'`, `'a'`--`'z'`, `'0'`--`'9'` and `'_'` are accepted at non-start positions.\n]\n\n[`normalization`]\n[\n[small [/ This option is not supported in the Silverlight version of FParsec.]][br]\nThe [url \"https://www.Unicode.org/reports/tr15/\" normalization form] to which identifier strings are normalized. The value must be one of the four enum values of `System.Text.NormalizationForm`. If no `normalization` parameter is given, no normalization is performed.\n\nThe normalization is performed with the [url \"https://msdn.microsoft.com/en-us/library/ebza6ck1.aspx\" `System.String.Normalize`] method provided by the Base Class Library.\n]\n\n[`normalizeBeforeValidation`]\n[\n[small [/ This option is not supported in the Silverlight version of FParsec.]][br]\nIndicates whether the identifier string should be normalized before validation (but after the pre-check). By default, identifiers are normalized after they have been validated. Normalization before validation will only work properly with non-default pre-check options.\n]\n\n[`allowJoinControlChars`]\n[\nIndicates whether the two join control characters ([url \"https://en.wikipedia.org/wiki/Zero-width_non-joiner\" zero-width non-joiner] and [url \"https://en.wikipedia.org/wiki/Zero-width_joiner\" zero-width joiner]) are allowed at any non-start character position in the identifier.\n]\n\n[`preCheckStart`, `preCheckContinue`]\n[\nThese two char predicates are used to identify potential identifier strings in the input. The first UTF-16 char of an identifier must satisfy `preCheckStart`, the following chars must satify `preCheckContinue`. Input chars that don't pass the pre-check aren't included in the identifier string, while characters that pass the pre-check but not the identifier validation trigger a parser error. For the `identifier` parser to work properly, the pre-check functions must accept a superset of valid identifier characters.\n\nIf you specify no `preCheckStart` (`preCheckContinue`) parameter, a default function will be used that accepts all chars that satisfy `isAsciiIdStart` (`isAsciiIdContinue`) as well as all non-ASCII characters in the Basic Multilingual Plane with the =XID_Start= (=XID_Continue=) property and all surrogate chars. `preCheckContinue` by default also accepts the two join control characters.\n\nIf you pass the option `allowAllNonAsciiCharsInPreCheck = true`, the pre-check predicates are only called once for each char in the range `'\\u0001'` - `'\\u007f'` during construction of the `IdentifierOptions` object (in order to construct a lookup table).\n]\n\n[`allowAllNonAsciiCharsInPreCheck`]\n[\nIndicates whether all non-ASCII chars should be accepted in the pre-check, irrespective of whether the (default) pre-check functions return `true` for these chars.\n]\n\n[`label`]\n[\nThe string label that is used in error messages if no identifier is found. The default is `\"identifier\"`.\n]\n\n[`invalidCharMessage`]\n[\nThe error message that is reported when an invalid char is found during validation of an identifier (after the pre-check). The default is `\"The identifier contains an invalid character at the indicated position.\"`.\n]\n]\n\nThe following example implements a parser for [url \"https://docs.python.org/3/index.html\" Python] identifiers as described in [url \"https://www.python.org/dev/peps/pep-3131/\" PEP-3131]:\n``\nlet pythonIdentifier =\n let isAsciiIdStart = fun c -> isAsciiLetter c || c = '_'\n let isAsciiIdContinue = fun c -> isAsciiLetter c || isDigit c || c = '_'\n\n identifier (IdentifierOptions(\n isAsciiIdStart = isAsciiIdStart,\n isAsciiIdContinue = isAsciiIdContinue,\n normalization = [url \"https://msdn.microsoft.com/en-us/library/system.text.normalizationform.aspx\" System.Text.NormalizationForm].FormKC,\n normalizeBeforeValidation = true,\n allowAllNonAsciiCharsInPreCheck = true))\n``\n\n\n]\n\n[``\n\nval @identifier@: IdentifierOptions -> Parser\n``]\n[The `identifier` parser is a configurable parser for the XID identifier syntax specified in the [url \"http://www.Unicode.org/reports/tr31/\" Unicode Standard Annex #31].\n\nBy default, a valid identifier string must begin with a Unicode character with the =XID_Start= property and continue with zero or more characters with the =XID_Continue= property. The specification of which characters have these properties can be found in the [url \"http://www.Unicode.org/Public/8.0.0/ucd/DerivedCoreProperties.txt\" DerivedCoreProperties] file in the [url \"http://www.Unicode.org/usc\" Unicode Character Database]. Currently FParsec implements the XID specification of Unicode 8.0.0.\n\nWithin the ASCII character range `'\\u0001'`--`'\\u007f'` you can customize the set of accepted characters through the `isAsciiIdStart` and `isAsciiIdContinue` parameters (the XID default allows `'a'`--`'z'` and `'A'`--`'Z'` at any position and `'_'` and `'0'`--`'9'` only in non-start positions). For example, to accept the same ASCII characters that are valid in F# identifiers, you could use the following `IdentifierOptions`:\n``\nlet isAsciiIdStart c =\n isAsciiLetter c || c = '_'\n\nlet isAsciiIdContinue c =\n isAsciiLetter c || isDigit c || c = '_' || c = '\\''\n\nidentifier (IdentifierOptions(isAsciiIdStart = isAsciiIdStart,\n isAsciiIdContinue = isAsciiIdContinue))\n``\n\nBy default, identifiers cannot contain the two join control characters [url \"https://en.wikipedia.org/wiki/Zero-width_non-joiner\" zero-width non-joiner] and [url \"https://en.wikipedia.org/wiki/Zero-width_joiner\" zero-width joiner]. While these characters can be abused to create distinct identifiers that look confusingly similar or even identical, they are also necessary to create identifiers with the correct visual appearance for common words or phrases in certain languages. [url \"http://www.Unicode.org/reports/tr31/#Layout_and_Format_Control_Characters\" Section 2.3] of the Unicode Standard Annex #31 recommends to accept join control characters if the identifier system is supposed to support \"natural representations of terms in modern, customary use\". However, in order to minimize the potential for abuse it also recommends accepting these characters only in some very specific contexts.\n\nUnfortunately, the proposed rules describing the contexts in which join control character should be allowed are rather difficult to implement, especially with the limited Unicode support in .NET. For this reason the `identifier` parser currently only supports a simpler option: if you set the parameter `allowJoinControlChars` to `true`, the parser accepts the two join control characters in any non-start position. Whether this setting is a reasonable compromise between not supporting join control characters at all and implementing the complicated rules proposed in Annex #31 obviously depends on the individual requirements of your project. An example of a programming language that [url \"https://mail.mozilla.org/pipermail/es5-discuss/2009-June/002832.html\" adopted] the same compromise is [url \"http://www.ecma-international.org/publications/standards/Ecma-262.htm\" ECMAScript 5].\n\nApart from the joint control characters, no layout or format control characters are allowed in identifiers. This is in accordance to the recommendation of the Unicode Standard Annex #31, but contrary to what Annex #15 [url \"http://www.Unicode.org/reports/tr15/tr15-23.html#Programming_Language_Identifiers\" recommended] prior to Unicode version 4.1. Programming languages whose identifier syntax is based on the recommendations of earlier versions of the Unicode standard may require that layout and format control characters are ignored or filtered out, as for example is the case for C\\#. However, since the identifier syntax of these languages isn't based on the XID properties, one can't parse their identifiers with this parser anyway.\n\nBy providing a value for the `normalization` parameter, you can ensure that identifiers are returned in a particular Unicode [url \"http://www.Unicode.org/reports/tr15/\" normalization form]. By default, an identifier is normalized *after* it has been validated. Since XID identifiers are \"closed under normalization\", a valid identifier is guaranteed to stay valid after normalization. The reverse, however, is not true, since not all identifier strings that are valid after normalization are also valid prior to normalization. If you want the identifier string to be normalized before validation, you have to set the `normalizeBeforeValidation` parameter to `true` and specify appropriate `preCheckStart` and `preCheckContinue` parameters.\n\nSilverlight does not support Unicode normalization, so the Silverlight version of FParsec does not support the `normalization` and `normalizeBeforeValidation` parameters.\n\nThe `identifier` parser uses the `preCheckStart` and `preCheckContinue` predicate functions to identify potential identifier strings in the input. The first UTF-16 char of the identifier must satisfy `preCheckStart`, the following chars must satifsy `preCheckContinue`. Input chars that don't pass the pre-check aren't included in the identifier string, while characters that pass the pre-check but not the identifier validation trigger a parser error (`FatalError`). For the `identifier` parser to work properly, the `preCheck` functions must accept a superset of valid identifier characters.\n\nIf you specify no `preCheckStart` (`preCheckContinue`) parameter, a default function will be used that accepts all chars that satisfy `isAsciiIdStart` (`isAsciiIdContinue`) as well as all non-ASCII characters in the Basic Multilingual Plane with the =XID_Start= (=XID_Continue=) property and all surrogate chars. `preCheckContinue` by default also accepts the two join control characters. If you set the parameter `allowAllNonAsciiCharsInPreCheck` to `true`, all non-ASCII chars will be accepted in the pre-check, irrespective of whether the (default) pre-check functions return `true` for these chars.\n\nBy passing custom `preCheckStart` and `preCheckContinue` functions you can modify the error reporting behaviour and support identifier strings that are only valid after normalization. You can also exclude specific UTF-16 chars that would otherwise be valid in identifiers, though you'd have to be careful to cover all (pre-)normalization forms.\n\nIn the following examples we will demonstrate the effect of custom pre-check functions on identifier parsing. For this we first set up two identifier parsers, `ident` and `identP`, with differing sets of options. Both parsers accept the same ASCII chars in identifiers. In particular, both do not accept the underscore char `'_'` in identifiers. However, only `identP` lets underscores through the pre-check.\n\n``\n// we don't allow underscores in identifiers ...\nlet isAsciiIdStart c = isAsciiLetter c\nlet isAsciiIdContinue c = isAsciiLetter c || isDigit c\n\n// ... but accept them in in the pre-check\nlet preCheckStart c = isAsciiLetter c || c = '_'\nlet preCheckContinue c = isAsciiLetter c || isDigit c || c = '_'\n\ntype NF = System.Text.NormalizationForm\n\nlet opts = IdentifierOptions(isAsciiIdStart = isAsciiIdStart,\n isAsciiIdContinue = isAsciiIdContinue,\n normalization = NF.FormKC,\n // The following option isn't really useful without\n // modified pre-check options. We only set the\n // option here to prove this point in an example below.\n normalizeBeforeValidation = true)\n\nlet optsWithPreCheck = IdentifierOptions(isAsciiIdStart = isAsciiIdStart,\n isAsciiIdContinue = isAsciiIdContinue,\n preCheckStart = preCheckStart,\n preCheckContinue = preCheckContinue,\n allowAllNonAsciiCharsInPreCheck = true,\n normalization = NF.FormKC,\n normalizeBeforeValidation = true)\n\nlet ident : Parser = identifier opts\nlet identP : Parser = identifier optsWithPreCheck\n``\n\nBoth `ident` and `identP` parse simple identifiers without a problem:\n``{fsi}\n> run (ident .>> eof) \"täst1\";;\nval it : ParserResult = Success: \"täst1\"\n> run (identP .>> eof) \"täst2\";;\nval it : ParserResult = Success: \"täst2\"\n``\n\nThe identifier parser with the default pre-check functions will treat underscores just like whitespace or any other non-identifier character:\n``{fsi}\n> run (ident .>> eof) \"test_id\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 5\ntest_id\n ^\nExpecting: end of input\n``\n\nSince `ident` only consumed the `\"test\"` part of the input string, the `eof` parser complained that it was expecting to be applied at the end of the input.\n\nWhen we use `identP` instead, we get a different error message:\n``{fsi}\n> run (identP .>> eof) \"test_id\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 5\ntest_id\n ^\nThe identifier contains an invalid character at the indicated position.\n``\nThis time the underscore passed the pre-check, but not the identifier validation.\n\nAs mentioned above, a custom pre-check is also neccessary to make the `normalizeBeforeValidation` option work properly. With the default pre-check options the identifier parser doesn't accept `\"MC²\"` as an identifier, even with the normalization set to NFKC:\n``{fsi}\n> run (ident .>> eof) \"MC²\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 3\nMC²\n ^\nExpecting: end of input\n``\n\n`identP` on the other hand doesn't have this issue, because it accepts all non-ASCII chars in the pre-check:\n``{fsi}\n> run (identP .>> eof) \"MC²\";;\nval it : ParserResult = Success: \"MC2\"\n``\n]\n\n[``\n\n// Parsing strings with the help of other parsers\n// ==============================================\n``]\n\n[``\n\nval @manyChars@: Parser -> Parser\n``]\n[\n`manyChars cp` parses a sequence of *zero* or more chars with the char parser `cp`. It returns the parsed chars as a string.\n\n`manyChars cp` is an optimized implementation of `many cp` that returns the chars as a string instead of a char list.\n\nMany string parsers can be conveniently implemented with both `manyChars` and `manySatisfy`. In these cases you should generally prefer the faster `manySatisfy`. For example, the parser `manySatisfyL isHex \"hex integer\"` is more efficient than `manyChars hex`.\n\nIf you are using `manyChars` for a parser similar to `manyChars (notFollowedBy endp >>. p)`, you should check whether this use of `manyChars` can be replaced with the more specialized `manyCharsTill` parser.\n]\n\n[``\nval @manyChars2@: Parser -> Parser -> Parser\n``]\n[\n`manyChars2 cp1 cp` behaves like `manyChars2 cp`, except that it parses the first char with `cp1` instead of `cp`.\n\nFor example, `manyChars2 letter (letter <|> digit)` will parse a letter followed by letters or digits and return the chars as a string. If the first char is not a letter, the parser succeeds with an empty string. Note, however, that this parser could be more efficiently implemented using `manySatisfy2L`.\n]\n\n[``\n\nval @many1Chars@: Parser -> Parser\n``]\n[\n`many1Chars cp` parses a sequence of *one* or more chars with the char parser `cp`. It returns the parsed chars as a string.\n\n`many1Chars cp` is an optimized implementation of `many1 cp` that returns the chars as a string instead of a char list.\n\nMany string parsers can be conveniently implemented with both `many1Chars` and `many1Satisfy`. In these cases you should generally prefer the faster `many1Satisfy`. For example, the parser `many1SatisfyL isHex \"hex integer\"` is more efficient than `many1Chars hex`.\n]\n\n[``\nval @many1Chars2@: Parser -> Parser -> Parser\n``]\n[\n`many1Chars2 cp1 cp` behaves like `many1Chars2 cp`, except that it parses the first char with `cp1` instead of `cp`.\n\nFor example, `many1Chars2 letter (letter <|> digit)` will parse a letter followed by letters or digits and return the chars as a string. Note, however, that this parser could be more efficiently implemented using `many1Satisfy2L`.\n]\n\n[``\n[#manyCharsTill-parsers]\nval @manyCharsTill@:\n Parser -> Parser<'b,'u> -> Parser\n``]\n[\n`manyCharsTill cp endp` parses chars with the char parser `cp` until the parser `endp` succeeds. It stops after `endp` and returns the parsed chars as a string.\n\n`manyCharsTill cp endp` is an optimized implementation of `manyTill cp endp` that returns the chars as a string instead of a char list.\n]\n[``\nval @manyCharsTill2@:\n Parser -> Parser -> Parser<'b,'u> -> Parser\n``]\n[\n`manyCharsTill2 cp1 cp endp` behaves like `manyCharsTill cp endp`, except that it parses the first char with `cp1` instead of `cp`.\n]\n\n\n[``\nval @manyCharsTillApply@:\n Parser -> Parser<'b,'u> -> (string -> 'b -> 'c)\n -> Parser<'c,'u>\n``]\n[\n`manyCharsTillApply cp endp f` behaves like `manyCharsTill cp endp`, except that it returns the result of the function application `f str b`, where `str` is the parsed string and `b` is result returned by `endp`.\n]\n\n[``\nval @manyCharsTillApply2@:\n Parser -> Parser -> Parser<'b,'u> -> (string -> 'b -> 'c)\n -> Parser<'c,'u>\n``]\n[\n`manyCharsTillApply2 cp1 cp endp f` behaves like `manyCharsTillApply cp endp f`, except that it parses the first char with `cp1` instead of `cp`.\n]\n\n\n\n[``\n\nval @many1CharsTill@:\n Parser -> Parser<'b,'u> -> Parser\n``]\n[\n`many1CharsTill cp endp` parses one char with the char parser `cp`. Then it parses more chars with `cp` until the parser `endp` succeeds. It stops after `endp` and returns the parsed chars as a string.\n\n`many1CharsTill cp endp` is an optimized implementation of `many1Till cp endp` that returns the chars as a string instead of a char list.\n]\n[``\nval @many1CharsTill2@:\n Parser -> Parser -> Parser<'b,'u> -> Parser\n``]\n[\n`many1CharsTill2 cp1 cp endp` behaves like `many1CharsTill cp endp`, except that it parses the first char with `cp1` instead of `cp`.\n]\n\n[``\nval @many1CharsTillApply@:\n Parser -> Parser<'b,'u> -> (string -> 'b -> 'c)\n -> Parser<'c,'u>\n``]\n[\n`many1CharsTillApply cp endp f` behaves like `many1CharsTill cp endp`, except that it returns the result of the function application `f str b`, where `str` is the parsed string and `b` is result returned by `endp`.\n]\n\n[``\nval @many1CharsTillApply2@:\n Parser -> Parser -> Parser<'b,'u> -> (string -> 'b -> 'c)\n -> Parser<'c,'u>\n``]\n[\n`many1CharsTillApply2 cp1 cp endp f` behaves like `many1CharsTillApply cp endp f`, except that it parses the first char with `cp1` instead of `cp`.\n]\n\n[``\n[#manyStrings-parsers]\nval @manyStrings@: Parser -> Parser\n``]\n[\n`manyStrings sp` parses a sequence of *zero* or more strings with the string parser `sp`. It returns the strings in concatenated form.\n\n `manyStrings sp` is an optimized implementation of `many sp |>> List.fold (fun acc s -> acc + s) \"\"`.\n]\n\n[``\nval @manyStrings2@: Parser -> Parser -> Parser\n``]\n[\n`manyStrings2 sp1 sp` behaves like `manyStrings sp`, except that it parses the first string with `sp1` instead of `sp`.\n]\n\n[``\nval @many1Strings@: Parser -> Parser\n``]\n[\n`many1Strings sp` parses a sequence of *one* or more strings with the string parser `sp`. It returns the strings in concatenated form.\nNote that `many1Strings sp` does not require the first string to be non-empty.\n\n`many1Strings sp` is an optimized implementation of `many1 sp |>> List.reduce (+)`.\n]\n\n\n[``\nval @many1Strings2@: Parser -> Parser -> Parser\n``]\n[\n`many1Strings2 sp1 sp` behaves like `many1Strings sp`, except that it parses the first string with `sp1` instead of `sp`.\n]\n\n\n[``\n\nval @stringsSepBy@: Parser -> Parser -> Parser\n``]\n[\n`stringsSepBy sp sep` parses *zero* or more occurrences of the string parser `sp` separated by `sep`\n(in EBNF: `{EBNF}(sp (sep sp)*)?`). It returns the strings parsed by `sp` *and* `sep` in concatenated form.\n\n`stringsSepBy` behaves like `sepBy`, except that instead of returning a list of the results of only the first argument parser it returns a concatenated string of all strings returned by both argument parsers (in the sequence they occurred).\n\nWith `stringsSepBy` you can for example implement an efficient parser for the following string literal format:\n\n``{other}\n stringLiteral: '\"' (normalChar|escapedChar)* '\"'\n normalChar: any char except '\\' and '\"'\n escapedChar: '\\\\' ('\\\\'|'\"'|'n'|'r'|'t')\n``\n\nThe parser implementation exploits the fact that two (possibly empty) normal char snippets must be separated by an escaped char:\n\n``\nlet stringLiteral =\n let str s = pstring s\n let normalCharSnippet = manySatisfy (fun c -> c <> '\\\\' && c <> '\"')\n let escapedChar = str \"\\\\\" >>. (anyOf \"\\\\\\\"nrt\" |>> function\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c)\n between (str \"\\\"\") (str \"\\\"\")\n (stringsSepBy normalCharSnippet escapedChar)\n``\n\n]\n\n[``\nval @stringsSepBy1@: Parser -> Parser -> Parser\n``]\n[\n`stringsSepBy1 sp sep` parses *one* or more occurrences of the string parser `sp` separated by `sep`\n(in EBNF: `{EBNF}(sp (sep sp)*)`). It returns the strings parsed by `sp` *and* `sep` in concatenated form.\n\n`stringsSepBy1` behaves like `stringsSepBy`, except that it fails without consuming input if `sp` does not succeed at least once.\n]\n\n[``\n\nval @skipped@: Parser -> Parser\n``]\n[`skipped p` applies the parser `p` and returns the chars skipped over by `p` as a string.\nAll newlines (`\"\\r\\n\"`, `\"\\r\"` or `\"\\n\"`) are normalized to `\"\\n\"`.]\n\n[``\n\nval @withSkippedString@: (string -> 'a -> 'b) -> Parser<'a,'u> -> Parser<'b,'u>\n``]\n[`p |> withSkippedString f` applies the parser `p` and returns the result `f str x`,\nwhere `str` is the string skipped over by `p` and `x` is the result returned by `p`.]\n\n[``\n\n// Parsing numbers\n// ===============\n``]\n\n[``\ntype @NumberLiteralOptions@``]\n[\n\nAn enum type that encodes the various options of the `numberLiteral` parser:\n\n``\ntype NumberLiteralOptions =\n| None = 0\n| AllowSuffix = 0b000000000001\n| AllowMinusSign = 0b000000000010\n| AllowPlusSign = 0b000000000100\n| AllowFraction = 0b000000001000\n| AllowFractionWOIntegerPart = 0b000000010000\n| AllowExponent = 0b000000100000\n| AllowHexadecimal = 0b000001000000\n| AllowBinary = 0b000010000000\n| AllowOctal = 0b000100000000\n| AllowInfinity = 0b001000000000\n| AllowNaN = 0b010000000000\n\n| IncludeSuffixCharsInString = 0b100000000000\n\n| DefaultInteger = 0b000111000110\n| DefaultUnsignedInteger = 0b000111000000\n| DefaultFloat = 0b011001101110\n``\n\nIf all flags are set any literal matching the following regular expression is accepted:\n\n``{regex}\n[+-]?((([0-9]+(\\.[0-9]*)?|\\.[0-9]+)([eE][+-]?[0-9]+)?\n |0[xX]([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n |0[oO][0-7]+\n |0[bB][01]+\n )[a-zA-Z]?[a-zA-Z]?[a-zA-Z]?[a-zA-Z]?\n |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n |[nN][aA][nN]\n )\n``\n\n\nHexadecimal literals must begin with `{regex}0x` or `{regex}0X`, octal literals with `{regex}0o` or `{regex}0O` and binary literals with `{regex}0b` or `{regex}0B`. If the respective flags are set, hexadecimal *floating-point* literals as supported by IEEE 754r, C99 and Java are accepted.\n\nSome remarks on the individual flags:\n[dl\n\n[`AllowSuffix`]\n[Allows up to 4 suffix chars. Such chars are used in many programming languages to determine the type of a number. For example, in F# the literal `\"123UL\"` represents the unsigned 64-bit integer 123.]\n\n[`AllowFraction`]\n[Allows a fraction in decimal and hexadecimal literals.]\n\n[`AllowFractionWOIntegerPart`]\n[Allows number literals with a fraction but no integer part, e.g. `\".123\"` or `\"0x.abc\"`. This flag can only be used together with `AllowFraction`.]\n\n[`AllowExponent`]\n[Allows exponents in decimal literals (beginning with an `\"e\"` or `\"E\"`) and in hexadecimal literals (beginning with a `\"p\"` or `\"P\"`).]\n\n[`AllowInfinity`]\n[Allows `\"Inf\"` or `\"Infinity\"` literals (case-insensitive).]\n\n[`AllowNaN`]\n[Allows `\"NaN\"` literals (case-insensitive).]\n\n[`IncludeSuffixCharsInString`]\n[Instructs the `numberLiteral` parser to include any parsed suffix chars in the `NumberLiteral.String` member.]\n\n]\n]\n\n[`` = //...\n``]\n\n\n[``\n\ntype @NumberLiteral@``]\n[\nThe return type of the `numberLiteral` parser. An instance contains the parsed number literal and various bits of information about it.\nNote that the `String` member contains the string literal *without* the suffix chars,\nexcept if the `NumberLiteralOptions` passed to the `numberLiteral` parser have the\n`IncludeSuffixCharsInString` flag set.\nAny parsed suffix chars are always available through the `SuffixChar1` - `4` members.\n\n``\ntype NumberLiteral =\n member String: string\n\n member #SuffixLength#: int\n member #SuffixChar1#: char // EOS if no suffix char was parsed\n member #SuffixChar2#: char // EOS if less than 2 suffix chars were parsed\n member #SuffixChar3#: char ...\n member #SuffixChar4#: char\n\n member Info: NumberLiteralResultFlags\n member #HasMinusSign#: bool\n member #HasPlusSign#: bool\n member #HasIntegerPart#: bool\n member #HasFraction#: bool\n member #HasExponent#: bool\n member #IsInteger#: bool // not (HasFraction || HasExponent)\n member #IsDecimal#: bool\n member #IsHexadecimal#: bool\n member #IsBinary#: bool\n member #IsOctal#: bool\n member #IsNaN#: bool\n member #IsInfinity#: bool\n\nand NumberLiteralResultFlags =\n | None = 0\n | SuffixLengthMask = 0b0000000000001111\n | HasMinusSign = 0b0000000000010000\n | HasPlusSign = 0b0000000000100000\n | HasIntegerPart = 0b0000000001000000\n | HasFraction = 0b0000000010000000\n | HasExponent = 0b0000000100000000\n | IsDecimal = 0b0000001000000000\n | IsHexadecimal = 0b0000010000000000\n | IsBinary = 0b0000100000000000\n | IsOctal = 0b0001000000000000\n | BaseMask = 0b0001111000000000\n | IsInfinity = 0b0010000000000000\n | IsNaN = 0b0100000000000000\n``\n]\n\n[`` = //...\n``]\n\n\n[``\n\nval @numberLiteral@: NumberLiteralOptions -> string -> Parser\n``]\n[\n`numberLiteral options label` parses a number literal and returns the result in form of a `NumberLiteral` value. The given `NumberLiteralOptions` argument determines the kind of number literals accepted. The string `label` is used in the `Expected` error message that is generated when the parser fails without consuming input.\n\nThe parser fails without consuming input if not at least one digit (including the =0= in the format specifiers `\"0x\"` etc.) can be parsed. It fails after consuming input, if no decimal digit comes after an exponent marker or no valid digit comes after a format specifier.\n\nThe parser in the following example employs `numberLiteral` to parse decimal numbers as either `integer` or `float` values:\n``\nopen FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\ntype Number = Int of int64\n | Float of float\n\n // -?[0-9]+(\\.[0-9]*)?([eE][+-]?[0-9]+)?\nlet numberFormat = NumberLiteralOptions.AllowMinusSign\n ||| NumberLiteralOptions.AllowFraction\n ||| NumberLiteralOptions.AllowExponent\n\nlet pnumber : Parser =\n numberLiteral numberFormat \"number\"\n |>> fun nl ->\n if nl.IsInteger then Int (int64 nl.String)\n else Float (float nl.String)\n``\n\nSome test runs:\n``{fsi}\n> run pnumber \"123\";;\nval it : ParserResult = Success: Int 123L\n\n> run pnumber \"-123.456E-7\";;\nval it : ParserResult = Success: Float -1.23456e-05\n\n> run pnumber \"-\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n-\n^\nExpecting: number\n\n> run pnumber \"123.456E-a\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 10\n123.456E-a\n ^\nExpecting: decimal digit\n\n> run pnumber \"1E9999\";;\nSystem.OverflowException:\n Value was either too large or too small for a Double.\n at (... stack trace ...)\nstopped due to error\n``\n\nThe next example improves on the error reporting in case of overflows. It also demonstrates how to support hexadecimal numbers and a suffix to indicate the integer format:\n``\nopen FParsec\nopen FParsec.Error\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\ntype Number = Int32 of int32\n | Int64 of int64\n | Float of float\n\n// We want to support decimal or hexadecimal numbers with an optional minus\n// sign. Integers may have an 'L' suffix to indicate that the number should\n// be parsed as a 64-bit integer.\nlet numberFormat = NumberLiteralOptions.AllowMinusSign\n ||| NumberLiteralOptions.AllowFraction\n ||| NumberLiteralOptions.AllowExponent\n ||| NumberLiteralOptions.AllowHexadecimal\n ||| NumberLiteralOptions.AllowSuffix\n\nlet pnumber : Parser =\n let parser = numberLiteral numberFormat \"number\"\n fun stream ->\n let reply = parser stream\n if reply.Status = Ok then\n let nl = reply.Result // the parsed NumberLiteral\n if nl.SuffixLength = 0\n || ( nl.IsInteger\n && nl.SuffixLength = 1 && nl.SuffixChar1 = 'L')\n then\n try\n let result = if nl.IsInteger then\n if nl.SuffixLength = 0 then\n Int32 (int32 nl.String)\n else\n Int64 (int64 nl.String)\n else\n if nl.IsHexadecimal then\n Float (floatOfHexString nl.String)\n else\n Float (float nl.String)\n Reply(result)\n with\n | :? System.OverflowException as e ->\n stream.Skip(-nl.String.Length)\n Reply(FatalError, messageError e.Message)\n else\n stream.Skip(-nl.SuffixLength)\n Reply(Error, messageError \"invalid number suffix\")\n else // reconstruct error reply\n Reply(reply.Status, reply.Error)\n``\n\nSome test runs:\n``{fsi}\n> run pnumber \"123\";;\nval it : ParserResult = Success: Int32 123\n\n> run pnumber \"-0xffL\";;\nval it : ParserResult = Success: Int64 -255L\n\n> run pnumber \"123.123\";;\nval it : ParserResult = Success: Float 123.123\n\n> run pnumber \"0xabc.defP-4\";;\nval it : ParserResult = Success: Float 171.8044281\n\n> run pnumber \"-0x\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 4\n-0x\n ^\nNote: The error occurred at the end of the input stream.\nExpecting: hexadecimal digit\n\n> run pnumber \"0x123UL\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 6\n0x123UL\n ^\ninvalid number suffix\n\n> run pnumber \"1E9999\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n1E9999\n^\nValue was either too large or too small for a Double.\n``\n]\n\n[``\nval @numberLiteralE@:\n NumberLiteralOptions -> errorInCaseNoLiteralFound: ErrorMessageList\n -> CharStream<'u> -> Reply\n``]\n[`numberLiteralE` is an uncurried version of `numberLiteral` that can be used to implement number parsers without having to construct a `numberLiteral` closure.]\n\n[``\n\nval @pfloat@: Parser\n``]\n[\nParses a floating point number in the decimal format (in regular expression notation)\n``{regex}\n[0-9]+(\\.[0-9]*)?([eE][+-]?[0-9]+)?\n``\nor the hexadecimal format\n``{regex}\n0[xX][0-9a-fA-F]+(\\.[0-9a-fA-F]*)?([pP][+-]?[0-9]+)?\n``\n(as supported by IEEE 754r, C99 and Java, where e.g. `{regex}0x1f.cP-5` represents 31.75 * 2[sup -5]).\n\nThe special values `{regex}NaN` and `{regex}Inf(inity)?` (case-insensitive) are also recognized. All recognized numbers may be prefixed with a plus or minus sign.\n\nFractions without a leading digit, as for example \".5\", are *not* supported.\n\nThe parser fails\n- without consuming input, if not at least one digit (including the `{regex}0` in `{regex}0x`) can be parsed,\n- after consuming input, if no digit comes after an exponent marker or no hex digit comes after `{regex}0x`.\n\n[note Values that can't be represented as a finite `float` after rounding are parsed as plus or minus infinity. This behaviour changed between FParsec versions 1.0.3 and 1.0.10, following the [url \"https://docs.microsoft.com/en-us/dotnet/core/compatibility/2.2-3.0#floating-point-parsing-operations-no-longer-fail-or-throw-an-overflowexception\" respective behaviour change of `System.Double.Parse` on .NET Core 3]. ]\n\n[note The `pfloat` parser is based on the configurable `numberLiteral` parser. If you'd like to support a different floating-point format, there's a good chance you can implement a parser for that format by some simple changes to a copy of the `pfloat` source.]\n\n]\n\n[``\n[#pint-parsers]\nval @pint64@: Parser\n``]\n[\nParses a 64-bit signed integer number in the decimal, hexadecimal (`{regex}0[xX]`), octal (`{regex}0[oO]`) and binary (`{regex}0[bB]`) formats (in regular expression notation):\n``{regex}\n[+-]?([0-9]+\n |0[xX][0-9a-fA-F]+\n |0[oO][0-7]+\n |0[bB][01]+\n )\n``\n\n\nThe parser fails\n- without consuming input, if not at least one digit (including the `{regex}0` in the format specifiers `{regex}0x` etc.) can be parsed,\n- after consuming input, if no digit comes after an exponent marker or no digit comes after a format specifier,\n- after consuming input, if the value represented by the input string is greater than `System.Int64.MaxValue` or less than `System.Int64.MinValue`.\n]\n\n[``\nval @pint32@: Parser\n``]\n[\n`pint32` parses a 32-bit signed integer and behaves like `pint64`, except for the different return type and smaller integer range.\n]\n\n[``\nval @pint16@: Parser\n``]\n[\n`pint16` parses a 16-bit signed integer and behaves like `pint64`, except for the different return type and smaller integer range.\n]\n\n[``\nval @pint8@: Parser\n``]\n[\n`pint8` parses an 8-bit signed integer and behaves like `pint64`, except for the different return type and smaller integer range.]\n\n[``\n[#puint-parsers]\nval @puint64@: Parser\n``]\n[\n Parses numbers in the decimal, hexadecimal (`{regex}0[xX]`), octal (`{regex}0[oO]`) and binary (`{regex}0[bB]`) formats (in regular expression notation):\n``{regex}\n[0-9]+\n|0[xX][0-9a-fA-F]+\n|0[oO][0-7]+\n|0[bB][01]+\n``\nNote that the parser does not accept a leading plus sign.\n\nThe parser fails\n- without consuming input, if not at least one digit (including the `{regex}0` in the format specifiers `{regex}0x` etc.) can be parsed,\n- after consuming input, if no digit comes after an exponent marker or no digit comes after a format specifier,\n- after consuming input, if the value represented by the input string is greater than `System.UInt64.MaxValue`.\n]\n\n[``\nval @puint32@: Parser\n``]\n[\n`puint32` parses a 32-bit unsigned integer and behaves like `puint64`, except for the different return type and smaller integer range.\n]\n\n[``\nval @puint16@: Parser\n``]\n[\n`puint16` parses a 16-bit unsigned integer and behaves like `puint64`, except for the different return type and smaller integer range.\n]\n\n[``\nval @puint8@: Parser\n``]\n[\n`puint8` parses an 8-bit unsigned integer and behaves like `puint64`, except for the different return type and smaller integer range.\n]\n\n[``\n\n// Conditional parsing\n// ===================\n``]\n\n[``\n\nval @notFollowedByEof@: Parser\n``]\n[\n`notFollowedByEof` is an optimized implementation of `notFollowedByL eof \"end of input\"`.\n]\n\n[``\n\nval @followedByNewline@: Parser\n``]\n[`followedByNewline` is an optimized implementation of `followedByL newline \"newline\"`.]\n[``\nval @notFollowedByNewline@: Parser\n``]\n[`notFollowedByNewline` is an optimized implementation of `notFollowedByL newline \"newline\"`.]\n\n\n[``\n[#followedByString-parsers]\nval @followedByString@: string -> Parser\n``]\n[`followedByString str` is an optimized implementation of `followedByL (pstring str) (\"'\" + str + \"'\")`.]\n\n[``\nval @followedByStringCI@: string -> Parser\n``]\n[`followedByStringCI str` is an optimized implementation of `followedByL (pstringCI str) (\"'\" + str + \"'\")`.]\n\n[``\nval @notFollowedByString@: string -> Parser\n``]\n[`notFollowedByString str` is an optimized implementation of `notFollowedByL (pstring str) (\"'\" + str + \"'\")`.]\n\n[``\nval @notFollowedByStringCI@: string -> Parser\n``]\n[`notFollowedByStringCI str` is an optimized implementation of `notFollowedByL (pstringCI str) (\"'\" + str + \"'\")`.]\n\n\n[``\n[#charSatisfies-parsers]\nval @nextCharSatisfies@: (char -> bool) -> Parser\n``]\n[\n`nextCharSatisfies f` is an optimized implementation of `followedBy (satisfy f)`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n\n]\n\n[``\nval @nextCharSatisfiesNot@: (char -> bool) -> Parser\n``]\n[\n`nextCharSatisfiesNot f` is an optimized implementation of `notFollowedBy (satisfy f)`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n]\n\n[``\nval @next2CharsSatisfy@: (char -> char -> bool) -> Parser\n``]\n[\n`next2CharsSatisfy f` succeeds if the predicate function `f` returns `true`\nwhen applied to the next 2 chars in the input stream, otherwise it fails.\nIf there aren't 2 chars remaining in the input stream, this parser fails (as opposed to `next2CharsSatisfyNot`).\nThis parser never changes the parser state.\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the input is interpreted as a single char `'\\n'`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n]\n\n[``\nval @next2CharsSatisfyNot@: (char -> char -> bool) -> Parser\n``]\n[\n`next2CharsSatisfy f` succeeds if the predicate function `f` returns `false`\nwhen applied to the next 2 chars in the input stream, otherwise it fails.\nIf there aren't 2 chars remaining in the input stream, this parser succeeds (as opposed to `next2CharsSatisfy`).\nThis parser never changes the parser state.\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the input is interpreted as a single char `'\\n'`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n]\n\n[``\nval @previousCharSatisfies@: (char -> bool) -> Parser\n``]\n[\n`previousCharSatisfies f` succeeds if the predicate function `f` returns `true`\nwhen applied to the previous char in the stream, otherwise it fails.\nIf there is no previous char (because the input stream is at the beginning),\nthis parser fails (as opposed to `previousCharSatisfiesNot`).\nThis parser never changes the parser state.\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the input is interpreted as a single char `'\\n'`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n]\n\n[``\nval @previousCharSatisfiesNot@: (char -> bool) -> Parser\n``]\n[\n`previousCharSatisfiesNot f` succeeds if the predicate function `f` returns `false`\nwhen applied to the previous char in the stream, otherwise it fails.\nIf there is no previous char (because the stream is at the beginning),If this parser fails, it returns no descriptive error message; hence it should only be used\nthis parser succeeds (as opposed to `previousCharSatisfies`).\nThis parser never changes the parser state.\nAny newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the input is interpreted as a single char `'\\n'`.\n\n[note\nIf this parser fails, it returns no descriptive error message; hence it should only be used together with parsers that take care of a potential error.\n]\n]\n\n[``\n\n// Helper functions\n// ================\n``]\n\n[``\n[]\nval #EOS#: char = CharStream.Iterator.EndOfStreamChar\n``]\n\n[``\n\nval @foldCase@: string -> string\n``]\n[\nForwards all calls to `FParsec.Text.FoldCase`.\n]\n\n[``\n\nval @normalizeNewlines@: string -> string\n``]\n[\nForwards all calls to `FParsec.Text.NormalizeNewlines`.\n]\n\n[``\n\nval @floatToHexString@: float -> string\n``]\n[\n Returns a hexadecimal string representation of the `float` argument. The hexadecimal format is the one supported by IEEE 754r, C99 and Java. This function produces the same output as the [url \"http://java.sun.com/javase/6/docs/api/java/lang/Double.html#toHexString(double)\" `Double.toHexString`] method in Java.\n]\n\n[``\nval @floatOfHexString@: string -> float\n``]\n[\n Returns the float value represented by the given string in hexadecimal format. The supported input format is (in regular expression notation):\n``{regex}\n[+-]?((0[xX])?([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n |[nN][aA][nN]\n )\n``\n\n\nNote that no leading or trailing whitespace is allowed, neither are trailing format specifiers such as `{regex}f` or `{regex}d`.\n\nFor example, a valid input string is `{regex}0x1f.cP-5`, which represents the value 31.75 * 2[sup -5].\n\nThe numerical value represented by the input string is conceptually converted to an \"infinitely precise\" binary value that is then rounded to type `float` by the usual round-to-nearest (and ties-to-even) rule of IEEE 754 floating-point arithmetic. The special values `{regex}NaN` and `{regex}Inf(inity)?` (case insensitive) are also recognized. Signs of zero and Infinity values are preserved.\n\n A `System.FormatException` is raised if the string representation is invalid. A `System.OverflowException` is raised, if the value represented by the input string (after rounding) is greater than `System.Double.MaxValue` or less than `System.Double.MinValue`.\n]\n\n[``\nval @float32ToHexString@: float32 -> string\n``]\n[\nReturns a hexadecimal string representation of the `float32` argument. The hexadecimal format is the one supported by IEEE 754r, C99 and Java. This function produces the same output as the `[url \"http://java.sun.com/javase/6/docs/api/java/lang/Float.html#toHexString(float)\" Float.toHexString] ` method in Java.\n]\n\n[``\nval @float32OfHexString@: string -> float32\n``]\n[\nReturns the `float32` value represented by the given string in hexadecimal format. The supported input format is (in regular expression notation):\n\n``{regex}\n[+-]?((0[xX])?([0-9a-fA-F]+(\\.[0-9a-fA-F]*)?|\\.[0-9a-fA-F]+)([pP][+-]?[0-9]+)?\n |[iI][nN][fF]([iI][nN][iI][tT][yY])?\n |[nN][aA][nN]\n )\n``\n\nNote that no leading or trailing whitespace is allowed, neither are trailing format specifiers such as `{regex}f` or `{regex}d`.\n\nFor example, a valid input string is `{regex}0x1f.cP-5`, which represents the value 31.75 * 2[sup -5].\n\nThe numerical value represented by the input string is conceptually converted to an \"infinitely precise\" binary value that is then rounded to type `float32` by the usual round-to-nearest (and ties-to-even) rule of IEEE 754 floating-point arithmetic. The special values `NaN` and `Inf(inity)?` (case insensitive) are also recognized. Signs of zero and Infinity values are preserved.\n\n\nNote that in general `float32OfHexString(str)` is *not* equivalent to `float32 (floatOfHexString(str))`, because the latter version rounds twice.\n\nA `System.FormatException` is raised if the string representation is invalid. A `System.OverflowException` is raised, if the value represented by the input string (after rounding) is greater than `System.Float.MaxValue` or less than `System.Float.MinValue`.\n]\n\n[/interface-members]\n[/section]\n\n[/interface-reference]\n[/section]\n\n" }, { "path": "Doc/src/reference-charstream.txt", "content": "\n[section#CharStream FParsec.CharStream]\n\n[toc]\n[auto-link{hide-outer-auto-links = [\"UserState\"],\n links = [\"UserState\" : CharStream_1.members.UserState]}]\n[section#CharStream CharStream]\n\nProvides read-access to a sequence of UTF-16 chars.\n\n[interface-reference]\n\n[section Interface]\n[$$interface]\n[/section]\n\n[section Remarks]\n\nThe `CharStream` class provides a unified interface for efficiently reading UTF-16 chars from a binary stream or an in-memory char buffer (e.g. a string). It is optimized for the use in backtracking parser applications and supports arbitrary *char-based* seeking, even for streams larger than the addressable memory (on 32-bit platforms).\n\n[** The `CharStream` class is the base class of `[^CharStream_1 CharStream<'TUserState>\\ ]`], which adds a user-definable state component and some convenience methods for working with the state of a `CharStream` instance.\n\n[#block-wise A `CharStream` constructed from a `[url \"https://msdn.microsoft.com/en-us/library/system.io.stream.aspx\" System.IO.Stream]` or a file path reads the stream block-wise and only holds the most recently accessed block in memory.] The blocks overlap in order to provide efficient access on the boundary between blocks.\n\nIf the char content is already available as a string or a char array, a `CharStream` can be directly constructed from the char buffer (without needing to copy the buffer). The overhead of accessing an in-memory char buffer through a `CharStream` is minimal.\n\n[dl\n\n[Position information]\n[\nThe position of the next char in the stream is described by the following 4 properties:\n- `Index`, the index of the UTF-16 char in the stream,\n- `Line`, the line number for the next char,\n- `LineBegin`, the index of the first char of the line that also contains the next char,\n- `Name`, a description or identifier for the stream.\n\nThe `LineBegin` can be combined with the `Index` to calculate a `Column` number.\n\nAmong these properties the char index is the most important one, as the `CharStream` uses it to uniquely identify a UTF-16 char in the stream.\n\nThe other 3 properties further describe the text location of the char identified by the index, but they are not necessary for the core functionality of the `CharStream` class. The `CharStream` class keeps track of this additional position information to provide a more convenient interface to higher-level library functions, in particular to assist debugging and error reporting purposes.\n]\n\n[Newlines]\n[\nFor performance reasons the most basic stream operations do *not* automatically recognize [url \"https://en.wikipedia.org/wiki/Newline\" newlines] (end-of-line markers) in the stream content. If you skip any newline with these methods, you have to manually register the newline afterwards with one of the [^RegisterNewline-methods `RegisterNewline` methods] (otherwise the line and column count becomes incorrect).\n\nIn order to provide a convenient interface for parser routines, the `CharStream` class also provides [^methods-that-register-newlines some more advanced methods] that automatically register any skipped standard newline (`\"\\n\"`, `\"\\r\\n\"` and `\"\\r\"`). Additionally, it provides two methods that automatically register any Unicode newline (`SkipUnicodeWhitespace` and `SkipUnicodeNewline`).\n\nIt should be obvious from the method names which methods automatically register newlines and which don't.\n]\n\n[#Case-insensitive matching#]\n[\nThe `MatchCaseFolded` and `SkipCaseFolded` members match the content of the stream \"case-insensitively\" with a reference string. In this instance \"case-insensitive\" means that before the chars are matched with the reference string they are [url \"http://unicode.org/reports/tr21/tr21-5.html#Caseless_Matching\" mapped to a canonical form where case differences are erased]. For performance reasons `MatchCaseFolded` only applies the (non-Turkic) 1-to-1 [url \"http://www.unicode.org/Public/8.0.0/ucd/CaseFolding.txt\" case folding mappings] (v. 8.0.0) for Unicode code points in the Basic Multilingual Plane, i.e. code points below 0x10000. These mappings are sufficient for many case-insensitive parser grammars encountered in practice, but they are not appropriate for matching arbitrary natural language content. Please also note that the `CharStream` class performs no Unicode [url \"http://unicode.org/reports/tr15/\" normalization].\n]\n\n[ #Non-sequential access#]\n[\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nIf you construct a `CharStream` from a `System.IO.Stream` or a file path and you backtrack over a distance long enough to require the `CharStream` to reread a previous block, then the underlying **byte stream needs to support seeking**, otherwise a `NotSupportedException` is thrown. Furthermore, the [** [^Decoder Decoder] for the input [^Encoding Encoding] must be serializable] if you backtrack to a block other than the first in the stream. Note that *file streams created for regular disk files are always seekable and all the .NET standard decoders are serializable*. In order to support non-seekable streams for applications which don't require extensive backtracking, no exception will be thrown before an operation actually requires backtracking and the necessary capabilities of the stream or decoder are not available.\n]\n\n[Decoder errors]\n[\nA `CharStream` constructed from a binary input stream decodes the input data with the help of a `Decoder` instance obtained via the `Encodings`'s `GetDecoder` method. Depending on the configuration of the encoding the decoder might throw an exception if it encounters invalid byte sequences, usually a `System.Text.DecoderFallbackException` or a `System.IO.ArgumentException`.\n [fn The detection of invalid byte sequences by the .NET decoders is not entirely reliable. For example, `System.Text.UnicodeEncoding` (UTF-16) has an alignment related bug in .NET versions prior to 4.0 that sometimes leads to invalid surrogate pairs not being detected. The implementations of more complicated encodings, like GB18030, ISO-2022 and ISCII, also have [url \"https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=432648\" several issues] with regard to the detection of invalid input data.]\n]\n\n[Disposable interface]\n[\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nA `CharStream` holds managed and unmanaged resources that need to be explicitly released. Hence, it is very important that `CharStream` objects are promptly disposed after use. Where possible `CharStream` objects should only be used within a \"using\" block (C#), a \"use\" expression( F#) or similar constructs in other languages.\n]\n\n[Thread safety]\n[\n`CharStream` members are not thread-safe.\n]\n\n[[#low-trust Low-Trust version]]\n[\nIf you compile FParsec with the `LOW_TRUST` conditional compiler symbol, the `CharStream` class differs from the normal version as follows:\n- No [url \"https://msdn.microsoft.com/en-us/library/t2yzs44b.aspx\" unverifiable code] involving pointers is used. *This allows FParsec to be executed in an environment with reduced trust*, such as medium trust ASP.NET applications or Silverlight applications.\n- A `CharStream` that is constructed from a `System.IO.Stream` or a file path reads the complete file into a single string during construction. *This severely limits the maximum practical stream size.*\n- Although the `CharStream` class still supports the `IDisposable` interface, disposing the `CharStream` instances is no longer necessary, since no resources are hold that need to be explicitly released.\nSee also [^download-and-installation.low-trust-version].\n]\n\n]\n\n[/section]\n\n[section#exceptions I/O exceptions]\n\nIf you construct a `CharStream` from a `System.IO.Stream` or a file path, the constructor and any `CharStream` operation that requires reading chars from the underlying byte stream may throw one of the following exceptions.\n\nIn the [^low-trust Low-Trust version], the constructor decodes the complete byte stream and hence only the constructor may throw one of these exceptions.\n\n[note Doing actual work in a constructor and potentially throwing exceptions seems to be a somewhat controversial design. We think it's the right choice for the `CharStream` class, because this way you can a have a reasonable expectation that the `CharStream` actually works after you've successfully constructed it.]\n\nIn general it is *not* safe to continue to use a `CharStream` instance after one of these exceptions was thrown, though calling `Dispose()` is always safe.\n\n[dl\n[`[no-auto-link NotSupportedException]`]\n[Seeking of the underlying byte stream is required, but the byte stream does not support seeking or the `Encoding`'s `Decoder` is not serializable. See also the remarks above on @non-sequential access@.]\n\n[`[no-auto-link IOException]`]\n[An I/O occurred while reading data from the underlying byte stream.]\n\n[`[no-auto-link ArgumentException]`]\n[The underlying byte stream contains invalid bytes and the `Encoding` was constructed with the `throwOnInvalidBytes` option.]\n\n[`[no-auto-link DecoderFallbackException]`]\n[The underlying byte stream contains invalid bytes for which the decoder fallback threw this exception.\n\nThe byte index of the invalid bytes in the stream is stored as a boxed `System.Int64` in the `\"Stream.Position\"` entry of the `[url \"https://msdn.microsoft.com/en-us/library/system.exception.data.aspx\" Data]` member of the exception instance. The precision of the index depends on the precision of the `DecoderFallbackException`'s `[url \"http://msdn.microsoft.com/en-us/library/system.text.decoderfallbackexception.index.aspx\" Index]` member. If the underlying `System.IO.Stream` is not seekable, the byte index only takes into account the bytes read by the `CharStream`, but not any bytes read before the `CharStream` was constructed.\n]\n]\n\n[/section]\n[section Members]\n[interface-members]\n\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype [+CharStream] =\n interface System.IDisposable\n\n``]\n\n[``\n @new@: chars: string * index: int * length: int\n -> CharStream\n``]\n[#new_string\n\nIs equivalent to `[^ new_string_offset new CharStream](chars, index, length, 0L)`.\n]\n\n[``\n @new@: chars: string * index: int * length: int * streamBeginIndex: int64\n -> CharStream\n``]\n[#new_string_offset\nConstructs a `CharStream` from the chars in the string argument between the indices `index` (inclusive) and `index + length` (exclusive). By directly referencing the chars in the string this constructor avoids any copy of the string content.\n\nThe first char in the stream is assigned the index `streamBeginIndex`. A positive `streamBeginIndex` allows you for example to create a substream of another `CharStream`, i.e. a `CharStream` instance that only contains a sub-segment of another char stream but is accessible through the same char indices.\n\n`chars` must not be null. An `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions:\n- `index` ≥ 0, `length` ≥ 0, `index` + `length` ≤ `chars.Length` and\n- 0 ≤ `streamBeginIndex` < 2[sup 60].\n\n[important\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nThe given string is \"[url \"https://msdn.microsoft.com/en-us/library/83y4ak54.aspx\" pinned]\" until the `CharStream` is disposed. Pinning the string prevents the GC from moving it around in memory during garbage collection. On .NET (at least in versions up to and including 4.0) the pinning has no effect if the string is large enough to be allocated on the Large Object Heap, i.e. has a length of about 42500 chars or more. However, pinning smaller strings does constrain the normal operations of the GC. Thus, **to minimize the negative impact on the GC, you should dispose `CharStream` instances constructed from small strings as soon as you're done parsing it**. If you keep a large number of `CharStream` instances constructed from small strings around for an extended period of time, you risk fragmenting the heap.\n]\n]\n\n[``\n\n @new@: chars: char[] * index: int * length: int\n -> CharStream\n``]\n[#new_char-array\n[small [/ This constructor is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nIs equivalent to `[^ new_char-array_offset new CharStream](chars, index, length, 0L)`.\n]\n\n[``\n @new@: chars: char[] * index: int * length: int * streamBeginIndex: int64\n -> CharStream\n``]\n[#new_char-array_offset\n[small [/ This constructor is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nConstructs a `CharStream` from the chars in the char array argument between the indices `index` (inclusive) and `index + length` (exclusive). By directly referencing the chars in the char array this constructor avoids any copy of the char array content.\n\nThe first char in the stream is assigned the index `streamBeginIndex`. A positive `streamBeginIndex` allows you for example to create a substream of another `CharStream`, i.e. a `CharStream` instance that only contains a sub-segment of another char stream but is accessible through the same char indices.\n\n`chars` must not be null. An `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions:\n- `index` ≥ 0, `length` ≥ 0, `index` + `length` ≤ `chars.Length` and\n- 0 ≤ `streamBeginIndex` < 2[sup 60].\n\n[note A `CharStream` constructed from a char array does not support .NET regex matching via the `[^Match_Regex Match]` method.]\n\n[important\nThe given char array is \"[url \"https://msdn.microsoft.com/en-us/library/83y4ak54.aspx\" pinned]\" until the `CharStream` is disposed. Pinning the char array prevents the GC from moving it around in memory during garbage collection. On .NET (at least in versions up to and including 4.0) the pinning has no effect if the char array is large enough to be allocated on the Large Object Heap, i.e. has a length of about 42500 chars or more. However, pinning smaller char arrays does constrain the normal operations of the GC. Thus, **to minimize the negative impact on the GC, you should dispose `CharStream` instances constructed from small char arrays as soon as you're done parsing it**. If you keep a large number of `CharStream` instances constructed from small char arrays around for an extended period of time, you risk fragmenting the heap.\n]\n\n\n]\n\n[``\n\n @new@: chars: NativePtr * length: int\n -> CharStream\n``]\n[#new_char-pointer\n[small [/ This constructor is not available in the [^low-trust Low-Trust version] of FParsec.]][br]\n\nIs equivalent to `[^ new_char-pointer_offset new CharStream](chars, length, 0L)`.\n]\n\n[``\n @new@: chars: NativePtr * length: int * streamBeginIndex: int64\n -> CharStream\n``]\n[#new_char-pointer_offset\n[small [/ This constructor is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nConstructs a `CharStream` from the `length` chars at the pointer address. By directly referencing the chars at the pointer address this constructor avoids any copy of the char buffer.\n\nThe first char in the stream is assigned the index `streamBeginIndex`. A positive `streamBeginIndex` allows you for example to create a substream of another `CharStream`, i.e. a `CharStream` instance that only contains a sub-segment of another char stream but is accessible through the same char indices.\n\n`chars` must not be null. An `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions:\n- `length` ≥ 0, `chars + length` must not overflow and\n- 0 ≤ `streamBeginIndex` < 2[sup 60].\n\n[note A `CharStream` constructed from a pointer does not support .NET regex matching via the `[^Match_Regex Match]` method.]\n\n]\n\n[``\n\n @new@: path: string * encoding: System.Text.Encoding\n -> CharStream\n``]\n[#new_file-path\n\nIs equivalent to `[^ new_file-path_2 new CharStream](path, encoding, true)`.\n]\n\n[``\n @new@: path: string\n * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n -> CharStream\n``]\n[#new_file-path_2\n\nIs equivalent to\n``\n[^ new_file-path_3 new CharStream](\n path, encoding, detectEncodingFromByteOrderMarks,\n blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n blockOverlap = DefaultBlockSize/3,\n minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n byteBufferLength = DefaultByteBufferLength\n)\n``\n]\n\n[``\n @new@: path: string\n * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n * blockSize: int * blockOverlap: int * minRegexSpace: int\n * byteBufferLength: int\n -> CharStream\n``]\n[#new_file-path_3\n\nConstructs a `CharStream` from a `FileStream` as if by calling\n``\n[^ new_stream_4 new CharStream](\n new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096,\n FileOptions.SequentialScan),\n leaveOpen = false,\n encoding = encoding,\n detectEncoding = true,\n blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n blockOverlap = DefaultBlockSize/3,\n minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n byteBufferLength = DefaultByteBufferLength\n)\n``\n\nIf an exception occurs after the `FileStream` is constructed but before the `CharStream` constructor is finished, the `FileStream` is disposed.\n\n[note The `FileStream` constructor might throw an exception, too.]\n\n]\n\n\n[``\n\n @new@: stream: System.IO.Stream * encoding: System.Text.Encoding\n -> CharStream\n``]\n[#new_stream\n\nIs equivalent to `[^ new_stream_3 new CharStream](stream, false, encoding, true)`.\n]\n\n[``\n @new@: stream: System.IO.Stream * leaveOpen: bool\n * encoding: System.Text.Encoding\n -> CharStream\n``]\n[#new_stream_2\n\nIs equivalent to `[^ new_stream_3 new CharStream](stream, leaveOpen, encoding, true)`.\n]\n\n[``\n @new@: stream: System.IO.Stream * leaveOpen: bool\n * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n -> CharStream\n``]\n[#new_stream_3\n\nIs equivalent to\n``\n[^ new_stream_4 new CharStream](\n stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks,\n blockSize = DefaultBlockSize (* = 3*2^16 ≈ 200k *),\n blockOverlap = DefaultBlockSize/3,\n minRegexSpace = ((DefaultBlockSize/3)*2)/3,\n byteBufferLength = DefaultByteBufferLength\n)\n``\n]\n\n[``\n @new@: stream: System.IO.Stream * leaveOpen: bool\n * encoding: System.Text.Encoding * detectEncodingFromByteOrderMarks: bool\n * blockSize: int * blockOverlap: int * minRegexSpace: int\n * byteBufferLength: int\n -> CharStream\n``]\n[#new_stream_4\n\nConstructs a `CharStream` from a `System.IO.Stream`.\n\nThe normal version of the `CharStream` class supports stream sizes up to approximately (2[sup 31]/p)×(`blockSize` - `blockOverlap`) chars, where p is 4 on a 32-bit CLR and 8 on a 64-bit CLR.[br]The [^low-trust Low-Trust version] only supports streams small enough that the complete content can be read into a single string.\n\n[note This constructor reads the first block of chars from the input stream and hence can throw any of the I/O related exceptions detailed in the @exceptions@ section above.]\n\nArguments:\n[dl\n[`stream`]\n[The byte stream providing the input. If `stream.[url \"https://msdn.microsoft.com/en-us/library/system.io.stream.canread.aspx\" CanRead]` returns `false`, an `ArgumentException` is thrown.]\n\n[`leaveOpen`]\n[Indicates whether the `stream` should be left open when the `CharStream` has finished reading it.]\n\n[`encoding`]\n[The default `Encoding` used for decoding the byte stream into chars.\n\nIf the preamble returned by `encoding.[url \"https://msdn.microsoft.com/en-us/library/system.text.encoding.getpreamble.aspx\" GetPreamble]()` is present at the beginning of the stream, the `CharStream` will skip over it.\n]\n\n[`detectEncodingFromByteOrderMarks`]\n[Indicates whether the constructor should detect the encoding from a unicode [url \"https://en.wikipedia.org/wiki/Byte-order_mark\" byte-order mark] at the beginning of the stream. An encoding detected from a byte-order mark overrides the default `encoding`. The standard byte-order marks for the following encodings are supported: UTF-8, UTF-16 LE/BE and UTF-32 LE/BE.]\n\n[`blockSize`]\n[The number of chars per block. The value is rounded up to the first positive multiple of 1536. The default is 3×2[sup 16] ≈ 200k.]\n\n[`[#blockOverlapParameter]blockOverlap`]\n[The number of chars at the end of a block that are preserved when reading the next block into into its internal char buffer. If this value is less than `encoding.GetMaxCharCount(1)` or not less than `blockSize/2`, the default value is used instead. The default is `blockSize/3`.]\n\n\n\n[byteBufferLength]\n[The size of the byte buffer used for decoding purposes. The default is 2[sup 12] = 4KB.]\n]\n]\n\n[``\n\n member @Dispose@: unit -> unit\n``]\n[Releases all resources used by the `CharStream`. If the `CharStream` was constructed from a `System.IO.Stream` or a file path and the constructor was not called with `leaveOpen = true`, the byte stream is closed.\n]\n\n\n[``\n\n member @BlockOverlap@: int\n``]\n[\nThe number of chars at the end of a block that are preserved when the `CharStream` reads the next block into its internal char buffer.\n\nThis value is only relevant for optimization purposes and as the maximum value for `MinRegexSpace`.\n\nThis value can only be set at construction time with the respective [^blockOverlapParameter constructor parameter].\n\nIf the `CharStream` is constructed from a string, char array or char pointer or only contains 1 block,\nthen this value is 0. In the [^low-trust Low-Trust version] this value is always 0.\n]\n\n\n\n\n[``\n\n member @IndexOfFirstChar@: int64\n``]\n[\nThe index of the first char in the stream. This value is determined by the `streamIndexOffset` argument of some of the `CharStream` constructors. By default this value is 0.\n]\n\n[``\n member @IndexOfLastCharPlus1@: int64\n``]\n[\nThe index of the last char of the stream plus 1, or `Int64.MaxValue` if the end of the stream has not yet been detected.\n]\n\n\n[``\n\n member @IsBeginOfStream@: bool\n``]\n[\nIndicates whether the next char in the stream is the first char, i.e. whether `Index` equals `IndexOfFirstChar`.\n\nIf the stream is empty, this value is always `true`.\n]\n\n[``\n member @IsEndOfStream@: bool\n``]\n[\nIndicates whether there is no char remaining in the stream, i.e. whether `Index` equals `IndexOfLastCharPlus1`.\n\nIf the stream is empty, this value is always `true`.\n]\n\n[``\n\n member @Index@: int64\n``]\n[\nThe stream index of the next char.\n]\n\n[``\n member @IndexToken@: CharStreamIndexToken\n``]\n[\nA `CharStreamIndexToken` value representing the current `Index` value.\n]\n\n[``\n member @Line@: int64\n``]\n[\nThe line number for the next char. (The line count starts with 1.)\n]\n\n[``\n member @LineBegin@: int64\n``]\n[\nThe stream index of the first char of the line that also contains the next char.\n]\n\n[``\n member @Column@: int64\n``]\n[The UTF-16 column number of the next char, i.e. `Index` - `LineBegin` + 1.\n]\n\n[``\n member @Name@: string with get, set\n``]\n[\nThis string is used in error messages to describe the input stream.\n\nIf the `CharStream` is constructed from a file path, the constructor initializes the `Name` value with the file path value. Otherwise, `Name` is initialized to `null`.\n\nIf the stream content is the concatenated content of multiple input files, you can improve error messages and help debugging by setting the name and resetting the line and column count at the transitions between the different content pieces.\n\nSetting the `Name` value increments the `StateTag` by 1, independent of whether the new value is different from the previous one.\n]\n\n[``\n member @Position@: [^ reference.Position Position]\n``]\n[\nReturns `new [^ reference.Position Position](Name, Index, Line, Column)`.\n]\n\n[``\n\n val mutable @StateTag@: uint64\n``]\n[\nThe `StateTag`'s purpose is to provide an efficient way to determine whether the publically visible state of the `CharStream` has changed after a series of method calls. For the purpose of this property, the state is defined as the aggregate of the `Index`, `Line`, `LineBegin` and `Name` values. The `UserState` value of `CharStream<'UserState>` instances is also part of the `CharStream` state. If a method or property setter changes one or more of these state values it increments the `StateTag`'s by 1. Thus, to determine whether a series of method calls has changed the `CharStream`, it is often enough to compare the `StateTag` values from before and after the method calls.\n\nThe `StateTag` property is primarily meant for use in the implementation of parser combinators. If you directly call `CharStream` methods, you normally don't need the `StateTag` to determine whether the state has changed, because that is usually obvious from either the method's return value or the context in which it was called. Please see [^ users-guide.applying-parsers-in-sequence.the-statetag] for more details on the design rationale behind the `StateTag`.\n]\n[``\n\n member @Seek@: index: int64 -> unit\n``]\n[\nSeeks the `CharStream` to the char with the specified index in the stream.\n\nIf you pass an index larger than the index of the last char in the stream, this method seeks the stream to the end of the stream, i.e. to one char past the last char in the stream.\n\nThe index is zero-based, except if the `CharStream` was constructed with a positive `streamIndexOffset` argument, in which case the index of the first char equals the value of the `streamIndexOffset` argument (and the `IndexOfFirstChar` value).\n\nWhen this method changes the stream position, it increments the `StateTag` by 1. When it does not change the position, it may or may not increment the `StateTag` by 1.\n\n\nAn `ArgumentOutOfRangeException` is thrown if the index is less than the `IndexOfFirstChar`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Seek@: indexToken: CharStreamIndexToken -> unit\n``]\n[#Seek_CharStreamIndexToken\n\nThis method is an optimized implementation of `@Seek@(GetIndex(indexToken))`.\n]\n\n[``\n\n static val @EndOfStreamChar@: char\n``]\n[\nThe char returned by `Peek` and `Read` at the end of the stream.\n\nThe value is `'\\uFFFF'`.\n]\n\n\n[``\n\n member @Peek@: unit -> char\n``]\n[Returns the next char without changing the state of the `CharStream`.\n\nAt the end of the `CharStream` the `EndOfStreamChar` (`'\\uFFFF'`) is returned.]\n\n[``\n member @Peek2@: unit -> TwoChars\n``]\n[\n`Peek2()` is an optimized implementation of `new TwoChars(Peek(), [^Peek_int Peek](1))`.\n]\n\n[``\n member @Peek@: utf16Offset: int -> char\n``]\n[#Peek_int\nReturns the char at the stream index `Index + utf16Offset`, without changing the state of the `CharStream`.\n\nIf `Index + utf16Offset` is smaller than the index of the first char in the stream or larger than the index of the last char in the stream, the `EndOfStreamChar` (`'\\uFFFF'`) is returned.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Peek@: utf16Offset: uint32 -> char\n``]\n[#Peek_uint32\nThis method is an optimized implementation of `[^Peek_int Peek](int)` for `uint32` arguments.\n]\n\n[``\n\n member @PeekString@: length: int -> string\n``]\n[#PeekString\nReturns a string with the next `length` stream chars, without changing the state of the `CharStream`.\n\nIf less than `length` chars are remaining in the stream, only the remaining chars are returned.\n\n[note\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nIf `length` is greater than the number of remaining chars in the stream, a temporary string with `length` chars may be allocated. For very large `length` values this might lead to an `OutOfMemoryException` even though a string with only the remaining chars in the stream would comfortably fit into memory.\n\nPlease also note that the maximum length of a string on .NET is less than 2[sup 30]. Allocating a string larger than the maximum length will always yield an `OutOfMemoryException`, even on 64-bit systems with enough physical memory.\n]\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n\n[``\n member @PeekString@: buffer: char[] * bufferIndex: int * length: int -> int\n``]\n[#PeekString_char-array\n\nCopies the next `length` stream chars into `buffer`, without changing the state of the `CharStream`. Returns the number of chars copied.\n\nThe chars are written into `buffer` beginning at the index `bufferIndex`. If less than `length` chars are remaining in the stream, only the remaining chars are copied.\n\n An `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions: `bufferIndex` ≥ 0, `length` ≥ 0 and `bufferIndex` + `length` ≤ `buffer.Length`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @PeekString@: buffer: NativePtr * length: int -> int\n``]\n[#PeekString_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nCopies the next `length` stream chars into the buffer at the specified pointer address, without changing the state of the `CharStream`. Returns the number of chars copied.\n\nIf less than `length` chars are remaining in the stream, only the remaining chars are copied.\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n\n[``\n\n member @Match@: char -> bool\n``]\n[\nReturns `true` if the next char in the stream matches the specified char. At the end of the stream `Match` always returns `false`.\n\nThis method does not change the state of the `CharStream`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Match@: chars: string -> bool\n``]\n[#Match_string\n\nReturns `true` if the passed string `chars` matches the next `chars.Length` stream chars.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, `false` is returned. If `chars` is empty, `true` is returned. `chars` must not be `null`.\n\nThis method does not change the state of the `CharStream`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Match@: chars: char[] * charsIndex: int * length: int -> bool\n``]\n[#Match_char-array\nReturns `true` if the next `length` stream chars match the chars in the array `chars` at the indices `charIndex` to `charsIndex + length - 1`.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, `false` is returned. If `length` is 0, `true` is returned. `chars` must not be `null`.\n\nThis method does not change the state of the `CharStream`.\n\nAn `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions: `charsIndex` ≥ 0, `length` ≥ 0 and `charsIndex` + `length` ≤ `chars.Length`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Match@: chars: NativePtr * length: int -> bool\n``]\n[#Match_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nReturns `true` if the next `length` stream chars match the chars at the specified pointer address.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, `false` is returned. If `length` is 0, `true` is returned.\n\nThis method does not change the state of the `CharStream`.\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n\n member @MatchCaseFolded@: caseFoldedChar: char -> bool\n``]\n[#MatchCaseFolded_char\nBehaves like `[^Match Match](caseFoldedChar)`, except that the next char in the stream is case-folded before it is compared with `caseFoldedChar`.\n\n[note\nWhile the char in the stream is case‐folded before it is matched, the char `caseFoldedChar` is assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n[``\n member @MatchCaseFolded@: caseFoldedChars: string -> bool\n``]\n[\nBehaves like `[^Match_string Match](caseFoldedChars)`, except that the chars in the stream are case-folded before they are compared with `caseFoldedChars`.\n\n[note\nWhile the chars in the `CharStream` are case‐folded before they are matched, the chars in the string argument `caseFoldedChars` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n[``\n member @MatchCaseFolded@: caseFoldedChars: NativePtr * length:int -> bool\n``]\n[#MatchCaseFolded_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nBehaves like `[^Match_char-pointer Match](caseFoldedChars, length)`, except that the chars in the stream are case-folded before they are compared with the chars at the pointer address `caseFoldedChars`.\n\n[note\nWhile the chars in the `CharStream` are case‐folded before they are matched, the chars at the pointer address `caseFoldedChars` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n\n[``\n\n member @Match@: System.Text.RegularExpressions.Regex\n -> System.Text.RegularExpressions.Match\n``]\n[#Match_Regex\n\nApplies the given regular expression to the stream chars beginning with the next char. Returns the resulting `Match` object.\n\nFor performance reasons you should specify the regular expression such that it can only match at the beginning of a string, for example by prepending `\"\\\\A\"`.\n\nFor `CharStream` instances constructed from strings the regular expression is applied to a string containing *all* the remaining chars in the stream.\n\nFor `CharStream` instances constructed from large binary streams (with more than 1 block) the regular expression is not applied to a string containing all the remaining chars in the stream. Here the `MinRegexSpace` value determines the *minimum* number of chars that are guaranteed to be visible to the regular expression (assuming there are still enough chars remaining in the stream). The exact number of chars visible to the regular expression may be affected even by calls to `CharStream` methods like `[^Peek_int Peek]` or `[^Match_string Match]` that otherwise guarantee to not change the (outwardly visible) state of the `CharStream`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[important\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nThis method is not supported by `CharStream` instances constructed directly from char arrays or pointers.\nA `NotSupportedException` is thrown if this method is called on such a `CharStream` instance.\n]\n\n[important\n[small [/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.]][br]\nIf the `CharStream` was constructed from a `System.IO.Stream` or a file path, the regular expression is applied to an internal *mutable* buffer. Since the `Match` object may work lazily, i.e. compute return values not before they are needed, you need to *retrieve all the required information from the `Match` object before you continue to access the `CharStream`*, otherwise you might get back invalid match results. Note that all strings returned by the `Match` object are, of course, immutable.]\n]\n\n[``\n member @MinRegexSpace@: int with get, set\n``]\n[\nThe number of chars that are guaranteed to be visible to a regular expression when it is matched by `[^Match_Regex Match]` (assuming there are enough chars remaining in the stream).\n\nThe value must be non-negative and not greater than `BlockOverlap`. The default value is 2/3 of `BlockOverlap`.\n\nIf the `CharStream` is constructed from a string, char array or char pointer or has only 1 block, then this value has no relevance and calling the property setter has no effect. (No [^low-trust Low-Trust version] `CharStream` instance has more than 1 block.)\n\nThe `MinRegexSpace` value is not recorded in `CharStreamState` instances and setting its value does not affect the `StateTag`.\n\nAn `ArgumentOutOfRangeException` is thrown if you try to set the property on a multi-block `CharStream` instance to a negative value or a value larger than the `BlockOverlap`.\n]\n\n[``\n[#RegisterNewline-methods]\n member @RegisterNewline@: unit -> bool\n``]\n[\nRegisters a newline (an end-of-line character) at the previous stream char, i.e. increments the `Line` value by 1 and sets the `LineBegin` to `Index`.\n\nThe previous `LineBegin` value must not equal `Index`. (For performance reasons this condition is only checked by an assert check in the debug build).\n\nThis method also increments the `StateTag` by 1.\n]\n\n[``\n member @RegisterNewlines@: lineOffset: int -> newColumnMinus1: int -> bool\n``]\n[\nIncrements the `Line` value by `lineOffset` and sets the `LineBegin` value to `Index - newColumnMinus1` (so that the `Column` value becomes `newColumnMinus1` + 1).\n\nThe `lineOffset` must not be 0, the new `Line` value must be greater than 0 and and the new `LineBegin` value must be different from the previous one. (For performance reasons these conditions are only checked by assert checks in the debug build).\n\nThis method also increments the `StateTag` by 1.\n]\n\n[``\n member @RegisterNewlines@: lineOffset: int64 -> newColumnMinus1: int64 -> bool\n``]\n[#RegisterNewlines_int64\nThis method is a variant of `@RegisterNewlines@` for `int64` arguments.\n]\n\n[``\n\n // The following methods require manual registration of skipped newlines\n``]\n\n[``\n\n [#Skip-members]member @Skip@: unit -> unit\n``]\n[\nAdvances the position within the stream by 1 char, except at the end of the stream, where it does nothing.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Skip@: utf16Offset: int -> unit\n``]\n[#Skip_int\nAdvances the position within the stream by `utf16Offset` chars.\n\nThe new position within the stream will be `min(Index + utf16Offset, IndexOfLastCharPlus1)`. This means you can't move past the end of the stream, because any position beyond the last char in the stream is interpreted as precisely one char beyond the last char.\n\nAn `ArgumentOutOfRangeException` is thrown if the new position would lie before the beginning of the `CharStream`, i.e. if the new index would be less than `IndexOfFirstChar`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1. When it does not change the position (because the given offset is 0 or because the stream has already reached the end and the offset is positive), it may or may not increment the `StateTag` by 1.\n]\n\n[``\n member @Skip@: utf16Offset: uint32 -> unit\n``]\n[#Skip_uint32\n\nThis method is an optimized implementation of `Skip` for `uint32` offsets.\n]\n\n[``\n member @Skip@: utf16Offset: int64 -> unit\n``]\n[#Skip_int64\nThis method is a variant of `Skip` for `int64` offsets.\n]\n\n[``\n\n [#SkipAndPeek-members]member @SkipAndPeek@: unit -> char\n``]\n[\n`c <- SkipAndPeek()` is an optimized implementation of `Skip(); c <- Peek()`.\n]\n\n\n[``\n member @SkipAndPeek@: utf16Offset: int -> char\n``]\n[#SkipAndPeek_int\n`c <- SkipAndPeek(utf16Offset)` is an optimized implementation of `Skip(utf16Offset); c <- Peek()`, with the following *exception for negative offsets* `n`:[br]\nIf the new position would lie before the beginning of the `CharStream`, i.e. if the new index would be less than `IndexOfFirstChar`, then `SkipAndPeek(n)` does not throw an exception like `stream.Skip(n)` would do. Instead it sets the position of the stream to `IndexOfFirstChar` and returns the `EndOfStreamChar` (`'\\uFFFF'`).\n]\n\n[``\n member @SkipAndPeek@: utf16Offset: uint32 -> char\n``]\n[#SkipAndPeek_uint32\n`c <- SkipAndPeek(utf16Offset)` is an optimized implementation of `[^Skip_uint32 Skip](utf16Offset); c <- Peek()`.\n]\n\n[``\n\n member @Skip@: char -> bool\n``]\n[#Skip_char\nSkips over the next char in the stream if this char matches the passed argument char. Returns `true` if the chars match; otherwise, `false`. At the end of the stream this method always returns `false`.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Skip@: TwoChars -> bool\n``]\n[#Skip_TwoChars\nSkips over the next two chars in the stream if these chars match the two chars in the passed `TwoChars` value. Returns `true` if the chars match.\n\nIf not both chars match or if there are less than 2 chars remaining in the stream, no char is skipped and `false` is returned.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Skip@: chars: string -> bool\n``]\n[#Skip_string\n\nSkips over the next `chars.Length` chars in the stream if these chars match the passed string `chars`. Returns `true` if the chars match.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, no char is skipped and `false` is returned. If `chars` is empty, `true` is returned. `chars` must not be `null`.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `chars` is empty, in which case it may or may not increment the `StateTag` by 1.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Skip@: chars: char[] * charsIndex: int * length: int -> bool\n``]\n[#Skip_char-array\n\nSkips over the next `length` chars in the stream if these chars match the chars in the passed array `chars` at the indices `charIndex` to `charsIndex + length - 1`. Returns `true` if the chars match.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, `false` is returned and the position within the `CharStream` is not changed. If `length` is 0, `true` is returned. `chars` must not be `null`.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `length` is 0, in which case it may or may not increment the `StateTag` by 1.\n\nAn `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions: `charsIndex` ≥ 0, `length` ≥ 0 and `charsIndex` + `length` ≤ `chars.Length`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Skip@: chars: NativePtr * length: int -> bool\n``]\n[#Skip_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nSkips over the next `length` chars in the stream if these chars match the chars at the pointer address `chars`. Returns `true` if the chars match.\n\nIf not all the chars match or if there are not enough chars remaining in the stream, `false` is returned and the position within the `CharStream` is not changed. If `length` is 0, `true` is returned.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `length` is 0, in which case it may or may not increment the `StateTag` by 1.\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n\n[``\n\n member @SkipCaseFolded@: caseFoldedChar: char -> bool\n``]\n[#SkipCaseFolded_char\nBehaves like `[^Skip_char Skip](caseFoldedChar)`, except that the next char in the stream is case-folded before it is compared with `caseFoldedChar`.\n\n[note\nWhile the char in the stream is case‐folded before it is matched, the char `caseFoldedChar` is assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n[``\n member @SkipCaseFolded@: caseFoldedChars: string -> bool\n``]\n[\nBehaves like `[^Skip_string Skip](caseFoldedChars)`, except that the chars in the stream are case-folded before they are compared with `caseFoldedChars`.\n\n[note\nWhile the chars in the `CharStream` are case‐folded before they are matched, the chars in the string argument `caseFoldedChars` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n[``\n member @SkipCaseFolded@: caseFoldedChars: NativePtr * length:int -> bool\n``]\n[#SkipCaseFolded_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nBehaves like `[^Skip_char-pointer Skip](caseFoldedChars)`, except that the chars in the stream are case-folded before they are compared with the chars at the pointer address `caseFoldedChars`.\n\n[note\nWhile the chars in the `CharStream` are case‐folded before they are matched, the chars at the pointer address `caseFoldedChars` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.\n]\n]\n\n[``\n\n member @Read@: unit -> char\n``]\n[Skips over the next char in the stream. Returns the skipped char.\n\nAt the end of the stream `Read()` does not change the stream position and returns the `EndOfStreamChar` (`'\\uFFFF'`).\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Read@: length: int -> string\n``]\n[#Read_int\nSkips over the next `length` chars in the stream. Returns the skipped chars as a string.\n\nIf less than `length` chars are remaining in the stream, only the remaining chars are skipped and returned.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `length` is 0, in which case it may or may not increment the `StateTag` by 1.\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n\n[``\n member @Read@: buffer: char[] * bufferIndex: int * length: int -> int\n``]\n[#Read_char-array\n\nSkips over the next `length` stream chars and copies the skipped chars into `buffer`. Returns the number of copied and skipped chars.\n\nThe chars are written into `buffer` beginning at the index `bufferIndex`. If less than `length` chars are remaining in the stream, only the remaining chars are copied and skipped.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `length` is 0, in which case it may or may not increment the `StateTag` by 1.\n\nAn `ArgumentOutOfRangeException` is thrown if the arguments do not satisfy the following conditions: `bufferIndex` ≥ 0, `length` ≥ 0 and `bufferIndex` + `length` ≤ `buffer.Length`. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @Read@: buffer: NativePtr * length: int -> int\n``]\n[#Read_char-pointer\n[small [/ This method is not available in the [^low-trust Low-Trust version] of FParsec.]]\n\nSkips over the next `length` stream chars and copies the skipped chars into the buffer at the given pointer address. Returns the number of copied and skipped chars.\n\nIf less than `length` chars are remaining in the stream, only the remaining chars are copied and skipped.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`, except if `length` is 0, in which case it may or may not increment the `StateTag` by 1.\n\nIf `length` is negative, an `ArgumentOutOfRangeException` is thrown. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n\n[``\n\n member @ReadFrom@: indexOfFirstChar: CharStreamIndexToken -> string\n``]\n[\nReturns a string with the chars between the stream index `indexOfFirstChar` (inclusive) and the current `Index` of the stream (exclusive).\n\nThis method trows\n- an `ArgumentOutOfRangeException`, if `Index < indexOfFirstChar`, and\n- an `ArgumentException`, if the `CharStreamIndexToken` is a zero-initialized instance (i.e. constructed with the default value type constructor).\nIt may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[note You may only pass `CharStreamToken` values that were retrieved from the `CharStream` instance on which you're calling `ReadFrom`. Passing a `CharStreamToken` value that was created for another `CharStream` instance triggers an assert exception in debug builds and will otherwise lead to undefined behaviour.]\n]\n\n[``\n[#methods-that-register-newlines]\n // The following methods automatically register skipped newlines\n``]\n\n[``\n\n member @SkipWhitespace@: unit -> bool\n``]\n[\nSkips over any sequence of space (`' '`), tab (`'\\t'`) or newline (`'\\r'`, `'\\n'`) chars.\nReturns `true` if it skips at least one char, otherwise `false`.\n\nThis method registers any skipped standard newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`).\n\nWhen this method skips at least one char, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @SkipUnicodeWhitespace@: unit -> bool\n``]\n[\nSkips over any sequence of unicode whitespace chars (as identified by `System.Char.IsWhiteSpace`).\nReturns `true` if it skips at least one char, otherwise `false`.\n\nThis method registers any skipped unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u000C\"`, `\"\\u2028\"` or `\"\\u2029\"`).\n\n[note This method recognizes the form feed char `'\\f'` (`'\\u000C'`) as a Unicode whitespace character, but not as a newline character.]\n\nWhen this method skips at least one char, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n\n member @SkipNewline@: unit -> bool\n``]\n[\nSkips over a standard newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`). Returns `true` if a newline is skipped, otherwise `false`.\n\nWhen this method skips a newline, it also registers it.\n\nWhen this method skips a newline, it increments the `StateTag` by 1, otherwise it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @SkipUnicodeNewline@: unit -> bool\n``]\n[\nSkips over a unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"`, or `\"\\u2029\"`). Returns `true` if a newline is skipped, otherwise `false`.\n\n[note This method does not recognize the form feed char `'\\f'` (`'\\u000C'`) as a newline character.]\n\nWhen this method skips a newline, it also registers it.\n\nWhen this method skips a newline, it increments the `StateTag` by 1, otherwise it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n\n member @SkipNewlineThenWhitespace@:\n powerOf2TabStopDistance: int * allowFormFeed: bool -> int\n``]\n[\nSkips over a newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) followed by any (possibly empty) sequence of whitespace chars (`' '`, `'\\t'`, `'\\r'`, `'\\n'` and optionally `'\\f'`).\n\nIf this method skips no chars because the next stream char is no newline char, it returns -1. Otherwise it returns the indentation of the first line with non-whitespace characters.\n\nThe *indentation* is calculated as follows:\n- Any newline char (`'\\r'` or `'\\n'`) or form feed char (`'\\f'`) resets the *indentation* to 0.\n- Any space char (`' '`) increments the *indentation* by 1.\n- Any tab char (`'\\t'`) increments the *indentation* by[br] `powerOf2TabStopDistance` - (*indentation* modulo `powerOf2TabStopDistance`).\n\nThe maximum indentation is 2[sup 31] - 1. If skipping a whitespace char would cause the indentation to overflow, the char is not skipped and the method returns the indentation up to that char.\n\nAn `ArgumentOutOfRangeException` is thrown if `powerOf2TabStopDistance` is not a positive power of 2.\n\nThe value of the `allowFormFeed` argument determines whether this method accepts the form feed char `'\\f'` as a whitespace char.\n\nThis method registers all skipped standard newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`).\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n\n member @SkipRestOfLine@: skipNewline: bool -> unit\n``]\n[\nSkips over any chars before the next newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) or\nthe end of the stream. If `skipNewline` is `true` and a newline is present, the newline is also skipped.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n[``\n member @ReadRestOfLine@: skipNewline: bool -> string\n``]\n[\n`ReadRestOfLine(skipNewline)` behaves like `SkipRestOfLine(skipNewline)`, except that it returns a string with the skipped chars (without a newline).\n]\n\n[``\n\n member @ReadCharOrNewline@: unit -> char\n``]\n[\nSkips over any single char or standard newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`).\n\nThis method returns `'\\n'` when it skips a newline. Otherwise, it returns the skipped char, except at the end of the stream, where it returns the `EndOfStreamChar` (`'\\uffff'`).\n\nWhen this method skips a newline, it also registers it.\n\nWhen this method skips a char or newline, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n\n member @SkipCharsOrNewlines@: maxCount: int -> int\n``]\n[\nSkips over up to `maxCount` chars. Returns the number of skipped chars.\n\nThe number of actually skipped chars is less than `maxCount` if the end of the stream is reached after less than `maxCount` chars.\n\nThis method counts standard newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) as single chars. When this method skips a newline, it also registers it.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nAn `ArgumentOutOfRangeException` is thrown if `maxCount` is negative. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @ReadCharsOrNewlines@: maxCount: int * normalizeNewlines: bool -> string\n``]\n[\nBehaves like `@SkipCharsOrNewlines@(maxCount)`, except that it returns a string with the skipped chars.\n\nThe `normalizeNewlines` parameter determines whether all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the returned string are normalized to `'\\n'` or whether they are preserved in the original form they are encountered in the input.\n]\n\n[``\n\n member @SkipCharsOrNewlinesWhile@:\n predicate: (char -> bool) -> int\n``]\n[\nSkips over a sequence of chars that satisfy the `predicate` function. Stops at the first char\n for which `predicate` returns `false`. Returns the number of skipped chars.\n\nThis method counts standard newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) as single chars and passes them to the predicate function as single `'\\n'` chars. When this method skips a newline, it also registers it.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\n[caution\nThe `predicate` function must not access the `CharStream` instance itself, because `SkipCharsOrNewlinesWhile` relies on `predicate` not having any side-effect on the internal state of the stream.\n]\n\nThis method may throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @SkipCharsOrNewlinesWhile@:\n predicateForFirstChar: (char -> bool) * predicate: (char -> bool) -> int\n``]\n[#SkipCharsOrNewlinesWhile2\nBehaves like `@SkipCharsOrNewlinesWhile@(predicate)`, except that the first char to be skipped must satisfy `predicateForFirstChar` instead of `predicate`.\n]\n\n[``\n member @SkipCharsOrNewlinesWhile@:\n predicate: (char -> bool) * minCount: int * maxCount: int -> int\n``]\n[#SkipCharsOrNewlinesWhile_int_int\nSkips over a sequence of up to `maxCount` chars that satisfy the `predicate` function,\nbut backtracks to the start if it can only skip less than `minCount` chars. Returns the number of skipped chars.\n\nThis method counts standard newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) as single chars and passes them to the predicate function as single `'\\n'` chars. When this method skips a newline, it also registers it.\n\nAn `ArgumentOutOfRangeException` is thrown if `maxCount` is negative. This method may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[caution\nThe `predicate` function must not access the `CharStream` instance itself, because `SkipCharsOrNewlinesWhile` relies on `predicate` not having any side-effect on the internal state of the stream.\n]\n]\n\n[``\n member @SkipCharsOrNewlinesWhile@:\n predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n * minCount: int * maxCount: int -> int\n``]\n[#SkipCharsOrNewlinesWhile2_int_int\nBehaves like `[^SkipCharsOrNewlinesWhile_int_int SkipCharsOrNewlinesWhile](predicate, nMin, nMax)`, except that the first char to be skipped must satisfy `predicateForFirstChar` instead of `predicate`.\n]\n\n\n[``\n\n member @ReadCharsOrNewlinesWhile@:\n predicate: (char -> bool)\n * normalizeNewlines: bool -> string\n``]\n[#ReadCharsOrNewlinesWhile\nBehaves like `@SkipCharsOrNewlinesWhile@(predicate)`, except that it returns a string with the skipped chars.\n\nThe `normalizeNewlines` parameter determines whether all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the returned string are normalized to `'\\n'` or whether they are preserved in the original form they are encountered in the input.\n]\n\n[``\n member @ReadCharsOrNewlinesWhile@:\n predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n * normalizeNewlines: bool -> string\n``]\n[#ReadCharsOrNewlinesWhile2\n\nBehaves like `@ReadCharsOrNewlinesWhile@(predicate, normalizeNewlines)`, except that the first char to be skipped must satisfy `predicateForFirstChar` instead of `predicate`.\n]\n\n[``\n member @ReadCharsOrNewlinesWhile@:\n predicate: (char -> bool)\n * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n``]\n[#ReadCharsOrNewlinesWhile_int_int\nBehaves like `[^SkipCharsOrNewlinesWhile_int_int SkipCharsOrNewlinesWhile](predicate, minCount, maxCount)`,\nexcept that it returns a string with the skipped chars.\n\nThe `normalizeNewlines` parameter determines whether all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the returned string are normalized to `'\\n'` or whether they are preserved in the original form they are encountered in the input.\n]\n\n[``\n member @ReadCharsOrNewlinesWhile@:\n predicateForFirstChar: (char -> bool) * predicate: (char -> bool)\n * minCount: int * maxCount: int * normalizeNewlines: bool -> string\n``]\n[#ReadCharsOrNewlinesWhile2_int_int\nBehaves like `[^ReadCharsOrNewlinesWhile_int_int ReadCharsOrNewlinesWhile](predicate, minCount, maxCount, normalizeNewlines)`, except that the first char to be skipped must satisfy `predicateForFirstChar` instead of `predicate`.\n]\n\n[``\n[#SkipCharsOrNewlinesUntilString-members]\n member @SkipCharsOrNewlinesUntilString@:\n str: string * maxCount: int\n * foundString: out -> int\n``]\n[\nSkips over all stream chars before the first occurrence of the specified string or the end of the stream, but not over more than `maxCount` chars.\nAssigns `true` to the output parameter if the string is found, otherwise `false`.\n\nThis method registers skipped newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) and counts them as single chars. However, no newline normalization takes place when the argument string `str` is matched with the stream chars. Hence, `str` should either contain no newlines or only in the form they occur in the stream.\nIf `str` starts with `'\\n'`, then `SkipCharsOrNewlinesUntilString` will not find occurences of `str` in the stream that start in the middle of an `\"\\r\\n\"` newline.\n\nWhen this method changes the stream position, it increments the `StateTag` by 1; otherwise, it does not change the `StateTag`.\n\nThis method throws\n- an `ArgumentException`, if the string argument is empty, and\n- an `ArgumentOutRangeException`, if `nMax` is negative.\nIt may also throw any of the [^exceptions I/O related exceptions] detailed above.\n]\n\n[``\n member @SkipCharsOrNewlinesUntilString@:\n str: string * maxCount: int * normalizeNewlines: bool\n * skippedCharsIfStringFoundOtherwiseNull: out -> int\n``]\n[#SkipCharsOrNewlinesUntilString_string\n\nBehaves like `@SkipCharsOrNewlinesUntilString@(str, maxCount, outBool)`, except that its output parameter is a string instead of a boolean. If `str` is found, a string with the skipped chars is assigned to this output parameter; otherwise, `null` is assigned to the output parameter.\n\nThe `normalizeNewlines` parameter determines whether all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the output string are normalized to `'\\n'` or are preserved in the original form they are encountered in the input.\n]\n\n[``\n[#SkipCharsOrNewlinesUntilCaseFoldedString-members]\n member @SkipCharsOrNewlinesUntilCaseFoldedString@:\n caseFoldedString: string * maxCount: int\n * foundString: out -> int\n``]\n[\nBehaves like `@SkipCharsOrNewlinesUntilString@(caseFoldedString, maxCount, foundString)`, except that the chars in the stream are case-folded before they are compared with `caseFoldedChars`.\n\n[note\nWhile the chars in the `CharStream` are case‐folded before they are matched, the chars in the string argument `caseFoldedString` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.]\n]\n\n[``\n member @SkipCharsOrNewlinesUntilCaseFoldedString@:\n caseFoldedString: string * maxCount: int * normalizeNewlines: bool\n * skippedCharsIfStringFoundOtherwiseNull: out -> int\n``]\n[#SkipCharsOrNewlinesUntilCaseFoldedString_string\n\nBehaves like `[^SkipCharsOrNewlinesUntilString_string SkipCharsOrNewlinesUntilString](caseFoldedString, maxCount,` `normalizeNewlines, skippedCharsIfStringFoundOtherwiseNull)`, except that the chars in the stream are case-folded before they are compared with `caseFoldedChars`.\n\n[note\nWhile the chars in the stream are case‐folded before they are matched, the chars in the string argument `caseFoldedString` are assumed to already be case-folded (e.g. with the help of `FParsec.Text.FoldCase`). Please also see the above remarks on @case-insensitive matching@.]\n]\n\n[/interface-members]\n[/section]\n[/interface-reference]\n[/section]\n\n[section#CharStream_1 CharStream]\n\nProvides read-access to a sequence of UTF-16 chars.\n\n[interface-reference]\n[section Interface]\n[$$interface]\n[/section]\n[section Remarks]\n\nThe `CharStream<'TUserState>` class adds a user definable state component to its base class `CharStream`.\n\nThe user state is accessible through the property `UserState`. It has the type `'TUserState`.\n\nYou can retrieve a snapshot of the complete stream state, including the user state, from the `State` property. The value returned from the `State` property has the type `CharStreamState<'TUserState>`. You can pass a `CharStreamState` value to the `BacktrackTo` method in order to restore a previous state of the `CharStream`.\n\n[important\n`'TUserState` must be an immutable type or at least be treated as an immutable type if you want `BacktrackTo` to completely restore old values of the user state. Hence, when you need to change the user state, you should set a new `'TUserState` value to the `UserState` property of the `CharStream` instance, *not* mutate the existing `'TUserState` value.\n]\n\n[/section]\n[section Members]\n[interface-members]\n[``\n[]\ntype CharStream<'TUserState> =\n inherit @CharStream@\n\n // has the same constructors as CharStream\n``]\n\n\n[``\n\n member @UserState@: 'TUserState with get, set\n``]\n[\nThe current user state value.\n\nSetting the `UserState` value increments the `StateTag` by 1, independent of whether the new value is different from the previous one.\n]\n\n[``\n\n member @State@: CharStreamState<'TUserState>\n``]\n[\nReturns a snapshot of the current `StateTag`, `Index`, `Line`, `LineBegin`, `Name`, and `UserState` values in the form of an immutable `CharStreamState` value.\n]\n\n[``\n\n member @BacktrackTo@: CharStreamState<'TUserState> -> unit\n``]\n[\nRestores the stream to the state represented by the given `CharStreamState` value.\n\nFor example:\n``\nfun (stream: CharStream<'u>) ->\n let state = stream.State\n // ... (do something with stream that might change the state)\n stream.BacktrackTo(state) // restores stream to previous state\n // ...\n``\n\nThis method throws an `ArgumentException` if the `CharStreamState` instance is zero-initialized (i.e. constructed with the default value type constructor). It may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[note You may only pass `CharStreamState` values that were retrieved from the `CharStream` instance on which you're calling `BacktrackTo`. Passing a `CharStreamState` value that was created for another `CharStream` instance triggers an assert exception in debug builds and will otherwise lead to undefined behaviour.]\n]\n\n[``\n\n member @ReadFrom@:\n stateWhereStringBegins: CharStreamState<'TUserState>\n * normalizeNewlines: bool\n -> string\n``]\n[\nReturns a string with the chars between the index of the `stateWhereStringBegins` (inclusive) and the current `Index` of the stream (exclusive).\n\nThe `normalizeNewlines` parameter determines whether all newlines (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`) in the returned string are normalized to `'\\n'` or whether they are preserved in the original form they are encountered in the input. (If `stateWhereStringBegins.[^CharStreamState.Line Line]` equals the current `Line`, this method will never normalize any newlines in the returned string.)\n\nThis method trows\n- an `ArgumentOutOfRangeException`, if `Index < [^ GetIndex_state GetIndex](stateWhereStringBegins)`, and\n- an `ArgumentException`, if the `CharStreamState` instance is zero-initialized (i.e. constructed with the default value type constructor).\nIt may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[note You may only pass `CharStreamState` values that were retrieved from the `CharStream` instance on which you're calling `ReadFrom`. Passing a `CharStreamState` value that was created for another `CharStream` instance triggers an assert exception in debug builds and will otherwise lead to undefined behaviour.]\n]\n\n[``\n\n member @CreateSubstream@<'TSubStreamUserState>:\n stateWhereSubstreamBegins: CharStreamState<'TUserState>\n -> CharStream<'TSubStreamUserState>\n``]\n[\nCreates a new `CharStream<'TUserState>` instance with the stream chars between the index of the `stateWhereSubstreamBegins` (inclusive) and the current `Index` of the stream (exclusive).\n\nThe state of the substream is initialized to `stateWhereSubstreamBegin`, so that the stream and the substream will report the same position (`Index`, `Line`, `LineBegin` and `Name`) for corresponding chars.\nHowever, the beginning and end will normally differ between stream and substream, in particular the `IndexOfFirstChar` and `IndexOfLastCharPlus1` values will normally differ between stream and substream.\n\nAn example:\n\n``\nopen FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\nopen FParsec.Error\n\nlet embeddedBlock (beginDelim: string) (endDelim: string) : Parser<_,_> =\n let expectedEmbeddedBlock = expected \"embedded block\"\n fun stream ->\n if stream.Skip(beginDelim) then\n let stateAtBegin = stream.State\n let mutable foundString = false\n let maxChars = System.Int32.MaxValue\n stream.SkipCharsOrNewlinesUntilString(endDelim, maxChars, &foundString)\n |> ignore\n if foundString then\n // create substream with content between beginDelim and endDelim\n use substream = stream.CreateSubstream(stateAtBegin)\n // here we would normally work with the substream,\n // in this example we will just extract the string content\n let str = substream.ReadCharsOrNewlines(System.Int32.MaxValue, true)\n Reply(str)\n else\n Reply(Error, expectedString endDelim)\n else\n Reply(Error, expectedEmbeddedBlock)\n``\n\n``{fsi}\n> run (embeddedBlock \"/*\" \"*/\") \"/*substream content*/\";;\nval it : ParserResult = Success: \"substream content\"\n``\n\n[note\n[/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.][br]\nIf you create a substream for a `CharStream` instance with more than one block, the content of the substream needs to be copied. Thus, you can minimize the overhead associated with creating a substream by ensuring that the `CharStream` has only one block, either by choosing a sufficiently large `blockSize`, or by creating the `CharStream` from a string or char buffer.]\n\nYou may use a stream and its substreams concurrently. However, notice the following warning:\n\n[caution\n[/ This note does not apply to the [^low-trust Low-Trust version] of FParsec.][br]\nYou may not dispose a stream before all of its substreams are disposed.\nDisposing a stream before all its substreams are disposed triggers an assert exception in debug builds and otherwise lead to undefined behaviour.]\n\nThis method trows\n- an `ArgumentOutOfRangeException`, if `Index < [^ GetIndex_state GetIndex](stateWhereSubstreamBegins)`, and\n- an `ArgumentException`, if the `CharStreamState` instance is zero-initialized (i.e. constructed with the default value type constructor).\nIt may also throw any of the [^exceptions I/O related exceptions] detailed above.\n\n[note You may only pass `CharStreamState` values that were retrieved from the `CharStream` instance on which you're calling `CreateSubstream`. Passing a `CharStreamState` value that was created for another `CharStream` instance triggers an assert exception in debug builds and will otherwise lead to undefined behaviour.]\n]\n[/interface-members]\n[/section]\n[/interface-reference]\n[/section]\n\n[interface-reference]\n[section#CharStreamIndexToken CharStreamIndexToken]\n\nAn opaque representation of a `CharStream` char index.\n\n[$$interface]\n\n`CharStream` methods can handle `CharStreamIndexToken` values more efficiently than integer char indices.\n\nYou can retrieve `CharStreamIndexToken` values from the `CharStream.IndexToken` and `CharStreamState<_>.IndexToken` properties.\n\nYou can get the char index corresponding to a given `CharStreamIndexToken` value by calling its `GetIndex` method with the `CharStream` instance from which the token was retrieved.\n\nZero-initialized `CharStreamIndexToken` values constructed with the default value type constructor are *not* valid and trying to call a `CharStream` method with such an instance will trigger an exception.\n\n[note\nA `CharStreamIndexToken` instance *may only be used together with the `CharSteam` instance it was created for*.\n]\n\n\n[interface-members]\n[``\ntype CharStreamIndexToken = struct\n``]\n[``\n member @GetIndex@: CharStream -> int64\n``]\n[\nReturns the stream index represented by the `CharStreamIndexToken` instance.\n\nThe `CharStream` instance passed as the argument must be the `CharStream` instance from which the `CharStreamIndexToken` was retrieved. Passing a different `CharStream` instance\ntriggers an assert exception in debug builds and will otherwise lead to undefined behaviour.\n\nAn `InvalidOperationException` is thrown if the `CharStreamIndexToken` value is zero-initialized (i.e. constructed with the default value type constructor).\n]\n[``\nend\n``]\n[/interface-members]\n\n\n[/section]\n[/interface-reference]\n\n[auto-link{hide-outer-auto-links = [\"IndexToken\", \"Line\", \"LineBegin\", \"Name\", \"UserState\"],\n do-not-pick-up-as-link-targets = [\"IndexToken\", \"Line\", \"LineBegin\", \"Name\", \"UserState\" ]}]\n[interface-reference]\n[section#CharStreamState CharStreamState]\n\nAn immutable value type representation of the state of a `CharStream`.\n\n[$$interface]\n\nYou can retrieve `CharStreamState` values from the `[^ State CharStream<_>.State]` property. By passing a `CharStreamState` value to the `BacktrackTo` method of a `@CharStream<_>\\ @` instance, you can restore the stream to the state represented by the `CharStreamState` value.\n\nZero-initialized `CharStreamState` values constructed with the default value type constructor are *not* valid and trying to call a `CharStream` method with such an instance will trigger an exception.\n\n[note\nA `CharStreamState` instance *may only be used together with the `CharSteam` instance it was created for*.\n]\n\n[interface-members]\n[``\ntype CharStreamState<'TUserState> = struct\n member #Tag#: int64\n member #IndexToken#: CharStreamIndexToken\n member #Line#: int64\n member #LineBegin#: int64\n member #Name#: string\n member #UserState#: 'TUserState\n``]\n[``\n\n member @GetIndex@: CharStream<'TUserState> -> int64\n``]\n[#GetIndex_state\n`state.GetIndex(stream)` is an optimized implementation of `state.[no-auto-link IndexToken].[^CharStreamIndexToken.GetIndex GetIndex](stream)`.\n\nThe `[^CharStream_1 CharStream<'TUserState>\\ ]` instance passed as the argument must be the `CharStream` instance from which the `CharStreamState` was retrieved. Passing a different `CharStream` instance\ntriggers an assert exception in debug builds and will otherwise lead to undefined behaviour.\n\nAn `InvalidOperationException` is thrown if the `CharStreamState` instance is zero-initialized (i.e. constructed with the default value type constructor).\n]\n\n[``\n member @GetPosition@: CharStream<'TUserState> -> Position\n``]\n[\n`state.GetPosition(stream)` is an optimized implementation of `new [^ reference.Position Position]([no-auto-link state.Name, state.[^GetIndex_state GetIndex](stream), state.Line, state.Column])`.\n\nThe `[^CharStream_1 CharStream<'TUserState>\\ ]` instance passed as the argument must be the `CharStream` instance from which the `CharStreamState` was retrieved. Passing a different `CharStream` instance\ntriggers an assert exception in debug builds and will otherwise lead to undefined behaviour.\n\nAn `InvalidOperationException` is thrown if the `CharStreamState` instance is zero-initialized (i.e. constructed with the default value type constructor).\n]\n\n[``\nend\n``]\n[/interface-members]\n\n[/section]\n[/interface-reference]\n\n[interface-reference]\n[section#TwoChars TwoChars]\n\nAn immutable value type representation of two chars:\n\n[$$interface]\n\n[interface-members]\n[``\ntype TwoChars = struct\n new: char0: char * char1: char -> [no-auto-link TwoChars]\n val #Char0#: char\n val #Char1#: char\nend\n``]\n[/interface-members]\n[/section]\n[/interface-reference]\n\n[/auto-link]\n[/auto-link]\n[/section]\n\n" }, { "path": "Doc/src/reference-error.txt", "content": "[auto-link{do-not-pick-up-as-link-targets = [\"Error\"], hide-outer-auto-links = [\"Position\"]}]\n[section#Error FParsec.Error]\n\n[interface-reference]\n[section Interface]\n[$$interface]\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsec.dll\n\n[] // module is automatically opened when FParsec namespace is opened\nmodule [no-auto-link FParsec.Error]\n``]\n\n[``\n[#discriminated-union-type]\n// The following type abbreviations and active patterns allow you to\n// treat the ErrorMessage type almost as if it was defined as:\n//\n// []\n// type ErrorMessage =\n// | Expected of string\n// | ExpectedString of string\n// | ExpectedStringCI of string\n// | Unexpected of string\n// | UnexpectedString of string\n// | UnexpectedStringCI of string\n// | Message of string\n// | NestedError of Position * obj * ErrorMessageList\n// | CompoundError of string * Position * obj * ErrorMessageList\n// | OtherErrorMessage of obj\n\ntype #Expected# = ErrorMessage.Expected\ntype #ExpectedString# = ErrorMessage.ExpectedString\ntype #ExpectedStringCI# = ErrorMessage.ExpectedCaseInsensitiveString\ntype #Unexpected# = ErrorMessage.Unexpected\ntype #UnexpectedString# = ErrorMessage.UnexpectedString\ntype #UnexpectedStringCI# = ErrorMessage.UnexpectedCaseInsensitiveString\ntype #Message# = ErrorMessage.Message\ntype #NestedError# = ErrorMessage.NestedError\ntype #CompoundError# = ErrorMessage.CompoundError\ntype #OtherErrorMessage# = ErrorMessage.Other\n\n// Unfortunately, F# currently doesn't support active patterns with more\n// than 7 cases, so we have to use partial patterns.\n\nval (|[no-auto-link Expected]|_|): ErrorMessage -> string option\nval (|[no-auto-link ExpectedString]|_|): ErrorMessage -> string option\nval (|[no-auto-link ExpectedStringCI]|_|): ErrorMessage -> string option\nval (|[no-auto-link Unexpected]|_|): ErrorMessage -> string option\nval (|[no-auto-link UnexpectedString]|_|): ErrorMessage -> string option\nval (|[no-auto-link UnexpectedStringCI]|_|): ErrorMessage -> string option\nval (|[no-auto-link Message]|_|): ErrorMessage -> string option\nval (|[no-auto-link NestedError]|_|): ErrorMessage\n -> (Position * obj * ErrorMessageList) option\nval (|[no-auto-link CompoundError]|_|): ErrorMessage\n -> (string * Position * obj * ErrorMessageList) option\nval (|[no-auto-link OtherErrorMessage]|_|): ErrorMessage -> obj option\n\n\n// The following literal definition and active pattern allow you to\n// treat the ErrorMessageList type as if it was defined as:\n//\n// []\n// type ErrorMessageList =\n// | AddErrorMessage of ErrorMessage * ErrorMessageList\n// | NoErrorMessages\n// with\n// static member Merge: ErrorMessageList * ErrorMessageList -> ErrorMessageList\n// static member ToHashSet: ErrorMessageList -> HashSet\n// static member ToSortedArray: ErrorMessageList -> ErrorMessage[]\n\n[]\nval #NoErrorMessages#: ErrorMessageList = null\n\nval (|[no-auto-link ErrorMessageList|NoErrorMessages]|):\n ErrorMessageList -> Choice\n``]\n\n[``\n\n\n// Helper functions for creating an ErrorMessageList with a single ErrorMessage\nval @expected@: string -> ErrorMessageList\n``]\n[`expected label` creates an `ErrorMessageList` with a single `Expected label` message.]\n\n[``\nval @expectedStringError@: string -> ErrorMessageList\n``]\n[`expectedStringError str` creates an `ErrorMessageList` with a single `ExpectedString str` message.]\n\n[``\nval @expectedStringCIError@: string -> ErrorMessageList\n``]\n[`expectedStringCIError str` creates an `ErrorMessageList` with a single `ExpectedStringCI str` message.]\n\n\n[``\nval @unexpected@: string -> ErrorMessageList\n``]\n[`unexpected label` creates an `ErrorMessageList` with a single `Unexpected label` message.]\n\n[``\nval @unexpectedStringError@: string -> ErrorMessageList\n``]\n\n[`unexpectedStringError str` creates an `ErrorMessageList` with a single `UnexpectedString str` message.]\n\n[``\nval @unexpectedStringCIError@: string -> ErrorMessageList\n``]\n[`unexpectedStringCIError str` creates an `ErrorMessageList` with a single `UnexpectedStringCI str` message.]\n\n[``\nval @messageError@: string -> ErrorMessageList\n``]\n[`messageError msg` creates an `ErrorMessageList` with a single `Message msg` message.]\n\n[``\nval @otherError@: obj -> ErrorMessageList\n``]\n[`otherError o` creates an `ErrorMessageList` with a single `OtherError o` message.]\n\n[``\nval @nestedError@:\n CharStream<_> -> ErrorMessageList -> ErrorMessageList\n``]\n[`nestedError stream msgs` creates an `ErrorMessageList` with a single `NestedError(stream.Position, stream.UserState, msgs)` message,\nexcept if `msgs` is already an `ErrorMessageList` with a single `NestedError` message, in which case `msgs` is returned instead.]\n\n[``\nval @compoundError@:\n string -> CharStream<_> -> ErrorMessageList -> ErrorMessageList\n``]\n[\n`compoundError label stream msgs` creates an `ErrorMessageList` with a single `CompoundError(label, stream.Position, stream.UserState, msgs)` message,\nexcept if `msgs` is an `ErrorMessageList` with a single `NestedError(pos2, ustate2, msgs2)` message,\nin which case an `ErrorMessageList` with a single `CompoundError(label, pos2, ustate2, msgs2)` message is returned instead.]\n\n[``\n\n\n// Two convenient helper functions\n``]\n\n[``\nval @mergeErrors@: ErrorMessageList -> ErrorMessageList -> ErrorMessageList\n``]\n[`mergeErrors error1 error2` is an abbreviation for `ErrorMessageList.Merge(error1, error2)`.]\n\n[``\nval @isSingleErrorMessageOfType@: ErrorMessageType -> ErrorMessageList\n``]\n[\n`isSingleErrorMessageOfType ty msgs` returns `true` if and only if`msgs` is an `ErrorMessageList` with a single `ErrorMessage` with the `ErrorMessageType` `ty`.\n]\n\n[``\n\n\n// A simple container type for holding an ErrorMessageList\n// together with its associated input stream position and user state\n``]\n[``\n[]\ntype @ParserError@``]\n[\n`ParserError` is a simple container type for holding an `ErrorMessageList` together with its associated input stream position and user state.\n\nThe `ParserError` class has the following members:\n\n[interface-members]\n[`` =\n``]\n\n[``\n @new@: position: [^reference.Position Position]\n * userState: obj\n * messages: ErrorMessageList\n -> ParserError\n``]\n[Constructs a `ParserError` from an `ErrorMessageList` and its associated position.]\n\n[``\n\n member @Position@: [^reference.Position Position]\n``]\n[\nThe input stream position of the parser error.\n]\n\n[``\n member @UserState@: obj\n``]\n[\nThe user state associated with the parser error.\n]\n\n[``\n member @Messages@: ErrorMessageList\n``]\n[#member-ErrorMessageList\nThe error messages of the parser error.\n]\n\n[``\n[#ToString/WriteTo]\n override @ToString@: unit -> string\n``]\n[\nIs equivalent to\n``\nuse sw = new System.IO.StringWriter()\nWriteTo(sw)\nsw.ToString()\n``\n]\n\n[``\n member @ToString@: streamWhereErrorOccurred: CharStream -> string\n``] [#ToString_CharStream\nIs equivalent to\n``\nuse sw = new System.IO.StringWriter()\n[^WriteTo_stream WriteTo](sw, streamWhereErrorOccurred)\nsw.ToString()\n``\n]\n\n[``\n\n member @WriteTo@:\n textWriter: System.IO.TextWriter\n * streamWhereErrorOccurred: CharStream\n * ?tabSize: int\n * ?columnWidth: int\n * ?initialIndentation: string * ?indentationIncrement: string\n -> unit\n``]\n[#WriteTo_stream\nIs equivalent to\n``\nlet getStream (pos: Position) =\n if pos.StreamName = @Position@.StreamName then streamWhereErrorOccurred\n else null\n\n[^WriteTo_getStream WriteTo](textWriter,\n [* getStream],\n ?tabSize = tabSize,\n ?columWidth = columnWidth,\n ?initialIndentation = initialIndentation,\n ?indentationIncrement = indentationIncrement)\n``\n]\n\n[``\n\n member @WriteTo@:\n textWriter: System.IO.TextWriter\n * getStream: (position -> CharStream)\n * ?tabSize: int\n * ?columnWidth: int\n * ?initialIndentation: string * ?indentationIncrement: string\n -> unit\n``]\n[#WriteTo_getStream\nWrites a string representation of the `ParserError` to the given `TextWriter` value.\n\nFor each error `getStream` is called with the error position.\nThe returned `CharStream` must be `null` or contain the content of the `CharStream` for which\nthe error was generated (at the original indices).\n\nIf `getStream` returns a non-null `CharStream`, the printed error position information is\naugmented with the line of text surrounding the error position, together with a '^'-marker\npointing to the exact location of the error in the input stream.\n\nThe `tabSize` parameter (default value: 8) specifies the tab stop distance that this method assumes when counting text columns. This parameter only has an effect for error positions where `getStream` returns a non-null `CharStream`.\n\nThe `columnWidth` parameter (default value: 79) specifies the number of char columns that this\nmethod should try to fit its output to.\n]\n\n[``\n\n member @WriteTo@:\n textWriter: System.IO.TextWriter\n * ?positionPrinter:\n (System.IO.TextWriter -> [^reference.Position Position] -> string -> int -> unit)\n * ?columnWidth: int\n * ?initialIndentation: string * ?indentationIncrement: string\n -> unit\n``]\n[\nWrites a string representation of the `ParserError` to the given `TextWriter` value.\n\nThe format of the position information can be customized by specifying the `positionPrinter`\nargument. The given function is expected to print a representation of the passed `[^reference.Position Position]` value\nto the passed `TextWriter` value. If possible, it should indent text lines with the passed string\nand take into account the maximum column count (including indentation) passed as the last argument.\n]\n\n[``\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n``]\n\n[/interface-members]\n]\n\n[/interface-members]\n[/section]\n[/interface-reference]\n[/section]\n[/auto-link]\n" }, { "path": "Doc/src/reference-errormessage.txt", "content": "\n\n[section#ErrorMessage FParsec.ErrorMessage]\n\n[interface-reference]\n\n[section Interface]\n[$$interface]\n[/section]\n\n[section Remarks]\n\n`ErrorMessage` is the abstract base class for FParsec error messages. `Parser` functions return `ErrorMessage` values within an `ErrorMessageList`.\n\nThere are [^members.nested-types several subtypes] of `ErrorMessage` that represent specific kind of error messages. These subtypes are defined as nested classes within `ErrorMessage`.\n\nThe [^Error.members.discriminated-union-type active patterns and type abbreviations] in the `FParsec.Error` module allow you to treat the `ErrorMessage` type almost as if it was defined as an F# discriminated union type.\n\n[/section]\n\n[section Members]\n[interface-members]\n\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype #ErrorMessageType# = [#ErrorMessageType..Expected Expected] = 0\n | [#ErrorMessageType..ExpectedString ExpectedString] = 1\n | [#ErrorMessageType..ExpectedCaseInsensitiveString ExpectedCaseInsensitiveString] = 2\n | [#ErrorMessageType..Unexpected Unexpected] = 3\n | [#ErrorMessageType..UnexpectedString UnexpectedString] = 4\n | [#ErrorMessageType..UnexpectedCaseInsensitiveString UnexpectedCaseInsensitiveString] = 5\n | [#ErrorMessageType..Message Message] = 6\n | [#ErrorMessageType..NestedError NestedError] = 7\n | [#ErrorMessageType..CompoundError CompoundError] = 8\n | [#ErrorMessageType..Other Other] = 9\n``]\n[``\n\ntype @ErrorMessage@``]\n[\n\n`ErrorMessage` is the abstract base class for FParsec error messages.\n\n``\ntype ErrorMessage =\n member [no-auto-link Type]: ErrorMessageType\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n interface System.IEquatable\n``\n\nPlease also see the @remarks@ above.\n\n]\n\n[`` =\n member [no-auto-link Type]: ErrorMessageType\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n interface System.IEquatable\n\n[#nested-types]// nested types\n``]\n\n[``\ntype @ErrorMessage.Expected@``]\n[\n\nParsers report this `ErrorMessage` when the input does not match the expected input.\n\n``\ntype ErrorMessage.Expected =\n inherit ErrorMessage\n new: label: string -> ErrorMessage.Expected\n member Label: string\n``\n\nThe string label describes the expected input.\n\nThis error message can be generated with the labeling operator ``.\n]\n[`` =\n inherit ErrorMessage\n new: label: string -> ErrorMessage.Expected\n member Label: string\n``]\n\n[``\n\ntype @ErrorMessage.ExpectedString@``]\n[\nParsers report this `ErrorMessage` when the input does not match an expected string constant.\n\n``\ntype ErrorMessage.ExpectedString =\n inherit ErrorMessage\n new: string -> ErrorMessage.ExpectedString\n member String: string\n``\n\nThis `ErrorMessage` is mainly generated by the `pstring` parser and its variants.\n]\n[`` =\n inherit ErrorMessage\n new: string -> ErrorMessage.ExpectedString\n member String: string\n``]\n[``\n\ntype @ErrorMessage.ExpectedCaseInsensitiveString@``]\n[\nParsers report this `ErrorMessage` when the input does not match an expected case-insensitive string constant.\n\n``\ntype ErrorMessage.ExpectedCaseInsensitiveString =\n inherit ErrorMessage\n new: string -> ErrorMessage.ExpectedCaseInsensitiveString\n member CaseInsensitiveString: string\n``\n\nThis `ErrorMessage` is mainly generated by the `pstringCI` parsers and its variants.\n]\n[`` =\n inherit ErrorMessage\n new: string -> ErrorMessage.ExpectedCaseInsensitiveString\n member String: string\n``]\n[``\n\ntype @ErrorMessage.Unexpected@``]\n[\nParsers report this `ErrorMessage` when they encounter some unexpected input.\n\n``\ntype ErrorMessage.Unexpected =\n inherit ErrorMessage\n new: label: string -> ErrorMessage.Unexpected\n member Label: string\n``\n\nThe string label describes the unexpected input.\n\nThis `ErrorMessage` is mainly generated by the `notFollowedByL` primitive.\n]\n[`` =\n inherit ErrorMessage\n new: label: string -> ErrorMessage.Unexpected\n member Label: string\n``]\n[``\n\ntype @ErrorMessage.UnexpectedString@``]\n[\nParsers report this `ErrorMessage` when they encounter an unexpected string constant.\n\n``\ntype ErrorMessage.UnexpectedString =\n inherit ErrorMessage\n new: string -> ErrorMessage.UnexpectedString\n member String: string\n``\n\nThis `ErrorMessage` is mainly generated by the `notFollowedByString` parser.\n]\n[`` =\n inherit ErrorMessage\n new: string -> ErrorMessage.UnexpectedString\n member String: string\n``]\n\n[``\n\ntype @ErrorMessage.UnexpectedCaseInsensitiveString@``]\n[\nParsers report this `ErrorMessage` when they encounter an unexpected case-insensitive string constant.\n\n``\ntype ErrorMessage.UnexpectedCaseInsensitiveString =\n inherit ErrorMessage\n new: string -> ErrorMessage.UnexpectedCaseInsensitiveString\n member CaseInsensitiveString: string\n``\n\nThis `ErrorMessage` is mainly generated by the `notFollowedByStringCI` parser.\n]\n[`` =\n inherit ErrorMessage\n new: string -> ErrorMessage.UnexpectedCaseInsensitiveString\n member String: string\n``]\n\n[``\n\ntype @ErrorMessage.Message@``]\n[\nParsers report this `ErrorMessage` when an the error does not fit the other `ErrorMessage` types.\n\n``\ntype ErrorMessage.Message =\n inherit ErrorMessage\n new: string -> ErrorMessage.Message\n member String: string\n``\n\nThis error message can be generated with the `fail` and `failFatally` primitives.\n]\n[`` =\n inherit ErrorMessage\n new: string -> ErrorMessage.Message\n member String: string\n``]\n\n[``\n\ntype @ErrorMessage.NestedError@``]\n[\n\nParsers report this `ErrorMessage` when they backtracked after an error occurred.\n\n``\ntype ErrorMessage.NestedError =\n inherit ErrorMessage\n\n new: position: Position * userState: obj * messages: ErrorMessageList\n -> ErrorMessage.NestedError\n\n member Position: Position\n member UserState: obj\n member Messages: ErrorMessageList\n``\n\nThe `Position` property describes the stream position where the original error occurred that triggered the backtracking. The `UserState` property contains the user state value from before the backtracking (upcasted to `obj`). The `Messages` property contains the error messages of the original error.\n\nThis error message is mainly generated by the `attempt`, `>>?` and `.>>?` primitives.\n]\n\n[`` =\n inherit ErrorMessage\n\n new: position: Position * userState: obj * messages: ErrorMessageList\n -> ErrorMessage.NestedError\n\n member Position: Position\n member UserState: obj\n member Messages: ErrorMessageList\n``]\n\n[``\n\ntype @ErrorMessage.CompoundError@``]\n[\nParsers report this `ErrorMessage` when a \"compound\" failed to parse.\n\n``\ntype ErrorMessage.CompoundError =\n inherit ErrorMessage\n\n new: labelOfCompound: string\n * nestedErrorPosition: Position\n * nestedErrorUserState: obj\n * nestedErrorMessages: ErrorMessageList\n -> ErrorMessage.CompoundError\n\n member LabelOfCompound: string\n member NestedErrorPosition: Position\n member NestedErrorUserState: obj\n member NestedErrorMessages: ErrorMessageList\n``\n\nThis error message is mainly generated by the compound-labelling operator ``.\n\n]\n[`` =\n inherit ErrorMessage\n\n new: labelOfCompound: string\n * nestedErrorPosition: Position\n * nestedErrorUserState: obj\n * nestedErrorMessages: ErrorMessageList\n -> ErrorMessage.CompoundError\n\n member LabelOfCompound: string\n member NestedErrorPosition: Position\n member NestedErrorUserState: obj\n member NestedErrorMessages: ErrorMessageList\n``]\n\n[``\n\ntype @ErrorMessage.Other@``]\n[\n\nUser-defined parsers can return this `ErrorMessage` to report application-specific error data.\n\n``\ntype ErrorMessage.Other =\n inherit ErrorMessage\n new: data: obj -> ErrorMessage.Other\n member Data: obj\n``\n\nTo display `OtherError` values in error messages, you will have to define your own error printer, as `ParserError.\\ @ToString/WriteTo@` ignores them.\n]\n[`` =\n inherit ErrorMessage\n new: data: obj -> ErrorMessage.Other\n member Data: obj\n``]\n\n\n[/interface-members]\n\n[/section]\n[/interface-reference]\n\n[/section]\n" }, { "path": "Doc/src/reference-errormessagelist.txt", "content": "\n\n\n[section#ErrorMessageList FParsec.ErrorMessageList]\n\nRepresents a list of error messages.\n\n[interface-reference]\n\n[section Interface]\n[$$interface]\n[/section]\n\n[section Remarks]\n\nThe `ErrorMessageList` represents a list of error messages in which the order of the messages carries no meaning and any duplicates and empty messages are ignored. Essentially, an `ErrorMessageList` is *constructed as a singly-linked list, but used as a set*.\n\nA `null` value represents an empty `ErrorMessageList`.\n\nThe `ErrorMessage` values in an `ErrorMessageList` are usually all associated with the same input stream position and user state. For example, the error messages returned by a parser in a `Reply` value describe an error at the `CharStream` position that is current when the parser returns.\n\nIn order to enforce set semantics in comparison operations, the `ErrorMessageList` overrides the `Equals` and `GetHashCode`.\n\n[/section]\n\n[section Members]\n[interface-members]\n\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\n[]\ntype ErrorMessageList``]\n[`` =\n``]\n[``\n member @Head@: ErrorMessage\n``]\n[\nThe first `ErrorMessage` in this list. This property is never `null`.\n]\n\n[``\n member @Tail@: ErrorMessageList\n``]\n[\nThe remaining `ErrorMessage` values in this list after the first `ErrorMessage`.\n\nIf there are no remaining `ErrorMessage` values, this property is `null`.\n]\n\n[``\n\n @new@: head: ErrorMessage -> ErrorMessageList\n``]\n[#new-1\nConstructs a new `ErrorMessageList` with a single `ErrorMessage` value.\n\nThis constructor throws a `NullReferenceException` if `head` is null.\n]\n\n[``\n @new@: head: ErrorMessage * tail: ErrorMessageList -> ErrorMessageList\n``]\n[#new-2\nConstructs a new `ErrorMessageList` with `Head` set to `head` and `Tail` set to `tail`.\n\nThis constructor throws a `NullReferenceException` if `head` is null.\n]\n\n[``\n @new@: head: ErrorMessage * tailMessage: ErrorMessage -> ErrorMessageList\n``]\n[#new-3\n`new ErrorMessageList(head, tailmessage)` is equivalent to `new [^new-2 ErrorMessageList](head, new [^new-1 ErrorMessageList](tailMessage))`.\n]\n\n[``\n\n static member @Merge@: ErrorMessageList * ErrorMessageList -> ErrorMessageList\n``]\n[\nCreates a new `ErrorMessageList` that contains the `ErrorMessage` values from both argument lists.\n\nThe order of the `ErrorMessage` values in the newly created list is an implementation detail that you should not depend on.\n]\n\n[``\n static member @ToHashSet@: ErrorMessageList -> HashSet\n``]\n[\nConverts the `ErrorMessageList` to a `HashSet`. Duplicate error messages and empty `Expected...`, `Unexpected...` and `Message` messages are filtered out when the list is converted to a set.\n]\n\n[``\n static member @ToSortedArray@: ErrorMessageList -> ErrorMessage[]\n``]\n[\nConverts the `ErrorMessageList` to a array that is sorted by a total order. Duplicate error messages and empty `Expected...`, `Unexpected...` and `Message` messages are filtered out when the list is converted to the array.\n\nThe order of the sorted array is an implementation detail and may change in the future.\n]\n\n[``\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n interface System.IEquatable\n``]\n\n\n[/interface-members]\n\n[/section]\n[/interface-reference]\n\n[/section]\n" }, { "path": "Doc/src/reference-operatorprecedenceparser.txt", "content": "\n[interface-reference]\n[section#OperatorPrecedenceParser FParsec.OperatorPrecedenceParser]\n\n[section Interface]\n[$$interface]\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype #Associativity# = [#Associativity..None None] = 0\n | [#Associativity..Left Left] = 1\n | [#Associativity..Right Right] = 2\n\ntype #OperatorType# = [#OperatorType..Infix Infix] = 0\n | [#OperatorType..Prefix Prefix] = 1\n | [#OperatorType..Postfix Postfix] = 2\n``]\n\n[``\n\n\ntype @Operator@<'TTerm, 'TAfterString, 'TUserState>``]\n[\n\nThe `Operator` type represents an immutable operator definition for the `OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState>` (OPP) class.\n\n``\n[]\ntype Operator<'TTerm, 'TAfterString, 'TUserState> =\n member Type: OperatorType\n member Associativity: Associativity\n member Precedence: int\n\n member IsAssociative: bool\n member IsTernary: bool\n\n member @String@: string\n member TernaryRightString: string // null for non-ternary operators\n``\n\n\nThe `Operator` class is the abstract base class of the `InfixOperator`, `PrefixOperator`, `PostfixOperator` and `TernaryOperator` classes. With these four concrete classes you can define binary infix (e.g. \"1 + 1\"), unary prefix (e.g. \"-1\"), unary postfix (e.g. \"1++\") and C-style ternary operators (e.g. \"a ? b : c\") for the `OperatorPrecedenceParser` (OPP) class.\n\nIf you have look at the constructors for the concrete operator classes, you'll see that operators are constructed from an operator string, an \"after-string-parser\", a precedence level, an associativity value and a mapping function that is applied after the expression is parsed.\n\nTernary operators are treated as special infix operators and require a string and associated after-string-parser parser for each of the two operator parts.\n\n[dl\n[Associativity and precedence]\n[\nWhile infix operators can be left-, right- and non-associative (see the `Associativity` type), prefix and postfix operators can only be associative (`true`) or non-associative (`false`). See below for details on [^precedence-associativity how precedence and associativity influence the operator precedence parser].\n]\n\n[Textual representation of operators]\n[\nThe operator string and the after-string-parser determine the textual representation of an operator. Usually, the after-string-parser is used for parsing the whitespace after an operator string.\n\nOPP instances have separate \"namespaces\" for prefix operators on the one hand and infix, postfix or ternary operators on the other hand. Hence, you can configure an OPP instance to recognize a prefix operator with the same string as the (first) string of an infix, postfix or ternary operator. However, no two prefix operators and no two infix, postfix or ternary operators can have the same (first) string. The second string of a ternary operator cannot be used for any other operator at the same time.\n\nThe OPP class parses operator strings greedily. This means, for example, that if you define a prefix operator with the string `\"-\"` and another prefix operator with the string `\"--\"`, then the input `--` in a prefix location will always be parsed as a `--` operator, never as two successive `-` operators.\n]\n\n[How the OPP applies the after-string-parser]\n[\nIf the OPP encounters the operator string in the input, it will apply the after-string-parser directly after the operator string. If the after-string-parser succeeds, the operator will be accepted. If the after-string-parser fails without consuming input (or changing the parser state any another way), the OPP will backtrack to before the operator string and will not try to parse any other operator at this location. If the after-string-parser parser fails after consuming input, the OPP will itself fail with this error.\n\nThis backtracking behaviour can be exploited to conditionally accept an operator depending on the input following the operator string. For example, the after-string-parser definition in `PrefixOperator(\"not\", notFollowedBy letter >>. spaces, 1, true, (* ... *))` will ensure that the `\"not\"` in `\"notAnOperator\"` cannot be parsed as an operator.\n]\n\n[The mapping function argument of the operator constructors]\n[\nWhen an OPP instance has finished parsing a sub-expresssion involving an operator, it uses the mapping function supplied as the last argument to the operator constructor to map the parsed term(s) to a new term. Usually this mapping function constructs an AST node or directly transforms the terminal values.\n\nThe operator classes `InfixOperator`, `PrefixOperator`, etc. all support two alternative types of mapping functions. The simpler type of mapping function only gets passed the parsed term(s). The other type of mapping function also gets passed the result(s) of the after-string-parser(s).\n]\n\n[More uses of the after-string-parser]\n[\nThe combination of individually configurable after-string-parsers and mapping functions make the OPP class quite flexible in addressing various practical parsing needs.\n\nOne use of the after-string-parser is discussed in the user's guide section on [@ parsing F# infix operators].\n\nAnother use is demonstrated in the following example. It shows [#get-position-with-after-string-parser how you can use the after-string-parser to get hold of the precise text location of the parsed operator] (which is often useful for diagnostic purposes in your application):\n\n``\nopen FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nlet opp = new OperatorPrecedenceParser<_,_,_>()\n\nlet ws = spaces\n\ntype Assoc = Associativity\n\nlet adjustPosition offset (pos: Position) =\n Position(pos.[^Position..StreamName StreamName], pos.[^Position..Index Index] + int64 offset,\n pos.[^Position..Line Line], pos.[^Position..Column Column] + int64 offset)\n\n// To simplify infix operator definitions, we define a helper function.\nlet addInfixOperator str prec assoc mapping =\n let op = InfixOperator(str, getPosition .>> ws, prec, assoc, (),\n fun opPos leftTerm rightTerm ->\n mapping\n (adjustPosition -str.Length opPos)\n leftTerm rightTerm)\n opp.AddOperator(op)\n\n// Of course, you can define similar functions for other operator types.\n\n// With the helper function in place, you can define an operator with\n// a mapping function that gets passed the text location of the\n// parsed operator as the first argument.\naddInfixOperator \"+\" 1 Assoc.Left (fun opPos leftTerm rightTerm -> (* ... *))\n``\n]\n]\n\n[br]Members of `Operator<'TTerm, 'TAfterString, 'TUserState>`:\n\n[interface-members]\n[``=\n``]\n[``\n member @Type@: OperatorType\n``]\n[\nThe operator's type: `[^OperatorType..Infix Infix]`, `[^OperatorType..Prefix Prefix]` or `[^OperatorType..Postfix Postfix]`.\n\nTernary operators are treated as special infix operators.\n]\n[``\n member @`Associativity@: @Associativity@\n``]\n[#Operator..Associativity\nThe operator's associativity: `[^Associativity..None None]`, `[^Associativity..None Left]` or `[^Associativity..None Right]`.\n\nFor associative prefix operators this value is `Associativity.Right`, for associative postfix operators this value is `Associativity.Left`.\n]\n\n[``\n member @Precedence@: int\n``]\n[\n The operator's precedence value. The value is always greater than zero. Operators with a numerically higher precedence value take precedence over operators with lower precedence values.\n]\n\n[``\n\n member @IsAssociative@: bool\n``]\n[Is equivalent to `[^Operator..Associativity Associativity] != Associativity.None`.]\n\n[``\n member @IsTernary@: bool\n``]\n[Indicates whether the operator is a `TernaryOperator`.]\n\n[``\n\n member @String@: string\n``]\n[The operator's string specified during construction.\n\nFor ternary operators this property returns the left string.\n]\n\n[``\n member @TernaryRightString@: string``]\n[\nThe right string of a `TernaryOperator`.\n\nFor non-ternary operators this property is null.\n]\n\n[`` // null for non-ternary operators\n``]\n[/interface-members]\n\n]\n\n[``\n\n// the following four types inherit from Operator<_,_,_>\n``]\n[``\ntype @InfixOperator@<'TTerm, 'TAfterString, 'TUserState>``]\n[\n\nThe `InfixOperator<'TTerm, 'TAfterString, 'TUserState>` type represents a binary infix operator definition (e.g. the `+` in `1 + 1`) for the `OperatorPrecedenceParser` class.\n\n``\ntype InfixOperator<'TTerm, 'TAfterString, 'TUserState> =\n inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * associativity: Associativity\n * mapping: 'TTerm -> 'TTerm -> 'TTerm\n -> InfixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * associativity: Associativity\n * dummy: unit // disambiguates overloads in F#\n * mapping: 'TAfterString -> 'TTerm -> 'TTerm -> 'TTerm\n -> InfixOperator<'TTerm, 'TAfterString, 'TUserState>\n``\n\n[* The two constructors only differ in the type of the mapping they accept. To help F#'s type inference discern both constructors, the second constructor accepts an additional dummy argument.]\n\nPlease see the documentation for the `Operator` base class for more information.\n\n]\n[`` = // ...\n``]\n[``\ntype @PrefixOperator@<'TTerm, 'TAfterString, 'TUserState>``]\n[\nThe `PrefixOperator<'TTerm, 'TAfterString, 'TUserState>` type represents a unary prefix operator definition (e.g. the `-` in `-1`) for the `OperatorPrecedenceParser` class.\n\n``\ntype PrefixOperator<'TTerm, 'TAfterString, 'TUserState> =\n inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * isAssociative: bool\n * mapping: 'TTerm -> 'TTerm\n -> PrefixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * isAssociative: bool\n * dummy: unit // disambiguates overloads in F#\n * mapping: 'TAfterString -> 'TTerm -> 'TTerm\n -> PrefixOperator<'TTerm, 'TAfterString, 'TUserState>\n``\n\n[* The two constructors only differ in the type of the mapping they accept. To help F#'s type inference discern both constructors, the second constructor accepts an additional dummy argument.]\n\nPlease see the documentation for the `Operator` base class for more information.\n]\n[`` = // ...\n``]\n[``\ntype @PostfixOperator@<'TTerm, 'TAfterString, 'TUserState>``]\n[\nThe `PostfixOperator<'TTerm, 'TAfterString, 'TUserState>` type represents a unary postfix operator definition (e.g. the `++` in `1++`) for the `OperatorPrecedenceParser` class.\n\n``\ntype PostfixOperator<'TTerm, 'TAfterString, 'TUserState> =\n inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * isAssociative: bool\n * mapping: 'TTerm -> 'TTerm\n -> PostfixOperator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * isAssociative: bool\n * dummy: unit // disambiguates overloads in F#\n * mapping: 'TAfterString -> 'TTerm -> 'TTerm\n -> PostfixOperator<'TTerm, 'TAfterString, 'TUserState>\n``\n\n[* The two constructors only differ in the type of the mapping they accept. To help F#'s type inference discern both constructors, the second constructor accepts an additional dummy argument.]\n\nPlease see the documentation for the `Operator` base class for more information.\n\n]\n[`` = // ...\n``]\n[``\ntype @TernaryOperator@<'TTerm, 'TAfterString, 'TUserState>``]\n[\nThe `TernaryOperator<'TTerm, 'TAfterString, 'TUserState>` type represents a C-style ternary operator definition (e.g. the `{none}? :` in `{c++}a ? b : c`) for the `OperatorPrecedenceParser` class.\n\n``\ntype TernaryOperator<'TTerm, 'TAfterString, 'TUserState> =\n inherit Operator<'TTerm, 'TAfterString, 'TUserState>\n\n new: leftString: string\n * afterLeftStringParser: Parser<'TAfterString,'TUserState>\n * rightString: string\n * afterRightStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * associativity: Associativity\n * mapping: 'TTerm -> 'TTerm -> 'TTerm -> 'TTerm\n -> TernaryOperator<'TTerm, 'TAfterString, 'TUserState>\n\n new: operatorString: string\n * afterStringParser: Parser<'TAfterString,'TUserState>\n * precedence: int\n * isAssociative: bool\n * dummy: unit // disambiguates overloads in F#\n * mapping: 'TAfterString -> 'TAfterString -> 'TTerm -> 'TTerm -> 'TTerm\n -> 'TTerm\n -> TernaryOperator<'TTerm, 'TAfterString, 'TUserState>\n``\n\n[* The two constructors only differ in the type of the mapping they accept. To help F#'s type inference discern both constructors, the second constructor accepts an additional dummy argument.]\n\nPlease see the documentation for the `Operator` base class for more information.\n\n]\n\n[`` = // ...\n``]\n\n[``\n\n\ntype @OperatorPrecedenceParser@<'TTerm, 'TAfterString, 'TUserState>``]\n[\n\nThe `OperatorPrecedenceParser` class (OPP) represents a dynamically configurable parser for parsing expression grammars involving binary infix (e.g. `1 + 1`), unary prefix (e.g. `-1`), unary postfix (e.g. `1++`) and C-style ternary operators (e.g. `{c++}a ? b : c`).\n\nYou can configure an OPP instance by adding and removing operator definitions in the form of [** `Operator`] values. If you add an operator that conflicts with a previous operator definition, `AddOperator` will raise an `ArgumentException`. The `Operators` property returns a snapshot of the currently defined set of operators. The `RemoveInfixOperator`, `RemovePrefixOperator`, etc. members remove operator definitions based only on their text representation. All `Remove...` members return `false` if no matching operator was previously defined, otherwise `true`.\n\nThe actual expression parser of the OPP is exposed through the `[** ExpressionParser]` property.\nThe `ExpressionParser` value is a constant closure that forwards all work to internal instance methods. This ensures that the behaviour of the expression parser always reflects the latest configuration of the OPP instance. You can safely call the `ExpressionParser` concurrently from multiple threads, as long as the configuration of the OPP instance is not changed at the same time.\n\nBefore you can call the `ExpressionParser` you first need to set the `[** TermParser]`. The OPP instance uses the `TermParser` to parse the terms in between the operators. Often the `TermParser` will not just parse terminal values but will also recursively call the `ExpressionParser`, for example to parse an expression between parentheses. Note that the `TermParser` also needs to consume any trailing whitespace.\n\nThis example shows how to define a parser for very simple arithmetic expressions:\n``\nopen FParsec\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nlet ws = spaces\nlet str_ws s = pstring s >>. ws\n\nlet opp = new OperatorPrecedenceParser()\nlet expr = opp.ExpressionParser\nlet term = (pfloat .>> ws) <|> between (str_ws \"(\") (str_ws \")\") expr\nopp.TermParser <- term\n\ntype Assoc = Associativity\n\nopp.AddOperator(InfixOperator(\"+\", ws, 1, Assoc.Left, fun x y -> x + y))\nopp.AddOperator(InfixOperator(\"*\", ws, 2, Assoc.Left, fun x y -> x * y))\n``\n``{fsi}\n> run expr \"1 + 2*(3 + 4)\";;\nval it : ParserResult = Success: 15.0\n``\n\n[#precedence-associativity]The following points explain *how expressions are parsed depending on precedence and associativity* of the involved operators:\n\n- Operators with higher precedence bind tighter. For example, if the prefix operator \"~\" has a lower precedence than the infix operator \"&\" then \"~x&y\" will be parsed as \"~(x&y)\".\n\n- Ternary operators are treated as special infix operators. The middle expression (e.g. \"expr2\" in \"expr1 ? expr2 : expr3\") is parsed as a \"fresh\" expression that is not influenced by the precedence of the surrounding operators.\n\n- Operators with identical precedence are parsed as follows:\n ``{none}\nHere o1, o2 are two infix operators,\n pre1, pre2 are two prefix operators,\n po1, po2 are two postfix operators\nand all operators have identical precedence.\n\npre1 x o1 y ==> (pre1 x) o1 y\nx o1 y po1 ==> x o1 (y po1)\nx o1 y o2 z ==> (x o1 y) o2 z if o1 and o2 are left-associative\nx o1 y o2 z ==> x o1 (y o2 z) if o1 and o2 are right-associative\npre1 x po1 ==> (pre1 x) po1 if pre1 or po1 is associative\npre1 pre2 x ==> pre1 (pre2 x) if pre1 or pre2 is associative\nx po1 po2 ==> (x po1) po2 if po1 or po2 is associative\n ``\n\n- If the parser encounters #conflicting operators#, e.g. if a right-associative infix operators follows a left-associative operator with the same precedence level, the OPP fails and returns with an error generated with the help of the `OperatorConflictErrorFormatter`.\n\n In the following situations the OPP will fail with an operator conflict error:\n ``{none}\n[Same notation as above, all operators have identical precedence.]\n\nx o1 y o2 z if o1 and o2 have different associativity\n or o1 and o2 are non-associative\npre1 pre2 x if pre1 and pre2 are non-associative\npre1 x po1 if pre1 and po1 are non-associative\nx po1 po2 if po1 and po2 are non-associative\n``\n\n By giving all operators different precedence levels and making all operators associative, you can exclude any possible operator conflict. A practical reason for defining operators that can lead to conflicts in the inputs (e.g. non-associative operators) is to force the user to explicitely parenthesize an expression involving such operators.\n\n[br]Members of `OperatorPrecedenceParser<'TTerm, 'TAfterString, 'TUserState>`:\n\n[interface-members]\n\n[`` =\n``]\n[``\n member @ExpressionParser@: Parser<'TTerm,'TUserState>\n``]\n[\nThe expression parser. This is a constant closure that forwards all work to internal instance methods, so that the behaviour of the expression parser always reflects the latest configuration of the OPP instance.\n\nYou can safely call the `ExpressionParser` concurrently from multiple threads, as long as the configuration of the `OperatorPrecedenceParser` instance is not changed at the same time.\n]\n\n[``\n member @TermParser@: Parser<'TTerm,'TUserState> with get, set\n``]\n[\nThis parser is called to parse the terms in between the operators. There is no default,\n so you must set this parser before you can call the `ExpressionParser`.\n Note that the term parser is also expected to parse any whitespace after a term.\n ]\n\n[``\n\n member @AddOperator@: Operator<'TTerm, 'TAfterString, 'TUserState> -> unit\n``]\n[\nAdds an operator to the grammar. Raises an `ArgumentException` if the operator definition conflicts with a previous definition.\n]\n[``\n\n member @RemoveOperator@: Operator<'TTerm, 'TAfterString, 'TUserState> -> bool\n``]\n[\nRemoves the given `Operator` instance from the grammar.\nReturns `false` if the `Operator` instance was not previously registered, otherwise `true`.\n]\n\n[``\n member @RemoveInfixOperator@: string -> bool\n``]\n[\nRemoves the `InfixOperator` with the given string from the grammar.\nReturns `false` if no infix operator with that string was previously registered, otherwise `true`.\n]\n\n[``\n member @RemovePrefixOperator@: string -> bool\n``]\n[\nRemoves the `PrefixOperator` with the given string from the grammar.\nReturns `false` if no prefix operator with that string was previously registered, otherwise `true`.\n]\n[``\n member @RemovePostfixOperator@: string -> bool\n``]\n[\nRemoves the `PostfixOperator` with the given string from the grammar.\nReturns `false` if no postfix operator with that string was previously registered, otherwise `true`.\n]\n\n[``\n member @RemoveTernaryOperator@: string * string -> bool\n``]\n[\nRemoves the `TernaryOperator` with the given left and right strings from the grammar.\nReturns `false` if no ternary operator with these strings was previously registered, otherwise `true`.\n]\n\n[``\n member @Operators@: seq>\n``]\n[\nReturns a sequence with a snapshot of the operators currently registered with the `OperatorPrecedenceParser`.\n]\n\n[``\n\n member @OperatorConflictErrorFormatter@:\n ( Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n -> Position * Operator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n -> ErrorMessageList)\n with get, set\n``]\n[\nThe `OperatorConflictErrorFormatter` function is called by the OPP instance when it encounters @conflicting operators@ in the input. The two passed tuples contain the stream positions, operator definitions and the after-string-parser values for the two conflicting operators. The returned `ErrorMessageList` will become part of the error messages returned by the OPP's `ExpressionParser`.\n\nYou can set this formatter to customize the error messages generated when the OPP instance encounters conflicting operators in the inputs. Of course, if your operator grammar doesn't allow for conflicting operators in the input, the `OperatorConflictErrorFormatter` will never be called and there's no need to customize it. The user's guide section on [@ parsing F# infix operators] contains an example with a custom `OperatorConflictErrorFormatter`.\n]\n\n[``\n\n member @MissingTernary2ndStringErrorFormatter@:\n ( Position * Position\n * TernaryOperator<'TTerm, 'TAfterString, 'TUserState> * 'TAfterString\n -> ErrorMessageList)\n with get, set\n``]\n[\nThe `MissingTernary2ndStringErrorFormatter` function is called by the OPP instance when it can't parse the second operator string of a C-style ternary operator (e.g. the `{c++}:` in `{c++}a ? b : c`). The passed tuple contains (in order) the position of the first operator string, the position where the the second string was expected, the operator definition and the after-string-parser value for the left operator part. The returned `ErrorMessageList` will become part of the error messages returned by the OPP's `ExpressionParser`.\n\n]\n\n[/interface-members]\n]\n\n[/interface-members]\n[/section]\n\n[/section]\n[/interface-reference]\n" }, { "path": "Doc/src/reference-overview.txt", "content": "[section Parser overview]\n\n- @Parsing single chars@\n- @Parsing strings directly@\n- @Parsing strings with the help of other parsers@\n- @Parsing numbers@\n- @Parsing whitespace@\n- @Chaining and piping parsers@\n- @Parsing sequences@\n- @Parsing alternatives and recovering from errors@\n- @Conditional parsing and looking ahead@\n- [^customizing-error-messages-table Customizing error messages]\n- @User state handling and getting the input stream position@\n\n[table{format=\"api-table\"} Parsing single chars\n\n[[Parser] [Description]]\n\n[[[** `pchar c`] [br][small (variants: `skipChar`, `charReturn`)]]\n [Parses the char `c`.]]\n\n[[[** `anyChar`] [br][small (variant: `skipAnyChar`)]]\n [Parses any one char.]]\n\n[[[** `satisfy f`] [br][small (variants: `[^satisfy-parsers (skipS|s)atisfy[L]]`)]]\n [Parses any one char for which the predicate function `f` returns `true`.]]\n\n[[[** `anyOf str`] [br][small (variant: `skipAnyOf`)]]\n [Parses any one char in the string `str`.]]\n\n[[[** `noneOf str`] [br][small (variant: `skipNoneOf`)]]\n [Parses any one char not in the string `str`.]]\n\n[[[** `letter`] [br][small (variants: `lower`, `upper`)]]\n [Parses any one unicode letter char identified by `System.Char.IsLetter`.]]\n\n[[[** `asciiLetter`] [br][small (variants: `asciiLower`, `asciiUpper`)]]\n [Parses any one char in the range `'a'` - `'z'` and `'A'` - `'Z'`.]]\n\n[[[** `digit`] [br][small (variants: `hex`, `octal`)]]\n [Parses any one char in the range `'0'` - `'9'`.]]\n\n]\n\n\n[table{format=\"api-table\"} Parsing strings directly\n\n[[Parser] [Description]]\n\n[[[** `pstring str`] [br][small (variants: `@skipString@`, `stringReturn`)]]\n [Parses the string `str`.]]\n\n[[[** `pstringCI str`] [br][small (variants: `skipStringCI`, `stringCIReturn`)]]\n [Parses any string that @case-insensitively@ matches the string `str`.]]\n\n[[[** `anyString n`] [br][small (variants: `skipAnyString`)]]\n [Parses any sequence of `n` chars.]]\n\n[[[** `restOfLine skipNewline`] [br][small (variant: `skipRestOfLine`)]]\n [Parses any chars before the end of the line and, if `skipNewline` is `true`, skips to the beginning of the next line (if there is one).]]\n\n[[[** `charsTillString str skipString nMax`] [br][small (variants: `charsTillStringCI`, `[^charsTillString-parsers skipCharsTillString[CI]]`)]]\n [Parses all chars before the first occurance of the string `str` and, if `skipString` is `true`, skips over `str`. Fails if more than `nMax` chars come before `str`.]]\n\n[[[** `manySatisfy f`] [br][small (variant: `skipManySatisfy`)]]\n [Parses a sequence of *zero* or more chars that satisfy the predicate function `f` (i.e. chars for which `f` returns `true`).]]\n\n[[[** `manySatisfy2 f1 f`] [br][small (variant: `skipManySatisfy2`)]]\n [Parses a sequence of *zero* or more chars, where the first char must satisfy the predicate function `f1` and the remaining chars must satisfy `f`.]]\n\n[[[** `many1Satisfy f`] [br][small (variants: `[^many1Satisfy-parsers (skipM|m)any1Satisfy[2][L]]`)]]\n [Parses a sequence of *one* or more chars that satisfy the predicate function `f`.]]\n\n[[[** `manyMinMaxSatisfy nMin nMax f`] [br][small (variants: `[^manyMinMaxSatisfy-parsers (skipM|m)anyMinMaxSatisfy[2][L]]`)]]\n [Parses a sequence of `nMin` or more chars that satisfy the predicate function `f`, but not more than `nMax` chars.]]\n\n[[[** `regex pattern`]]\n [Parses a sequence of *one* or more chars matched by the .NET regular expression string `pattern`.]]\n\n[[[** `identifier options`]]\n [Parses a Unicode identifier.]]\n\n]\n\n[table{format=\"api-table\"} Parsing strings with the help of other parsers\n\n[[Parser] [Description]]\n\n[[[** `manyChars cp`] [br][small (variants: `[^manyChars-parsers manyChars2]`)]]\n [Parses a sequence of *zero* or more chars with the char parser `cp`.]]\n\n[[[** `many1Chars cp`] [br][small (variants: `[^many1Chars-parsers many1Chars2]`)]]\n [Parses a sequence of *one* or more chars with the char parser `cp`.]]\n\n[[[** `manyCharsTill cp endp`] [br][small (variants: `[^manyCharsTill-parsers manyCharsTill[Apply][2]]`)]]\n [Parses chars with the char parser `cp` until the parser `endp` succeeds. Stops after `endp`.]]\n\n\n[[[** `manyStrings sp`] [br][small (variant: `[^manyStrings-parsers many[1]Strings[2]]`)]]\n [Parses a sequence of *zero* or more strings with the parser `sp`. Returns the parsed strings in concatenated form.]]\n\n[[[** `stringsSepBy sp sep`]]\n [Parses a sequence of *zero* or more occurrences of `sp` separated by `sep`. Returns the strings parsed with `sp` *and* `sep` in concatenated form.]]\n\n[[[** `skipped p`]]\n [Applies the parser `p`. Returns the chars skipped over by `p` as a string.]]\n\n[[[** `p |> withSkippedString f`]]\n [Applies the parser `p`. Returns `f str x`, where `str` is the string skipped over by `p` and `x` is the result returned by `p`.]]\n\n]\n\n\n[table{format=\"api-table\"} Parsing numbers\n\n[[Parser] [Description]]\n\n[[[** `pfloat`]]\n [Parses a double-precision floating-point number.]]\n\n[[[** `pint64`][br][small (variants: `[^pint-parsers pint(8|16|32)]`)]]\n [Parses a 64-bit signed integer.]]\n\n[[[** `puint64`][br][small (variants: `[^puint-parsers puint(8|16|32)]`)]]\n [Parses a 64-bit unsigned integer.]]\n\n[[[** `numberLiteral options label`]]\n [Parses a number literal and returns the result in form of a `NumberLiteral` value.]]\n]\n\n\n[table{format=\"api-table\"} Parsing whitespace\n\n[[Parser] [Description]]\n\n[[[** `newline`] [br][small (variants: `@skipNewline@`, `newlineReturn`, `unicodeNewline`)]]\n [Parses a newline (`\"\\n\"`, `\"\\r\\n\"` or `\"\\r\"`). Returns `'\\n'`.]]\n\n[[[** `unicodeNewline`] [br][small (variants: `@skipUnicodeNewline@`, `unicodeNewlineReturn`)]]\n [Parses a Unicode newline (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"` or `\"\\u2029\"`). Returns `'\\n'`.]]\n\n[[[** `spaces`] [br][small (variant: `spaces1`)]]\n [Skips over any sequence of whitespace chars (`' '`, `'\\t'` or a newline).]]\n\n[[[** `unicodeSpaces`] [br][small (variant: `unicodeSpaces1`)]]\n [Skips over any sequence of Unicode whitespace chars and recognizes (`\"\\n\"`, `\"\\r\\n\"`, `\"\\r\"`, `\"\\u0085\"`, `\"\\u2028\"` and `\"\\u2029\"`) as newlines.]]\n\n[[[** `eof`]]\n [Only succeeds at the end of the input.]]\n\n]\n\n\n[table{format=\"api-table\"} Chaining and piping parsers\n\n[[Parser] [Description]]\n\n[[[** `preturn x`]]\n [Returns `x`.]]\n\n[[[** `p >>% x`]]\n [Applies the parser `p`. Returns `x`.]]\n\n[[[** `p |>> f`]]\n [Applies the parser `p`. Returns `f x`, where `x` is the result returned by `p`.]]\n\n[[[** `p1 >>. p2`]]\n [Applies the parsers `p1` and `p2` in sequence. Returns the result of `p2`.]]\n\n[[[** `p1 .>> p2`]]\n [Applies the parsers `p1` and `p2` in sequence. Returns the result of `p1`.]]\n\n[[[** `p1 .>>. p2`]]\n [Applies the parsers `p1` and `p2` in sequence. Returns the results in a tuple.]]\n\n[[[** `between pBegin pEnd p`]]\n [Applies the parsers `pBegin`, `p` and `pEnd` in sequence. Returns the result of `p`.]]\n\n[[[** `pipe2 p1 p2 f`] [br][small (variants: `[^pipe-parsers pipe(3|4|5)]`]]\n [Applies the parsers `p1` and `p2` in sequence. Returns `f x1 x2`, where `x1` and `x2` are the results returned by `p1` and `p2`.]]\n\n[[[** `p >>= f`]]\n [First applies the parser `p`, then applies the function `f` to the result returned by `p` and finally applies the parser returned by `f`.]]\n\n]\n\n[table{format=\"api-table\"} Parsing sequences\n\n[[Parser] [PEG] [Description]]\n\n[[[** `tuple2 p1 p2`] [br][small (variants: `[^tuple-parsers tuple(3|4|5)]`)]]\n [`{PEG}p1 p2`]\n [Applies the parsers `p1` and `p2` in sequence. Returns the results in a tuple.]]\n\n[[[** `parray n p`] [br][small (variants: `skipArray`)]]\n []\n [Parses `n` occurrences of `p`. Returns the results in an array.]]\n\n[[[** `many p`] [br][small (variant: `skipMany`)]]\n [`{PEG}p*`]\n [Parses *zero* or more occurrences of `p`. Returns the results in a list.]]\n\n[[[** `many1 p`] [br][small (variant: `skipMany1`)]]\n [`{PEG}p+`]\n [Parses *one* or more occurrences of `p`. Returns the results in a list.]]\n\n[[[** `sepBy p sep`] [br][small (variants: `sepBy1`, `[^sepBy-parsers skipSepBy[1]]`)]]\n [`{PEG}(p (sep p)*)?`]\n [Parses *zero* or more occurrences of `p`, separated by `sep`. Returns the results in a list.]]\n\n\n[[[** `sepEndBy p sep`] [br][small (variants: `sepEndBy1`, `[^sepEndBy-parsers skipSepEndBy[1]]`)]]\n [`{PEG}(p (sep p)* sep?)?`]\n [Parses *zero* or more occurrences of `p`, separated and optionally ended by `sep`. Returns the results in a list.]]\n\n[[[** `manyTill p endp`] [br][small (variants: `many1Till`, `[^manyTill-parsers skipMany[1]Till]`)]]\n [`{PEG}(!endp p)* endp`]\n [Parses *zero* or more occurrences of `p` for as long as `endp` does not succeed. Stops after `endp` succeeded. Returns the results returned by `p` in a list.]]\n\n[[[** `chainl1 p op`] [br][small (variants: `[^chain-parsers chain(l|r)[1]]`)]]\n [`{EBNF}p (op p)*`]\n [Parses *one* or more occurrences of `p`, separated by `sep`. Returns [small `f_n (... (f_2 (f_1 x_1 x_2) x_3) ...) x_n+1`], where `f_1` to `f_n` are the functions returned by the parser `op` and `x_1` to `x_n+1` are the values returned by `p`.]]\n]\n\n[table{format=\"api-table\"} Parsing alternatives and recovering from errors\n\n[[Parser] [Description]]\n\n[[[** `p1 <|> p2`]]\n [Parses `p1` or `p2`. The parser `p2` is only tried if `p1` fails with a non-fatal error and *without changing the parser state*. The stream position is part of the parser state, so *if `p1` fails after consuming input, `p2` will not be tried*.]]\n\n[[[** `choice ps`] [br][small (variant: `choiceL`)]]\n [Is equivalent to `p1 <|> p2 <|> ... <|> pn <|> pzero`, where `p1` ... `pn` are the parsers in the sequence `ps`.]]\n\n[[[** `p <|>% x`]]\n [Parses `p` or returns `x`. Is equivalent to `p1 <|> preturn x`.]]\n\n[[[** `opt p`] [br][small (variant: `optional`)]]\n [Parses an optional occurrence of `p` as an option value. Is equivalent to `(p |>> Some) <|>% None`]]\n\n[[[** `attempt p`]]\n [\nParses `p`. If `p` fails after changing the parser state, `attempt p` will *backtrack* to the original parser state before reporting a (non-fatal) error. Thus, `attempt p1 <|> p2` will continue to try to parse `p2` even if `p1` fails after consuming input.]]\n\n[[[** `p1 >>? p2`] [br][small (variants: `.>>?`, `.>>.?`, `>>=?`)]]\n [Behaves like `p1 >>. p2`, but will *backtrack* to the beginning if `p2` fails with a non-fatal error and with an unchanged parser state, even if `p1` has changed the parser state.]]\n\n]\n\n[table{format=\"api-table\"} Conditional parsing and looking ahead\n\n[[Parser] [Description]]\n\n[[[** `notEmpty p`]]\n [Behaves like `p`, but fails when `p` succeeds without consuming input or changing the parser state in any other way.]]\n\n[[[** `followedBy p`] [br][small (variant: `notFollowedBy`)]]\n [Succeeds without changing the parser state if the parser `p` succeeds at the current position.]]\n\n[[[** `followedByL p label`] [br][small (variant: `notFollowedByL`)]]\n [Behaves like `followedBy p`, but uses the string `label` to generate a more descriptive error message in case `p` fails. The string `label` should describe `p`.]]\n\n[[[** `notFollowedByEof`]]\n [Is an optimized version of `notFollowedByL eof \"end of input\"`.]]\n\n\n[[[** `followedByString str`] [br][small (variants: `[^followedByString-parsers (notF|f)ollowedByString[CI]]`)]]\n [Is an optimized version of `followedByL (pstring str) (\"'\" + str + \"'\")`.]]\n\n[[[** `nextCharSatisfies f`] [br][small (variants: `next2CharsSatisfy`, `previousCharSatisfies`)]]\n [Is an optimized version of `followedBy (satisfy f)`.]]\n\n[[[** `nextCharSatisfiesNot f`] [br][small (variants: `next2CharsSatisfyNot`, `previousCharSatisfiesNot`)]]\n [Is an optimized version of `notFollowedBy (satisfy f)`.]]\n\n[[[** `lookAhead p`]]\n [Parses `p` and restores the original parser state afterwards.]]\n\n]\n\n[table#customizing-error-messages-table{format=\"api-table\"} Customizing error messages\n\n[[Parser] [Description]]\n\n[[[** `p label`]]\n [Applies the parser `p`. If `p` does not change the parser state (usually because `p` failed), the error messages are replaced with `expectedError label`. The string `label` should describe `p`.]]\n\n[[[** `p label`]]\n [Behaves like `p label`, but when `p` fails after changing the parser state, a `CompoundError` message is generated with both the given `label` and the error messages generated by `p`.]]\n\n[[[** `fail msg`]]\n [Always fails with a `messageError msg`. The string `msg` will be displayed together with other error messages generated for the same input position.]]\n\n\n[[[** `failFatally msg`]]\n [Always fails with a `messageError msg`. Returns with a `FatalError`, so that no error recovery is attempted (except via backtracking constructs).]]\n\n]\n\n[table{format=\"api-table\"} User state handling and getting the input stream position\n\n[[Parser] [Description]]\n\n[[[** `getUserState`]]\n [Returns the current user state.]]\n\n[[[** `setUserState u`]]\n [Sets the user state to `u`.]]\n\n[[[** `updateUserState f`]]\n [Sets the user state to `f u`, where `u` is the current user state.]]\n\n[[[** `userStateSatisfies f`]]\n [Succeeds if the current user state satisfies the predicate function `f`.]]\n\n[[[** `getPosition`]]\n [Returns the current position in the input stream.]]\n\n]\n\n[/section]\n" }, { "path": "Doc/src/reference-position.txt", "content": "\n\n[auto-link{hide-outer-auto-links = [\"Position\", \"Index\", \"Line\", \"Column\"]}]\n[interface-reference]\n[section#Position FParsec.Position]\n[$$interface]\n\n[interface-members]\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype Position =\n new: streamName: string * index: int64 * line: int64 * column: int64 -> Position\n\n member [#Position..StreamName StreamName]: string\n member [#Position..Index Index]: int64\n member [#Position..Line Line]: int64\n member [#Position..Column Column]: int64\n\n override ToString: unit -> string\n\n interface System.IEquatable\n interface [url \"https://msdn.microsoft.com/en-us/library/4d7sx9hd.aspx\" System.IComparable]\n interface System.IComparable\n``]\n[/interface-members]\n[/section]\n[/interface-reference]\n[/auto-link]" }, { "path": "Doc/src/reference-primitives.txt", "content": "[section#Primitives FParsec.Primitives]\n\n[interface-reference]\n[section Interface]\n\n[$$interface]\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsec.dll\n\n[] // module is automatically opened when FParsec namespace is opened\nmodule [no-auto-link FParsec.Primitives]\n\n``]\n[``[] val @Ok@: ReplyStatus = ReplyStatus.Ok\n``]\n[This `ReplyStatus` value indicates that a parser succeeded.]\n[``[] val @Error@: ReplyStatus = ReplyStatus.Error\n``]\n[This `ReplyStatus` value indicates that a parser failed.]\n\n[``[] val @FatalError@: ReplyStatus = ReplyStatus.FatalError\n``]\n[This `ReplyStatus` value indicates that a parser failed and\nno error recovery (except after backtracking) should be tried.]\n\n[``\n\n\ntype @Parser@<'TResult, 'TUserState> = CharStream<'TUserState> -> Reply<'TResult>\n``]\n[The type of the parser functions supported throughout the FParsec library.]\n\n\n[``\n\n\n// Two basic primitives that are only seldomly directly used in user code:\n``]\n\n[``\nval @preturn@: 'a -> Parser<'a,'u>\n``]\n[The parser `preturn x` always succeeds with the result `x` (without changing the parser state).\n\n`preturn x` is defined as `fun stream -> Reply(x)`.\n]\n\n[``val @pzero@: Parser<'a,'u>\n``]\n[The parser `pzero` always fails with an empty error message list, i.e. an unspecified error.\n\n`pzero x` is defined as `fun stream -> Reply(Error, NoErrorMessage)`.\n]\n\n\n[``\n\n\n// Chaining and piping parsers\n// ==============================\n``]\n\n[``val (@\\ >>=\\ @): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n``]\n[The parser `p >>= f` first applies the parser `p` to the input, then applies the function `f` to the result returned by `p` and finally applies the parser returned by `f` to the input.\n\n[note\nPlease see the user's guide chapter @Applying parsers in sequence@ for an in-depth discussion of\nthe behaviour of this and other sequencing combinators.\n]\n\nThe `>>=` combinator is the conceptual foundation for all combinators that consecutively apply multiple parsers to the input. In order to precisely define its behaviour we give an equivalent definition:\n\n``\nlet (>>=) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) =\n fun stream ->\n let reply1 = p stream\n if reply1.Status = Ok then\n let p2 = f reply1.Result\n let stateTag = stream.StateTag\n let mutable reply2 = p2 stream\n if stateTag = stream.StateTag then\n reply2.[^RError Error] <- mergeErrors reply1.[^RError Error] reply2.[^RError Error]\n reply2\n else\n Reply(reply1.Status, reply1.[^RError Error])\n``\n]\n\n[``\n\nval (@\\ >>%\\ @): Parser<'a,'u> -> 'b -> Parser<'b,'u>\n``]\n[\nThe parser `p >>% x` applies the parser `p` and returns the result `x`.\n\n `p >>% x` is an optimized implementation of `p >>= fun _ -> preturn x`.\n]\n\n[``val (@\\ >>.\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n``]\n[\nThe parser `p1 >>. p2` applies the parsers `p1` and `p2` in sequence and returns the result of `p2`.\n\n`p1 >>. p2` is an optimized implementation of `p1 >>= fun _ -> p2`.\n]\n\n[``val (@\\ .>>\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n``]\n[\nThe parser `p1 .>> p2` applies the parsers `p1` and `p2` in sequence and returns the result of `p1`.\n\n`p1 .>> p2` is an optimized implementation of `p1 >>= fun x -> p2 >>% x`.\n]\n\n[``val (@\\ .>>.\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n``]\n[\nThe parser `p1 .>>. p2` applies the parsers `p1` and `p2` in sequence and returns the results in a tuple.\n\n`p1 .>>. p2` is an optimized implementation of\n``\np1 >>= fun a ->\np2 >>= fun b -> preturn (a, b)\n``\n]\n\n[``val @between@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'c,'u>\n``]\n[\nThe parser `between popen pclose p` applies the parsers `pOpen`, `p` and `pEnd` in sequence. It returns the result of `p`.\n\n`between popen pclose p` is an optimized implementation of `popen >>. p .>> pclose`.\n]\n\n[``\n\nval (@\\ |>>\\ @): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b,'u>\n``]\n[\nThe parser `p |>> f` applies the parser `p` and returns the result of the function application `f x`, where `x` is the result returned by `p`.\n\n`p |>> f` is an optimized implementation of `p >>= fun x -> preturn (f x)`.\n]\n\n[``\n[#pipe-parsers]\nval @pipe2@: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>\n``]\n[\nThe parser `pipe2 p1 p2 f` applies the parsers `p1` and `p2` in sequence.\nIt returns the result of the function application `f a b`, where `a` and `b` are the results returned by `p1` and `p2`.\n\n`pipe2 p1 p2 f` is an optimized implementation of\n``\np1 >>= fun a ->\np2 >>= fun b -> preturn (f a b)\n``\n]\n\n[``\n\nval @pipe3@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n -> ('a -> 'b -> 'c -> 'd) -> Parser<'d,'u>\n``]\n[\nThe parser `pipe3 p1 p2 p3 f` applies the parsers `p1`, `p2` and `p3` in sequence.\nIt returns the result of the function application `f a b c`, where `a`, `b` and `c` are the results returned by `p1`, `p2` and `p3`.\n\n`pipe3 p1 p2 p3 f` is an optimized implementation of\n``\np1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c -> preturn (f a b c)\n``\n]\n\n[``\n\nval @pipe4@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n -> ('a -> 'b -> 'c -> 'd -> 'e) -> Parser<'e,'u>\n``]\n[\nThe parser `pipe4 p1 p2 p3 p4 f` applies the parsers `p1`, `p2`, `p3` and `p4` in sequence.\nIt returns the result of the function application `f a b c d`, where `a`, `b`, `c` and `d` are the results returned by `p1`, `p2`, `p3` and `p4`.\n\n`pipe4 p1 p2 p3 p4 f` is an optimized implementation of\n``\np1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c ->\np4 >>= fun d -> preturn (f a b c d)\n``\n]\n\n[``\n\nval @pipe5@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n -> Parser<'e,'u>\n -> ('a -> 'b -> 'c -> 'd -> 'e -> 'f) -> Parser<'f,'u>\n``]\n[\nThe parser `pipe5 p1 p2 p3 p4 p5 f` applies the parsers `p1`, `p2`, `p3`, `p4` and `p5` in sequence.\nIt returns the result of the function application `f a b c d e`, where `a`, `b`, `c`, `d` and `e` are the results returned by `p1`, `p2`, `p3`, `p4` and `p5`.\n\n`pipe5 p1 p2 p3 p4 p5 f` is an optimized implementation of\n``\np1 >>= fun a ->\np2 >>= fun b ->\np3 >>= fun c ->\np4 >>= fun d ->\np5 >>= fun e -> preturn (f a b c d e)\n``\n]\n\n[``\n\n\n// Parsing alternatives and recovering from errors\n// ===============================================\n``]\n\n\n[``val (@\\ <|>\\ @): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>\n``]\n[The parser `p1 <|> p2` first applies the parser `p1`. If `p1` succeeds, the result of `p1` is returned. If `p1` fails with a non-fatal error and *without changing the parser state*, the parser `p2` is applied.\nNote: The stream position is part of the parser state, so if `p1` fails after consuming input, `p2` will not be applied.\n\nThe `choice` combinator is a generalization of `<|>` to more than two parsers.\n\n[note\nPlease see the user's guide chapter on @Parsing alternatives@ for an in-depth discussion of the behaviour of this combinator.\n]\n]\n\n[``val @choice@: seq> -> Parser<'a,'u>\n``]\n[The parser `choice ps` is an optimized implementation of `p1 <|> p2 <|> ... <|> pn `, where `p1` ... `pn` are the parsers in the sequence `ps`.\n\n`choice [p]` is equivalent to `p`.[br]\n`choice []` is equivalent to `pzero`.\n]\n\n[``val @choiceL@: seq> -> string -> Parser<'a,'u>\n``]\n[\nThe parser `choiceL ps label` is an optimized implementation of `choice ps label`.\n\n`choiceL` is slightly faster than `choice`, because it doesn't have to aggregate error messages.\n]\n\n\n[``\n\nval (@\\ <|>%\\ @): Parser<'a,'u> -> 'a -> Parser<'a,'u>\n``]\n[\nThe parser `p <|>% x` is an optimized implementation of `p <|> preturn x`.\n]\n\n[``val @opt@: Parser<'a,'u> -> Parser<'a option,'u>\n``]\n[\nThe parser `opt p` parses an optional occurrence of `p` as an option value.\n\n`opt p` is an optimized implementation of `(p |>> Some) <|>% None`.\n]\n\n[``val @optional@: Parser<'a,'u> -> Parser\n``]\n[\nThe parser `optional p` skips over an optional occurrence of `p`.\n\n`optional p` is an optimized implementation of `(p >>% ()) <|>% ()`.\n]\n\n[``\n\nval @attempt@: Parser<'a,'u> -> Parser<'a,'u>\n``]\n[\nThe parser `attempt p` applies the parser `p`. If `p` fails after changing the parser state or with a fatal error, `attempt p` will backtrack to the original parser state and report a non-fatal error.\n]\n\n[``\n\nval (@\\ >>=?\\ @): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n``]\n[\nThe parser `p >>=? f` behaves like `p >>= f`, except that it will backtrack to the beginning if the parser returned by `f` fails with a non-fatal error and without changing the parser state, even if `p1` has changed the parser state.\n]\n\n[``val (@\\ >>?\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n``]\n[\nThe parser `p1 >>? p2` behaves like `p1 >>. p2`, except that it will backtrack to the beginning if `p2` fails with a non-fatal error and without changing the parser state, even if `p1` has changed the parser state.\n]\n\n[``val (@\\ .>>?\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n``]\n[\nThe parser `p1 .>>? p2` behaves like `p1 .>> p2`, except that it will backtrack to the beginning if `p2` fails with a non-fatal error and without changing the parser state, even if `p1` has changed the parser state.\n]\n\n[``val (@\\ .>>.?\\ @): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n``]\n[\nThe parser `p1 .>>.? p2` behaves like `p1 .>>. p2`, except that it will backtrack to the beginning if `p2` fails with a non-fatal error and without changing the parser state, even if `p1` has changed the parser state.\n]\n\n\n[``\n\n\n// Conditional parsing and looking ahead\n// =====================================\n``]\n\n[``val @notEmpty@: Parser<'a,'u> -> Parser<'a,'u>\n``]\n[The parser `notEmpty p` behaves like `p`, except that it fails when `p` succeeds without consuming input\nor changing the parser state in any other way.\n\n`notEmpty` is useful for forcing sequence parsers to consume input. For example, `notEmpty (manySatisfy f)` behaves like `many1Satisfy f`.\n]\n\n\n[``\n\nval @followedBy@: Parser<'a,'u> -> Parser\n``]\n[The parser `followedBy p` succeeds if the parser `p` succeeds at the current position. Otherwise it fails with a non-fatal error. This parser never changes the parser state.\n\nIf the parser `followedBy p` fails, it returns no descriptive error message. Hence it should only be used together with other parsers that take care of a potential error. Alternatively, `followedByL p label` can be used to ensure a more descriptive error message.]\n\n[``val @followedByL@: Parser<'a,'u> -> string -> Parser\n``]\n[The parser `followedByL p` behaves like `followedBy p`, except that it returns an `Expected label` error message when the parser `p` fails.]\n\n[``val @notFollowedBy@: Parser<'a,'u> -> Parser\n``]\n[The parser `notFollowedBy p` succeeds if the parser `p` fails to parse at the current position. Otherwise it fails with a non-fatal error. This parser never changes the parser state.\n\nIf the parser `notFollowedBy p` fails, it returns no descriptive error message. Hence it should only be used together with other parsers that take care of a potential error. Alternatively, `notFollowedByL p label` can be used to ensure a more descriptive error message.]\n\n[``val @notFollowedByL@: Parser<'a,'u> -> string -> Parser\n``]\n[The parser `notFollowedByL p` behaves like `notFollowedBy p`, except that it returns an `Unexpected label` error message when the parser `p` fails.]\n\n[``\n\nval @lookAhead@: Parser<'a,'u> -> Parser<'a,'u>\n``]\n[\n The parser `lookAhead p` parses `p` and restores the original parser state afterwards.\n If `p` fails after changing the parser state, the error messages are wrapped in a `NestedError`. If it succeeds,\n any error messages are discarded. Fatal errors are turned into normal errors.\n]\n\n[``\n\n\n// Customizing error messages\n// ==========================\n``]\n\n[``\nval (@\\ \\ @): Parser<'a,'u> -> string -> Parser<'a,'u>\n``]\n[\nThe parser `p label` applies the parser `p`. If `p` does not change the parser state\n(usually because `p` failed), the error messages are replaced with `expected label`.\n\nPlease also see the user's guide chapter on @customizing error messages@.\n]\n\n[``val (@\\ \\ @): Parser<'a,'u> -> string -> Parser<'a,'u>\n``]\n[The parser `p label` behaves like `p label`, except that when `p` fails after changing the parser state (for example, because `p` consumes input before it fails), a `CompoundError` message is generated with both the given string `label` and the error messages generated by `p`.\n\nPlease also see the user's guide chapter on @customizing error messages@.\n]\n\n[``\n\nval @fail@: string -> Parser<'a,'u>\n``]\n[\nThe parser `fail msg` always fails with a `messageError msg`. The string `msg` will be displayed together with other error messages generated for the same input position.\n\n`fail msg` is equivalent to `fun stream -> Reply(Error, messageError msg)`.\n]\n\n[``val @failFatally@: string -> Parser<'a,'u>\n``]\n[\nThe parser `failFatally msg` always fails with a `messageError msg`. It returns with a `FatalError`, so that no error recovery is attempted (except via backtracking constructs).\n\n`failFatally msg` is equivalent to `fun stream -> Reply(FatalError, messageError msg)`.\n]\n\n[``\n\n\n// Parsing sequences\n// =================\n``]\n\n[``\n[#tuple-parsers]val @tuple2@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n``]\n[\nThe parser `tuple2 p1 p2` applies the parsers `p1` and `p2` in sequence and returns the results in a tuple.\n\n`tuple2 p1 p2` is defined as `p1 .>>. p2` and is equivalent to `pipe2 p1 p2 (fun a b -> (a, b))`.\n]\n\n[``\nval @tuple3@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u>\n -> Parser<('a * 'b * 'c),'u>\n``]\n[\nThe parser `tuple3 p1 p2 p3` applies the parsers `p1`, `p2` and `p3` in sequence and returns the results in a tuple.\n\n`tuple3 p1 p2 p3` is equivalent to `pipe3 p1 p2 p3 (fun a b c -> (a, b, c))`.\n]\n\n[``\nval @tuple4@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n -> Parser<('a * 'b * 'c * 'd),'u>\n``]\n[\nThe parser `tuple4 p1 p2 p3 p4` applies the parsers `p1`, `p2`, `p3` and `p4` in sequence and returns the results in a tuple.\n\n`tuple4 p1 p2 p3 p4` is equivalent to `pipe4 p1 p2 p3 p4 (fun a b c d -> (a, b, c, d))`.\n]\n\n[``\nval @tuple5@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u>\n -> Parser<'e,'u> -> Parser<('a * 'b * 'c * 'd * 'e),'u>\n``]\n[\nThe parser `tuple5 p1 p2 p3 p4 p5` applies the parsers `p1`, `p2`, `p3`, `p4` and `p5` in sequence and returns the results in a tuple.\n\n`tuple5 p1 p2 p3 p4 p5` is equivalent to `pipe5 p1 p2 p3 p4 p5 (fun a b c d e -> (a, b, c, d, e))`.\n]\n\n[``\n\nval @parray@: int -> Parser<'a,'u> -> Parser<'a[],'u>\n``]\n[The parser `parray n p` parses `n` occurrences of `p` and returns the results in an array.\n\nFor example, `parray 3 p` is equivalent to `pipe3 p p p (fun a b c -> [|a;b;c|])`.]\n\n[``val @skipArray@: int -> Parser<'a,'u> -> Parser\n``]\n[The parser `skipArray n p` is an optimized implementation of `parray n p |>> ignore`.]\n\n[``\n[#many-parsers]\nval @many@: Parser<'a,'u> -> Parser<'a list,'u>\n``]\n[The parser `many p` repeatedly applies the parser `p` until `p` fails. It returns a list of the results returned by `p`.\nAt the end of the sequence `p` must fail without changing the parser state and without signalling a `FatalError`, otherwise `many p` will fail with the error reported by `p`.\n\n`many p` tries to guard against an infinite loop by raising an exception if `p` succeeds without changing the parser state.\n\nIgnoring efficiency issues, stack space usage and the infinite recursion case, `many` could be defined as follows:\n``\nlet rec many p =\n (p >>= fun hd ->\n many p\n |>> fun tl -> hd::tl)\n <|>% []\n``\n]\n\n[``\nval @many1@: Parser<'a,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `many1 p` behaves like `many p`, except that it requires `p` to succeed at least one time.\n\n`many1 p` is equivalent to `pipe2 p (many p) (fun hd tl -> hd::tl)`.\n]\n\n[``\nval @skipMany@: Parser<'a,'u> -> Parser\n``]\n[\nThe parser `skipMany p` is an optimized implementation of `many p |>> ignore`.\n]\n\n[``\nval @skipMany1@: Parser<'a,'u> -> Parser\n``]\n[\nThe parser `skipMany1 p` is an optimized implementation of `many1 p |>> ignore`.\n]\n\n[``\n[#sepBy-parsers]\nval @sepBy@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `sepBy p sep` parses *zero* or more occurrences of `p` separated by `sep`\n(in EBNF: `{EBNF}(p (sep p)*)?`). It returns a list of the results returned by `p`.\n\n`sepBy p sep` is almost equivalent to `pipe2 p (many (sep >>. p)) (fun hd tl -> hd::tl) <|>% []`,\nexcept with regard to a case rarely encountered in practice:\nIf `sep` succeeds without changing the parser state and `p` then fails without changing the state,\nthen `sepBy p sep` fails too, while the parser given by the almost equivalent definition would succeed.\n]\n\n[``\nval @sepBy1@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `sepBy1 p sep` parses *one* or more occurrences of `p` separated by `sep`\n(in EBNF: `{EBNF}p (sep p)*`).\n\nThe parser `sepBy1 p` behaves like `sepBy p`, except that it requires `p` to succeed at least one time. Hence, if `sepBy1` succeeds, the returned list always contains at least one value.\n]\n\n[``\nval @skipSepBy@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipSepBy p sep` is an optimized implementation of `sepBy p sep |>> ignore`.\n]\n\n\n[``\nval @skipSepBy1@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipSepBy1 p sep` is an optimized implementation of `sepBy1 p sep |>> ignore`.\n]\n\n[``\n[#sepEndBy-parsers]\nval @sepEndBy@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `sepEndBy p sep` parses *zero* or more occurrences of `p` separated and optionally ended by `sep` (in EBNF: `{EBNF}(p (sep p)* sep?)?`). It returns a list of the results returned by `p`.\n\n`sepEndBy p sep` tries to guard against an infinite loop by raising an exception if `p` and `sep` succeed without changing the parser state.\n\nIgnoring efficiency issues, stack space usage and the infinite recursion case, `sepEndBy` could be defined as follows:\n``\nlet rec sepEndBy p sep =\n (p >>= fun hd ->\n sep >>. sepEndBy p sep <|>% []\n |>> fun tl -> hd::tl)\n <|>% []\n``\n]\n\n[``\nval @sepEndBy1@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `sepEndBy1 p sep` parses *one* or more occurrences of `p` separated and optionally ended by `sep` (in EBNF: `{EBNF}p (sep p)* sep?`). It returns a list of the results returned by `p`.\n\nThe parser `sepEndBy1 p` behaves like `sepEndBy p`, except that it requires `p` to succeed at least one time. Hence, if `sepEndBy1` succeeds, the returned list always contains at least one value.\n]\n\n\n[``\nval @skipSepEndBy@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipSepEndBy p sep` is an optimized implementation of `sepEndBy p sep |>> ignore`.\n]\n\n[``\nval @skipSepEndBy1@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipSepEndBy1 p sep` is an optimized implementation of `sepEndBy1 p sep |>> ignore`.\n]\n\n[``\n[#manyTill-parsers]\nval @manyTill@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `manyTill p endp` repeatedly applies the parser `p` for as long as `endp` fails (without changing the parser state). It returns a list of the results returned by `p`.\n\n`manyTill p endp` is an optimized variant of `many (notFollowedBy endp >>. p) .>> endp` that doesn't have to apply `endp` twice at the end of the sequence and that fails with the error reported by `endp` if `endp` fails after changing the parser state.\n]\n\n[``\nval @many1Till@: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n``]\n[\nThe parser `many1Till p endp` behaves like `manyTill p endp`, except that it requires `p` to succeed at least one time.\n\n`many1Till p endp` is an optimized implementation of `pipe2 p (manyTill p endp) (fun hd tl -> hd::tl)`.\n]\n\n[``\nval @skipManyTill@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipManyTill p endp` is an optimized implementation of `manyTill p endp |>> ignore`.\n]\n\n[``\nval @skipMany1Till@: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n``]\n[\nThe parser `skipMany1Till p endp` is an optimized implementation of `many1Till p endp |>> ignore`.\n]\n\n[``\n\n[]\ntype @Inline@ =\n``]\n[\n\n`Inline` is a static class that contains the following inline helper methods for defining optimized sequence parsers:\n- `Inline.Many`\n- `Inline.SepBy`\n- `Inline.ManyTill`\n]\n\n[``\n static member inline @Many@: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'T -> 'State)\n * resultFromState: ('State -> 'Result)\n * elementParser: Parser<'T,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: (unit -> 'Result)\n -> Parser<'Result,'U>\n``]\n[#Inline..Many{no-auto-link = [\"Inline\", \"Inline.Many\"]}\n`Inline.Many` is an inline helper method for defining optimized sequence parsers.\n\n`Inline.Many(stateFromFirstElement, foldState, resultFromState, elementParser)` expands to an optimized implementation of\n``\nmany1 elementParser // requires at least 1 element\n|>> function hd::tl ->\n resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n``\n\n[note\nThe `'State` argument to the `foldState` function is completely independent of FParsec's usual parser state. The term \"accumulator\" would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.]\n\nIf you pass a value for the optional argument [** `resultForEmptySequence`], the parser expands to an optimized implementation of\n``\nmany elementParser // accepts empty sequence\n|>> function\n | [] -> resultForEmptySequence()\n | hd::tl -> resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n``\n\nIf you pass a value for the optional argument [** `firstElementParser`], the first element of the sequence will be parsed with `firstElementParser` instead of `elementParser`.\n\nThe following example shows how you can use `Inline.Many` to define an optimized parser that behaves like `many1 p |>> List.reduce f` but avoids the temporary allocation of a list:\n\n``\nlet many1Reduce p f =\n Inline.Many(elementParser = p,\n stateFromFirstElement = (fun x0 -> x0),\n foldState = (fun acc y -> f acc y),\n resultFromState = (fun acc -> acc))\n``\n\nA simple test run:\n``{fsi}\n> run (many1Reduce (pint32 .>> spaces) (+)) \"1 2 3\";;\nval it : ParserResult = Success: 6\n``\n\nThe following example shows how you can use `Inline.Many` to create an optimized sequence parser that returns an array instead of a list:\n\n``\nlet manyA2 p1 p =\n Inline.Many(firstElementParser = p1,\n elementParser = p,\n stateFromFirstElement = (fun x0 ->\n let ra = ResizeArray<_>()\n ra.Add(x0)\n ra),\n foldState = (fun ra x -> ra.Add(x); ra),\n resultFromState = (fun ra -> ra.ToArray()),\n resultForEmptySequence = (fun () -> [||]))\n\nlet manyA p = manyA2 p p\n``\n\nTwo simple test runs:\n``{fsi}\n> run (manyA (pint32 .>> spaces)) \"\";;\nval it : ParserResult = Success: [||]\n> run (manyA (pint32 .>> spaces)) \"1 2 3\";;\nval it : ParserResult = Success: [|1; 2; 3|]\n``\n\n]\n\n[``\n\n static member inline @SepBy@: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'Separator -> 'T -> 'State)\n * resultFromState: ('State -> 'Result)\n * elementParser: Parser<'T,'U>\n * separatorParser: Parser<'Separator,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: (unit -> 'Result)\n * ?separatorMayEndSequence: bool\n -> Parser<'Result,'U>\n``]\n[#Inline..SepBy{no-auto-link = [\"Inline\", \"Inline.SepBy\"]}\n`Inline.SepBy` is an inline helper method for defining optimized sequence parsers. By default, parsers defined with `Inline.SepBy` parse sequences of the form (in EBNF):\n`{EBNF}element (separator element)*`\n\n`Inline.SepBy(stateFromFirstElement, foldState, resultFromState, elementParser,` `separatorParser)` expands to an optimized implementation of\n``\npipe2 elementParser (many (separatorParser .>>. elementParser))\n (fun elem0 sepsAndElems ->\n sepsAndElems\n |> List.fold (fun acc (sep, e) -> foldState acc sep e)\n (stateFromFirstElement elem0)\n |> resultFromState)\n``\n\n[small For most practical purposes the behaviour of the expanded `Inline.SepBy` parser and the above definition based on `many` can be considered equivalent, but there is a fringe case where the behaviour differs:\nIf `separatorParser` succeeds without changing the parser state and `elementParser` then fails without changing the parser state, then the `Inline.SepBy` parser fails too, while the parser given by the definition based on `many` would succeed.]\n\n[note\nThe `'State` argument to the `foldState` function is completely independent of FParsec's usual parser state. The term \"accumulator\" would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.]\n\nIf you pass `true` as the value for the optional argument `separatorMayEndSequence`, a separator may also end the sequence, i.e. the parser will accept sequences of the following form (in EBNF):\n``{EBNF}element (separator element)* separator?``\nNote that `foldState` is not called with the value of an ending separator.\n\nIf you pass a value for the optional argument `resultForEmptySequence`, the parser returned by `Inline.SepBy`\nwill call `resultForEmptySequence` to create the parser result when it encounters an empty sequence. If you don't pass a `resultForEmptySequence` function, the parser will fail for an empty sequence.\n\nIf you pass a value for the optional argument `firstElementParser`, the first element of a sequence will be parsed with `firstElementParser` instead of `elementParser`.\n\nThe following example shows how you can use `Inline.SepBy` to define an optimized parser that behaves like `sepBy1 p sep |>> List.reduce f` but avoids the temporary allocation of a list:\n\n``\nlet sepBy1Reduce p sep f =\n Inline.SepBy(elementParser = p,\n separatorParser = sep,\n stateFromFirstElement = (fun x0 -> x0),\n foldState = (fun acc _ y -> f acc y),\n resultFromState = (fun acc -> acc))\n``\n\nA simple test run:\n``{fsi}\n> run (sepBy1Reduce pint32 (pstring \",\" >>. spaces) (+)) \"1, 2, 3\";;\nval it : ParserResult = Success: 6\n``\n\nThe following example shows how one could define `CharParsers.stringsSepBy` using `Inline.SepBy`:\n\n``\nlet stringsSepBy p sep =\n Inline.SepBy(elementParser = p,\n separatorParser = sep,\n stateFromFirstElement =\n (fun str -> let sb = System.Text.StringBuilder()\n sb.Append(str : string)),\n // sb.Append returns sb\n foldState =\n (fun sb sep str -> sb.Append(sep : string)\n .Append(str : string)),\n resultFromState = (fun sb -> sb.ToString()))\n\nlet testParser : Parser =\n stringsSepBy (manySatisfy isLetter) (pstring @\"\\\\\" >>% @\"\\\")\n``\n\n``{fsi}\n> run testParser \"\";;\nval it : ParserResult = Success: \"\"\n> run testParser @\"abc\\\\def\\\\\\\\\";;\nval it : ParserResult = Success: \"abc\\def\\\\\"\n``\n]\n\n[``\n\n static member inline @ManyTill@: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'T -> 'State)\n * resultFromStateAndEnd: ('State -> 'E -> 'Result)\n * elementParser: Parser<'T,'U>\n * endParser: Parser<'E,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: ('E -> 'Result)\n -> Parser<'Result,'U>\n``]\n[#Inline..ManyTill{no-auto-link = [\"Inline\", \"Inline.ManyTill\"]}\n`Inline.ManyTill` is an inline helper method for defining optimized sequence parsers.\n\n`Inline.ManyTill(stateFromFirstElement, foldState, resultFromState, elementParser,` `endParser)` expands to an optimized implementation of\n``\nmany1Till elementParser endParser // requires at least 1 element\n|>> function hd::tl ->\n resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n``\n\n[note\nThe `'State` argument to the `foldState` function is completely independent of FParsec's usual parser state. The term \"accumulator\" would be a more accurate name for the argument, but that is just too unwieldy to use in the method signature.]\n\nIf you pass a value for the optional argument [** `resultForEmptySequence`], the parser expands to an optimized implementation of\n``\nmanyTill elementParser endParser // accepts empty sequence\n|>> function\n | [] -> resultForEmptySequence()\n | hd::tl -> resultFromState (List.fold foldState (stateFromFirstElement hd) tl)\n``\n\nIf you pass a value for the optional argument [** `firstElementParser`], the first element of the sequence will be parsed with `firstElementParser` instead of `elementParser`.\n\nThe following example shows how one could define `CharParsers.manyCharsTill2` using `Inline.ManyTill`:\n\n``\nlet myManyCharsTillApply2 cp1 cp endp f =\n Inline.ManyTill(firstElementParser = cp1,\n elementParser = cp,\n endParser = endp,\n stateFromFirstElement =\n (fun c -> let sb = System.Text.StringBuilder()\n sb.Append(c : char)),\n // sb.Append returns sb\n foldState = (fun sb c -> sb.Append(c : char)),\n resultFromStateAndEnd = (fun sb e -> f (sb.ToString()) e),\n resultForEmptySequence = (fun e -> f \"\" e))\n\nlet myManyCharsTillApply cp endp f = myManyCharsTillApply2 cp cp endp f\n\nlet myRestOfLine : Parser =\n myManyCharsTillApply anyChar ((newline >>% \"\\\\n\") <|> (eof >>% \"\"))\n (fun str nl -> str + nl)\n``\n\n``{fsi}\n> run myRestOfLine \"\";;\nval it : ParserResult = Success: \"\"\n> run myRestOfLine \"abc\\r\\ndef\";;\nval it : ParserResult = Success: \"abc\\n\"\n``\n\n]\n\n[``\n[#chain-parsers]\nval @chainl1@: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n``]\n[The parser `chainl1 p op` parses one or more occurrences of `p` separated by `op`\n(in EBNF: `p (op p)*`). It returns the value obtained by *left* associative application of all functions returned by `op` to the results returned by `p`, i.e. `f_n (...(f_2 (f_1 x_1 x_2) x_3) ...) x_n+1`, where `f_1` to `f_n` are the functions returned by theparser `op` and `x_1` to `x_n+1` are the values returned by `p`. If only a single occurance of `p` and no occurance of `op` is parsed, the result of `p` is returned directly.\n\nThe `chainl1` implementation uses constant stack space.\n]\n\n[``\nval @chainl@: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n``]\n[The parser `chainl p op defVal` is equivalent to `chainl1 p op <|>% defVal`.]\n\n\n[``\nval @chainr1@: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n``]\n[The parser `chainr1 p op` parses one or more occurrences of `p` separated by `op`\n(in EBNF: `p (op p)*`).\n It returns the value obtained by *right* associative application of all functions returned by `op` to the results returned by `p`, i.e. `f1 x_1 (f_2 x_2 (... (f_n x_n x_n+1) ...))`, where `f_1` to `f_n` are the functions returned by the parser `op` and `x_1` to `x_n+1` are the values returned by `p`. If only a single occurance of `p` and no occurance of `op` is parsed, the result of `p` is returned directly.\n\nThe `chainr1` implementation uses constant stack space.\n]\n\n\n[``\nval @chainr@: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n``]\n[The parser `chainr p op defVal` is equivalent to `chainr1 p op <|>% defVal`.]\n\n\n\n\n[``\n\n\n// Building parsers using F#'s computation expression syntax\ntype @ParserCombinator@ = // ...\n``]\n[\nThis class is defined as\n\n``\n[]\ntype ParserCombinator() =\n member t.Delay(f) = fun state -> (f ()) state\n member t.Return(x) = preturn x\n member t.Bind(p, f) = p >>= f\n member t.Zero() = pzero\n member t.ReturnFrom(p) = p\n member t.TryWith(p, cf) = fun state -> try p state\n with e -> (cf e) state\n member t.TryFinally(p, ff) = fun state -> try p state\n finally ff ()\n``\n\nInstances of this class can be used to build parsers using F#'s @computation expression@ syntax. The default instance for this purpose is `parse`.\n\nPlease see the user's guide chapter \"@Where is the monad?@\" for an introduction to the `parse {...}` syntax.\n\n\nSome constructs supported by `parse` and their translations are\n``\nlet! pat = expr in pexpr ==> expr >>= (fun pat -> pexpr)\n\nlet pat = expr in pexpr ==> let pat = expr in pexpr\n\ndo! expr in pexpr ==> expr >>= (fun () -> pexpr)\n\ndo expr in pexpr ==> expr; pexpr\n\nif expr then pexpr1 ==> if expr then pexpr1\nelse pexpr2 else pexpr2\n\nif expr then pexpr ==> if expr then pexpr1 else pzero\n\nreturn exp ==> preturn rexpr\n\nreturn! expr ==> expr\n``\nwhere `expr` is any F# expression and `pexpr` is an expression of type `Parser<_,_>`. You need to use the `!`-constructs whenever you have a right hand side expression that evaluates to a parser.\n]\n\n[``\nval @parse@: ParserCombinator\n``]\n[\nA builder object of type `ParserCombinator` for building parsers using F#'s @computation expression@ syntax.\n]\n\n[``\n\n// Building mutually recursive parser values\n``]\n\n[``\nval @createParserForwardedToRef@: unit -> Parser<'a,'u> * Parser<'a,'u> ref\n``]\n[\n`let p, pRef = createParserForwardedToRef()` creates a parser `p` that forwards all calls to the parser in the reference cell `pRef`. Initially, `pRef` holds a reference to a dummy parser that raises an exception on any invocation.\n\nThe JSON parser example in the tutorial [^createParserForwardedToRef-example shows] how you can use `createParserForwardedToRef` to define a parser for a recursive grammar.\n]\n\n[/interface-members]\n[/section]\n[/interface-reference]\n[/section]\n\n\n" }, { "path": "Doc/src/reference-reply.txt", "content": "\n\n[section#Reply FParsec.Reply]\n\nRepresents the return value of a `Parser` function.\n\n[interface-reference]\n\n[section Interface]\n[$$interface]\n[/section]\n\n[section Remarks]\nThe `Reply` type is the return type of `Parser` functions. Similar to a tuple, a `Reply` value can be viewed as a simple aggregate of its three fields `Status`, `Result` and `@Error@`.\n\nThe value of the `Status` field indicates whether the parser returning the reply succeeded (`ReplyStatus.Ok`) or failed (`ReplyStatus.Error` or `ReplyStatus.FatalError`). If the value of the `Status` field is `Ok`, the `Result` field contains a parser result value; otherwise, its value is undefined.\n\nThe `Equals` override ignores the `Result` value when it compares two `Reply` values with `Status <> Ok`.\n\n[note\n\nThe `Reply` fields are mutable because that allows us to implement library primitives with more compact code, for which the .NET JIT produces faster machine code.\n\nOf course, if you object to mutable structs [url \"https://stackoverflow.com/questions/441309/why-are-mutable-structs-evil\" on religious grounds] or if you're not familiar with the somewhat subtle behaviour of mutable structs in certain sitations, you can always treat the `Reply` type as if it was immutable.\n]\n\n[/section]\n\n[section Members]\n[interface-members]\n\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype #ReplyStatus# = [#ReplyStatus..Ok Ok] = 1\n | [#ReplyStatus..Error Error] = 0\n | [#ReplyStatus..FatalError FatalError] = -1\n\n\ntype Reply<'TResult> = struct\n``]\n\n[``\n @new@: 'TResult -> Reply<'TResult>\n``]\n[#new-1\n\nConstructs a `Reply` instance with the `Status` field set to `Ok`, the `Result` field set to the argument value and the `@Error@` field set to `null`.\n]\n\n[``\n @new@: ReplyStatus * ErrorMessageList -> Reply<'TResult>\n``]\n[#new-2\nConstructs a `Reply` instance with the `Status` and `@Error@` fields set to the respective argument values and the `Result` field initialized to `Unchecked.defaultof<'TResult>`.\n\nThis constructor is usually used for constructing an error reply, like in `Reply(Error, expected \"something\")`.\n]\n\n[``\n @new@: ReplyStatus * 'TResult * ErrorMessageList -> Reply<'TResult>\n``]\n[#new-3\nConstructs a `Reply` instance with the `Status`, `Result` and `@Error@` fields set to the respective argument values.\n]\n\n[``\n\n val mutable @Status@: ReplyStatus\n``]\n[\nThe `Status` field contains a `ReplyStatus` enum value indicating whether a parser succeeded (`Ok`) or failed (`Error` or `FatalError`). By returning a `FatalError` instead of an `Error` a parser can signal that no error recovery should be tried (except through backtracking mechanisms).\n]\n\n[``\n /// If Status <> Ok then the Result value is undefined and may be null.\n``]\n\n[``\n val mutable @Result@: 'TResult\n``]\n[\nIf the value of the `Status` field is `Ok`, the `Result` field contains a parser result value; otherwise, its value is undefined and may be equal to `Unchecked.defaultof<'TResult>`. (The result value in a `Reply` returned by an unsuccessful parser is generally an implementation detail of the parser that you should not depend on.)\n]\n\n[``\n val mutable @Error@: ErrorMessageList\n``]\n[\nThe `Error` field holds a list of error messages in the form of an `ErrorMessageList` value. An empty `ErrorMessageList` is represented as a `null` value.\n\nThe error messages returned by a parser in a `Reply` value implicitly refer to the state of the `CharStream` as it is when the parser returns. Since the `ErrorMessage` values stored in the `ErrorMessageList` do not themselves contain an error position, they can only be interpreted together with the position of the `CharStream` as it is when the parser returns.\n]\n\n[``\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n interface System.IEquatable>\nend\n``]\n[/interface-members]\n\n[/section]\n[/interface-reference]\n\n[/section]\n" }, { "path": "Doc/src/reference-staticmapping.txt", "content": "\n[section#StaticMapping FParsec.StaticMapping]\n\nThis module defines functions for creating optimized mapping functions between keys and values.\n\nThis module is not available in the @Low-Trust version@ of FParsec.\n\n[interface-reference]\n[section Interface]\n[$$interface]\n[/section]\n\n[section Remarks]\nThe functions in the `StaticMapping` module use runtime code generation via `System.Reflection.Emit` to create optimized mapping functions between keys and values.\n\n[note\nRuntime code generation is relatively expensive, so the functions in this module should only be used for optimizing static mappings that are potentially called a (very) large number of times.]\n\n[note\nThe code generated for the optimized mapping functions will occupy memory until the associated AppDomain is unloaded.\n]\n\nThe performance of the generated functions depends a lot on the individual key-value mapping and the application-specific call pattern. Ignoring the overhead of the function call, the generated mapping functions should generally be as fast as an equivalent statically compiled switch-statement in C# or F#. In certain cases they will even be faster.\n\nThe code size of the generated functions increases about linearly with the number of key ranges (i.e. continuous sequences of keys with the same value). Hence, you should only use the `StaticMapping` module for small mappings. If you try to turn arbitrarily large key-value mappings into static mapping functions, you'll likely hit upon certain implementation limitations (of this module's code, of `Reflection.Emit` or of the CLR's JIT).\n\n[#DEBUG_STATIC_MAPPING]\nIf the conditional compilation symbol `DEBUG_STATIC_MAPPING` is defined when compiling FParsec, the generated mapping functions will compute each result with two different methods and check the results against each other. Of course, this means that they will take more than twice the time than without the `DEBUG_STATIC_MAPPING` symbol and will also consume more memory. In Debug builds of FParsec `DEBUG_STATIC_MAPPING` is switched on by default, since the `StaticMapping` module is still relatively new.\n\n[note Measuring and comparing the performance of the generated mapping functions only makes sense in Release builds.]\n\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsec.dll\n\nnamespace FParsec\n\n``]\n[``\ntype @Range@``]\n[\nRepresents an immutable range between the integer values `Min` and `Max` (inclusive).\n\n``\ntype Range = struct\n new: min: int * max: int -> Range\n\n val Min: int\n val Max: int\nend\n``\n\nThe `Min` value must not be larger than the `Max` value. In a Debug build this condition is checked by an assert-check in the `Range` constructor.\n\n]\n\n[`` = struct\n new: min: int * max: int -> Range\n\n val Min: int\n val Max: int\nend\n\nmodule [no-auto-link StaticMapping] =\n\n``]\n\n[``\n val @createStaticCharIndicatorFunction@:\n invert: bool -> charsInSet: seq -> (char -> bool)\n``]\n[\nCreates an optimized indicator function for the chars specified by the `charsInSet` sequence.\n\nIf `invert` is `false` (`true`), the returned indicator function will return `true` (`false`)\nif and only if it is called with a char contained in `charsInSet`.\n\n`charsInSet` may contain duplicate char values.\n\nInternally, this function collects continuous ranges of chars into `Range` values and then uses the same compilation strategy as `createStaticCharRangeIndicatorFunction`.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n\n\n[``\n\n val @createStaticCharRangeIndicatorFunction@:\n invert: bool -> rangesInSet: seq -> (char -> bool)\n``]\n[\nCreates an optimized indicator function for the chars in the ranges specified by the `rangesInSet` sequence.\n\nIf `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\ncalled with a char contained in at least one of the ranges of `rangesInSet`.\n\n`rangesInSet` may contain overlapping or duplicate ranges. However, the ranges must not contain values less than `0` or greater than `0xffff` (the minimum and maximum UTF-16 char values), otherwise an `ArgumentException` is thrown.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n[``\n\n val @createStaticIntIndicatorFunction@:\n invert: bool -> valuesInSet: seq -> (int -> bool)\n``]\n[\nCreates an optimized indicator function for the integers specified by the `valuesInSet` sequence.\n\nIf `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\ncalled with an integer contained in `valuesInSet`.\n\n`valuesInSet` may contain duplicate integer values.\n\nInternally, this function collects continues ranges of integer into `Range` values and then uses the same compilation strategy as `createStaticIntRangeIndicatorFunction`.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n\n\n[``\n\n val @createStaticIntRangeIndicatorFunction@:\n invert: bool -> rangesInSet: seq -> (int -> bool)\n``]\n[\nCreates an optimized indicator function for the integers in the ranges specified by the `rangesInSet` sequence.\n\nIf `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\ncalled with an `int` contained in at least one of the ranges of `rangesInSet`.\n\n`rangesInSet` may contain overlapping or duplicate ranges.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n\n[``\n\n\n val @createStaticIntMapping@:\n defaultValue: 'T -> keyValues: seq -> (int -> 'T)\n``]\n[\nCreates an optimized mapping function that maps integer keys to values.\n\nThe `keyValues` sequence specifies the key-value pairs for the mapping. All keys not specified in `keyValues` are mapped to `defaultValue`.\n\nThis function throws an `ArgumentException` if `keyValues` contains a duplicate key.\n\nInternally, this function collects continues ranges of integer keys with equal values[fn In the case of a reference type the values are only compared for reference-equality. In the case of a value type the values are only compared if the type implements `System.IEquality<_>` or is an `int` enum type.] into `Range` values and then uses the same compilation strategy as `createStaticIntRangeMapping`.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n[``\n\n val @createStaticIntRangeMapping@:\n defaultValue: 'T -> keyValues: seq -> (int -> 'T)\n``]\n[\nCreates an optimized mapping function that maps integer key ranges to values.\n\nThe `keyValues` sequence specifies the range-value pairs for the mapping. All keys not contained in one of the ranges in `keyValues` are mapped to `defaultValue`.\n\nThis function throws an `ArgumentException` if `keyValues` contains an overlapping or duplicate key range.\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n\n[``\n\n\n val @createStaticStringMapping@:\n defaultValue: 'T -> keyValues: seq -> (string -> 'T)\n``]\n[\nCreates an optimized mapping function that maps string keys to values.\n\nThe `keyValues` sequence specifies the key-value pairs for the mapping. All keys not specified in `keyValues` are mapped to `defaultValue`. A `null` key is not supported.\n\n`createStaticStringMapping` throws an `ArgumentException` if `keyValues` contains a duplicate key or a `null` key. If the generated mapping function is called with a `null` string, it throws a `NullReferenceException`.\n\n[note The compilation strategy employed by `createStaticStringMapping` does not handle all mappings equally well. It is optimized for mapping a relatively small set of string symbols to constants. If you want to use `createStaticStringMapping` to optimize a frequently used mapping in your program, you should test how well `createStaticStringMapping` handles your situation (in a Release build!) and see whether the performance is worth the compilation costs and the additional code dependency.]\n\nPlease also see the @remarks@ at the beginning of this section.\n]\n\n\n\n\n[``\n\n``]\n[/interface-members]\n[/section]\n\n[/interface-reference]\n[/section]\n\n" }, { "path": "Doc/src/reference-text.txt", "content": "\n[section#Text FParsec.Text]\n\n[interface-reference]\n[section Interface]\n[$$interface]\n[/section]\n\n[section Members]\n[interface-members]\n[``\n// FParsecCS.dll\n\nnamespace FParsec\n\ntype Text =\n``]\n\n[``\n static member @CountTextElements@: string -> int\n``]\n[\n`FParsec.Text.CountTextElements(str)` is equivalent to `System.Globalization.StringInfo(str).LengthInTextElements`.\n]\n\n\n[``\n static member @FoldCase@: char -> char\n``]\n[#FoldCase_char\n`FParsec.Text.FoldCase(chr)` is an optimized implementation of `FParsec.Text.FoldCase(string chr)`.\n]\n\n[``\n static member @FoldCase@: string -> string\n``]\n[\nReturns a case-folded copy of the string argument. All chars are mapped using\nthe (non-Turkic) 1-to-1 [url \"http://www.unicode.org/Public/8.0.0/ucd/CaseFolding.txt\" case folding mappings] (v. 8.0.0) for Unicode code points in the Basic Multilingual Plane, i.e. code points below 0x10000. If the case-folded string equals the argument string, the original argument is returned (to preserve its reference identity). If the argument is `null`, `null` is returned.\n]\n\n[``\n static member @IsWhitespace@: char -> bool\n``]\n[\n`FParsec.Text.IsWhitespace(chr)` is a faster implementation of `System.Char.IsWhiteSpace(chr)`.\n\nIn the `LOW_TRUST`-version of FParsec this method simply forwards all calls to `System.Char.IsWhiteSpace(chr)`.\n]\n\n[``\n static member @NormalizeNewlines@: string -> string\n``]\n[\nReturns the passed string with all occurrences of `\"\\r\\n\"` and `\"\\r\"` replaced by `\"\\n\"`. If the normalized string equals the argument string, the original argument is returned (to preserve its reference identity). If the argument is `null`, `null` is returned.\n]\n\n[``\n\n``]\n[/interface-members]\n[/section]\n\n[/interface-reference]\n[/section]\n\n" }, { "path": "Doc/src/reference.txt", "content": "\n[section Reference]\n [split-section]\n [output-in-subdirectory]\n\n [include \"reference-overview.txt\"]\n [include \"reference-primitives.txt\"]\n [include \"reference-charparsers.txt\"]\n [include \"reference-operatorprecedenceparser.txt\"]\n [include \"reference-staticmapping.txt\"]\n [include \"reference-reply.txt\"]\n [include \"reference-error.txt\"]\n [include \"reference-errormessage.txt\"]\n [include \"reference-errormessagelist.txt\"]\n [include \"reference-position.txt\"]\n [include \"reference-charstream.txt\"]\n [include \"reference-text.txt\"]\n\n[/section]\n" }, { "path": "Doc/src/status-and-roadmap.txt", "content": "[section Status and roadmap]\n\n[section Status]\nFParsec has been in development for several years and can now be considered \"stable\".\n\nVersion 1.0 of FParsec was released on 19 July 2012.\n\n[note\n\nAlthough FParsec has rather comprehensive unit tests (with code coverage close to 100% for many components), it likely still contains bugs. If you want to use FParsec in a production environment, you need to test your parsers thoroughly.\n]\n[/section]\n\n[section Future development]\n\nThere are no firm plans for any major new features yet.\n\nOne goal for the future development of FParsec is to support a more declarative parser definition syntax without compromising on FParsec's performance or language-integrated nature.\n\nFor example, it would be nice if FParsec provided a way to automatically create optimized lexer functions from a series of typed regular expressions and associated mapping functions, ideally at compile time. Using such a feature could maybe look similar to\n\n``\nlet lexer : Parser = \n lex [\"regex-with-1-capture-group\", (fun x -> AstNode1(x))\n \"regex-with-2-capture-groups\", (fun x y -> AstNode2(x, y))\n (* ... *)]\n``\n\n[/section]\n\n[/section]\n" }, { "path": "Doc/src/template.html", "content": "\n\n\n \n {title; text-only, no-number}\n \n \n \n \n \n\n\n
\n
\n
\n \n
\n {navigation-tree}\n
\n
\n Copyright © 2022 Stephan Tolksdorf\n
\n
\n
\n
\n
\n
\n
\n
\n \n {parent-section-short-titles;\n prefix = '',\n separator = ' > ',\n postfix = ' > '\n }{short-title; no-number}\n \n
\n {content}\n
\n
\n
\n\n\n\n" }, { "path": "Doc/src/tutorial.txt", "content": "[section Tutorial]\n\nThis tutorial introduces you to the basic concepts of FParsec. Our goal is to give you an intuition for how you can build parser applications using the FParsec library. We will only cover the basic ideas and only cursorily explore FParsec's API, but hopefully we will cover enough ground to enable you to further explore FParsec with the help of the @user's guide@, the @API reference@ and the sample parsers in the =Samples= folder.\n\n[*A Japanese translation of this tutorial by Gab_km is available [url \"http://blog.livedoor.jp/gab_km/archives/1437534.html\" here].][br]\n[*A Russian translation of this tutorial by Dmitry Vlasov is available [url \"http://dmitriyvlasov.ru/publication/fparsec-tutorial/\" here].]\n\n[toc]\n\n[section Preliminaries]\n\nFParsec is built as two DLLs: =FParsec.dll= and =FParsecCS.dll=. To use FParsec in your project, you can either let [url \"http://nuget.org\" NuGet] install one of the [^nuget-packages NuGet packages], or you can build the two FParsec DLLs from source. The easiest way to build FParsec from source is using the Visual Studio solution files in the [=Build/VS11] folder of the [url \"https://github.com/stephan-tolksdorf/fparsec/archive/master.zip\" source code package]. Any project that uses FParsec has to reference both DLLs. See @Download and Installation@ for more details.\n\nAll FParsec types and modules are declared in the `FParsec` namespace. This namespace contains some basic classes (such as `CharStream` and `Reply`) and four F# modules, namely\n- `Primitives`, containing basic type definitions and parser combinators,\n- `CharParsers`, containing parsers for chars, strings and numbers, and functions for applying parsers to input streams,\n- `Error`, containing types and helper functions for creating, processing and formatting parser error messages,\n- `StaticMapping`, containing functions for compiling static key-value mappings into optimized functions.\n\nAll code snippets in this tutorial assume that you've opened the `FParsec` namespace:\n``\nopen FParsec\n``\nOpening the `FParsec` namespace also automatically opens the `Primitives`, `CharParsers` and `[^FParsec..Error Error]` modules.\n\n[note\nAll code snippets in this tutorial are contained in the [=Samples/Tutorial] project. Having this project open while reading the tutorial can be quite helpful. For example, you can hover the mouse over an identifier to get an Intellisense popup with the inferred type. And if you're curious how a library function is implemented, you can click the /Go to definition/ context menu option to view its source code.\n]\n\n[/section]\n\n[section Parsing a single float]\n\nParsing input with FParsec involves two steps:\n1) building a parser and\n2) applying the parser to the input.\n\nLet's start with a simple example: parsing a single floating-point number in a string.\n\nIn this case the first step, building the parser, is trivial, because the `CharParsers` module already comes with a built-in float parser:\n``val pfloat: Parser``\n\nThe generic type `Parser<'Result,'UserState>` is the type of all parsers in FParsec. If you follow the hyperlink into the reference, you'll see that `Parser` is a type abbreviation for a function type. However, at this point we don't need to go into the details of the `Parser` type. It's enough to note that the first type argument represents the type of the parser result. Thus, in the case of `pfloat` the type tells us that if the parser succeeds it returns a floating-point number of type `float`. We won't use a \"user state\" in this tutorial, so you can just ignore the second type argument for the time being.\n\nTo apply the `pfloat` parser to a string, we can use the `run` function from the `CharParsers` module:\n``val run: Parser<'Result, unit> -> string -> ParserResult<'Result,unit>``\n\n`run` is the simplest function out of [^runparser-functions several] provided by the `CharParsers` module for running parsers on input. Other functions allow you, for example, to run parsers directly on the contents of a file or a `System.IO.Stream`.\n\n`run` applies the parser passed as the first argument to the string passed as the second argument and returns the return value of the parser in form of a `ParserResult` value. The `ParserResult` type is a discriminated union type with the two cases: `Success` and `Failure`. In case the parser succeeds, the `ParserResult` value contains the result value, otherwise it contains an error message.\n\nTo simplify testing we write a little helper function that prints the result value or error message:\n``\nlet test p str =\n match run p str with\n | Success(result, _, _) -> printfn \"Success: %A\" result\n | Failure(errorMsg, _, _) -> printfn \"Failure: %s\" errorMsg\n``\n\nWith this helper function in place, we can test `pfloat` by executing\n``test pfloat \"1.25\"``\nwhich produces the output\n``{fsi}Success: 1.25``\n\nTesting `pfloat` with a number literal that has an invalid exponent\n``test pfloat \"1.25E 3\"``\nyields the error message\n``{fsi}\nFailure: Error in Ln: 1 Col: 6\n1.25E 3\n ^\nExpecting: decimal digit\n``\n\n[/section]\n\n[section Parsing a float between brackets]\n\nImplementing parsers with FParsec typically means combining higher-level parsers from lower-level ones. You start with the parser primitives provided by the library and then successively combine these into higher-level parsers until you finally have a single parser for the complete input.\n\nIn the following sections we will illustrate this approach by discussing various sample parsers that build on each other. In this section we will begin with a very simple parser for a floating-point number between brackets:\n``\nlet str s = pstring s\nlet floatBetweenBrackets = str \"[\" >>. pfloat .>> str \"]\"\n``\n\n[note\nIf you're trying to compile this or another code snippet and you get a compiler error mentioning F#'s \"value restriction\", please see [^fs-value-restriction].\n]\n\nThe definition of `str` and `floatBetweenBrackets` involves three library functions that we haven't yet introduced: `pstring`, `>>.` and `.>>`.\n\nThe function\n``val pstring: string -> Parser``\ntakes a string as the argument and returns a parser for that string. When this parser is applied to an input stream it checks whether the following chars in the input stream match the given string. If the chars match the complete string, the parser consumes them, i.e. skips over them. Otherwise it fails without consuming any input. When the parser succeeds, it also returns the given string as the parser result, but since the string is a constant, you'll rarely make use of the result.\n\nThe `pstring` function isn't named `string` because otherwise it would hide the built-in F# function `string`. In general, parser names in FParsec that would otherwise conflict with built-in F# function names are prefixed by a single p char. `pfloat` is another example for this naming convention.\n\nTo save a few keystrokes we abbreviate `pstring` as `str`. So, for instance, `str \"[\"` is a parser that skips over the char `'['`.\n\nThe binary operators `>>.` and `.>>` have the following types:\n``\nval (>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n``\n\nAs you can see from these signatures, both operators are parser combinators that construct a new parser from the two argument parsers. The parser `p1 >>. p2` parses `p1` and `p2` in sequence and returns the result of `p2`. The parser `p1 .>> p2` also parses `p1` and `p2` in sequence, but it returns the result of `p1` instead of `p2`. In each case the point points to the side of the parser whose result is returned. By combining both operators in `p1 >>. p2 .>> p3` we obtain a parser that parses `p1`, `p2` and `p3` in sequence and returns the result from `p2`.\n\n[note\nWith the somewhat imprecise wording \"parses `p1` and `p2` in sequence\" we actually mean: The parser `p1` is applied to the input and if `p1` succeeds then `p2` is applied to the remaining input; in case any of the two element parsers fails, the aggregate parser immediately propagates the error message.\n\nIn the documentation for FParsec we often use expressions such as \"parses `p`\" or \"parses an occurrence of `p`\" instead of the technically more accurate \"applies the parser `p` to the remaining input and if `p` succeeds ...\", hoping that the exact meaning is obvious from the context.\n]\n\nThe following tests show that `floatBetweenBrackets` parses valid input as expected and produces informative error messages when it encounters invalid input:\n\n``{fsi}\n> test floatBetweenBrackets \"[1.0]\";;\nSuccess: 1.0\n\n> test floatBetweenBrackets \"[]\";;\nFailure: Error in Ln: 1 Col: 2\n[]\n ^\nExpecting: floating-point number\n\n> test floatBetweenBrackets \"[1.0\";;\nFailure: Error in Ln: 1 Col: 5\n[1.0\n ^\nNote: The error occurred at the end of the input stream.\nExpecting: ']'\n``\n\n[(*\n[\nNote that all infix F# operators that begin with `<` or '>' (leading '.' characters are ignored) are left-associative. Hence,\n`p1 >>. p2 .>> p3` is equivalent with `(p1 >>. p2) .>> p3`. However, in this case the associativity has no effect on the aggregate parser behaviour.\n]\n*)]\n\n[/section]\n\n\n\n[section Abstracting parsers]\n\nOne of FParsec's greatest strengths is the ease with which you can define your own parser abstractions.\n\nTake for instance the `floatBetweenBrackets` from the previous section. If you intend to also parse other elements between strings, you could define your own specialized combinator for this purpose:\n``\nlet betweenStrings s1 s2 p = str s1 >>. p .>> str s2\n``\n\nYou could then define `floatInBrackets` and other parsers with the help of this combinator:\n``\nlet floatBetweenBrackets = pfloat |> betweenStrings \"[\" \"]\"\nlet floatBetweenDoubleBrackets = pfloat |> betweenStrings \"[[\" \"]]\"\n``\n\n[note In case you're new to F#:[br] `pfloat |> betweenStrings \"[\" \"]\"` is just another way to write `betweenStrings \"[\" \"]\" pfloat`.]\n\nOnce you notice that you frequently need to apply a parser between two others, you could go a step further and factor `betweenStrings` as follows:\n``\nlet [no-auto-link between] pBegin pEnd p = pBegin >>. p .>> pEnd\nlet betweenStrings s1 s2 p = p |> [no-auto-link between] (str s1) (str s2)\n``\n\nActually, you don't need to define `between`, because this is already a built-in FParsec combinator.\n\nThese are all trivial examples, of course. But since FParsec is merely an F# library and not some external parser generator tool, there are no limits to the abstractions you can define. You can write functions that take whatever input you need, do some arbitrarily complex computations on the input and then return a special purpose parser or parser combinator.\n\nFor example, you could write a function that takes a regular-expression pattern as the input and returns a `Parser` for parsing input conforming to that pattern. This function could use another parser to parse the pattern into an AST and then compile this AST into a special-purpose parser function. Alternatively, it could construct a .NET regular expression from the pattern and then return a parser function that uses FParsec's `CharStream` API to directly apply the regex to the input stream (which is what the built-in `regex` parser actually does).\n\nAnother example are extensible parser applications. By storing parser functions in dictionaries or other data structures and defining an appropriate extension protocol, you could allow plugins to dynamically register new parsers or modify existing ones.\n\nThe possibilities are really endless. But before you can fully exploit these possibilities, you first need to be familiar with the fundamentals of FParsec.\n\n[/section]\n\n[section Parsing a list of floats]\n\nWe've already spent three sections on discussing how to parse a single floating-point number, so it's about time we try something more ambitious: parsing a list of floating-point numbers.\n\nLet us first assume that we need to parse a sequence of floating-point numbers in brackets, i.e. text in the following EBNF format: `{EBNF}(\"[\" float \"]\")*`. Valid input strings in this format are for example: `\"\"`, `\"[1.0]\"`, `\"[2][3][4]\"`.\n\nSince we already have a parser for a float between brackets, we only need a way to repeatedly apply this parser to parse a sequence. This is what the `many` combinator is for:\n``val many: Parser<'a,'u> -> Parser<'a list,'u>``\n\nThe parser `many p` repeatedly applies the parser `p` until `p` fails, i.e. it \"greedily\" parses as many occurrences of `p` as possible. The results of `p` are returned as a list in the order of occurrence.\n\nSome simple tests show that `many floatInBrackets` works as expected:\n``{fsi}\n> test (many floatBetweenBrackets) \"\";;\nSuccess: []\n> test (many floatBetweenBrackets) \"[1.0]\";;\nSuccess: [1.0]\n> test (many floatBetweenBrackets) \"[2][3][4]\";;\nSuccess: [2.0; 3.0; 4.0]\n``\n\nIf `floatBetweenBrackets` fails *after consuming input*, then the combined parser fails too:\n``{fsi}\n> test (many floatBetweenBrackets) \"[1][2.0E]\";;\nFailure: Error in Ln: 1 Col: 9\n[1][2.0E]\n ^\nExpecting: decimal digit\n``\n\nNote that `many` also succeeds for an empty sequence. If you want to require at least one element, you can use `many1` instead:\n``{fsi}\n> test (many1 floatBetweenBrackets) \"(1)\";;\nFailure: Error in Ln: 1 Col: 1\n(1)\n^\nExpecting: '['\n``\n\n[tip\n\nIf you'd prefer the last error message to be worded in terms of the higher level `floatBetweenBrackets` parser instead of the lower level `str \"[\"` parser, you could use the `` operator as in the following example:\n\n``{fsi}\n> test (many1 (floatBetweenBrackets \"float between brackets\")) \"(1)\";;\nFailure: Error in Ln: 1 Col: 1\n(1)\n^\nExpecting: float between brackets\n``\n\nPlease see [^users-guide.customizing-error-messages] of the user's guide to learn more about customizing error messages.\n]\n\nIf you just want to skip over a sequence and don't need the list of parser results, you could use the optimized combinators `skipMany` or `skipMany1` instead of `many` and `many1`.\n\nAnother frequently used combinator for parsing sequences is `sepBy`:\n``val sepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list, 'u>``\n\n`sepBy` takes an \"element\" parser and a \"separator\" parser as the arguments and returns a parser for a list of elements separated by separators. In EBNF notation `sepBy p pSep` could be written as `{EBNF}(p (pSep p)*)?`. Similar to `many`, there are [^sepBy-parsers several variants] of `sepBy`.\n\nWith the help of `sepBy` we can parse a more readable list format, where floating-point numbers are separated by a comma:\n``{EBNF}floatList: \"[\" (float (\",\" float)*)? \"]\"``\n\nValid input strings in this format are for example: `\"[]\"`, `\"[1.0]\"`, `\"[2,3,4]\"`.\n\nThe straightforward implementation of this format is\n``\nlet floatList = str \"[\" >>. sepBy pfloat (str \",\") .>> str \"]\"\n``\n\nTesting `floatList` with valid test strings yields the expected result:\n``{fsi}\n> test floatList \"[]\";;\nSuccess: []\n> test floatList \"[1.0]\";;\nSuccess: [1.0]\n> test floatList \"[4,5,6]\";;\nSuccess: [4.0; 5.0; 6.0]\n``\n\nTesting with invalid input shows that `floatList` produces helpful error messages:\n``{fsi}\n> test floatList \"[1.0,]\";;\nFailure: Error in Ln: 1 Col: 6\n[1.0,]\n ^\nExpecting: floating-point number\n\n> test floatList \"[1.0,2.0\";;\nFailure: Error in Ln: 1 Col: 9\n[1.0,2.0\n ^\nNote: The error occurred at the end of the input stream.\nExpecting: ',' or ']'\n``\n\n\n[/section]\n\n[section Handling whitespace]\n\nFParsec treats whitespace (spaces, tabs, newlines, etc) just as any other input, so our `floatList` parser can't yet deal with whitespace:\n``{fsi}\n> test floatBetweenBrackets \"[1.0, 2.0]\";;\nFailure: Error in Ln: 1 Col: 5\n[1.0, 2.0]\n ^\nExpecting: ']'\n``\n\nIf we want the parser to ignore whitespace, we need to make this explicit in the parser definition.\n\nFirst, we need to define what we want to accept as whitespace. For simplicity we will just use the built-in `spaces` parser, which skips over any (possibly empty) sequence of `' '`, `'\\t'`, `'\\r'` or `'\\n'` chars.\n``\nlet ws = spaces\n``\n\nNext, we need to insert the `ws` parser at every point where we want to ignore whitespace. In general it's best to skip whitespace *after* one parses elements, i.e. skip trailing instead of leading whitespace, because that reduces the need for backtracking (which will be explained below). Hence, we insert `ws` at two places to skip over any whitespace after brackets or numbers:\n\n``\nlet str_ws s = pstring s .>> ws\nlet float_ws = pfloat .>> ws\nlet numberList = str_ws \"[\" >>. sepBy float_ws (str_ws \",\") .>> str_ws \"]\"\n``\n\nA simple test shows that `numberList` ignores whitespace:\n``{fsi}\n> test numberList @\"[ 1 ,\n 2 ] \";;\nSuccess: [1.0; 2.0]\n``\n\nIf we introduce an error on the second line, we see that `FParsec` automatically keeps track of the line count:\n``{fsi}\n> test numberList @\"[ 1,\n 2; 3]\";;\n\nFailure: Error in Ln: 2 Col: 27\n 2; 3]\n ^\nExpecting: ',' or ']'\n``\n\nOur `numberList` parser still doesn't skip leading whitespace, because that's not necessary when we put it together with other parsers that skip all trailing whitespace. If we wanted to parse a whole input stream with only a list of floating-point numbers, we could use the following parser:\n``\nlet numberListFile = ws >>. numberList .>> eof\n``\n\nThe end-of-file parser `eof` will generate an error if the end of the stream hasn't been reached. This is useful for making sure that the complete input gets consumed. Without the `eof` parser the following test wouldn't produce an error:\n``{fsi}\n> test numberListFile \" [1, 2, 3] [4]\";;\nFailure: Error in Ln: 1 Col: 12\n [1, 2, 3] [4]\n ^\nExpecting: end of input\n``\n\n[/section]\n\n[section Parsing string data]\n\nFParsec contains various built-in parsers for chars, strings, numbers and whitespace. In this section we will introduce a few of the char and string parsers. For an overview of all available parsers please refer to the @parser overview@ in the reference.\n\nYou've already seen several applications of the `pstring` parser (abbreviated as `str`), which simply skips over a constant string in the input. When the `pstring` parser succeeds, it also returns the skipped string as the parser result. The following example demonstrates this:\n\n``{fsi}\n> test (many (str \"a\" <|> str \"b\")) \"abba\";;\nSuccess: [\"a\"; \"b\"; \"b\"; \"a\"]\n``\n\nIn this example we also used the `<|>` combinator to combine two alternative parsers. We'll discuss this combinator in more detail below.\n\n[note\nWe refer to both `pstring` and `pstring \"a\"` as \"parsers\". Strictly speaking, `pstring` is function taking a string argument and returning a `Parser`, but it's more convenient to just refer to it as a (parametric) parser.\n]\n\n\n\nWhen you don't need the result of the `pstring` parser, you can alternatively use the `skipString` parser, which returns the `unit` value `()` instead of the argument string. In this case it doesn't make any difference to performance whether you use `pstring` or `skipString`, since the returned string is a constant. However, for most other built-in parsers and combinators you should prefer the variants with the \"skip\" name prefix when you don't need the parser result values, because these will generally be faster. If you look at the @parser overview@, you'll see \"skip\" variants for many of the built-in parsers and combinators.\n\nIf you want to parse a [+ c]ase [+ i]nsensitive string constant you can use `pstringCI` and `skipStringCI`. For example:\n``{fsi}\n> test (skipStringCI \"\" >>. pfloat) \"1.0\";;\nSuccess: 1.0\n``\n\nFrequently one needs to parse string variables whose chars have to satisfy certain criteria. For instance, identifiers in programming languages often need to start with a letter or underscore and then need to continue with letters, digits or underscores. To parse such an identifier you could use the following parser:\n\n``\nlet identifier =\n let isIdentifierFirstChar c = isLetter c || c = '_'\n let isIdentifierChar c = isLetter c || isDigit c || c = '_'\n\n many1Satisfy2L isIdentifierFirstChar isIdentifierChar \"identifier\"\n .>> ws // skips trailing whitespace\n``\n\nHere we have used the `many1Satisfy2L` string parser, which is one of several primitives for parsing strings based on char predicates (i.e. functions that take a char as input and return a boolean value). It parses any sequence of one or more chars (hence the \"many1\" in the name) whose first char satisfies the first predicate function and whose remaining chars satisfy the second predicate (hence the \"Satisfy2\"). The string label given as the third argument (hence the \"L\") is used in error message to describe the expected input.\n\nThe following tests show how this parser works:\n``{fsi}\n> test identifier \"_\";;\nSuccess: \"_\"\n> test identifier \"_test1=\";;\nSuccess: \"_test1\"\n> test identifier \"1\";;\nFailure: Error in Ln: 1 Col: 1\n1\n^\nExpecting: identifier\n``\n\n[tip If you want to parse identifiers based on the Unicode XID syntax, consider using the built-in `identifier` parser.]\n\nMany string formats are complicated enough that you need to combine several char and string parser primitives. For example, consider the following string literal format:\n``{EBNF}\n stringLiteral: '\"' (normalChar|escapedChar)* '\"'\n normalChar: any char except '\\' and '\"'\n escapedChar: '\\\\' ('\\\\'|'\"'|'n'|'r'|'t')\n``\n\nA straightforward translation of this grammar to FParsec looks like:\n``\nlet stringLiteral =\n let normalChar = satisfy (fun c -> c <> '\\\\' && c <> '\"')\n let unescape c = match c with\n | 'n' -> '\\n'\n | 'r' -> '\\r'\n | 't' -> '\\t'\n | c -> c\n let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> unescape)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (manyChars (normalChar <|> escapedChar))\n``\n\nIn this example we use several library functions that we haven't yet introduced:\n- `satisfy` parses any char that satisfies the given predicate function.\n- `anyOf` parses any char contained in the argument string.\n- The pipeline combinator `|>>` applies the function on the right side (`unescape`) to the result of the parser on the left side (`anyOf \"\\\\nrt\\\"\"`).\n- The choice combinator `<|>` applies the parser on the right side if the parser on the left side fails, so that `normalChar <|> escapedChar` can parse both normal and escaped chars. (We will discuss this operator in more detail two sections below.)\n- `manyChars` parses a sequence of chars with the given char parser and returns it as a string.\n\nLet's test the `stringLiteral` parser with a few test inputs:\n``{fsi}\n> test stringLiteral \"\\\"abc\\\"\";;\nSuccess: \"abc\"\n> test stringLiteral \"\\\"abc\\\\\\\"def\\\\\\\\ghi\\\"\";;\nSuccess: \"abc\"def\\ghi\"\n> test stringLiteral \"\\\"abc\\\\def\\\"\";;\nFailure: Error in Ln: 1 Col: 6\n\"abc\\def\"\n ^\nExpecting: any char in ‘\\nrt\"’\n``\n\nInstead of parsing the string literal char-by-char we could also parse it \"snippet-by-snippet\":\n``\nlet stringLiteral2 =\n let normalCharSnippet = many1Satisfy (fun c -> c <> '\\\\' && c <> '\"')\n let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> function\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (manyStrings (normalCharSnippet <|> escapedChar))\n``\n\nHere we have used the `manyStrings` combinator, which parses a sequence of strings with the given string parser and returns the strings in concatenated form.\n\n[note\nWe have to require `normalCharSnippet` to consume at least one char, i.e. use `many1Satisfy` instead of `manySatisfy`. Otherwise `normalCharSnippet` would succeed even if doesn't consume input, `escapedChar` would never be called and `manyStrings` would eventually throw an exception to prevent an infinite loop.\n]\n\nParsing a string chunk-wise using an optimized parser like `many1Satisfy` is usually a bit faster than parsing it char-wise using `manyChars` and `satisfy`. In this case we can optimize our parser even a bit further -- once we realize that two normal char snippets must be separated by at least one escaped char:\n``\nlet stringLiteral3 =\n let normalCharSnippet = manySatisfy (fun c -> c <> '\\\\' && c <> '\"')\n let escapedChar = (* like in stringLiteral2 *)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (stringsSepBy normalCharSnippet escapedChar)\n\n``\nThe `stringsSepBy` combinator parses a sequence of strings (with the first argument parser) separated by other strings (parsed with the second argument parser). It returns all parsed strings, including the separator strings, as a single, concatenated string.\n\nNote that `stringLiteral3` uses `manySatisfy` instead of `many1Satisfy` in its `normalCharSnippet` definition, so that it can parse escaped chars that are not separated by normal chars. This can't lead to an infinite loop because `escapedChar` can't succeed without consuming input.\n\n[/section]\n\n[section Sequentially applying parsers]\n\nWhenever you need to apply multiple parsers in sequence and only need the result of one of them, a suitable combination of `>>.` and `.>>` operators will do the job. However, these combinators won't suffice if you need the result of more than one of the involved parsers. In that case you can use the `pipe2`, ..., `pipe5` combinators, which apply multiple parsers in sequence and pass all the individual results to a function that computes the aggregate result.\n\nFor instance, with the `pipe2` combinator\n``val pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> b -> 'c) -> Parser<'c,'u>``\nyou can construct a parser `pipe2 p1 p2 f` that sequentially applies the two parsers `p1` and `p2` and then returns the result of the function application `f x1 x2`, where `x1` and `x2` are the results returned by `p1` and `p2`.\n\nIn the following example we use `pipe2` to parse a product of two numbers:\n``\nlet product = pipe2 float_ws (str_ws \"*\" >>. float_ws)\n (fun x y -> x * y)\n``\n``{fsi}\n> test product \"3 * 5\";;\nSuccess: 15.0\n``\n\nThe `pipe2-5` combinators are particularly useful for constructing AST objects. In the following example we use `pipe3` to parse a string constant definition into a `StringConstant` object:\n``\ntype StringConstant = StringConstant of string * string\n\nlet stringConstant = pipe3 identifier (str_ws \"=\") stringLiteral\n (fun id _ str -> StringConstant(id, str))\n``\n``{fsi}\n> test stringConstant \"myString = \\\"stringValue\\\"\";;\nSuccess: StringConstant (\"myString\",\"stringValue\")\n``\n\nIf you just want to return the parsed values as a tuple, you can use the predefined `tuple2-5` parsers. For instance, `tuple2 p1 p2` is equivalent to `pipe2 p1 p2 (fun x1 x2 -> (x1, x2))`.\n\nThe `tuple2` parser is also available under the operator name `.>>.`, so that you can write `p1 .>>. p2` instead of `tuple2 p1 p2`. In the following example we parse a pair of comma separated numbers with this operator:\n``{fsi}\n> test (float_ws .>>. (str_ws \",\" >>. float_ws)) \"123, 456\";;\nSuccess: (123.0, 456.0)\n``\n\nHopefully you find the [/ `>>`-with-1-or-2-dots-notation] intuitive by now.\n\nIf you need a `pipe` or `tuple` parser with more than 5 arguments, you can easily construct one using the existing ones. For example, do you have an idea how to define a `pipe7` parser? This footnote gives a possible solution: [fn\n``\nlet pipe7 p1 p2 p3 p4 p5 p6 p7 f =\n pipe4 p1 p2 p3 (tuple4 p4 p5 p6 p7)\n (fun x1 x2 x3 (x4, x5, x6, x7) -> f x1 x2 x3 x4 x5 x6 x7)\n``\n]\n\n[/section]\n\n[section Parsing alternatives]\n\nIn the section on `Parsing string data` we already shortly introduced the choice combinator `<|>`:\n``\nval (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,u>\n``\n\nThis combinator allows you to support multiple alternative input formats at a given input position. For example, in the above section we used `<|>` to combine a parser for unescaped chars and a parser for escaped chars into a parser that supports both: `normalChar <|> escapedChar`.\n\nAnother example that shows how `<|>` works is the following parser for boolean variables:\n``\nlet boolean = (stringReturn \"true\" true)\n <|> (stringReturn \"false\" false)\n``\n\nHere we have also used the `stringReturn` parser, which skips the string constant given as the first argument and, if successful, returns the value given as the second argument.\n\nTesting the `boolean` parser with some inputs yields:\n``{fsi}\n> test boolean \"false\";;\nSuccess: false\n> test boolean \"true\";;\nSuccess: true\n> test boolean \"tru\";;\nFailure: Error in Ln: 1 Col: 1\ntru\n^\nExpecting: 'false' or 'true'\n``\n\nThe behaviour of the `<|>` combinator has two important characteristics:\n- `<|>` only tries the parser on the right side if the parser on the left side fails. It does *not* implement a longest match rule.\n- However, it only tries the right parser if the left parser fails *without consuming input*.\n\nA consequence of the second point is that the following test fails because the parser on the left side of `<|>` consumes whitespace before it fails:\n``{fsi}\n> test ((ws >>. str \"a\") <|> (ws >>. str \"b\")) \" b\";;\nFailure: Error in Ln: 1 Col: 2\n b\n ^\nExpecting: 'a'\n``\n\nFortunately, we can easily fix this parser by factoring out `ws`:\n``{fsi}\n> test (ws >>. (str \"a\" <|> str \"b\")) \" b\";;\nSuccess: \"b\"\n``\n\nIf you're curious why `<|>` behaves this way and how you can handle situations where you need `<|>` to try the alternative parser even if the first parser fails after consuming input, please see [^users-guide.parsing-alternatives] and [^users-guide.looking-ahead-and-backtracking] in the user's guide.\n\nIf you want to try more than two alternative parsers, you can chain multiple `<|>` operators, like in `p1 <|> p2 <|> p3 <|> ...`, or you can use the `choice` combinator, which accepts a sequence of parsers as the argument, like in `choice [p1; p2; p3; ...]`.\n\n[/section]\n\n[section F#'s value restriction]\n\nWhen you start writing your own parsers with FParsec or try to compile some individual code snippets from above, you'll come across a compiler issue that\noften causes some head-scratching among new users of F# and FParsec: the /value restriction/. In this section we'll explain the value restriction and how you can handle it in your FParsec programs.\n\n[note If you find the discussion in this section too technical for the moment, just skip to the next section and come back later when you actually see a compiler message mentioning \"value restriction\" for the first time.]\n\nF#'s value restriction is the reason that the following code snippet does not compile\n``\nopen FParsec\nlet p = pstring \"test\"\n``\neven though the following snippet compiles without a problem[fn Assuming you referenced the two FParsec DLLs.]:\n``\nopen FParsec\nlet p = pstring \"test\"\nrun p \"input\"\n``\n\nThe compiler error generated for the first sample is the following:\n``{fsi}\nerror FS0030: Value restriction.\nThe value 'p' has been inferred to have generic type\n val p : Parser\nEither make the arguments to 'p' explicit or,\nif you do not intend for it to be generic, add a type annotation.\n``\n\nWhen you work with FParsec you'll sooner or later see this or similar error messages, in particular if you work with the interactive console prompt. Fortunately, this kind of error is usually easy to workaround.\n\nThe problem with the first snippet above is that the F# compiler infers the `p` value to have an unresolved generic type, although F# doesn't permit a generic value in this situation. The return type of the `pstring` function is `Parser`, where the type parameter `'u` represents the type of the `CharStream` user state. Since there is nothing in the first snippet that constrains this type parameter, the compiler infers the type `Parser` for the parser value `p`, with `'_a` representing an unresolved type parameter.\n\nIn the second snippet this problem doesn't occur, because the use of `p` as the first argument to the `run` function constrains the user state type. Since `run` only accepts parsers of type `Parser<'t,unit>`, the compiler infers the non-generic type `Parser` for `p`.\n\nThis example suggests two ways to handle the value restriction in FParsec programs:\n- Either make sure that the type of a parser value is constrained to a non-generic type by subsequent uses of this parser value *in the same compilation unit*,\n- or provide an explicit type annotation to manually constrain the type of the parser value (usually, a few type annotations in key locations are enough for a whole parser module).\n\nOften it is convenient to define some type abbreviations like the following\n``\ntype UserState = unit // doesn't have to be unit, of course\ntype Parser<'t> = Parser<'t, UserState>\n``\n\nWith such an abbreviation in place, type annotations become as simple as\n``\nlet p : Parser<_> = pstring \"test\"\n``\n\nOf course, constraining the type of a parser value to a non-generic type is only a solution if you don't actually need a generic type. If you do need a generic value, you'll have to apply other techniques, as they are for example explained in the [url \"http://msdn.microsoft.com/en-us/library/dd233183.aspx\" F# reference] or in a [url \"http://blogs.msdn.com/b/mulambda/archive/2010/05/01/value-restriction-in-f.aspx\" blog entry] by Dmitry Lomov. However, FParsec `Parser` values (not parametric parser functions) are usually only used in the context of a specific parser application with a fixed user state type. In that situation constraining the type is indeed the appropriate measure to avoid a value restriction error.\n\n[/section]\n\n[section Parsing JSON]\n\nNow that we have discussed the basics of FParsec we are well prepared to work through a real world parser example: a JSON parser.\n\nJSON (JavaScript Object Notation) is a text-based data interchange format with a simple and lightweight syntax. You can find descriptions of the syntax on [url \"http://json.org\" json.org] and in [url \"http://www.ietf.org/rfc/rfc4627.txt\" RFC 4626].\n\nIn many applications one only has to deal with JSON files describing one particular kind of object. In such a context it sometimes can be appropriate to write a specialized parser just for that specific kind of JSON file. In this tutorial, however, we will follow a more general approach. We will implement a parser that can parse any general JSON file into an AST, i.e. an intermediate data structure describing the contents of the file. Applications can then conveniently query this data structure and extract the information they need. This is an approach comparable to that of XML parsers which build a data structure describing the document tree of an XML document. The great advantage of this approach is that the JSON parser itself becomes reusable and the document specific parsing logic can be expressed in the form of simple functions processing the AST of the JSON document.\n\nThe natural way to implement an AST in F# is with the help of a discriminated union type. If you look at the [url \"http://json.org\" JSON specification], you can see that a JSON value can be a string, a number, a boolean, null, a comma-separated list of values in square brackets, or an object with a sequence of key-value pairs in curly brackets.\n\nIn our parser we will use the following union type to represent JSON values:\n``\ntype Json = JString of string\n | JNumber of float\n | JBool of bool\n | JNull\n | JList of Json list\n | JObject of Map\n``\n\nHere we've chosen the F# `list` type to represent a sequence of values and the `Map` type to represent a sequence of key-value pairs, because these types are particularly convenient to process in F#.[fn If you need to parse huge sequences and objects, it might be more appropriate to use an array and dictionary for JList and JObject respectively.] Note that the `Json` type is recursive, since both `JList` and `JObject` values can themselves contain `Json` values. Our parser will have to reflect this recursive structure.\n\n[tip If you're new to FParsec and have a little time, it would be a good exercise to try to implement the JSON parser on your own (with the help of the reference documentation). This tutorial already covered almost everything you need and the JSON grammar is simple enough that this shouldn't take too much time. Of course, you can always peek at the implementation below if you get stuck.]\n\nWe start the actual parser implementation by covering the simple `null` and boolean cases:\n``\nlet jnull = stringReturn \"null\" JNull\nlet jtrue = stringReturn \"true\" (JBool true)\nlet jfalse = stringReturn \"false\" (JBool false)\n``\n\nHandling the number case is just as simple, because the JSON number format is based on the typical floating-point number format used in many programming languages and hence can be parsed with FParsec's built-in `pfloat` parser:\n``\nlet jnumber = pfloat |>> JNumber\n``\n(Note that F# allows us to pass the object constructor `JNumber` as a function argument.)\n\nIf you compare the precise number format supported by `pfloat` with that in the JSON spec, you'll see that `pfloat` supports a superset of the JSON format. In contrast to the JSON format the `pfloat` parser also recognizes `NaN` and `Infinity` values, accepts a leading plus sign, accepts leading zeros and even supports the hexadecimal float format of Java and C99. Depending on the context this behaviour can be considered a feature or a limitation of the parser. For most applications it probably doesn't matter, and the JSON RFC clearly states that a JSON parser may support a superset of the JSON syntax. However, if you'd rather only support the exact JSON number format, you can implement such a float parser rather easily based on the configurable `numberLiteral` parser (just have a look at how this is currently done in the `pfloat` source).\n\nThe JSON string format takes a little more effort to implement, but we've already parsed a similar format with the `stringLiteral` parsers in [^parsing-string-data], so we can just adapt one of those parsers for our purpose:\n\n``\nlet str s = pstring s\n\nlet stringLiteral =\n let escape = anyOf \"\\\"\\\\/bfnrt\"\n |>> function\n | 'b' -> \"\\b\"\n | 'f' -> \"\\u000C\"\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c // every other char is mapped to itself\n\n let unicodeEscape =\n \t/// converts a hex char ([0-9a-fA-F]) to its integer number (0-15)\n let hex2int c = (int c &&& 15) + (int c >>> 6)*9\n\n str \"u\" >>. pipe4 hex hex hex hex (fun h3 h2 h1 h0 ->\n (hex2int h3)*4096 + (hex2int h2)*256 + (hex2int h1)*16 + hex2int h0\n |> char |> string\n )\n\n let escapedCharSnippet = str \"\\\\\" >>. (escape <|> unicodeEscape)\n let normalCharSnippet = manySatisfy (fun c -> c <> '\"' && c <> '\\\\')\n\n between (str \"\\\"\") (str \"\\\"\")\n (stringsSepBy normalCharSnippet escapedCharSnippet)\n\nlet jstring = stringLiteral |>> JString\n``\n\n`stringLiteral` parses string literals as a sequence of normal char snippets separated by escaped char snippets. A normal char snippet is any sequence of chars that does not contain the chars `'\"'` and `'\\\\'`. An escaped char snippet consists of a backslash followed by any of the chars `'\\\\'`, `'\\\"'`, `'/'`, `'b'`, `'f'`, `'n'`, `'r'`, `'t'`, or an Unicode escape. An Unicode escape consists of an `'u'` followed by four hex chars representing an UTF-16 code point.\n\n[#createParserForwardedToRef-example]\nThe grammar rules for JSON lists and objects are recursive, because any list or object can contain itself any kind of JSON value. Hence, in order to write parsers for the list and object grammar rules, we need a way to refer to the parser for any kind of JSON value, even though we haven't yet constructed this parser. Like it is so often in computing, we can solve this problem by introducing an extra indirection:\n``\nlet jvalue, jvalueRef = createParserForwardedToRef()\n``\n\nAs you might have guessed from the name, `createParserForwardedToRef` creates a parser (`jvalue`) that forwards all invocations to the parser in a reference cell (`jvalueRef`). Initially, the reference cell holds a dummy parser, but since the reference cell is mutable, we can later replace the dummy parser with the actual value parser, once we have finished constructing it.\n\nThe JSON RFC sensibly only permits spaces, (horizontal) tabs, line feeds and carriage returns as whitespace characters, which allows us to use the built-in `spaces` parser for parsing whitespace:\n``\nlet ws = spaces\n``\n\nBoth JSON lists and objects are syntactically represented as a comma-separated lists of \"elements\" between brackets, where whitespace is allowed before and after any bracket, comma and list element. We can conveniently parse such lists with the following helper function:\n``\nlet listBetweenStrings sOpen sClose pElement f =\n between (str sOpen) (str sClose)\n (ws >>. sepBy (pElement .>> ws) (str \",\" >>. ws) |>> f)\n``\n\nThis function takes four arguments: an opening string, a closing string, an element parser and a function that is applied to the parsed list of elements.\n\nWith the help of this function we can define the parser for a JSON list as follows:\n``\nlet jlist = listBetweenStrings \"[\" \"]\" jvalue JList\n``\n\nJSON objects are lists of key-value pairs, so we need a parser for a key-value pair:\n``\nlet keyValue = stringLiteral .>>. (ws >>. str \":\" >>. ws >>. jvalue)\n``\n(Remember, the points on both sides of `.>>.` indicate that the results of the two parsers on both sides are returned as a tuple.)\n\nBy passing the `keyValue` parser to `listBetweenStrings` we obtain a parser for JSON objects:\n``\nlet jobject = listBetweenStrings \"{\" \"}\" keyValue (Map.ofList >> JObject)\n``\n\n[#json-value-parser]\nHaving defined parsers for all the possible kind of JSON values, we can combine the different cases with a `choice` parser to obtain the finished parser for JSON values:\n\n``\ndo jvalueRef := choice [jobject\n jlist\n jstring\n jnumber\n jtrue\n jfalse\n jnull]\n``\n\nThe `jvalue` parser doesn't accept leading or trailing whitespace, so we need to define our parser for complete JSON documents as follows:\n``\nlet json = ws >>. jvalue .>> ws .>> eof\n``\n\nThis parser will try to consume a complete JSON input stream and, if successful, will return a `Json` AST of the input as the parser result\n\nAnd that's it, we're finished with our JSON parser. If you want to try this parser out on some sample input, please take a look at the JSON project in the =Samples= folder.\n\n\n[/section]\n\n[section What now?]\n\nIf this tutorial has whet your appetite for a more in-depth introduction to FParsec, just head over to the @user's guide@. If you can't wait to write your own parser, then bookmark the @parser overview@ page, maybe take a short look at the example parsers in the =Samples= folder and just start hacking. You can always consult the user's guide at a later point should you get stuck somewhere.\n[/section]\n\n\n\n[/section]" }, { "path": "Doc/src/users-guide.txt", "content": "\n[section User's Guide]\n[split-section]\n[output-in-subdirectory]\n\n\n[#^ RError reference.Reply.members.Error]\n\nThis user's guide is an in-depth introduction to parsing with FParsec. It explains how `Parser` functions work, covers the most important parser combinators in detail, explains how you can customize error messages, and discusses some important practical aspects of parser writing, such as debugging and performance optimizations.\n\nThe aim of this user's guide is to prepare you for writing \"real world\" parser applications with FParsec. It doesn't try to cover every feature of the library, but focuses on covering the core concepts such that you can gain a deep understanding of the library design.\n\nAlthough there is some overlap between the @tutorial@ and this user's guide, it's probably a good idea to read the tutorial first, since it will give you a quick overview of the library that will later help you put things into perspective. You might also want to experiment with some small parsers before you start reading the user's guide, or maybe in parallel to reading it, so that it becomes easier for you to relate the dry technical discussions to exciting practical applications ☺\n\nThe first seven chapters of this user's guide build on each other. The remaining chapters are rather independent and can be read in any order.\n\n[section Parser functions]\n\nAn FParsec parser is a function that reads input from a text stream. When it succeeds, it returns a result value (e.g. a parsed number or an [url \"https://en.wikipedia.org/wiki/Abstract_syntax_tree\" AST] node); when it fails, it returns error messages describing what went wrong.\n\nThe following type abbreviation from the `Primitives` module defines the basic type of parser function supported throughout the FParsec library:\n``type Parser<'Result,'UserState> = CharStream<'UserState> -> Reply<'Result>``\n\nAs you can see from this definition, parser functions only accept a single argument: a `CharStream<'UserState>` instance. The `CharStream` class is FParsec's specialized stream type for \"text\" streams, i.e. streams of Unicode chars. A `CharStream` can either be created directly from a string or it can be created from a file path or `System.IO.Stream`. In the latter cases the `CharStream` will take care of decoding the binary input into UTF-16 chars, similar to what a `System.IO.StreamReader` does. What separates `CharStream` from the `StreamReader` and similar classes is that it comes with some advanced features that make it especially suitable for backtracking parser applications.\n\nWe will discuss the purpose of the `'UserState` type in more detail in later chapters. For now it's enough to note that the user state is a user-definable component of the `CharStream` state. If you don't need a user state, you will normally define `'UserState` to be `unit`. To save some key strokes and screen real estate, we usually abbreviate `'UserState` as `'u`.\n\nThe `Reply<'Result>` value returned from a parser function is a a simple value type container for the parser result and possible error messages. It contains a status field indicating whether the parser succeeded or not, a field for the result value (of type `'Result`) and a field with a possibly empty list of error messages. We will explain these fields in more details in [^internals-of-a-simple-parser-function].\n\nA very basic example of a parser is the `asciiLower` parser from the `CharParsers` module:\n``val asciiLower: Parser``\nIt parses any lower case ASCII char, i.e. any char in the range `'a'` - `'z'`, and, if successful, returns the parsed char as part of its reply.\n\nMany predefined parsers expect one or more parameter values as arguments. Take for instance the `skipString` function:\n``val skipString: string -> Parser``\nIt takes a string as an argument and returns a parser that skips over this (and only this) string in the input.\n\n[note\nImplementing parser grammars with FParsec usually means composing parsers for higher-level grammar rules from parsers for lower-level rules. You start with simple parsers for the leaf nodes of your grammar and then work your way up step-by-step until you eventually obtain a parser for the complete grammar. The simple representation of parsers as functions makes this composition particularly easy and allows for a straightforward and intuitive implementation of the library primitives.\n]\n\n[/section]\n\n[section Running parsers on input]\n\nWhile it is not difficult to construct a `CharStream` instance yourself, then apply a parser function to the `CharStream`, then interpret the returned `Reply` value and finally dispose the `CharStream` again, it takes less effort to instead use one of the several @`runParser...` functions@ from the `CharParsers` module.\n\nAmong the `runParser...` functions `run` is the most convenient for simple testing purposes:\n``val @run@: Parser<'a, unit> -> string -> ParserResult<'a,unit>``\n\n`run` applies the parser given as the first argument to a `CharStream` constructed from the string argument and then captures the return value as `ParserResult` value. The `ParserResult` type is a simple discriminated union that is a bit more convenient to interpret than the `Reply` values returned by `Parser` functions.\n\nFor example:\n``{fsi}\n> run pint32 \"0xff\";;\nval it : ParserResult = Success: 255\n\n> run pint32 \"0xgf\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 3\n0xgf\n ^\nExpecting: hexadecimal digit\n``\n\nThe text messages displayed in these examples after the `=` signs are the default string representations of the returned `ParserResult` values, just like they are printed in the [@ F# Interactive] console. The reference documentation describes the two union cases `Success` and `Failure` of the `ParserResult` type in more detail.\n\n`run` only supports parser functions with no user state, i.e. with a `unit` user state. If you want to test parsers that depend on a user state, you will need to use one of the other `runParser...` functions, e.g. `runParserOnString`. Please see the reference for more details on the @`runParser...` functions@.\n\nNote that the `runParser...` functions are primarily meant for the \"end-users\" of parsers, i.e. those users that apply an aggregate parser on the content of a complete input stream. This is a situation different from the one where you implement a `Parser` function yourself. In the latter case you typically work directly with the input `CharStream` and output `Reply` values.\n\n[/section]\n\n\n[section Internals of a simple `Parser` function]\n\nIn the beginning of this user's guide we noted that `asciiLower` \"parses\" a lower case ASCII char and that `skipString` \"skips\" over a string, but we haven't yet explained what it actually means for a `Parser` function to \"parse\" a letter or \"skip\" a string. That's what we will do in this chapter. To explain how `Parser` functions work, we will discuss the implementation of a simple string parser. This also gives us the opportunity to explain some important details about the `Reply` and `CharStream` types.\n\n[section The code]\nThe parser whose implementation we will discuss in this chapter is\n``\nval stringReturn: string -> 'a -> Parser<'a,'u>\n``\n\nLike `skipString str` the parser `stringReturn str result` skips over the string `str`, but it returns `result` as part of its reply value, instead of the `unit` value `()` that `skipString str` returns. This makes `stringReturn` a bit more general than `skipString`. Indeed, the two library parsers `pstring` and `skipString` are actually implemented with the help of `stringReturn`. For example, `skipString str` is defined as `stringReturn str ()`.\n\nA simplified version[fn The library version is a bit more complicated because it contains optimized paths for argument strings with only 1 or 2 chars.] of the actual implementation of `stringReturn` in the library is\n``\nlet [no-auto-link stringReturn] str result : Parser<_,_> = // 1\n checkStringContainsNoNewlineChar str \"pstring/skipString/stringReturn\" // 2\n let error = expectedString str // 3\n fun stream -> // 4\n if stream.Skip(str) then // 5\n Reply(result) // 6\n else // 7\n Reply(Error, error) // 8\n``\n\nLet's start with the general structure of this implementation: We define a function [no-auto-link `stringReturn`] with two parameters that returns a function closure. The type annotation `: Parser<_,_>` on /line 1/ fixes the type of the returned function closure to `Parser<'a,'u>` and in particular constrains the type of its argument to `CharStream<'u>`. Remember, the type `Parser<'a,'u>` is simply an abbreviation for `CharStream<'u> -> Reply<'a>`, where `'a` represents the result type and `'u` the user state type.\n\nImplementing our parameterized parser as a function returning a parser closure allows us to factor out common setup work that only needs to be done once for every parser.[fn Even parsers without a parameter, like e.g. `asciiLower`, are actually compiled as properties returning a new function object every time they are called. This is because the user state type variable makes `asciiLower` generic, while function values can only have a non-generic type.] In this case we only need to check once (/line 2/) whether the string contains a newline char, i.e. `'\\r'` or `'\\n'`, (we'll explain below why this is necessary) and in /line 3/ we preconstruct the error message that is later used whenever the parser is applied and doesn't find `str` in the input (we'll write more about error messages in later chapters).\n\nThe actual parsing logic is completely straightforward: On /line 5/ the parser calls the CharStream's `Skip` method with the argument `str`. If the next chars in the stream match `str`, `Skip` advances the stream's position by the length of the passed string and returns `true`; otherwise, it doesn't change the position of the stream and returns `false`. Thus, if the string is skipped, the parser returns with a `Reply` value containing the result (/line 6/). Otherwise, it returns a `Reply` with the preconstructed error message (/line 8/).\n\n[/section]\n\n[section The `Reply` type]\n\nThis is a good time to discuss the `Reply` type in a little more detail.\n\n``\ntype Reply<'TResult> = struct\n new: 'TResult -> Reply<'TResult>\n new: ReplyStatus * ErrorMessageList -> Reply<'TResult>\n new: ReplyStatus * 'TResult * ErrorMessageList -> Reply<'TResult>\n\n val mutable Status: ReplyStatus\n /// If Status <> Ok then the Result value is undefined and may be null.\n val mutable Result: 'TResult\n val mutable Error: ErrorMessageList\nend\n``\n\nSimilar to a tuple, a `Reply` can be seen as an aggregate of it three fields: `Status`, `Result` and [no-auto-link `Error`]. The `Status` field contains a `ReplyStatus` enum value indicating whether the parser succeeded (`Ok`) or failed (`Error` or `FatalError`). By returning a `FatalError` instead of an `Error` a parser can signal that no error recovery should be tried (except through backtracking mechanisms, which we explain later). If the `Status` is `Ok`, the `Result` field contains the parser result; otherwise, its value is undefined (and `null`). The `Error` field holds a list of error messages in the form of an `ErrorMessageList` value. An empty `ErrorMessageList` is represented as a `null` value.\n\nThe 1-argument constructor we use in /line 6/ sets the status to `Ok` and the result value to `result`. The 2-argument constructor we use in /line 8/ sets the status to `Error` and the error message to `error`. The `Reply` type also defines a 3-argument constructor, which simply sets the fields to the respective argument values. The default valuetype constructor with 0 arguments initializes the `Reply` value to `Reply(Error, null)`.\n\nThe error messages returned in the `Reply` value implicitly refer to the current stream position. Since the `ErrorMessage` values stored in the `ErrorMessageList` do not themselves contain an error position, they can only be interpreted together with the position of the `CharStream` as it is when the parser returns.\n\n[/section]\n\n[section The parser state and the line and column count]\n\nUsually one `CharStream<'u>` instance is created per input file and all parser functions involved in parsing elements of the same file are passed the same `CharStream` instance. Since calling the methods of a `CharStream` may change its state, parser functions have to be careful about when and how they change the `CharStream` state, because it obviously may affect all parsers subsequently called.\n\nIn the example above, `stringReturn` only advances the stream position when it succeeds. This makes it an *atomic* string parser, because it does not consume input if only the beginning of the argument string matches the input. Whether or not a parser consumes input before it fails has important implications for the error handling, as we will discuss later in this user's guide.\n\nExcept for the freely customizable `UserState`, all the mutable state information in the `CharStream<'u>` instance pertains to the location of the next char in the text stream. The most important element of the state is the char `Index`, which uniquely identifies the UTF-16 char in the stream. In addition to the index of the next char, the `CharStream` also keeps track of char's `Line` number and the index of the first char in the line, the `LineBegin`. By combining the `Index` and `LineBegin` we can calculate a `Column`. The `CharStream`'s `Name` serves as a description or identifier for the stream.\n\nOnly the char index is strictly necessary for the core stream functionality. We also store the other pieces of state information in a `CharStream<'u>` instance because having all parser state information in one place reduces complexity and allows us to expose a more convenient API to `Parser` functions.\n\n[note The `CharStream<'u>.State` property returns a snapshot of all the mutable state components in the form of a `CharStreamState<'u>` value.\n\nThe state information that is exposed through the `CharStream<'u>.State` property is *all* the state that is tracked by `FParsec` parsers, which is why we also refer to it as *the parser state*.[fn Strictly speaking, a `CharStream<'a>` instance has a little more publically observable mutable state than the one that is also exposed through the `State` property. For example, the `MinRegexSpace` configuration parameter is not tracked in the `State` parameter. Another example is the value of the `IndexOfLastCharPlus1` property which changes once the last char of the stream is detected. However, there shouldn't be a reason that a parser needs to restore the old values of these properties upon backtracking, so we just treat these properties as constant and ignore them when we discuss the mutable `CharStream` state.]\n]\n\nIdeally, the `CharStream` class would keep track of the column and line count in a completely automated fashion. Ideally, the `CharStream` class would give the user a way to freely specify the recognized set of newline character sequences and all `CharStream` methods then would automatically detect such newlines in the input. Unfortunately, such a configuration option would be difficult to implement efficiently and would likely have a severe impact on performance (at least in comparison to the hard-coded alternative, and with the current language and compiler support).\n\nSince the `CharStream` can't provide automatic support for all possible notions of a newline, it exposes two sets of methods in its interface. One set provides the basic stream operations, such as skipping a certain number of UTF-16 chars or matching a string with the stream content. These methods come without any automatic newline detection, but they offer optimal performance and give the user complete freedom to manually register any kind of newline. The other set of methods provides some frequently needed higher-level text operations, such as skipping over a sequence of whitespace chars or reading a sequence of chars satisfying a given predicate function. These other methods automatically detect any of the 3 standard newline char sequences `\"\\n\"`, `\"\\r\\n\"` and `\"\\r\"`, because that's the notion of a newline used by most text applications. In combination both sets of methods cover the needs of a majority of text parsers in a convenient and efficient manner.\n\n[note Maybe you wonder why we don't just leave the line and column count completely to the user instead of complicating the `CharStream` API. The reason we keep track of a line count in the `CharStream` class is that most non-trivial text-parsing applications require a line count for error reporting purposes. Implementing it at a relatively low API level brings significant performance advantages and relieves higher-level API users from constantly having to code around the special case of newline chars.]\n\nIf you have a look at the reference documentation for `CharStream`, you'll see that the `CharStream` methods that automatically detect newlines are easily discernible by their name. The `Skip` method we used in the above example does *not* belong to these methods, which is why we have to make sure in /line 2/ that the string doen't contain any newlines. In practice one hardly ever uses a parser like `stringReturn` with a string containing a newline, hence lifting this restriction wouldn't be worth the effort, especially since simple workarounds are available.[fn For example, `stringReturn \"str1\\nstr2\" result` can be replaced with `attempt (skipString \"str1\" >>. newline >>. stringReturn \"str2\" result)`.]\n[/section]\n\n[/section]\n[section Applying parsers in sequence]\n\n\nNow that we have discussed how `Parser` functions work, we can start explaining how FParsec's parser combinators work.\n\nIn this chapter we will discuss combinators that allow you to apply multiple parsers in sequence, i.e. parse the beginning of the input with the first parser, then parse the following input with the second parser, and so on.\n\n[section The definition of `>>.`]\n\nThe simplest combinators for sequentially applying two parsers are\n``\nval (>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\nval (.>>): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n``\n\nBoth operators take two parsers as arguments and return a combined parser that applies the two parsers in sequence. As you can infer from the type signatures, `p1 >>. p2` returns the result of `p2` and `p1 .>> p2` the result of `p1`. In each case the point points to the parser whose result is returned.\n\nIn order to explain exactly what it means to apply two parser in sequence, we give a full definition of the `>>.` operator:\n``\nlet [no-auto-link (>>.)] (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) =\n fun stream ->\n let reply1 = p1 stream\n if reply1.Status = Ok then\n let stateTag = stream.StateTag\n let mutable reply2 = p2 stream\n if stateTag = stream.StateTag then // (1)\n reply2.[^RError Error] <- mergeErrors reply1.[^RError Error] reply2.[^RError Error] // (2)\n reply2\n else // reconstruct error reply with new result type\n Reply(reply1.Status, reply1.[^RError Error])\n``\n\nThe implementation of `p1 >>. p2` should be quite self-explanatory: First `p1` is applied to the input `stream`. If `p1` succeeds, i.e. if the status of `reply1` is `Ok`, `p2` is applied to stream and the reply of `p2` then becomes the reply of the combined parser. However, if `p1` fails, its reply is immediately propagated as the reply of the combined parser. Since `reply1` has type `Reply<'a,'u>` but `p1 >>. p2` needs to return a `Reply<'b,'u>`, the error reply needs to be reconstructed with a new result type before it can be returned.\n[/section]\n\n[section Merging error messages]\n\nWe mentioned earlier that the error messages returned in the `Reply.[^RError Error]` field implicitly refer to the state of the `CharStream` at the time the parser returns. In particular the error messages refer to the then current *stream position*. Since the messages do not contain themselves a separate record of the error position they can only be interpreted together with the `CharStream` state.\n\nWhen `p2` does not change the parser state, the error messages from both replies refer to the state of the `CharStream` as it is when\n`p1 >>. p2` returns. Thus, the combinator needs to merge the (immutable) `ErrorMessageList`s from both replies, so that the returned list contains all the relevant error messages (see the line marked with `(2)`).\n\nIn order to check whether the `CharStream` state has changed, the combinator does not compare the full states from before and after `p2` is invoked. Instead it only compares the `StateTag` values (see line `(1)`). This improves performance and --- for most practical purpose --- is almost equivalent to comparing the full state, as we will discuss below.\n\n[note The way `[no-auto-link (>>.)]` handles errors and merges error messages is a template for all combinators in FParsec that perform multiple sequential parser invocations.]\n\nYou may wonder why the error messages get merged even though `p1` succeeded. The somewhat counterintuitive reason is that parsers can return nonempty error message lists even when they don't fail. For example, a parser that skips over the *optional* string `\"str\"` will return `Reply(Ok, (), expectedString \"str\")` if it doesn't find the string in the input. In this case the error message describes what further input the parser could have parsed at the current stream position. If subsequently a parser fails at the same position, all error messages for the same position can be aggregated to give the user as much information as possible about what went wrong and what alternative inputs could have been parsed at the given position.\n\nThe following sample demonstrates the helpful effect of this error handling behaviour:\n``\nlet str s = pstring s\n\nlet oneOrTwoInts =\n str \"(\" >>. tuple2 pint32 (opt (str \",\" >>. spaces >>. pint32)) .>> str \")\"\n``\n``{fsi}\n> run oneOrTwoInts \"(1 2)\";;\nval it : ParserResult<(int32 * int32 option),unit> = Failure:\nError in Ln: 1 Col: 3\n(1 2)\n ^\nExpecting: ')' or ','\n``\nThis error message wouldn't mention the possibility of a missing comma if the `.>>` combinator did not merge error messages for the same position when the left-hand side parser succeeds.\n\n[/section]\n\n[section The `StateTag`]\n\nParser combinators often need to check whether a parser has changed the `CharStream` state. In a typical FParsec application these checks are performed so frequently that an efficient implementation is important for the overall parser performance. Since a straightforward comparison of the complete `CharStream` states can be quite expensive, the `CharStream` class provides a shortcut for this purpose: the `StateTag`.\n\nThe `StateTag` is a simple integer counter that is incremented every time a `CharStream` method changes the state. Thus, if the `StateTag` hasn't changed, you can safely infer that the state hasn't changed either.[fn Of course, this doesn't apply if you manually set back the `StateTag` to the old value. There is also the purely theoretical possibility that the `StateTag` has overflown and was incremented exactly 2[sup 64] times (or 2[sup 32] if you define the `SMALL_STATETAG` conditional compiler symbol).] Except for some special cases, the opposite is also true: if the `StateTag` has changed, the state has changed too.\n\nIn the following special cases checking whether the `StateTag` has changed is not equivalent to checking whether the `CharStream` state has changed, because the tag may change even though the state doesn't:\n- A parser calls the basic `Skip` or `Read` methods with a 0 offset or an empty argument string.\n- A parser seeks the `CharStream` to the current position or replaces the user state with the current value.\n- A parser makes several calls to `CharStream` methods and in later calls undoes the changes it made in earlier calls.\n\nThe first and second cases only have practical relevance for generic or parameterized parsers and can be simply avoided by checking the arguments before calling the respective `CharStream` methods. The third case only arises in the context of backtracking and it too can be easily avoided, either by using the `BacktrackTo` method for backtracking or by manually restoring the `StateTag` after the backtracking.\n\nIn practice these special cases are extremely rare, usually without consequences for the parser behaviour and always easily avoidable. Hence, FParsec combinators make free use of the `StateTag` to check whether a parser has changed the `CharStream` state.\n\n[/section]\n\n[section Generalizing `>>.`]\n\nThe parsers `p1 .>> p2` and `p1 >>. p2` only return the results of `p1` and `p2` respectively. If you want to combine the results from both `p1` and `p2`, you could use the `pipe2` combinator instead:\n``val pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>``\n\nThe parser `pipe2 p1 p2 f` will apply `p1` and `p2` in sequence, exactly like `>>.`, but instead of returning one of the result values of `p1` and `p2` it will return the result of the function application `f x1 x2`, where `x1` and `x2` are the results returned by `p1` and `p2`.\n\nThere are also `pipe3`, `pipe4` and `pipe5` combinators, in case you need more than two arguments. Often these combinators are used to pass arguments to object constructors, like in the following example of a parser for a comma-separated list of XYZ coordinates:\n``\ntype Data = Point of float*float*float\n\nlet ws = spaces\nlet str s = pstring s .>> ws\nlet number = pfloat .>> ws\nlet point = pipe3 number (str \",\" >>. number) (str \",\" >>. number)\n (fun x y z -> Point(x, y, z))\n``\n``{fsi}\n> run point \"1, 2, 3\";;\nval it : ParserResult = Success: Point (1.0,2.0,3.0)\n``\n\nIf you just want to return the parsed values as a tuple, you can use the predefined `tuple2-5` parsers. For example, `tuple2 p1 p2` is equivalent to `pipe2 p1 p2 (fun x1 x2 -> (x1, x2)`.\n\n`tuple2` is also available under the operator name `.>>.`, so that you can write `p1 .>>. p2` instead of `tuple2 p1 p2`.\n\nThere is no `pipe1` combinator, but there is an operator for the same purpose:\n``\nval (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b, 'u>\n``\n\nThis operator is used similarly to the F#'s ubiquitous pipeline operator `|>`:\n``\ntype Expression = Number of int\n | Identifier of string\n\nlet number = pint32 |>> Number\n``\n``{fsi}\n> run number \"123\";;\nval it : ParserResult = Success: Number 123\n``\n[/section]\n\n[section The `>>=` combinator]\n\nAll the sequencing and piping combinators we have discussed so far could be implemented with the help of the \"bind\" combinator:\n``val (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>``\n\nInstead of two parsers this combinator takes a parser and a *function producing a parser* as arguments. The combined parser `p >>= f` first applies the parser `p` to the input, then it applies the function `f` to the result returned by `p` and finally it applies the parser returned by `f` to the input. If we knew in advance that `p` returns `x` then `p >>= f` would be equivalent to `p >>. (f x)`.\n\nThe `>>=` combinator is quite versatile. For example, the following code\nimplements five of the previously discussed combinators in terms of `>>=` and the trivial `preturn` primitive:\n\n``\nlet preturn x = fun stream -> Reply(x)\n\nlet (|>>) p f = p >>= fun x -> preturn (f x)\nlet (.>>) p1 p2 = p1 >>= fun x -> p2 >>= fun _ -> preturn x\nlet (>>.) p1 p2 = p1 >>= fun _ -> p2 >>= fun y -> preturn y\nlet (.>>.) p1 p2 = p1 >>= fun x -> p2 >>= fun y -> preturn (x, y)\nlet pipe2 p1 p2 f = p1 >>= fun x -> p2 >>= fun y -> preturn (f x y)\n``\n\nIn typical FParsec code `>>=` is only seldomly used, because in many situations where `>>=` could in principle be used one of the other specialized operators is more convenient to use and faster. However, on a conceptual level this combinator is important, because its generality allows us to define and test many combinators through their equivalence with a parser defined in terms of `>>=`.\nThis combinator is also significant for the role it plays in the monadic parser construction syntax, see [^where-is-the-monad].\n\n[/section]\n\n[/section]\n\n[section Parsing sequences]\n\nIn the previous chapter we discussed various ways to sequentially apply two or more parsers. In this section we will explain how to repeatedly apply the same parser in order to parse a sequence with an arbitrary number of elements.\n\n[section The `many` parser]\nIn regular expressions and many grammar formalisms a [url \"https://en.wikipedia.org/wiki/Kleene_star\" Kleene Star] marks a parser rule that can be repeatedly applied. For example, `number*` could represent a sequence of zero or more numbers.\n\nIn FParsec the `many` combinator takes the place of the Kleene Star:\n``val many: Parser<'a,'u> -> Parser<'a list, 'u>``\n\nWith `many` the number example could be translated into the following FParsec code:\n``\nlet ws = spaces\nlet number = pint32 .>> ws\n``\n``{fsi}\n> run (many number) \"1 2 3 4\";;\nval it : ParserResult = Success: [1; 2; 3; 4]\n``\n\nThe parser `many p` repeatedly applies the parser `p` until `p` fails, i.e. it \"greedily\" parses as many occurrences of `p` as possible. The results of `p` are returned as a list in the order of occurrence.\n\nAt the end of a sequence parsed with `many p` the argument parser `p` must fail without consuming input (or changing the parser state in any other way). When `p` fails after consuming input, `many p` fails with the error returned by `p`.\n\nThe following example illustrates this behaviour:\n\n``\nlet ws = spaces\nlet str s = pstring s\nlet numberInBrackets = str \"[\" >>. pint32 .>> str \"]\" .>> ws\n``\n\nThe `many numberInBrackets` parser successfully parses the first two numbers in this test run:\n``{fsi}\n> run (many numberInBrackets .>> str \"(c)\") \"[1] [2] (c)\";;\nval it : ParserResult = Success: [1; 2]\n``\n\nHowever, the same parser fails while trying to parse the 3rd number in this test run:\n``{fsi}\n> run (many numberInBrackets >>. str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 10\n[1] [2] [c]\n ^\nExpecting: integer number (32-bit, signed)\n``\n\nThe `many` parser failed here because the `numberInBrackets` parser failed *after consuming input*. In the chapter on @looking ahead and backtracking@ we'll come back to this example and discuss how you can modify the `numberInBrackets` parser such that it fails without consuming input if an opening bracket is not followed by a number.[fn `many` doesn't automatically backtrack when the argument parser fails after changing the parser state for two reasons:\n- In most situations automatic backtracking would only obscure error messages, because the reported input error was indeed severe and backtracking would only trigger secondary error messages that detract from the main error.\n- In the few instances where you rely on backtracking behaviour you can easily introduce it using the combinators detailed in [^looking-ahead-and-backtracking]. Marking the occasions where you rely on backtracking with these combinators makes your parser implementations easier to debug and optimize.\n]\n\nSince `many p` continues until `p` fails, you have to be a little careful not to supply an argument parser `p` that can succeed without consuming input. The following example shows what happens if you accidentally supply such an argument parser:\n\n``{fsi}\n> run (many (many digit .>> ws)) \"123 456\";;\nSystem.InvalidOperationException: (Ln: 1, Col: 8): The combinator 'many' was\napplied to a parser that succeeds without consuming input and without changing\nthe parser state in any other way. (If no exception had been raised, the\ncombinator likely would have entered an infinite loop.)\n (... stack trace ...)\nStopped due to error\n``\n\nThe problem here is that `many digit .>> ws` will succeed without changing the parser state if it can't parse any digits or trailing whitespace. Thus, if the combined parser hadn't have thrown an exception, it would have entered an infinite loop at the end of the input.\n\nWe can easily avoid the error in the last example by requiring the inner parser to consume at least one digit. Instead of `many digit`, which succeeds with an empty list if can't parse any digits, we can use `many1 digit`, which fails if it can't parse at least one digit:\n``{fsi}\n> run (many (many1 digit .>> ws)) \"123 456\";;\nval it : ParserResult =\n Success: [['1'; '2'; '3']; ['4'; '5'; '6']]\n``\n\nBefore we continue, we should point out that an example like `many1 digit` is somewhat artificial, because you hardly ever want to parse digit chars into a list. If you want to parse numbers, one of the [^parsing-numbers number parsers] is usually the best way forward. If you actually need the individual chars, you normally need them as a string, not as a list.\n\n[tip\nIf you want to parse a sequence of chars, you should generally prefer one of the specialized [^parsing-strings-directly string parsers].\n]\n\nIf you just want to skip over a sequence and don't need the list of parser results, you can use the optimized combinators `skipMany` or `skipMany1`.\n\n[/section]\n\n[section `sepBy` and `sepEndBy`]\n\nOften the elements of a sequence are separated by some separator. A convenient way to parse such sequences are the `sepBy` and `sepEndBy` combinators.\n\n`sepBy p sep` parses a sequence of `p` separated by `sep` and returns the results in a list. `sepEndBy` parses a sequence of `p` separated *and optionally ended* by `sep`.\n\nWith these combinators you could for example define the following two parsers for a semicolon-separated list of numbers in brackets:\n``\nlet str s = pstring s\nlet sepList = between (str \"[\") (str \"]\") (sepBy pint32 (str \";\"))\nlet sepEndList = between (str \"[\") (str \"]\") (sepEndBy pint32 (str \";\"))\n``\n\nThe `sepList` parser only accepts lists where the semicolons only occur between two numbers:\n``{fsi}\n> run sepList \"[]\";;\nval it : ParserResult = Success: []\n> run sepList \"[1;2;3]\";;\nval it : ParserResult = Success: [1; 2; 3]\n> run sepList \"[1;2;3;]\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 8\n[1;2;3;]\n ^\nExpecting: integer number (32-bit, signed)\n``\n\nThe `sepEndList` parser also accepts a terminating semicolon:\n``{fsi}\n> run sepEndList \"[1;2;3]\";;\nval it : ParserResult = Success: [1; 2; 3]\n> run sepEndList \"[1;2;3;]\";;\nval it : ParserResult = Success: [1; 2; 3]\n``\n\nLike for the `many` combinator, there are also variants of the `sepBy` and `sepEndBy` parsers that require at least one element in the sequence and/or skip over a sequence without returning the results. Have a look at the [^parsing-sequences parser overview].\n\n[/section]\n\n[section Parsing a sequence without creating a list]\n\nIf you want to parse a sequence and you don't need the results as an F# list, you can avoid the allocation of a temporary list by defing a custom sequence parser using the inline helper methods `Inline.Many` and `Inline.SepBy`.\n\nFor example, if you wanted to define a variant of `many` that parses the elements directly into a `ResizeArray`, i.e. a `System.Collections.Generic.List`, you could use the following definition:\n\n``\nlet manyRA p =\n // the compiler expands the call to Inline.Many to an optimized sequence parser\n Inline.Many(elementParser = p,\n stateFromFirstElement = (fun x0 ->\n let ra = ResizeArray<_>()\n ra.Add(x0)\n ra),\n foldState = (fun ra x -> ra.Add(x); ra),\n resultFromState = (fun ra -> ra),\n resultForEmptySequence = (fun () -> ResizeArray<_>()))\n``\n\nA test run:\n``{fsi}\n> run (manyRA (pint32 .>> spaces)) \"1 2 3\";;\nval it : ParserResult,unit> =\n Success: seq [1; 2; 3]\n``\n\nThe reference documentation for the `Inline` class contains some more examples.\n[/section]\n\n[/section]\n\n[section Parsing alternatives]\nFParsec's main operator for trying to parse input with alternative parsers is\n``\nval (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>\n``\n\nThis operator implements a form of *prioritized choice*: it only tries to parse input with the second parser if the first parser fails.\n\nThe following example illustrates this behaviour:\n``\ntype Char = AsciiChar of char\n | Char of char\n\nlet asciiLetter = asciiLetter |>> AsciiChar\nlet letter = letter |>> Char\n``\n``{fsi}\n> run (asciiLetter <|> letter) \"a\";;\nval it : ParserResult = Success: AsciiChar 'a'\n> run (letter <|> asciiLetter) \"a\";;\nval it : ParserResult = Success: Char 'a'\n> run (asciiLetter <|> letter) \"ä\";;\nval it : ParserResult = Success: Char 'ä'\n``\n\nThe prioritized choice also implies that FParsec doesn't enforce a longest-match rule like in regular expressions:\n``{fsi}\n> run (pstring \"a\" <|> pstring \"ab\") \"ab\";;\nval it : ParserResult = Success: \"a\"\n``\n\nIf you want to accept more than two alternatives, you can either chain multiple `<|>` operators, like in `p1 <|> p2 <|> p3`, or you can use the `choice` combinator, which accepts a sequence of parsers as the argument, like in `choice [p1; p2; p3]`. In both cases the argument parsers are tried from left to right until a parser succeeds.\n\nA good understanding of the `<|>` operator is important for productively working with FParsec, so let's have a look at its implementation:\n\n``\nlet (<|>) (p1: Parser<'a,'u>) (p2: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let mutable reply = p1 stream\n if reply.Status = Error && stateTag = stream.StateTag then\n let error1 = reply.[^RError Error]\n reply <- p2 stream\n if stateTag = stream.StateTag then\n reply.[^RError Error] <- mergeErrors reply.[^RError Error] error1\n reply\n``\n\nAs you can see, the parser `p1 <|> p2` works as follows: First, it applies the parser `p1` to the input stream. If `p1` succeeds, the reply of `p1` is returned. If `p1` fails with a non-fatal error (i.e. with the status `Error`, not `FatalError`) and *without changing the parser state*, the parser `p2` is applied. If `p2` does not change the parser state, the error messages from both parsers are merged. (We compare the `StateTag` values instead of the actual parser states for optimization reasons, see [^applying-parsers-in-sequence.the-statetag].)\n\n\nThe most important point to note here is that `p1 <|> p2` will always return with the reply of `p1` if `p1` changes the parser state, even if `p1` eventually fails. Remember that the stream position is part of the parser state, so if `p1` fails after consuming input, `p2` will not be applied. Since a parser usually consumes input as soon as it can accept at least one atomic token from the input, this means that `p1 <|> p2` by default implements backtracking with only a \"one token look-ahead\".\n\nConsider the following example:\n``\nlet parserA = spaces >>. pstring \"a\"\nlet parserB = spaces >>. pstring \"b\"\nrun (parserA <|> parserB) \" b\";;\n``\n``{fsi}\n> run (parserA <|> parserB) \" b\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 2\n b\n ^\nExpecting: 'a'\n``\n\nThe combined parser fails because `parserA` fails after consuming the whitespace, so that `parserB` never gets tried.\n\nOf course, this simple parser could be easily fixed by factoring out the common prefix:\n``{fsi}\n> run (spaces >>. (pstring \"a\" <|> pstring \"b\")) \" b\";;\nval it : ParserResult = Success: \"b\"\n``\n\nThe restriction of the look-ahead in `p1 <|> p2` may strike you as odd at first, but it has two big advantages:\n1) The error reporting is simplified and error messages are easier to understand because terminal errors can only occur at one position at a time.\n2) Parser developers are guided towards more efficient grammar implementations because parsers requiring more than a one token look-ahead need to be explicitly annotated with the `attempt` or `>>?` combinators (see the [^looking-ahead-and-backtracking next chapter]).[fn In case you're wondering: No, we're not trying to sell a design limitation as a feature here. In Parsec, the Haskell library on which FParsec's design was originally based, the limited look-ahead is essential for the library design, because it allows Parsec to exploit Haskell's laziness in order to ensure space efficiency. FParsec has a different implementation in which the limited look-ahead has [^block-wise no effect on space efficiency]. We stick to the limited look-ahead because we think it's the appropriate default behaviour for a parser combinator library like FParsec. Now, admittedly, if FParsec could automatically optimize the implementation of a parser in a way that minimized backtracking, e.g. by automatically left-factoring grammars, then backtracking would be less of a problem and a different default behaviour might become more attractive.]\n\n[/section]\n\n[section Looking ahead and backtracking]\n\n[section Backtracking]\nSometimes you need more than the default one token look-ahead of `<|>`, either because it really can't be avoided or because avoiding it would be too inconvenient. In those instances you can use one of the combinators `attempt`, `>>?`, `.>>?` or `>>=?` to force a parser to backtrack after an error.\n\nThe `attempt` combinator\n``\nval attempt: Parser<'a,'u> -> Parser<'a,'u>\n``\ntakes a parser as the argument and returns a wrapped parser that behaves exactly like the argument, except that if the argument parser fails with an output state different from the input state or with a fatal error, the wrapped parser will backtrack to the original input state and report a non-fatal error.\n\nYou can observe the effect of the `attempt` combinator in the following error message:\n``{fsi}\n> run (attempt (pstring \"a\" >>. pstring \"b\")) \"ac\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\nac\n^\n\nThe parser backtracked after:\n Error in Ln: 1 Col: 2\n ac\n ^\n Expecting: 'b'\n``\n\nThe next example demonstrates the effect of `attempt` on the choice combinator.\n\n``\nlet str s = pstring s\nlet ab = str \"a\" .>>. str \"b\"\nlet ac = str \"a\" .>>. str \"c\"\n``\n\nWithout `attempt` the following test produces an error:\n``{fsi}\n> run (ab <|> ac) \"ac\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 2\nac\n ^\nExpecting: 'b'\n``\n\nBy introducing `attempt` we allow the `<|>` combinator to recover from the error in the first branch:\n``{fsi}\n> run ((attempt ab) <|> ac) \"ac\";;\nval it : ParserResult<(string * string),unit> = Success: (\"a\", \"c\")\n``\n\nSometimes it can be a disadvantage that `attempt` will trigger backtracking after any error returned by the argument parser, no matter how much content the parser has consumed. Consider for example a parser like `prefix >>. expr`, where `expr` is a parser for a potentially large and deeply nested expression. If you wrap this parser with `attempt` then the wrapped parser will not only backtrack if an error occurs within the prefix or directly after the prefix, but also if it occurs anywhere in the expression. However, in most cases you only want the parser to backtrack if the error occurs directly after the prefix, not if the error occurs deeply inside the expression parser. For situations like this FParsec defines the `>>?`, `.>>?`, `.>>.?` and `>>=?` operators.\n\nThe `>>?` combinator\n``\nval (>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n``\nbehaves like the `>>.` operator, except that `p1 >>? p2` will backtrack to the beginning if `p2` fails with a non-fatal error and without changing the parser state, even if `p1` has changed the parser state. Similarly, `.>>?`, `.>>.?` and `>>=?` behave like `.>>`, `.>>.` and `>>=`, except that they will backtrack to the beginning if the second parser fails with a non-fatal error and without changing the parser state\n\nThe following tests illustrate the differences between backtracking implemented via `attempt` and `.>>.?`.\n\n``\nlet bInBrackets = str \"[\" >>. str \"b\" .>> str \"]\"\n``\n\nA test with `attempt` on the left side of `<|>`:\n``{fsi}\n> run ((attempt (str \"a\" .>>. bInBrackets)) <|> ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 2\na[B]\n ^\nExpecting: 'c'\n``\n\nA test with `attempt` on both sides of `<|>`:\n``{fsi}\n> run ((attempt (str \"a\" .>>. bInBrackets)) <|> attempt ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 1\na[B]\n^\n\nThe parser backtracked after:\n Error in Ln: 1 Col: 2\n a[B]\n ^\n Expecting: 'c'\n\nThe parser backtracked after:\n Error in Ln: 1 Col: 3\n a[B]\n ^\n Expecting: 'b'\n``\n\nA test with `.>>.?` instead of `attempt` on the left side of `<|>`:\n``{fsi}\n> run (str \"a\" .>>.? bInBrackets <|> ac) \"a[B]\";;\nval it : ParserResult<(string * string),unit> = Failure:\nError in Ln: 1 Col: 3\na[B]\n ^\nExpecting: 'b'\n``\n\nYou can of course chain multiple of the `>>?` and `.>>?` operators to backtrack longer distances, like in `prefix1 >>? prefix2 >>? p .>>? postfix`.\n\n[note\nWhen implementing backtracking parsers you should generally prefer the `>>?`, `.>>?` and `.>>.?` combinators to the `attempt` combinator, because the former combinators offer finer control over the exact backtracking behaviour and hence will often lead to better error reporting. Note however that neither can completely replace the other.\n]\n\nBacktracking combinators can also be useful when parsing sequences. In the chapter \"Parsing sequences\" we briefly discussed the following example:\n\n``\nlet ws = spaces\nlet str s = pstring s\nlet numberInBrackets = str \"[\" >>. pint32 .>> str \"]\" .>> ws\n``\n``{fsi}\n> run (many numberInBrackets >>. str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 10\n[1] [2] [c]\n ^\nExpecting: integer number (32-bit, signed)\n``\n\nThe problem here is that the argument parser to `many` fails after consuming input if it encounters a bracket that is not followed by a digit. If we decided that this is a defect of the parser as opposed to the grammar, we could fix it by simply replacing a `>>.` with `>>?`.\n\n``\nlet numberInBrackets = str \"[\" >>? pint32 .>> str \"]\" .>> ws\n``\n``{fsi}\n> run (many numberInBrackets .>> str \"[c]\") \"[1] [2] [c]\";;\nval it : ParserResult = Success: [1; 2]\n``\n\nA similar example is the `sepEndBy1` combinator for parsing a sequence of one or more elements separated and optionally ended by a separator. If FParsec didn't provide this combinator, you could define it yourself using `many` and `>>?`:\n``\nlet sepEndBy1_ p sep =\n pipe2 p (many (sep >>? p)) (fun hd tl -> hd::tl) .>> opt sep\n``\n\nThe following tests show that our `sepEndBy1` replacement works as expected:\n``{fsi}\n> run (sepEndBy1_ pint32 (str \";\")) \"1;2;3\";;\nval it : ParserResult = Success: [1; 2; 3]\n> run (sepEndBy1_ pint32 (str \";\")) \"1;2;3;\";;\nval it : ParserResult = Success: [1; 2; 3]\n``\n\nNote however that in contrast to `sepEndBy1_` the version of `sepEndBy1` provided by FParsec doesn't need to parse the separator twice when it terminates a sequence.\n\n[/section]\n[section Parser predicates]\n\nThe backtracking combinators allow you to \"look ahead\" by tentatively parsing input and then backtracking if an error occurs. However, they don't allow you to conditionally parse the input with one parser depending on the success or failure of another parser. This is what the following two combinators are for:\n\n``\nval followedBy: Parser<'a,'u> -> Parser\nval notFollowedBy: Parser<'a,'u> -> Parser\n``\n\nThe parser `followedBy p` (`notFollowedBy p`) succeeds *without changing the parser state* if `p` succeeds (fails) when applied at the current position.\n\nFor example, both the following parser definitions only parse positive integer literals without a leading zero:\n``\nlet p1 = followedBy (satisfy ((<>) '0')) >>. pint32\nlet p2 = notFollowedBy (pstring \"0\") >>. pint32\n``\n\nBoth definitions will correctly parse `\"123\"` and fail to parse `\"01\"`:\n``{fsi}\n> run p1 \"123\";;\nval it : ParserResult = Success: 123\n> run p1 \"01\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnknown Error(s)\n> run p2 \"123\";;\nval it : ParserResult = Success: 123\n> run p2 \"01\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnknown Error(s)\n``\n\nWhile both parsers work as expected, the generated error messages aren't very helpful. The problem is that `followedBy` and `notFollowedBy` can't generate better error messages, because they don't know what kind of input their argument parsers accept.[fn In the case of `notFollowedBy p` the problem is clear: `notFollowedBy p` fails if `p` succeeds and when `p` succeeds, `p` doesn't generate an error message that `notFollowedBy` could reuse. In the case of `followedBy p` the situation is different: `followedBy p` fails if `p` fails, so `followedBy` could try to reuse the error messages generated by `p`. However, the error messages generated by the argument parser will in practice often not suffice to explain what kind of input is expected. So, for reasons of consistency and performance, `followedBy` doesn't even try to reuse the error messages generated by the argument parser.]\nTo improve the error messages you can either use the \"labeled\" combinator variants `followedByL` and `notFollowedByL` or you could use the labelling operator `` that we will discuss in the next chapter.\n\nFor example:\n\n``\n> run (followedByL (satisfy ((<>) '0')) \"positive int w/o leading 0\" >>. pint32)\n \"01\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n01\n^\nExpecting: positive int w/o leading 0\n\n> run (followedBy (satisfy ((<>) '0')) >>. pint32 \"positive int w/o leading 0\")\n \"01\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n01\n^\nExpecting: positive int w/o leading 0\n\n> run (notFollowedByL (pstring \"0\") \"'0'\" >>. pint32) \"01\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n01\n^\nUnexpected: '0'\n``\n\nThe parser `notFollowedByL (pstring \"0\") \"'0'\"` from the last example could actually be simplified to `notFollowedByString \"0\"`, which uses the specialized parser predicate `notFollowedByString`. In [^reference.parser-overview.conditional-parsing-and-looking-ahead] you'll find an overview of all available parser predicates.\n\nA frequent application for the `notFollowedBy` predicate are sequence parsers similar to `many (notFollowedBy pEnd >>. p) .>> pEnd`. If you are writing such a parser, you should check whether you can replace it with an application of one of the [^manyTill-parsers `manyTill` parsers]. Please consult the reference for more details.\n\nBefore we conclude this chapter we want to emphasize that you're not limited to the built-in (backtracking) combinators of FParsec. A great advantage of FParsec is the simplicity with which you can write custom combinators using the low-level API.\n\nFor example, you could define a combinator that backtracks if the result of the argument parser doesn't satisfy a predicate function:\n\n``\nlet resultSatisfies predicate msg (p: Parser<_,_>) : Parser<_,_> =\n let error = messageError msg\n fun stream ->\n let state = stream.State\n let reply = p stream\n if reply.Status <> Ok || predicate reply.Result then reply\n else\n stream.BacktrackTo(state) // backtrack to beginning\n Reply(Error, error)\n``\n\nWith this combinator you could conveniently define a parser for positive ints:\n\n``\nlet positiveInt = pint32 |> resultSatisfies (fun x -> x > 0)\n \"The integer must be positive.\"\n``\n``{fsi}\n> run positiveInt \"1\";;\nval it : ParserResult = Success: 1\n> run positiveInt \"-1\";;\nError in Ln: 1 Col: 1\n-1\n^\nThe integer must be positive.\n``\n\n[/section]\n\n[/section]\n\n[section Customizing error messages]\n\nGenerating relevant and informative parser error messages is one of FParsec's greatest strengths. The top-down approach of recursive-descent parsing guarantees that there is always enough context to describe the exact cause of a parser error and how it could be avoided. FParsec exploits this context to automatically generate descriptive error messages whenever possible. This chapter explains how you can ensure with minimal efforts that your parser always produces understandable error messages.\n\nAs we already described in detail in [^applying-parsers-in-sequence.merging-error-messages], error reporting in FParsec is based on the following two principles:\n- Parsers that fail or could have consumed more input return as part of their `Reply` an `ErrorMessageList` describing the input they expected or the reason they failed.\n- Parser combinators aggregate all error messages that apply to the same input position and then propagate these error messages as appropriate.\n\nThe various error messages in the previous chapters demonstrate that the built-in error reporting usually works quite well even without any intervention by the parser author. However, sometimes FParsec lacks the information necessary to produce an informative error message by itself.\n\nConsider for example the `many1Satisfy f` parser, which parses a string consisting of one or more chars satisfying the predicate function `f`. If this parser fails to parse at least one char, the generated error is not very helpful:\n``{fsi}\n> run (many1Satisfy isLetter) \"123\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n123\n^\nUnknown Error(s)\n``\n\nThe problem here is that `many1Satisfy` can't describe what chars the function predicate accepts. Hence, when you don't use `many1Satisfy` as part of a combined parser that takes care of a potential error, you better replace it with `many1SatisfyL`, which allows you to describe the accepted input with a label (hence the \"L\"):\n\n``{fsi}\n> run (many1SatisfyL isLetter \"identifier\") \"123\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: identifier\n``\n\nThere are also labelled variants of other parsers and combinators, for example `choiceL` and `notFollowedByL`.\n\nIf there is no labelled parser variant or you want to replace a predefined error message, you can always use the labelling operator\n``\nval (): Parser<'a,'u> -> string -> Parser<'a,'u>\n``\n\nThe parser `p label` behaves like `p`, except that the error messages are replaced with `expectedError label` if `p` does not change the parser state (usually because `p` failed).\n\nFor example, if FParsec didn't provide `many1SatisfyL`, you could define it yourself as\n``\nlet many1SatisfyL f label = many1Satisfy f label\n``\n\nThe labelling operator is particularly useful for producing error messages in terms of higher-level grammar productions instead of error messages in terms of lower-level component parsers. Suppose you want to parse a string literal with the following parser\n``\nlet literal_ = between (pstring \"\\\"\") (pstring \"\\\"\")\n (manySatisfy ((<>) '\"'))\n``\n\nIf this parser encounters input that doesn't start with a double quote it will fail with the error message produced by the parser for the opening quote:\n``{fsi}\n> run literal_ \"123\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: '\"'\n``\n\nIn situations like these an error message that mentions the aggregate thing you're trying to parse will often be more helpful:\n``\nlet literal = literal_ \"string literal in double quotes\"\n``\n``{fsi}\n> run literal \"123\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n123\n^\nExpecting: string literal in double quotes\n``\n\nNote that `` only replaces the error message if the parser doesn't consume input. For example, our `literal` parser won't mention that we're trying to parse a string literal if it fails after the initial double quote:\n``{fsi}\n> run literal \"\\\"abc def\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 9\n\"abc def\n ^\nNote: The error occurred at the end of the input stream.\nExpecting: '\"'\n``\n\nWith the compound labelling operator `` you can make sure that the compound gets mentioned even if the parser fails after consuming input:\n``\nlet literal = literal_ \"string literal in double quotes\"\n``\n``{fsi}\n> run literal \"\\\"abc def\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n\"abc def\n^\nExpecting: string literal in double quotes\n\nstring literal in double quotes could not be parsed because:\n Error in Ln: 1 Col: 9\n \"abc def\n ^\n Note: The error occurred at the end of the input stream.\n Expecting: '\"'\n``\n\n[tip\nIf you don't like the formatting of these error messages, you can write a custom formatter for your application. The data structure in which error messages are stored is easy to query and process. See the reference for the [^FParsec..Error `Error` module].\n]\n\nThe parsers we discussed so far in this chapter only generated `Expected` error messages, but FParsec also supports other type of error messages. For example, the `notFollowedByL` parser generates `Unexpected` error messages:\n``{fsi}\n> run (notFollowedByL spaces \"whitespace\") \" \";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\n\n^\nUnexpected: whitespace\n``\n\nError messages that don't fit into the `Expected` and `Unexpected` categories can be produced with the `fail` and `failFatally` primitives:\n\n``\nlet theory =\n charsTillString \"3) \" true System.Int32.MaxValue\n >>. (pstring \"profit\" <|> fail \"So much about that theory ... ;-)\")\n\nlet practice = \"1) Write open source library 2) ??? 3) lot's of unpaid work\"\n\n``\n``{fsi}\n> run theory practice;;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 40\n1) Write open source library 2) ??? 3) lot's of unpaid work\n ^\nExpecting: 'profit'\nOther error messages:\n So much about that theory... ;-)\n``\n\nIf you can't get the built-in operators and parsers to produce the error message you need, you can always drop down one API level and write a special-purpose parser combinator.\n\nThe following example shows how you can define a custom `between` combinator that includes the position of the opening delimiter as part of the error message that gets generated when the closing delimiter cannot be parsed.\n\n``\nlet betweenL (popen: Parser<_,_>) (pclose: Parser<_,_>) (p: Parser<_,_>) label =\n let expectedLabel = expected label\n let notClosedError (pos: Position) =\n messageError (sprintf \"The %s opened at %s was not closed.\"\n label (pos.ToString()))\n fun (stream: CharStream<_>) ->\n // The following code might look a bit complicated, but that's mainly\n // because we manually apply three parsers in sequence and have to merge\n // the errors when they refer to the same parser state.\n let state0 = stream.State\n let reply1 = popen stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = p stream\n let error2 = if stateTag1 <> stream.StateTag then reply2.[^RError Error]\n else mergeErrors reply1.[^RError Error] reply2.[^RError Error]\n if reply2.Status = Ok then\n let stateTag2 = stream.StateTag\n let reply3 = pclose stream\n let error3 = if stateTag2 <> stream.StateTag then reply3.[^RError Error]\n else mergeErrors error2 reply3.[^RError Error]\n if reply3.Status = Ok then\n Reply(Ok, reply2.Result, error3)\n else\n Reply(reply3.Status,\n mergeErrors error3 (notClosedError (state0.GetPosition(stream))))\n else\n Reply(reply2.Status, reply2.[^RError Error])\n else\n let error = if state0.Tag <> stream.StateTag then reply1.[^RError Error]\n else expectedLabel\n Reply(reply1.Status, error)\n``\n\nThe behaviour of the `betweenL` combinator differs from that of the standard `between` combinator in two ways:\n- If `popen` fails without changing the parser state, `betweenL popen p pclose label` fails with `expected label`, just\n like `between popen p pclose label` would have.\n- If `pclose` fails without changing the parser state, `betweenL` additionally prints the opening position of the compound.\n\nThe following tests demonstrate this behaviour:\n\n``\nlet stringLiteral = betweenL (str \"\\\"\") (str \"\\\"\")\n (manySatisfy ((<>) '\"'))\n \"string literal in double quotes\"\n``\n``{fsi}\n> run stringLiteral \"\\\"test\\\"\";;\nval it : ParserResult = Success: \"test\"\n\n> run stringLiteral \"\\\"test\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 6\n\"test\n ^\nNote: The error occurred at the end of the input stream.\nExpecting: '\"'\nOther messages:\n The string literal in double quotes opened at (Ln: 1, Col: 1) was not closed.\n\n> run stringLiteral \"test\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 1\ntest\n^\nExpecting: string literal in double quotes\n``\n\n[/section]\n\n[section Parsing with user state]\n\nEach `[^CharStream_1 CharStream<'u>\\ ]` holds a value of the freely definable user state type `'u`. In previous chapters we just ignored the user state and always assumed `'u` to be `unit`. In this section we finally get to discuss the purpose of the user state and how you can use it in your parsers.\n\n[section Overview]\nThe user state allows you to introduce additional variables into the state tracked by FParsec parsers. It has the following two important properties:\n- The user state is stored in the `[^CharStream_1 CharStream<'u>\\ ]` instance and hence associated with the input. It is not shared globally and not associated with particular parser instances. The same parser instances can be concurrently applied to different `[^CharStream_1 CharStream<'u>\\ ]` instances with different user state instances.\n- The user state is tracked by FParsec parsers together with the input stream position. This means in particular that a parser restores the previous user state value when it backtracks.\n\n[important\nIf you want changes to the user state to be undone during backtracking, you must change the user state by assigning a new value to the user state, not by mutating an existing user state value.\n]\n\nWith the help of the user state you can implement context sensitive parsers, i.e. parsers whose behaviour not only depends on the immediate input but also on the context of the input. In general this works as follows:\n# You establish a context by defining variables in the user state.\n# You update the context depending on the input by letting parsers update the user state.\n# You parse input depending on the context by making the parser behaviour dependent on the user state variables.\n\nThe user state is exposed through the `UserState` property of the `[^CharStream_1 CharStream<'u>\\ ]`. You can implement parsers using the low-level API that directly access this property, or you can use the following parser primitives from the `CharParsers` module:\n- `getUserState`,\n- `setUserState`,\n- `updateUserState`,\n- `userStateSatisfies`.\n\nThe next section contains an example employing `updateUserState` to change the user state and `userStateSatisfies` to check for parser preconditions.\n\n[/section]\n[section Recursive grammars with nesting restrictions]\n\nAn important area of application for context sensitive parsers are recursive grammars where certain grammar elements cannot nest within others or where grammar elements need to be parsed differently depending on the nesting context.\n\nConsider for example a textual markup languages like HTML. Many such markup languages support various \"inline tags\" to annotate text in a paragraph. Usually these inline tags can nest arbitrarily, except for a few tags with special restrictions. One of these restrictions often is that hyperlinks must not contain hyperlinks, even though they can contain any other inline content. Other restrictions may apply to elements allowed in superscript text or footnotes. A convenient way to enforce such restrictions during parsing is to introduce variables into the user state that keep track of the nesting context. The following example demonstrates this approach.[fn An alternative way to handle such restrictions at the parser level would be to define separate instances of the parser for each possible combination of restrictions, e.g. separate parsers for inline elements at the top level, for inline elements within hyperlinks, for elements within hyperlinks within superscript text and so on. However, with an increasing number of restrictions this approach quickly falls victim to the combinatorial explosion caused by the recursive nature of the involved parsers.]\n\nThe following parser for a tiny markup-language employs the user state\n# to ensure that nested hyperlinks are not accepted and\n# to parse potentially nested quotations between matching pairs of `'\\''` or `'\\\"'` chars.\n\n``\nopen FParsec\n\ntype Element = Text of string\n | Bold of Element list\n | Italic of Element list\n | Url of string * Element list\n | Quote of char * Element list\n\ntype UserState =\n {InLink: bool\n QuoteStack: char list}\n with\n static member Default = {InLink = false; QuoteStack = []}\n\nlet ws = spaces\nlet ws1 = spaces1\nlet str s = pstring s\n\nlet elements, elementsR = createParserForwardedToRef()\n\nlet text = many1Satisfy (isNoneOf \"<>'\\\"\\\\\") |>> Text\nlet escape = str \"\\\\\" >>. (anyChar |>> (string >> Text))\n\nlet quote (q: char) =\n let pq = str (string q)\n\n let pushQuote =\n updateUserState (fun us -> {us with QuoteStack = q::us.QuoteStack})\n\n let popQuote =\n updateUserState (fun us -> {us with QuoteStack = List.tail us.QuoteStack})\n\n let isNotInQuote =\n userStateSatisfies (fun us -> match us.QuoteStack with\n | c::_ when c = q -> false\n | _ -> true)\n\n isNotInQuote >>. between pq pq\n (between pushQuote popQuote\n (elements |>> fun ps -> Quote(q, ps)))\n\n// helper functions for defining tags\n\nlet tagOpenBegin tag =\n str (\"<\" + tag)\n >>? nextCharSatisfiesNot isLetter // make sure tag name is complete\n \"<\" + tag + \"> tag\"\n\nlet tagOpen tag = tagOpenBegin tag >>. str \">\"\nlet tagClose tag = str (\"\")\n\nlet tag t p f =\n between (tagOpen t) (tagClose t)\n (p |>> f)\n\nlet attributeValue =\n ws >>. str \"=\" >>. ws\n >>. between (str \"\\\"\") (str \"\\\"\")\n (manySatisfy (isNoneOf \"\\n\\\"\"))\n\nlet attribute s = str s >>. attributeValue\n\nlet nonNestedTag tag pAttributesAndClose pBody f\n isInTag setInTag setNotInTag =\n tagOpenBegin tag\n >>. ((fun stream ->\n if not (isInTag stream.UserState) then\n stream.UserState <- setInTag stream.UserState\n Reply(())\n else // generate error at start of tag\n stream.Skip(-tag.Length - 1)\n Reply(FatalError,\n messageError (\"Nested <\" + tag + \"> tags are not allowed.\")))\n >>. pipe2 pAttributesAndClose pBody f\n .>> (tagClose tag >>. updateUserState setNotInTag))\n\n// the tags\n\nlet bold = tag \"b\" elements Bold\nlet italic = tag \"i\" elements Italic\n\nlet url = nonNestedTag \"a\" (ws >>. attribute \"href\" .>> (ws >>. str \">\"))\n elements\n (fun url phrases -> Url(url, phrases))\n (fun us -> us.InLink)\n (fun us -> {us with InLink = true})\n (fun us -> {us with InLink = false})\n\n\nlet element = choice [text\n escape\n quote '\\''\n quote '\\\"'\n bold\n italic\n url]\n\ndo elementsR:= many element\n\nlet document = elements .>> eof\n``\n``{fsi}\n\n> runParserOnString document UserState.Default \"\"\n \"A \\\"'text' with 'nested \\\"quotes\\\"'.\\\"\";;\nval it : ParserResult = Success:\n[Text \"A \";\n Quote\n ('\"',\n [Quote ('\\'',[Text \"text\"]); Text \" with \";\n Quote ('\\'',[Text \"nested \"; Quote ('\"',[Bold [Text \"quotes\"]])]); Text \".\"])]\n\n> runParserOnString document UserState.Default \"\"\n @\"Text with 'link' but no \\\";;\nval it : ParserResult = Success:\n[Bold [Text \"Text \"; Italic []; Text \"with\"]; Text \" \";\n Url(\"url\",\n [Quote ('\\'',[Text \"link\"]); Text \" but no \"; Text \"<\"; Text \"blink\";\n Text \">\"])]\n\n> runParserOnString document UserState.Default \"\"\n \"test\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 15\ntest\n ^\nNested tags are not allowed.\n``\n\n[/section]\n\n[section Parameterizing a parser through the user state]\n\nThe user state is also a good place to store parser configuration data that is specific to a \"parser job\". For example, a compiler that processes multiple compilation units could put configuration data that is specific to the compilation unit, e.g. include paths, into the user state and then parse different compilation units with the same `Parser` instance, like in the following code:\n\n``\ntype CompilationUnitAST = (* ... *)\n\ntype UserState = {\n IncludePaths = string list\n (* ... *)\n}\n\nlet parser : Parser = (* ... *)\n\nlet parseCompilationUnit file encoding includePaths (* ... *) =\n let initialUserState = {IncludePaths = includePaths; (* ... *)}\n runParserOnFile parser initialUserState file encoding\n``\n[/section]\n\n[/section]\n\n[section Where is the monad?]\n\nIf you have previously used Haskell's Parsec library or an early version of FParsec you're probably wondering by now where the \"monadic syntax\" has gone. There's also a chance that you've stumbled upon FParsec while searching for a \"monadic parser library\" for F#/.Net and you're now wondering whether FParsec actually is one.\n\nTo answer these questions right away: FParsec supports a monadic parser construction syntax, but this syntax is only an optional feature, not the foundation of the library design. FParsec doesn't use the monadic syntax internally and we no longer recommend using it for new parser projects when performance is a concern.\n\n[section An example using the monadic syntax]\n\nWith the monadic syntax you can, for example, write a parser for a pair of floating-point numbers as follows:\n\n``\nopen FParsec\n\nlet ws = spaces // whitespace parser\n\nlet str_ws str = parse {do! skipString str\n do! ws\n return ()}\n\nlet number_ws = parse {let! number = pfloat\n do! ws\n return number}\n\nlet pairOfNumbers = parse {do! str_ws \"(\"\n let! number1 = number_ws\n let! number2 = number_ws\n do! str_ws \")\"\n return (number1, number2)}\n``\n\nWe'll explain how the F# compiler handles the `parse {...}` expressions in the next section. For now, just compare the previous implementation with the following one using the usual FParsec combinators:\n\n``\nopen FParsec\n\nlet ws = spaces // whitespace parser\n\nlet str_ws str = skipString str >>. ws\n\nlet number_ws = pfloat .>> ws\n\nlet pairOfNumbers = between (str_ws \"(\") (str_ws \")\")\n (tuple2 number_ws number_ws)\n``\n\nThe latter implementation is obviously more concise, but -- at least for users without prior exposure to FParsec -- the first implementation is probably a bit more intuitive and self-explanatory. What makes the first implementation so intuitive is that the syntax of the `parse {...}` expressions is a) very close to what developers are used to from their normal work with F# and b) expressive enough that it obviates the need for many of FParsec's basic combinators. Unfortunately, the intuitiveness of the monadic syntax comes at the price of a large performance penalty.\n\n[/section]\n\n[section How the monadic syntax works]\n\nTo explain how the monadic syntax works, we need to take a look at how the F# compiler translates the `parse {...}` expressions.\n\nThe foundation for the monadic syntax is the `>>=` combinator introduced in [^applying-parsers-in-sequence.the-combinator]:\n``val (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b>``\n\nThis operator takes a parser and a function returning a parser as arguments. The combined parser `p >>= f` first applies the parser `p` to the input, then it applies the function `f` to the result returned by `p` and finally it applies the parser returned by `f` to the input. As we exlained in [^applying-parsers-in-sequence.the-combinator], this way to combine parsers is powerful enough that we can express many other sequencing combinators in terms of `>>=` and `preturn`.\n\nFor example, we could implement the `pipe3` combinator for sequentially applying three parsers as follows:\n\n``\nlet pipe3 p1 p2 p3 f =\n p1 >>= fun x1 ->\n p2 >>= fun x2 ->\n p3 >>= fun x3 ->\n preturn (f x1 x2 x3)\n``\n\nDirectly using the `>>=` and `preturn` combinators obviously leads to somewhat unwieldy and unreadable expressions. Fortunately, F#'s @computation expressions@ allow us to rewrite this expression in a more intuitive way:\n\n``\nlet pipe3 p1 p2 p3 f =\n parse {let! x1 = p1\n let! x2 = p2\n let! x3 = p3\n return f x1 x2 x3}\n``\n\nThe `parse` object that we reference in this and other code snippets of this chapter is a so-called \"builder\" object for computation expressions. It is defined in FParsec's `Primitives` module. Using the methods of this object, the F# compiler translates the computation expression in the curly braces to the following equivalent expression:\n\n``\nlet pipe3 p1 p2 p3 f =\n parse.Delay(fun () ->\n parse.Bind(p1, fun x1 ->\n parse.Bind(p2, fun x2 ->\n parse.Bind(p3, fun x3 ->\n parse.Return(f (x1 x2 x3))))))\n``\n\nWhen we replace the `parse` object method calls with the respective method bodies, we will see that this definition is equivalent to our original definition using `>>=` and `preturn`.\n\nThe `Bind`, `Return` and `Delay` methods of the `parse` object are defined as:\n``\n member t.Bind(p, f) = p >>= f\n member t.Return(x) = preturn x\n member t.Delay(f:(unit -> Parser<'a,'u>)) = fun stream -> (f ()) stream\n``\n\nSubstituting these method bodies into the previous expression yields an expression that is very similar to the original one (except for the additional indirection introduced by the `Delay` method[fn The computation expression specification does not require a `Delay` method.\nSo, we could avoid the overhead associated with the additional indirection by removing the `Delay` method from the `ParserCombinator` class. However, this would make the behaviour of `parse` expressions somewhat counter-intuitive, as the behaviour would differ from the behaviour of F#'s `seq` and `async` expressions.]):\n\n``\nlet pipe3 p1 p2 p3 f =\n fun stream ->\n (p1 >>= fun x1 ->\n p2 >>= fun x2 ->\n p3 >>= fun x3 ->\n preturn (f x1 x2 x3)) stream\n``\n\nIn summary, the `parse {...}` syntax is syntactic sugar for defining parsers with the `>>=` operator. The expressiveness of this syntax stems from the power of the `>>=` operator.\n\n[/section]\n\n[section The term \"monad\"]\n\nA function with a signature like the one of the `>>=` operator is often called \"bind\". The above examples make it obvious why: the `>>=` combinator binds the result of the parser on the left-hand side to the function argument on the right-hand side.\n\nThe `Parser` type together with the `>>=` and `preturn` operations constitute a @monad@, which is an abstraction in type theory that denotes this kind of combination of a generic type with associated bind and return operations.\n\nDiscussing the theoretical background of monads would be outside the scope of this user's guide. For our purposes it is enough to note that the monad abstraction is so useful for certain applications that F# comes with built-in syntax support for monadic expressions. FParsec utilizes this language feature (@computation expressions@) to enable `parse {...}` expressions.\n\nBe assured that you don't need to know anything about monads in general in order to use FParsec's `parse {...}` expressions. To fully understand this feature all you need to know to is how the F# compiler translates `parse {...}` expressions into normal code.\n\nBesides `let!`, `do!` and `return` there are some more language constructs that are supported inside `parse {...}` expressions. Please refer to the [^reference.Primitives.members.parse reference documentation] for more information.\n[/section]\n\n[section Why the monadic syntax is slow]\n\nCompared to parsers implemented with only the usual FParsec operators and functions, parsers implemented with `parse {...}` expressions can be up to several times slower.\n\nThe relatively bad performance can be directly attributed to the way `parse {...}` expressions are compiled. As you have seen above, a `parse {...}` expression is simply translated into a series of nested closures that are chained through calls to the `>>=` operator. *With the current compiler technology and the current implementation of FParsec* this introduces some significant overhead.\n\n*Every time* a `Parser` function constructed with the `parse {...}` syntax is called:\n- Two function closures get newly instantiated for each invocation of the `>>=` operator: the closure that is passed as the second argument to `>>=` and the closure that is returned by `>>=`.\n- Any parser created inside a `parse {...}` expression gets (re-)created every time execution reaches that point in the expression.\n\nIn principle, you can avoid the overhead described in the second point by moving the construction of parser functions out of the `parse {...}` expression.\n\nFor example, you can avoid the repeated construction of the `skipString` parsers in\n``\nlet numberInParens = parse {do! skipString \"(\"\n let! number = pfloat\n do! skipString \")\"\n return number}\n``\nby rewriting the code as\n``\nlet parenOpen = skipString \"(\"\nlet parenClose = skipString \")\"\nlet numberInParens = parse {do! parenOpen\n let! number = pfloat\n do! parenClose\n return number}\n``\n\nHowever, if you wanted to factor out any parser construction from a `parse {...}` expression, you'd also have to factor out any use of parser combinators, which would take away a lot from the attractiveness of the syntax.\n\nIf performance is not that important for your application, you can just ignore that a parser like `skipString \"(\"` is repeatedly constructed, since its construction is relatively cheap. But if you do the same for parsers based on `regex` or `anyOf`, where the construction potentially involves some relatively expensive compilation or runtime code generation, you might be surprised just how slow your parsers can become.\n\nBecause of the described performance issues, we recommend not to use `parse {...}` expressions and instead work with FParsec's rich set of operators and other combinators. Not only does the operator-based notation (which is used everywhere else in FParsec's documentation) lead to faster parsers, it also allows for more concise parser code with a higher signal-to-noise ratio.\n\n[/section]\n\n[/section]\n\n\n[section Debugging a parser]\n\nDebugging a parser implemented with the help of a combinator library has its special challenges. In particular, setting a breakpoint and stepping through the code is not as straightforward as in a regular recursive descent parser. Furthermore, stack traces can be difficult to decipher because of the ubiquitous use of anonymous functions.[fn Although, debugging a parser written with a combinator library is often still easier than debugging one generated by an opaque parser generator tool.] However, with the help of the techniques we explain in this chapter, working around these issues should be easy.\n\n[section Setting a breakpoint]\n\nSuppose you have a combined parser like\n``let buggyParser = pipe2 parserA parserB (fun a b -> ...)``\nand you would like to break into the debugger whenever `buggyParser` calls `parserB`. One thing you could try is to set a breakpoint at the beginning of `parserB`. However, that's only possible if `parserB` is not itself a combined parser, and even then you still have the problem that your breakpoint is also triggered whenever `parserB` is called from any other place in your source. Similarly, a breakpoint you set in `pipe2` will probably be triggered by many other parsers besides `buggyParser`.\n\nFortunately there's a simple workaround if you can modify and recompile the code. Just define a wrapper function like the following\n``\nlet BP (p: Parser<_,_>) stream =\n p stream // set a breakpoint here\n``\n\nThen redefine the buggy parser as\n``\nlet buggyParser = pipe2 parserA (BP parserB) (fun a b -> ...)\n``\n\nIf you now set a breakpoint at the body of the BP function, it will be triggered whenever `parserB` is called from `buggyParser`.\n\nWith such a wrapper it's also easy define a precise conditional breakpoint. For example, if you only want to break once the parser has reached line 100 of the input file, you could use the breakpoint condition `stream.Line >= 100`.\n\nBy the way, you don't need to set the breakpoint in the debugger. You can also write it directly into the code:\n``\nlet BP (p: Parser<_,_>) (stream: CharStream<_>) =\n // this will execute much faster than a\n // conditional breakpoint set in the debugger\n if stream.Line >= 100L then\n System.Diagnostics.Debugger.Break()\n p stream\n``\n\n[note\nThere are some issues with setting breakpoints in or stepping into anonymous or curried F# functions in Visual Studio 2008. In Visual Studio 2010 many of these issues have been fixed.\n\nIf you're using Visual Studio, don't forget to switch on the \"Suppress JIT optimization on module load\" option in the Tools -- Options -- Debugging -- General dialog. And, when possible, use a debug build (of FParsec) for debugging.\n]\n\n[/section]\n\n[section Tracing a parser]\nOccasionally you have a parser that doesn't work as expected and playing around with the input or staring at the code long enough just isn't enough for figuring out what's wrong. In such cases the best way to proceed usually is to trace the execution of the parser. Unfortunately, stepping through the parser under a debugger can be quite tedious, because it involves stepping through long sequences of nested invocations of parser combinators. A more convenient approach often is to output tracing information to the console or a logging service.\n\nA simple helper function for printing trace information to the console could like the following example:\n``\nlet () (p: Parser<_,_>) label : Parser<_,_> =\n fun stream ->\n printfn \"%A: Entering %s\" stream.Position label\n let reply = p stream\n printfn \"%A: Leaving %s (%A)\" stream.Position label reply.Status\n reply\n``\n\nTo demonstrate how you could use such a tracing operator, let's try to debug\nthe following buggy (and completely silly) parser:\n\n``\nlet number = many1Satisfy isDigit\n\nlet emptyElement = pstring \"[]\" : Parser<_,unit>\nlet numberElement = pstring \"[\" >>. number .>> pstring \"]\"\nlet nanElement = pstring \"[NaN]\"\n\nlet element = choice [emptyElement\n numberElement\n nanElement] .>> spaces\n\nlet elements : Parser<_,unit> = many element\n``\n\nThe following test run shows that the above parser is indeed buggy:\n``{fsi}\n> run elements \"[] [123] [NaN]\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 11\n[] [123] [NaN]\n ^\nUnknown Error(s)\n``\n\nYou probably don't need trace information to figure out why the `\"NaN\"` bit of the string doesn't get parsed, but let's pretend you do. Obviously, there's something wrong with the `element` parser. To find out what's wrong, let's decorate the `element` parser and all subparsers with the `` operator and an appropriate label:\n\n``\nlet number = many1Satisfy isDigit \"number\"\n\nlet emptyElement = pstring \"[]\" \"emptyElement\"\nlet numberElement = pstring \"[\" >>. number .>> pstring \"]\" \"numberElement\"\nlet nanElement = pstring \"[NaN]\" \"nanElement\"\n\nlet element = choice [emptyElement\n numberElement\n nanElement] .>> spaces \"element\"\n\nlet elements : Parser<_,unit> = many element\n``\n\nIf you now run the parser on the same input as before, you get the following output:\n``{fsi}\n> run elements \"[] [123] [NaN]\";;\n(Ln: 1, Col: 1): Entering element\n(Ln: 1, Col: 1): Entering emptyElement\n(Ln: 1, Col: 3): Leaving emptyElement (Ok)\n(Ln: 1, Col: 4): Leaving element (Ok)\n(Ln: 1, Col: 4): Entering element\n(Ln: 1, Col: 4): Entering emptyElement\n(Ln: 1, Col: 4): Leaving emptyElement (Error)\n(Ln: 1, Col: 4): Entering numberElement\n(Ln: 1, Col: 5): Entering number\n(Ln: 1, Col: 8): Leaving number (Ok)\n(Ln: 1, Col: 9): Leaving numberElement (Ok)\n(Ln: 1, Col: 10): Leaving element (Ok)\n(Ln: 1, Col: 10): Entering element\n(Ln: 1, Col: 10): Entering emptyElement\n(Ln: 1, Col: 10): Leaving emptyElement (Error)\n(Ln: 1, Col: 10): Entering numberElement\n(Ln: 1, Col: 11): Entering number\n(Ln: 1, Col: 11): Leaving number (Error)\n(Ln: 1, Col: 11): Leaving numberElement (Error)\n(Ln: 1, Col: 11): Leaving element (Error)\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 11\n[] [123] [NaN]\n ^\nUnknown Error(s)\n``\n\nThis trace log clearly reveals that the `element` parser failed because the `numberElement` parser failed after consuming the left bracket and thus the `choice` parser never got to try the the `nanElement` parser. Of course, this issue could be easily avoided by factoring out the bracket parsers from the `emptyElement`, `numberElement` and `nanElement` parsers. Also, if we had used `many1SatisfyL` instead of `manySatisfy` for the `number` parser, we would have gotten an error message more descriptive than \"Unknown error(s)\" (see the chapter on @customizing error messages@).\n[/section]\n\n[/section]\n\n\n[section Performance optimizations]\n\nIn the past, the relatively poor performance of parser combinator libraries has often been cited as the primary impediment to their more widespread adoption. For this reason optimal performance stood front and center as a design goal during the development of FParsec and a lot of effort has been spent on optimizing parsing speed. As a result, FParsec has become so fast that parsers implemented with FParsec often significantly outperform parsers created by parser generator tools like fslex & fsyacc.\n\nIn general, a parser implemented in FParsec can get close to the performance of a hand-optimized recursive-descent parser written in C#. Due to the multi-layered architecture of the FParsec API, you always have the option to fall back to the lower-level API should a particular parser component implemented with the high-level API turn out to be too slow. Hence, if you choose FParsec for implementing your parsers, you don't have to worry that performance will become a reason for switching away from FParsec.\n\n[section Performance guidelines]\n\nIf you strive for optimal performance in your parser applications, try to adhere to the following guidelines:\n[dl\n[Avoid backtracking]\n[\nTry to avoid backtracking where possible. Sometimes it's already enough to factor out a common prefix from a parser expression to avoid backtracking, e.g. by transforming `(prefix >>? p1) <|> (prefix >>? p2)` to `prefix >>. (p1 <|> p2)`. Some simple backtracking can also be avoided by parsing whitespace as trailing whitespace instead of leading whitespace.\n\nIf you're designing a programming or markup language, you should try to minimize the need for backtracking, both to simplify parsing and to avoid exponential worst-case behaviour.\n]\n\n[Prefer specialized parsers]\n[\nFParsec provides a number of specialized parsers and combinators for various purposes. Using more specialized primitives instead of reimplementing them with generic combinators will often safe you time and improve parsing speed.\n\nIn particular:\n- Prefer the `skip...` variants of parsers and combinators if you don't need the parser results.\n- Parse whitespace with the built-in whitespace parsers.\n- Parse numbers with the built-in number parsers.\n- Prefer to parse strings with the `many[1]Satisfy[2][L]` parsers.\n- Consider parsing unicode identifiers with the `identifier` parser.\n]\n\n[[# Construct parsers once]]\n[\nConstructing a parser can be relatively expensive in comparison to a single invocation of the parser. Hence, if you repeatedly apply the same parser, you should make sure that you construct the parser only once, either by preconstructing it at the beginning or by lazily constructing the parser and then caching it.\n\nUsually the place where parsers get inadvertently constructed more than once is inside closures.\n\nFor example, if you have a local function like\n``\nfun stream ->\n let reply = (parser1 >>. parser2) stream\n if reply.Status = Ok then // ...\n else // ...\n``\nyou should avoid the repeated construction of `parser1 >>. parser2` every time the closure is called by moving the construction outside of the closure, as in\n``\nlet parser = parser1 >>. parser2\nfun stream ->\n let reply = parser stream\n if reply.Status = Ok then //...\n else // ...\n``\n\nAlso, you shouldn't wrap a parser expression inside a function just to avoid F#'s value restriction if you can achieve the same goal with a type annotation. For example, you should **not** try to fix the compiler error in the first example of the [^fs-value-restriction tutorial chapter on F#'s value restriction] by replacing\n``let p = pstring \"test\"``\nwith\n``let p stream = pstring \"test\" stream``\n]\n\n[Avoid `parse {...}` expressions]\n[\nSee @Why the monadic syntax is slow@.\n]\n\n[Avoid `regex` parsers]\n[\nThe `regex` parser parses a string by applying a .NET regular expression to the input. Since .NET regular expressions are relatively slow, you should reserve the use of the `regex` parser for patterns that you can't easily express with other FParsec parsers and combinators.\n]\n\n[Consider optimizing large `choice` parsers]\n[\nFormal grammars for programming languages or DSLs often have one or two grammar rules at their core that essentially just enumerate a long list of possible ways to form a statement or expression in that language. A straightforward FParsec implementation of such a grammar rule typically uses the `choice` combinator to combine a list of parsers for all the alternatives.\n\nUsually such an implementation with a large `choice`-based parser will do just fine. However, if parsing performance is critical for your application, replacing a large `choice` parser with a custom-made combinator can be an optimization with a high benefit-cost ratio. The next section explains this optimization in more detail.\n]\n]\n\n[/section]\n\n[section Low-level parser implementations]\n\nFParsec's high-level API consists of its built-in parsers and combinators in the `Primitives` and `CharParsers` module. The high-level API allows you to easily construct parsers in a concise and rather declarative way. Usually you will author most of your parsers using the high-level API, because that's the most productive way to do it.\n\nHowever, sometimes you might find that a specific piece of parser functionality is a bit inconvenient to express through the high-level API or that the high-level implementation isn't as fast as you had hoped for. In those situations it's a great advantage that FParsec allows you to drop down to the low-level API, so that you can implement your own special-purpose parser and combinator primitives.\n\nWe have already covered the basics of the low-level API in the chapters on the @internals of a simple parser function@ and [@ applying parsers in sequence]. In this section we will discuss some examples that demonstrate how you can use low-level parser implementations for optimization purposes.\n\nOne example of a parser implemented using the low-level API is contained in the samples folder of the FParsec distribution in [= samples/FSharpParsingSample/FParsecVersion/parser.fs]. It is a parser for an identifier string that is not identical with a keyword.\n\nThe low-level implementation uses another parser, `identifierString`, to parse an identifier string and then backtracks when the parsed string is a keyword:\n``\nlet identifier : Parser =\n let expectedIdentifier = expected \"identifier\"\n fun stream ->\n let state = stream.State\n let reply = identifierString stream\n if reply.Status <> Ok || not (isKeyword reply.Result) then reply\n else // result is keyword, so backtrack to before the string\n stream.BacktrackTo(state)\n Reply(Error, expectedIdentifier)\n``\n\nThe same parser could also be implemented with the high-level API:\n``\nlet identifier =\n attempt (identifierString\n >>= fun str ->\n if not (isKeyword str) then preturn str\n else pzero) \"identifier\"\n\n``\n\nThe high-level version is a bit more concise, but whether it is also easier to understand is debatable. The low-level version seems at least a bit more self-explanatory and hence is probably more accessible to new FParsec users. Since the low-level implementation is also significantly faster than the high-level one, this is a good example for a parser that can be improved through a low-level implementation.\n\nIf you wanted to optimize the performance of the identifier parser even more, you could replace the `identifierString` parser invocation with direct calls to `CharStream` methods. However, whether the potential performance gain would be worth the loss in code modularity and maintainability is questionable. A more promising optimization often is to integrate the identifier parser into a higher-level `choice`-based parser, like it is done below in the last example of this section.\n\n`choice` parsers with long list of argument parsers are performance-wise one of the weakest spots of FParsec's high-level API. As we noted in the previous section, formal grammars for programming languages or DSLs often have one or two grammar rules at their core that essentially just enumerate a long list of possible ways to form a statement or expression in that language. A straightforward implementation of such a grammar rule using the `choice` combinator yields only sub-optimal performance, since the `choice` parser has no knowledge about its argument parsers and has to try one parser after another.\n\nThis makes large `choice`-based parsers an excellent optimization opportunity. With your knowledge about the parser grammar you can often narrow down the set of possible parsers just by peeking at the following one or two chars in the input. Having identified the set of possible parsers (often only consisting of one parser), you can then considerably speed up the dispatch to the right subparser.\n\nFor example, take a look at the @JSON-value parser@ from the tutorial:\n``\nchoice [jobject\n jlist\n jstring\n jnumber\n jtrue\n jfalse\n jnull]\n``\n\nIf you look at the definitions for the argument parsers, you'll see that in almost all cases one can decide which parser should handle the input just based on the next char in the input. Hence, we could replace the `choice`-based parser with the following low-level implementation:\n\n``\nlet error = expected \"JSON value\"\nfun (stream: CharStream<_>) ->\n match stream.Peek() with\n | '{' -> jobject stream\n | '[' -> jlist stream\n | '\"' -> jstring stream\n | 't' when stream.Skip(\"true\") -> Reply(JBool true)\n | 'f' when stream.Skip(\"false\") -> Reply(JBool false)\n | 'n' when stream.Skip(\"null\") -> Reply(JNull)\n | _ ->\n let stateTag = stream.StateTag\n let mutable reply = jnumber stream\n if reply.Status = Error && stateTag = stream.StateTag then\n reply.[^RError Error] <- error\n reply\n``\n\nA drawback of such a low-level implementation is that you have to be a bit careful not to overlook any of the possible grammar cases. This is why we applied the `jnumber` parser in the \"catch-all\" case, so that we don't depend on the precise grammar rules for numbers.\n\nYou also need to consider how the low-level implementation affects error messages. When a `choice` parser fails, it will generate an error message with the error messages from all the argument parsers it tried. This gives a human reader usually enough context to understand the error. For a low-level implementation it can take a little more effort to ensure that the error messages for every case contain enough information about the grammar context. For example, in our implementation above we had to replace the default error message by `jnumber` with a custom one, so that the error message generated by the catch-all case doesn't create the impression that a JSON value can only be a number.\n\nBy now it is probably obvious that a low-level parser implementation can actually be quite simple to write, but that it also comes at a certain cost in terms of code modularity and maintainability. Having the option of a low-level implementation can certainly be what saves a project in certain situations and should give you some peace of mind with regard to parser performance, but generally you should only consider it as a backup option for those cases where you really need it.\n\nThe following example shows again how you can replace a `choice`-based parser with a low-level implementation, this time with a grammar that is a bit more representative of a typical programming language:\n\n``\ntype Expr = Number float\n | LetBinding ...\n | IfThenElse ...\n | ...\n\ntype UserState = //...\n\ntype Parser<'result> = Parser<'result, UserState>\n\ntype Keyword = None = 0\n | If = 1\n | Let = 2\n // ...\n\nlet stringToKeyword = createStaticStringMapping\n Keyword.None\n [\"if\", Keyword.If\n \"let\", Keyword.Let\n // ...\n ]\n\nlet str s = pstring s\n\nlet identifierString : Parser = // ...\n\nlet identifierRest (id: string) : Parser = ...\n\nlet number : Parser = // ... (parser for floating-point number)\n\nlet ifThenElseRest : Parser = // ...\nlet letBindingRest : Parser = // ...\nlet exprInParensRest : Parser = // ...\n\n// The parser after this comment is a replacement for\n// let identifierStringButNoKeyword =\n// (* implementation like identifier parser in the first example above *)\n//\n// let identifier : Parser = identifierStringButNoKeyword\n// >>= identifierRest\n//\n// let ifThenElse : Parser = str \"if\" >>. ifThenElseRest\n// let letBinding : Parser = str \"let\" >>. letBindingRest\n// let exprInParens : Parser = str \"(\" >>. exprInParensRest\n//\n// let expr = choice [identifierStringNoKeyword\n// number\n// ifThenElse\n// exprInParens\n// // ...\n// ]\n//\nlet expr : Parser =\n fun stream ->\n let stateTag = stream.StateTag\n let reply = identifierString stream\n if reply.Status = Ok then\n match stringToKeyword reply.Result with\n | Keyword.None -> identifierRest reply.Result stream\n | Keyword.If -> ifThenElseRest stream\n | Keyword.Let -> letBindingRest stream\n // ...\n elif reply.Status = Error && stateTag = stream.StateTag then // no identifier\n match stream.Peek() with\n | '(' -> stream.Skip(); exprInParensRest stream\n | c when isDigit c -> number stream\n // ...\n else // error within identifier string\n Reply(reply.Status, reply.[^RError Error])\n\n``\n\n[/section]\n\n[/section]\n\n[section Tips and tricks]\n\n[toc]\n\n[section Parallel parsing]\n\nIf your parser grammar is suitable for parallel parsing, parallelizing the parser has the potential to dramatically accelerate parsing on multi-core machines. In the following we will shortly discuss requirements and strategies for parallelizing an FParsec parser.\n\nFor a parser grammar to be well suited for parallel parsing, the grammar and the typical input must satisfy the following two criteria:\n- Parts of the input must be independently parseable, i.e. parts must be parseable without knowlege about the other parts.\n- These parts must be large enough and easily enough identifiable within the total input.\n\nOften, the easiest and most beneficial way to parallelize the parsing stage of an application is to parse multiple input files in parallel. In the simplest case you have multiple independent \"compilation units\" that can be parsed in parallel. This works even for C/C++, where a badly designed preprocesser generally makes efficient parsing quite hard. In many programming languages and markup languages you can also parse in parallel files that are \"included\", \"opened\" or \"imported\" within source files. However, this usually only works if the language allows such includes only at well-defined points in the grammar. In languages like C/C++, where the unstructured text content of other files can be included at essentially arbitrary positions in the source, parsing the included files in parallel is generally quite hard. (In C/C++ it's even hard to avoid parsing the same file multiple times when it is included multiple times).\n\nIf you're dealing with large input files or very slow parsers, it might also be worth trying to parse multiple sections within a single file in parallel. For this to be efficient there must be a fast way to find the start and end points of such sections. For example, if you are parsing a large serialized data structure, the format might allow you to easily skip over segments within the file, so that you can chop up the input into multiple independent parts that can be parsed in parallel. Another example could be a programming languages whose grammar makes it easy to skip over a complete class or function definition, e.g. by finding the closing brace or by interpreting the indentation. In this case it *might* be worth not to parse the definitions directly when they are encountered, but instead to skip over them, push their text content into a queue and then to process that queue in parallel.\n\nHere are some tips for parallel parsing with FParsec:\n- All FParsec parsers are thread-safe and can be safely applied concurrently to different `CharStream` instances, as long as you don't introduce mutable shared state yourself.\n- `CharStream` instances are not thread-safe and a single instance must not be accessed concurrently.\n- However, you can call the `CreateSubstream` method to create a substream for a `CharStream`. A `CharStream` and its substreams can be safely accessed concurrently.\n- If you want to parse multiple files in parallel, you should also create the `CharStream` instances in parallel, because the `CharStream` constructors that accept file paths or binary streams perform I/O operations that benefit from parallelization.\n- If you parallelize your parser, consider introducing an option for switching off parallel execution, since debugging a multi-threaded parser is harder than debugging a single-threaded one.\n\n[/section]\n\n[section Dispatching parsers through a dictionary]\n\nA technique that is often useful for making a parser modular and easily extensible is to store `Parser` functions in dictionaries and then to delegate parsing to one of the `Parser` functions in the dictionary based on the input.\n\nFor example, a parser for a markup language could be implemented by defining a generic tag parser that delegates the parsing of the tagged content to a specific parser for the respective tag name. The following code shows how this could be done:\n\n``\nopen FParsec\nopen System.Collections.Generic\n\n// For simplicity we don't define a full-blown markup language here,\n// just a parser for two simple non-recursive \"tags\" in square brackets.\n// The chapter on \"parsing with user state\" contains a slightly more developed\n// sample for a markup language, though without a dictionary-based tag parser.\n\ntype Tag = Bold of string\n | Url of string * string\n\n// We store the tag parser dictionary in the user state, so that we can\n// concurrently parse multiple input streams with the same parser instance\n// but differerent tag dictionaries.\n\ntype TagParserMap = Dictionary>\n\nand UserState = {\n TagParsers: TagParserMap\n }\n\nlet defaultTagParsers = TagParserMap()\n\nlet isTagNameChar1 = fun c -> isLetter c || c = '_'\nlet isTagNameChar = fun c -> isTagNameChar1 c || isDigit c\nlet expectedTag = expected \"tag starting with '['\"\n\nlet tag : Parser =\n fun stream ->\n if stream.Skip('[') then\n let name = stream.ReadCharsOrNewlinesWhile(isTagNameChar1, isTagNameChar,\n false)\n if name.Length <> 0 then\n let mutable p = Unchecked.defaultof<_>\n if stream.UserState.TagParsers.TryGetValue(name, &p) then p stream\n else\n stream.Skip(-name.Length)\n Reply(Error, messageError (\"unknown tag name '\" + name + \"'\"))\n else Reply(Error, expected \"tag name\")\n else Reply(Error, expectedTag)\n\nlet str s = pstring s\nlet ws = spaces\nlet text = manySatisfy (function '['|']' -> false | _ -> true)\n\ndefaultTagParsers.Add(\"b\", str \"]\" >>. text .>> str \"[/b]\" |>> Bold)\n\ndefaultTagParsers.Add(\"url\", (str \"=\" >>. manySatisfy ((<>)']') .>> str \"]\")\n .>>. (text .>> str \"[/url]\")\n |>> Url)\n\nlet parseTagString str =\n runParserOnString tag {TagParsers = TagParserMap(defaultTagParsers)} \"\" str\n\n``\n``{fsi}\n> parseTagString \"[b]bold text[/b]\";;\nval it : ParserResult = Success: Bold \"bold text\"\n\n> parseTagString \"[url=http://tryfsharp.org]try F#[/url]\";;\nval it : ParserResult =\n Success: Url (\"http://tryfsharp.org\",\"try F#\")\n\n> parseTagString \"[bold]test[/bold]\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 2\n[bold]test[/bold]\n ^\nunknown tag name 'bold'\n``\n[/section]\n\n[section Memoizing parsers]\n\nIf your parser implementation backtracks a lot when parsing typical inputs and as a result repeatedly applies some `Parser` functions at the same input position, it can be beneficial to memoize these `Parser` functions, i.e. cache their results for each input position.\n\nIn the extreme case, [url \"https://en.wikipedia.org/wiki/Memoization\" memoization] can mean the difference between linear and exponential execution times. In practice, FParsec is typically used for formal grammars that hardly require any extensive backtracking, so that memoization would usually only have a negative affect on performance.\n\nIn situation where you really do need to memoize parsers, you can work with a generic `memoize` combinator like the one in the following example:\n\n``\nopen FParsec\nopen System.Collections.Generic\n\n// We need a place to store the cached parser results. Since we want parser\n// instances to be able to concurrently access different caches for different\n// input streams, we will use a user state variable for this purpose. Since we\n// don't want the backtracking to undo changes to the cache, we will use a\n// mutable dictionary for this purpose.\n\ntype UserState = {\n MemoCache: Dictionary\n // ...\n }\n\n// An entry in the MemoCache must be uniquely identified by its MemoKey. In this\n// example the MemoKey includes the stream index value and a reference to the\n// memoized parser instance. Should the result of a memoized Parser function in\n// your implementation also depend on the UserState value, you will have to\n// extend the MemoKey with a UserState member. Similarly, if you want to cache\n// results for more than one stream in the MemoCache, you'll have to extend the\n// MemoKey with an identifier for the stream.\n\nand []\n MemoKey = struct\n new (parser: obj, stream: CharStream) =\n {[no-auto-link Parser] = parser; [no-auto-link Index] = stream.Index}\n\n val [no-auto-link Parser]: obj\n val [no-auto-link Index]: int64\n\n interface System.IEquatable with\n member t.Equals(other: MemoKey) =\n t.[no-auto-link Index] = other.[no-auto-link Index] && t.[no-auto-link Parser] = other.[no-auto-link Parser]\n\n override t.Equals(otherObj: obj) =\n match otherObj with\n | :? MemoKey as other ->\n t.[no-auto-link Index] = other.[no-auto-link Index] && t.[no-auto-link Parser] = other.[no-auto-link Parser]\n | _ -> false\n\n override t.GetHashCode() = int32 t.[no-auto-link Index]\nend\n\n/// Returns a memoized version of the argument parser\nlet memoize (p: Parser<'a,UserState>) : Parser<'a,UserState> =\n fun stream ->\n let key = MemoKey(p, stream)\n let memoCache = stream.UserState.MemoCache\n let mutable boxedReply = null\n if memoCache.TryGetValue(key, &boxedReply) then\n boxedReply :?> Reply<'a>\n else\n let reply = p stream\n memoCache.Add(key, box reply)\n reply\n``\n\n[/section]\n\n[section Parsing F# infix operators]\nF# supports user-definable infix operators whose precedence and associativity depend on the first chars of the operator name. For example, the [url \"http://fsharp.org/specs/language-spec/4.0/FSharpSpec-4.0-latest.pdf\" F# spec] states that operators that start with `*` are left-associative, while operators that start with `**` are right associative and have a higher precedence, so that `1*2*.3**4**.5` is parsed as `((1*2)*.(3**(4 **.5)))`.\n\nSince the precedence and associativity rules are fixed, you can parse F# expressions with a static operator precedence grammar, i.e. without having to reconfigure the parser when a new operator is defined in the parsed source code. However, it's probably not immediately obvious how to do this with FParsec's `OperatorPrecedenceParser` class (OPP), since the OPP normally expects all possible operators to be (individually) specified before they are used.\n\nThe trick to supporting whole classes of operator names without having to reconfigure the OPP at run-time is to shift part of the operator parsing to the [^Operator after-string-parser], like in the following example:\n\n``\nopen FParsec\n\ntype Expr = InfixOpExpr of string * Expr * Expr\n | Number of int\n\nlet ws = spaces // whitespace parser\n\nlet isSymbolicOperatorChar = isAnyOf \"!%&*+-./<=>@^|~?\"\nlet remainingOpChars_ws = manySatisfy isSymbolicOperatorChar .>> ws\n\nlet opp = new OperatorPrecedenceParser()\nopp.TermParser <- pint32 .>> ws |>> Number\n\n// a helper function for adding infix operators to opp\nlet addSymbolicInfixOperators prefix precedence associativity =\n let op = InfixOperator(prefix, remainingOpChars_ws,\n precedence, associativity, (),\n fun remOpChars expr1 expr2 ->\n InfixOpExpr(prefix + remOpChars, expr1, expr2))\n opp.AddOperator(op)\n\n// the operator definitions:\naddSymbolicInfixOperators \"*\" 10 Associativity.Left\naddSymbolicInfixOperators \"**\" 20 Associativity.Right\n// ...\n\n``\n``{fsi}\n> run opp.ExpressionParser \"1*2*.3**4**.5\";;\nval it : ParserResult = Success\nInfixOpExpr\n (\"*.\", InfixOpExpr (\"*\", Number 1, Number 2),\n InfixOpExpr (\"**\", Number 3, InfixOpExpr (\"**.\", Number 4, Number 5)))\n``\n\n\nIf you use the after-string-parser in this manner for operators that can lead to operator conflicts in the input, e.g. non-associative operators, then you also need to replace the default `OperatorConflictErrorFormatter`, since otherwise the default formatter may print truncated operator names:\n\n``\naddSymbolicInfixOperators \"<\" 1 Associativity.None\n``\n``{fsi}\n> run opp.ExpressionParser \"1 <= 2 <=. 3\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 9\n1 <= 2 <=. 3\n ^\nThe infix operator '<' (precedence: 1, non-associative) conflicts with the\ninfix operator '<' (precedence: 1, non-associative) on the same line at column 3.\n``\n\nAn error formatter that prints the full operator names could look like the following:\n\n``\nopp.OperatorConflictErrorFormatter <-\n fun (pos1, op1, afterString1) (pos2, op2, afterString2) ->\n let msg = sprintf \"The operator '%s' conflicts with the previous operator '%s' at %A.\"\n (op2.[@ String] + afterString2)\n (op1.[@ String] + afterString1) pos1\n messageError msg\n``\n``{fsi}\n> run opp.ExpressionParser \"1 <= 2 <=. 3\";;\nval it : ParserResult = Failure:\nError in Ln: 1 Col: 9\n1 <= 2 <=. 3\n ^\nThe operator '<=.' conflicts with the previous operator '<=' at (Ln: 1, Col: 3).\n``\n\n[/section]\n\n[/section]\n[/section]\n\n" }, { "path": "FParsec/AssemblyInfo.fs", "content": "namespace FParsec\n\nopen System.Reflection\nopen System.Runtime.CompilerServices\nopen System.Runtime.InteropServices\n\n[]\n\n#if LOW_TRUST\n []\n []\n#endif\n[]\ndo ()" }, { "path": "FParsec/CharParsers.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2019\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.CharParsers\n\nopen System.Diagnostics\nopen System.Text\nopen System.Text.RegularExpressions\nopen System.Runtime.CompilerServices // for MethodImplAttribute\n\n#if !LOW_TRUST\nopen Microsoft.FSharp.NativeInterop\n#endif\n\nopen FParsec\nopen FParsec.Internals\nopen FParsec.Error\nopen FParsec.Primitives\n\n#nowarn \"9\" // \"Uses of this construct may result in the generation of unverifiable .NET IL code.\"\n#nowarn \"51\" // \"The address-of operator may result in non-verifiable code.\"\n\n// ================\n// Helper functions\n// ================\n\n[]\nlet EOS = '\\uffff'\n\nlet foldCase = Text.FoldCase : string -> string\nlet normalizeNewlines = Text.NormalizeNewlines\n\nlet floatToHexString = HexFloat.DoubleToHexString\nlet floatOfHexString = HexFloat.DoubleFromHexString\n\nlet float32ToHexString = HexFloat.SingleToHexString\nlet float32OfHexString = HexFloat.SingleFromHexString\n\n// ========================\n// Running parsers on input\n// ========================\n\n[]\ntype ParserResult<'Result,'UserState> =\n | Success of 'Result * 'UserState * Position\n | Failure of string * ParserError * 'UserState\n with\n member private t.StructuredFormatDisplay =\n match t with\n | Success(r,_,_) ->\n if typeof<'Result> = typeof then \"Success: ()\"\n else sprintf \"Success: %A\" r\n | Failure(msg,_,_) ->\n sprintf \"Failure:\\n%s\" msg\n\nlet internal applyParser (parser: Parser<'Result,'UserState>) (stream: CharStream<'UserState>) =\n let reply = parser stream\n if reply.Status = Ok then\n Success(reply.Result, stream.UserState, stream.Position)\n else\n let error = ParserError(stream.Position, stream.UserState, reply.Error)\n Failure(error.ToString(stream), error, stream.UserState)\n\nlet runParserOnString (parser: Parser<'Result,'UserState>) (ustate: 'UserState) (streamName: string) (chars: string) =\n CharStream.ParseString(chars, 0, chars.Length, applyParser parser, ustate, streamName)\n\nlet runParserOnSubstring (parser: Parser<'Result,'UserState>) (ustate: 'UserState) (streamName: string) (chars: string) (index: int) length =\n CharStream.ParseString(chars, index, length, applyParser parser, ustate, streamName)\n\nlet runParserOnStream (parser: Parser<'Result,'UserState>) (ustate: 'UserState) (streamName: string) (byteStream: System.IO.Stream) (encoding: System.Text.Encoding) =\n#if LOW_TRUST\n let\n#else\n use\n#endif\n stream = new CharStream<'UserState>(byteStream, encoding)\n stream.UserState <- ustate\n stream.Name <- streamName\n applyParser parser stream\n\nlet runParserOnFile (parser: Parser<'Result,'UserState>) (ustate: 'UserState) (path: string) (encoding: System.Text.Encoding) =\n#if LOW_TRUST\n let\n#else\n use\n#endif\n stream = new CharStream<'UserState>(path, encoding)\n stream.UserState <- ustate\n applyParser parser stream\n\nlet run parser (string: string) =\n runParserOnString parser () \"\" string\n\n// =======\n// Parsers\n// =======\n\n// -------------------------------------------------------------\n// Reading the input stream position and handling the user state\n// -------------------------------------------------------------\n\nlet getPosition : Parser =\n fun stream -> Reply(stream.Position)\n\nlet getUserState : Parser<'u,'u> =\n fun stream -> Reply(stream.UserState)\n\nlet setUserState (newUserState: 'u) : Parser =\n fun stream ->\n stream.UserState <- newUserState\n Reply(())\n\nlet updateUserState (f: 'u -> 'u) : Parser =\n fun stream ->\n stream.UserState <- f stream.UserState\n Reply(())\n\nlet userStateSatisfies f : Parser =\n fun stream ->\n let status = if f stream.UserState then Ok else Error\n Reply(status, (), NoErrorMessages)\n\n// --------------------\n// Parsing single chars\n// --------------------\n\nlet newlineReturn result : Parser<_,'u> =\n fun stream ->\n if stream.SkipNewline() then Reply(result)\n else Reply(Error, Errors.ExpectedNewline)\n\nlet newline<'u> = newlineReturn '\\n' : Parser<_,'u>\nlet skipNewline<'u> = newlineReturn () : Parser<_,'u>\n\nlet unicodeNewlineReturn result : Parser<_,'u> =\n fun stream ->\n if stream.SkipUnicodeNewline() then Reply(result)\n else Reply(Error, Errors.ExpectedNewline)\n\nlet unicodeNewline<'u> = unicodeNewlineReturn '\\n' : Parser<_,'u>\nlet skipUnicodeNewline<'u> = unicodeNewlineReturn () : Parser<_,'u>\n\nlet internal charReturnE (c: char) result error : Parser<'a,'u> =\n fun stream ->\n if stream.Skip(c) then Reply(result)\n else Reply(Error, error)\n\nlet charReturn c result : Parser<'a,'u> =\n match c with\n | '\\r' | '\\n' -> newlineReturn result\n | EOS -> invalidArg \"c\" \"The char '\\uffff' (EOS) is not a valid argument for the pchar/skipChar/charReturn parser. If you want to check for the end of the stream, consider using the `eof` parser.\"\n | _ -> charReturnE c result (expectedString (string c))\n\nlet pchar c = charReturn c c\nlet skipChar c = charReturn c ()\n\n\n/// returns true for chars '\\u000E' - '\\ufffe'\nlet inline internal isCertainlyNoNLOrEOS (c: char) =\n // '\\n' = '\\u000A', '\\r' = '\\u000D'\n unativeint c - 0xEun < unativeint EOS - 0xEun\n\nlet anyChar : Parser =\n fun stream ->\n let c = stream.ReadCharOrNewline()\n if c <> EOS then Reply(c)\n else Reply(Error, Errors.ExpectedAnyChar)\n\nlet skipAnyChar : Parser =\n fun stream ->\n if stream.ReadCharOrNewline() <> EOS then Reply(())\n else Reply(Error, Errors.ExpectedAnyChar)\n\n\n// doesn't check for newlines or EOS\nlet\n#if !NOINLINE\n inline\n#endif\n internal fastInlineSatisfyE f error : Parser =\n fun stream ->\n let c = stream.Peek()\n if f c then\n stream.Skip()\n Reply(c)\n else\n Reply(Error, error)\n\nlet internal satisfyE f error : Parser =\n fun stream ->\n let mutable reply = Reply()\n match stream.Peek() with\n | c when isCertainlyNoNLOrEOS c ->\n if f c then\n stream.Skip()\n reply.Status <- Ok\n reply.Result <- c\n else\n reply.Error <- error\n | '\\r' | '\\n' ->\n if f '\\n' then\n stream.SkipNewline() |> ignore\n reply.Status <- Ok\n reply.Result <- '\\n'\n else\n reply.Error <- error\n | c ->\n if c <> EOS && f c then\n stream.Skip()\n reply.Status <- Ok\n reply.Result <- c\n else\n reply.Error <- error\n reply\n\nlet internal skipSatisfyE f error : Parser =\n fun stream ->\n let mutable reply = Reply()\n match stream.Peek() with\n | c when isCertainlyNoNLOrEOS c ->\n if f c then\n stream.Skip()\n reply.Status <- Ok\n else\n reply.Error <- error\n | '\\r' | '\\n' ->\n if f '\\n' then\n stream.SkipNewline() |> ignore\n reply.Status <- Ok\n else\n reply.Error <- error\n | c ->\n if c <> EOS && f c then\n stream.Skip()\n reply.Status <- Ok\n else\n reply.Error <- error\n reply\n\nlet satisfy f = satisfyE f NoErrorMessages\nlet satisfyL f label = satisfyE f (expected label)\n\nlet skipSatisfy f = skipSatisfyE f NoErrorMessages\nlet skipSatisfyL f label = skipSatisfyE f (expected label)\n\n\nlet private charsToString (chars: seq) =\n match chars with\n | :? string as str -> str\n | _ -> new string(Array.ofSeq chars)\n\nlet isAnyOf (chars: seq) =\n#if LOW_TRUST\n let cs = new CharSet(charsToString chars)\n fun c -> cs.Contains(c)\n#else\n #if USE_STATIC_MAPPING_FOR_IS_ANY_OF\n StaticMapping.createStaticCharIndicatorFunction false chars\n #else\n let cs = new CharSet(charsToString chars)\n fun c -> cs.Contains(c)\n #endif\n#endif\n\nlet isNoneOf (chars: seq) =\n#if LOW_TRUST\n let cs = new CharSet(charsToString chars)\n fun c -> not (cs.Contains(c))\n#else\n #if USE_STATIC_MAPPING_FOR_IS_ANY_OF\n StaticMapping.createStaticCharIndicatorFunction true chars\n #else\n let cs = new CharSet(charsToString chars)\n fun c -> not (cs.Contains(c))\n #endif\n#endif\n\nlet anyOf (chars: seq) =\n let str = charsToString chars\n satisfyE (isAnyOf str) (Errors.ExpectedAnyCharIn(str))\n\nlet skipAnyOf (chars: seq) =\n let str = charsToString chars\n skipSatisfyE (isAnyOf str) (Errors.ExpectedAnyCharIn(str))\n\nlet noneOf (chars: seq) =\n let str = charsToString chars\n satisfyE (isNoneOf str) (Errors.ExpectedAnyCharNotIn(str))\n\nlet skipNoneOf (chars: seq) =\n let str = charsToString chars\n skipSatisfyE (isNoneOf str) (Errors.ExpectedAnyCharNotIn(str))\n\nlet inline isAsciiUpper (c: char) =\n uint32 c - uint32 'A' <= uint32 'Z' - uint32 'A'\n\nlet inline isAsciiLower (c: char) =\n uint32 c - uint32 'a' <= uint32 'z' - uint32 'a'\n\nlet inline isAsciiLetter (c: char) =\n let cc = uint32 c ||| uint32 ' '\n cc - uint32 'a' <= uint32 'z' - uint32 'a'\n\nlet inline isUpper (c: char) =\n isAsciiUpper c || (c > '\\u007F' && System.Char.IsUpper(c))\n\nlet inline isLower (c: char) =\n isAsciiLower c || (c > '\\u007F' && System.Char.IsLower(c))\n\nlet inline isLetter (c: char) =\n isAsciiLetter c || (c > '\\u007F' && System.Char.IsLetter(c))\n\nlet inline isDigit (c: char) =\n uint32 c - uint32 '0' <= uint32 '9' - uint32 '0'\n\nlet inline isHex (c: char) =\n let cc = uint32 c ||| uint32 ' '\n isDigit c || cc - uint32 'a' <= uint32 'f' - uint32 'a'\n\nlet inline isOctal (c: char) =\n uint32 c - uint32 '0' <= uint32 '7' - uint32 '0'\n\nlet asciiUpper stream = fastInlineSatisfyE isAsciiUpper Errors.ExpectedAsciiUppercaseLetter stream\nlet asciiLower stream = fastInlineSatisfyE isAsciiLower Errors.ExpectedAsciiLowercaseLetter stream\nlet asciiLetter stream = fastInlineSatisfyE isAsciiLetter Errors.ExpectedAsciiLetter stream\n\n// unicode is the default for letters and ascii the default for numbers\nlet upper stream = fastInlineSatisfyE isUpper Errors.ExpectedUppercaseLetter stream\nlet lower stream = fastInlineSatisfyE isLower Errors.ExpectedLowercaseLetter stream\nlet letter stream = fastInlineSatisfyE isLetter Errors.ExpectedLetter stream\n\nlet digit stream = fastInlineSatisfyE isDigit Errors.ExpectedDecimalDigit stream\nlet hex stream = fastInlineSatisfyE isHex Errors.ExpectedHexadecimalDigit stream\nlet octal stream = fastInlineSatisfyE isOctal Errors.ExpectedOctalDigit stream\n\nlet tab stream = fastInlineSatisfyE ((=) '\\t') Errors.ExpectedTab stream\n\nlet spaces : Parser =\n fun stream ->\n stream.SkipWhitespace() |> ignore\n Reply(())\n\nlet spaces1 : Parser =\n fun stream ->\n if stream.SkipWhitespace() then Reply(())\n else Reply(Error, Errors.ExpectedWhitespace)\n\nlet unicodeSpaces : Parser =\n fun stream ->\n stream.SkipUnicodeWhitespace() |> ignore\n Reply(())\n\nlet unicodeSpaces1 : Parser =\n fun stream ->\n if stream.SkipUnicodeWhitespace() then Reply(())\n else Reply(Error, Errors.ExpectedWhitespace)\n\nlet eof : Parser=\n fun stream ->\n if stream.IsEndOfStream then Reply(())\n else Reply(Error, Errors.ExpectedEndOfInput)\n\n\n// ------------------------\n// Parsing strings directly\n// ------------------------\n\nlet internal newlineOrEOSCharInStringArg name (arg: string) i =\n let msg2 = match arg[i] with\n |'\\r'|'\\n' -> \" may not contain newline chars ('\\r' or '\\n').\"\n | EOS -> \" may not contain the char '\\uffff' (EOS)\"\n | _ -> failwith \"newlineOrEOSCharInStringArg\"\n raise (System.ArgumentException(concat3 \"The string argument to \" name msg2))\n\nlet internal checkStringContainsNoNewlineOrEOSChar s name =\n let i = findNewlineOrEOSChar s\n if i >= 0 then newlineOrEOSCharInStringArg name s i\n\nlet stringReturn s result : Parser<'a,'u> =\n let inline checkNoNewlineOrEOSChar c i =\n if not (isCertainlyNoNLOrEOS c) then\n match c with\n |'\\r'|'\\n'|EOS -> newlineOrEOSCharInStringArg \"pstring/skipString/stringReturn\" s i\n | _ -> ()\n\n let error = expectedString s\n match s.Length with\n | 0 -> preturn result\n | 1 ->\n let c = s[0]\n checkNoNewlineOrEOSChar c 0\n charReturnE c result error\n | 2 ->\n let c0, c1 = s[0], s[1]\n checkNoNewlineOrEOSChar c0 0\n checkNoNewlineOrEOSChar c1 1\n let cs = TwoChars(c0, c1)\n fun stream ->\n if stream.Skip(cs) then Reply(result)\n else Reply(Error, error)\n | _ ->\n checkStringContainsNoNewlineOrEOSChar s \"pstring/skipString/stringReturn\"\n fun stream ->\n if stream.Skip(s) then Reply(result)\n else Reply(Error, error)\n\nlet pstring s = stringReturn s s\nlet skipString s = stringReturn s ()\n\nlet pstringCI s : Parser =\n checkStringContainsNoNewlineOrEOSChar s \"pstringCI\"\n let error = expectedStringCI s\n let cfs = foldCase s\n fun stream ->\n let index0 = stream.IndexToken\n if stream.SkipCaseFolded(cfs) then\n Reply(stream.ReadFrom(index0))\n else Reply(Error, error)\n\nlet stringCIReturn (s: string) result : Parser<'a,'u> =\n let error = expectedStringCI s\n if s.Length = 1 then\n let c = s[0]\n if not (isCertainlyNoNLOrEOS c) then\n match c with '\\r'|'\\n'|EOS -> newlineOrEOSCharInStringArg \"skipStringCI/stringCIReturn\" s 0 | _ -> ()\n let cfc = Text.FoldCase(c)\n fun stream ->\n if stream.SkipCaseFolded(cfc) then Reply(result)\n else Reply(Error, error)\n else\n checkStringContainsNoNewlineOrEOSChar s \"skipStringCI/stringCIReturn\"\n let cfs = foldCase s\n fun stream ->\n if stream.SkipCaseFolded(cfs) then Reply(result)\n else Reply(Error, error)\n\nlet skipStringCI s = stringCIReturn s ()\n\n\nlet anyString n : Parser =\n let error = Errors.ExpectedAnySequenceOfNChars(n)\n fun stream ->\n let state = stream.State\n let str = stream.ReadCharsOrNewlines(n, true)\n if str.Length = n then Reply(str)\n else\n stream.BacktrackTo(state)\n Reply(Error, error)\n\nlet skipAnyString n : Parser =\n let error = Errors.ExpectedAnySequenceOfNChars(n)\n fun stream ->\n let state = stream.State\n if stream.SkipCharsOrNewlines(n) = n then Reply(())\n else\n stream.BacktrackTo(state)\n Reply(Error, error)\n\nlet restOfLine skipNewline : Parser<_,_> =\n fun stream ->\n Reply(stream.ReadRestOfLine(skipNewline))\n\nlet skipRestOfLine skipNewline : Parser<_,_> =\n fun stream ->\n stream.SkipRestOfLine(skipNewline)\n Reply(())\n\nlet charsTillString (s: string) skipString maxCount : Parser =\n checkStringContainsNoNewlineOrEOSChar s \"charsTillString\"\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n let error = Errors.CouldNotFindString(s)\n fun stream ->\n let mutable charsBeforeString = null\n stream.SkipCharsOrNewlinesUntilString(s, maxCount, true, &charsBeforeString) |> ignore\n if isNotNull charsBeforeString then\n if skipString then stream.Skip(s.Length)\n Reply(charsBeforeString)\n else\n Reply(Error, error)\n\nlet charsTillStringCI (s: string) skipString maxCount : Parser =\n checkStringContainsNoNewlineOrEOSChar s \"charsTillStringCI\"\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n let cfs = foldCase s\n let error = Errors.CouldNotFindCaseInsensitiveString(s)\n fun stream ->\n let mutable charsBeforeString = null\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(cfs, maxCount, true, &charsBeforeString) |> ignore\n if isNotNull charsBeforeString then\n if skipString then stream.Skip(s.Length)\n Reply(charsBeforeString)\n else\n Reply(Error, error)\n\n\nlet skipCharsTillString (s: string) skipString maxCount : Parser =\n checkStringContainsNoNewlineOrEOSChar s \"skipCharsTillString\"\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n let error = Errors.CouldNotFindString(s)\n fun stream ->\n let mutable foundString = false\n stream.SkipCharsOrNewlinesUntilString(s, maxCount, &foundString) |> ignore\n if foundString then\n if skipString then stream.Skip(s.Length)\n Reply(())\n else\n Reply(Error, error)\n\nlet skipCharsTillStringCI (s: string) skipString maxCount : Parser =\n checkStringContainsNoNewlineOrEOSChar s \"skipCharsTillStringCI\"\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n let cfs = foldCase s\n let error = Errors.CouldNotFindCaseInsensitiveString(s)\n fun stream ->\n let mutable foundString = false\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(cfs, maxCount, &foundString) |> ignore\n if foundString then\n if skipString then stream.Skip(s.Length)\n Reply(())\n else\n Reply(Error, error)\n\nlet\n#if !NOINLINE\n inline\n#endif\n internal manySatisfyImpl require1 (f1: char -> bool) (f: char -> bool) error : Parser =\n fun stream ->\n let str = stream.ReadCharsOrNewlinesWhile(f1, f, true)\n if not require1 || str.Length <> 0 then Reply(str)\n else Reply(Error, error)\n\nlet\n#if !NOINLINE\n inline\n#endif\n internal skipManySatisfyImpl require1 (f1: char -> bool) (f: char -> bool) error : Parser =\n fun stream ->\n let n = stream.SkipCharsOrNewlinesWhile(f1, f)\n if not require1 || n <> 0 then Reply(())\n else Reply(Error, error)\n\nlet manySatisfy2 f1 f = manySatisfyImpl false f1 f NoErrorMessages\nlet many1Satisfy2 f1 f = manySatisfyImpl true f1 f NoErrorMessages\nlet many1Satisfy2L f1 f label = manySatisfyImpl true f1 f (expected label)\n\nlet skipManySatisfy2 f1 f = skipManySatisfyImpl false f1 f NoErrorMessages\nlet skipMany1Satisfy2 f1 f = skipManySatisfyImpl true f1 f NoErrorMessages\nlet skipMany1Satisfy2L f1 f label = skipManySatisfyImpl true f1 f (expected label)\n\nlet manySatisfy f = manySatisfy2 f f\nlet many1Satisfy f = many1Satisfy2 f f\nlet many1SatisfyL f label = many1Satisfy2L f f label\n\nlet skipManySatisfy f = skipManySatisfy2 f f\nlet skipMany1Satisfy f = skipMany1Satisfy2 f f\nlet skipMany1SatisfyL f label = skipMany1Satisfy2L f f label\n\n\nlet internal manyMinMaxSatisfy2E minCount maxCount f1 f error : Parser =\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n if minCount > 0 then\n fun stream ->\n let str = stream.ReadCharsOrNewlinesWhile(f1, f, minCount, maxCount, true)\n if str.Length <> 0 then Reply(str)\n else Reply(Error, error)\n else\n fun stream ->\n Reply(stream.ReadCharsOrNewlinesWhile(f1, f, 0, maxCount, true))\n\nlet internal skipManyMinMaxSatisfy2E minCount maxCount f1 f error : Parser =\n if maxCount < 0 then raise (System.ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\"))\n if minCount > 0 then\n fun stream ->\n let n = stream.SkipCharsOrNewlinesWhile(f1, f, minCount, maxCount)\n if n <> 0 then Reply(())\n else Reply(Error, error)\n else\n fun stream ->\n stream.SkipCharsOrNewlinesWhile(f1, f, 0, maxCount) |> ignore\n Reply(())\n\nlet manyMinMaxSatisfy minCount maxCount f = manyMinMaxSatisfy2E minCount maxCount f f NoErrorMessages\nlet manyMinMaxSatisfyL minCount maxCount f label = manyMinMaxSatisfy2E minCount maxCount f f (expected label)\nlet manyMinMaxSatisfy2 minCount maxCount f1 f = manyMinMaxSatisfy2E minCount maxCount f1 f NoErrorMessages\nlet manyMinMaxSatisfy2L minCount maxCount f1 f label = manyMinMaxSatisfy2E minCount maxCount f1 f (expected label)\n\nlet skipManyMinMaxSatisfy minCount maxCount f = skipManyMinMaxSatisfy2E minCount maxCount f f NoErrorMessages\nlet skipManyMinMaxSatisfyL minCount maxCount f label = skipManyMinMaxSatisfy2E minCount maxCount f f (expected label)\nlet skipManyMinMaxSatisfy2 minCount maxCount f1 f = skipManyMinMaxSatisfy2E minCount maxCount f1 f NoErrorMessages\nlet skipManyMinMaxSatisfy2L minCount maxCount f1 f label = skipManyMinMaxSatisfy2E minCount maxCount f1 f (expected label)\n\n\nlet internal regexE pattern error : Parser =\n let regex = new Regex(\"\\\\A\" + pattern, RegexOptions.Multiline |||\n RegexOptions.ExplicitCapture)\n fun stream ->\n let m = stream.Match(regex)\n if m.Success then\n let str = m.Value\n if findNewlineOrEOSChar str < 0 then\n if str.Length <> 0 then stream.Skip(str.Length)\n Reply(str)\n else\n let nStr = normalizeNewlines str\n let mutable nSkippedChars = 0\n let n = stream.SkipCharsOrNewlines(nStr.Length)\n if n = nStr.Length then Reply(nStr)\n else Reply(FatalError, messageError \"Internal error in the regex parser. Please report this error to fparsec@quanttec.com.\")\n else Reply(Error, error)\n\nlet regex pattern = regexE pattern (Errors.ExpectedStringMatchingRegex(pattern))\nlet regexL pattern label = regexE pattern (expected label)\n\ntype private IdFlags = IdentifierValidator.IdentifierCharFlags\n\ntype IdentifierOptions(?isAsciiIdStart, ?isAsciiIdContinue,\n ?normalization,\n ?normalizeBeforeValidation,\n ?allowJoinControlChars, ?preCheckStart, ?preCheckContinue, ?allowAllNonAsciiCharsInPreCheck, ?label, ?invalidCharMessage) =\n // we use match instead of defaultArg here, so that the function wrapper objects only get constructed when needed\n let isAsciiIdStart = match isAsciiIdStart with Some v -> v | _ -> IdentifierValidator.IsXIdStartOrSurrogate\n let isAsciiIdContinue = match isAsciiIdContinue with Some v -> v | _ -> IdentifierValidator.IsXIdContinueOrSurrogate\n let normalizationForm = defaultArg normalization (enum 0)\n let normalizeBeforeValidation = defaultArg normalizeBeforeValidation false\n let allowJoinControlChars = defaultArg allowJoinControlChars false\n let expectedIdentifierError = expected (defaultArg label Strings.Identifier)\n let invalidCharError = messageError (defaultArg invalidCharMessage Strings.IdentifierContainsInvalidCharacterAtIndicatedPosition)\n let allowAllNonAsciiCharsInPreCheck = defaultArg allowAllNonAsciiCharsInPreCheck false\n\n let preCheckStart = if preCheckStart.IsSome then preCheckStart.Value\n elif allowAllNonAsciiCharsInPreCheck then isAsciiIdStart\n else Unchecked.defaultof<_>\n let preCheckContinue = if preCheckContinue.IsSome then preCheckContinue.Value\n elif allowAllNonAsciiCharsInPreCheck then isAsciiIdContinue\n else Unchecked.defaultof<_>\n\n let asciiOptions = Array.zeroCreate 128\n do for i = 1 to 127 do\n let c = char i\n let mutable v = IdFlags.None\n if isAsciiIdStart c then v <- v ||| IdFlags.NonContinue\n if isAsciiIdContinue c then v <- v ||| IdFlags.Continue\n if allowAllNonAsciiCharsInPreCheck then\n if preCheckStart c then v <- v ||| IdFlags.PreCheckNonContinue\n if preCheckContinue c then v <- v ||| IdFlags.PreCheckContinue\n asciiOptions[i] <- v\n\n let iv = new IdentifierValidator(asciiOptions)\n do\n iv.NormalizationForm <- normalizationForm\n iv.NormalizeBeforeValidation <- normalizeBeforeValidation\n iv.AllowJoinControlCharsAsIdContinueChars <- allowJoinControlChars\n\n let preCheck1 =\n if allowAllNonAsciiCharsInPreCheck then\n fun c -> let i = int c\n if i <= 0x7f then\n // not (x = y) currently yields better code here than (x <> y)\n not (asciiOptions[int c] &&& IdFlags.PreCheckNonContinue = IdFlags.None)\n else true\n elif isNotNull preCheckStart then preCheckStart\n else iv.IsIdStartOrSurrogateFunc\n\n let preCheck =\n if allowAllNonAsciiCharsInPreCheck then\n fun c -> let i = int c\n if i <= 0x7f then\n not (asciiOptions[i] &&& IdFlags.PreCheckContinue = IdFlags.None)\n else true\n elif isNotNull preCheckContinue then preCheckContinue\n else iv.IsIdContinueOrJoinControlOrSurrogateFunc\n\n member internal t.IdentifierValidator = iv\n member internal t.PreCheck1 = preCheck1\n member internal t.PreCheck = preCheck\n member internal t.ExpectedIdentifierError = expectedIdentifierError\n member internal t.InvalidCharError = invalidCharError\n\nlet identifier (identifierOptions: IdentifierOptions) : Parser =\n let validator = identifierOptions.IdentifierValidator\n let preCheck1 = identifierOptions.PreCheck1\n let preCheck = identifierOptions.PreCheck\n let expectedIdentifierError = identifierOptions.ExpectedIdentifierError\n let invalidCharError = identifierOptions.InvalidCharError\n fun stream ->\n let str = stream.ReadCharsOrNewlinesWhile(preCheck1, preCheck, true)\n if str.Length <> 0 then\n let mutable errorPos = 0\n let nstr = validator.ValidateAndNormalize(str, &errorPos)\n if isNotNull nstr then Reply(nstr)\n else\n stream.Skip(errorPos - str.Length)\n Reply(FatalError, invalidCharError)\n else\n Reply(Error, expectedIdentifierError)\n\n// ----------------------------------------------\n// Parsing strings with the help of other parsers\n// ----------------------------------------------\n\nlet manyChars2 p1 p = ManyChars(p1, p).AsFSharpFunc\nlet manyChars p = manyChars2 p p\n\nlet many1Chars2 p1 p = Many1Chars(p1, p).AsFSharpFunc\nlet many1Chars p = many1Chars2 p p\n\nlet manyCharsTillApply2 p1 p endp f = ManyCharsTill(p1, p, endp, f).AsFSharpFunc\nlet manyCharsTillApply p endp f = manyCharsTillApply2 p p endp f\nlet manyCharsTill2 p1 p endp = manyCharsTillApply2 p1 p endp (fun str _ -> str)\nlet manyCharsTill p endp = manyCharsTill2 p p endp\n\nlet many1CharsTillApply2 p1 p endp f = Many1CharsTill(p1, p, endp, f).AsFSharpFunc\nlet many1CharsTillApply p endp f = many1CharsTillApply2 p p endp f\nlet many1CharsTill2 p1 p endp = many1CharsTillApply2 p1 p endp (fun str _ -> str)\nlet many1CharsTill p endp = many1CharsTill2 p p endp\n\n\n\nlet\n#if !NOINLINE\n inline\n#endif\n internal manyStringsImpl require1 (p1: Parser) (p: Parser) : Parser =\n fun stream ->\n let mutable stateTag = stream.StateTag\n let mutable reply = p1 stream\n if reply.Status = Ok then\n let result1 = reply.Result\n let mutable error = reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status <> Ok then reply.Result <- result1\n else\n let result2 = reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status <> Ok then reply.Result <- result1 + result2\n else\n let result3 = reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status <> Ok then reply.Result <- concat3 result1 result2 result3\n else\n let result4 = reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status <> Ok then reply.Result <- concat4 result1 result2 result3 result4\n else\n let n = 2*(result1.Length + result2.Length + result3.Length + result4.Length) + reply.Result.Length\n let sb = new StringBuilder(n)\n sb.Append(result1).Append(result2).Append(result3).Append(result4).Append(reply.Result) |> ignore\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n while reply.Status = Ok do\n if stateTag = stream.StateTag then\n raiseInfiniteLoopException \"manyStrings\" stream\n error <- reply.Error\n sb.Append(reply.Result) |> ignore\n stateTag <- stream.StateTag\n reply <- p stream\n reply.Result <- sb.ToString()\n // We assume that the string parser changes the state when it succeeds,\n // so we don't need to merge more than 2 error message lists.\n if stateTag = stream.StateTag then\n if reply.Status = Error then\n reply.Status <- Ok\n if isNotNull error then\n reply.Error <- mergeErrors error reply.Error\n elif not require1 && reply.Status = Error && stateTag = stream.StateTag then\n reply.Status <- Ok\n reply.Result <- \"\"\n reply\n\nlet manyStrings2 p1 p = manyStringsImpl false p1 p\nlet manyStrings p = manyStrings2 p p\nlet many1Strings2 p1 p = manyStringsImpl true p1 p\nlet many1Strings p = many1Strings2 p p\n\nlet\n#if !NOINLINE\n inline\n#endif\n internal stringsSepByImpl require1 (p: Parser) (sep: Parser) : Parser =\n fun stream ->\n let mutable stateTag = stream.StateTag\n let mutable reply = p stream\n if reply.Status = Ok then\n let result1 = reply.Result\n let mutable error = reply.Error\n stateTag <- stream.StateTag\n reply <- sep stream\n if reply.Status <> Ok then\n if stateTag = stream.StateTag then\n if reply.Status = Error then\n reply.Status <- Ok\n reply.Result <- result1\n if isNotNull error then\n reply.Error <- mergeErrors error reply.Error\n else\n // We assume that at least one of the parsers sep and p consume\n // input when both are called consecutively and succeed. This\n // way we only have to merge a maximum of 3 error message lists.\n let mutable result = null\n let mutable error0 = error\n let mutable stateTag0 = stateTag\n let result2 = reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status = Ok then\n let result3 = reply.Result\n error0 <- error\n stateTag0 <- stateTag\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- sep stream\n if reply.Status <> Ok then result <- concat3 result1 result2 result3\n else\n let result4 = reply.Result\n error0 <- error\n stateTag0 <- stateTag\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if reply.Status = Ok then\n let n = 2*(result1.Length + result2.Length + result3.Length + result4.Length) + reply.Result.Length\n let sb = new StringBuilder(n)\n sb.Append(result1).Append(result2).Append(result3).Append(result4) |> ignore\n while reply.Status = Ok do\n sb.Append(reply.Result) |> ignore\n error0 <- error\n stateTag0 <- stateTag\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- sep stream\n if reply.Status <> Ok then result <- sb.ToString()\n else\n sb.Append(reply.Result) |> ignore\n if stateTag0 = stream.StateTag then\n raiseInfiniteLoopException \"stringsSepBy\" stream\n error0 <- error\n stateTag0 <- stateTag\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- p stream\n if stateTag = stream.StateTag then\n if isNotNull result && reply.Status = Error then\n reply.Status <- Ok\n reply.Result <- result\n error <- mergeErrors error reply.Error\n if stateTag0 = stateTag then\n error <- mergeErrors error0 error\n reply.Error <- error\n elif not require1 && reply.Status = Error && stateTag = stream.StateTag then\n reply.Status <- Ok\n reply.Result <- \"\"\n reply\n\nlet stringsSepBy p sep = stringsSepByImpl false p sep\nlet stringsSepBy1 p sep = stringsSepByImpl true p sep\n\nlet skipped (p: Parser) : Parser =\n fun stream ->\n let index0 = stream.IndexToken\n let line0 = stream.Line\n let reply = p stream\n if reply.Status = Ok then\n let str = stream.ReadFrom(index0)\n let nstr = if line0 = stream.Line then str\n else Text.NormalizeNewlines(str)\n Reply(Ok, nstr, reply.Error)\n else\n Reply(reply.Status, reply.Error)\n\nlet withSkippedString (f: string -> 'a -> 'b) (p: Parser<'a,'u>) : Parser<'b,'u> =\n let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n fun stream ->\n let index0 = stream.IndexToken\n let line0 = stream.Line\n let reply = p stream\n if reply.Status = Ok then\n let str = stream.ReadFrom(index0)\n let nstr = if line0 = stream.Line then str\n else Text.NormalizeNewlines(str)\n let result = optF.Invoke(nstr, reply.Result)\n Reply(Ok, result, reply.Error)\n else\n Reply(reply.Status, reply.Error)\n\n// ---------------\n// Parsing numbers\n// ---------------\n\n[]\ntype NumberLiteralOptions =\n | None = 0\n | AllowSuffix = 0b000000000001\n | AllowMinusSign = 0b000000000010\n | AllowPlusSign = 0b000000000100\n | AllowFraction = 0b000000001000\n | AllowFractionWOIntegerPart = 0b000000010000\n | AllowExponent = 0b000000100000\n | AllowHexadecimal = 0b000001000000\n | AllowBinary = 0b000010000000\n | AllowOctal = 0b000100000000\n | AllowInfinity = 0b001000000000\n | AllowNaN = 0b010000000000\n\n | IncludeSuffixCharsInString = 0b100000000000\n\n | DefaultInteger = 0b000111000110\n | DefaultUnsignedInteger = 0b000111000000\n | DefaultFloat = 0b011001101110\n\ntype internal NLO = NumberLiteralOptions\n\n[]\ntype NumberLiteralResultFlags =\n | None = 0\n | SuffixLengthMask = 0b0000000000001111\n | HasMinusSign = 0b0000000000010000\n | HasPlusSign = 0b0000000000100000\n | HasIntegerPart = 0b0000000001000000\n | HasFraction = 0b0000000010000000\n | HasExponent = 0b0000000100000000\n | IsDecimal = 0b0000001000000000\n | IsHexadecimal = 0b0000010000000000\n | IsBinary = 0b0000100000000000\n | IsOctal = 0b0001000000000000\n | BaseMask = 0b0001111000000000\n | IsInfinity = 0b0010000000000000\n | IsNaN = 0b0100000000000000\n\ntype internal NLF = NumberLiteralResultFlags\n\ntype NumberLiteral(string, info, suffixChar1, suffixChar2, suffixChar3, suffixChar4) =\n member t.String = string\n\n member t.SuffixLength = int (info &&& NLF.SuffixLengthMask)\n member t.SuffixChar1 = suffixChar1\n member t.SuffixChar2 = suffixChar2\n member t.SuffixChar3 = suffixChar3\n member t.SuffixChar4 = suffixChar4\n\n member t.Info = info\n\n member t.HasMinusSign = int (info &&& NLF.HasMinusSign) <> 0\n member t.HasPlusSign = int (info &&& NLF.HasPlusSign) <> 0\n member t.HasIntegerPart = int (info &&& NLF.HasIntegerPart) <> 0\n member t.HasFraction = int (info &&& NLF.HasFraction) <> 0\n member t.HasExponent = int (info &&& NLF.HasExponent) <> 0\n member t.IsInteger = int (info &&& (NLF.HasFraction ||| NLF.HasExponent)) = 0 // HasIntegerPart must be set if HasFraction and HasExponent both aren't\n member t.IsDecimal = int (info &&& NLF.IsDecimal) <> 0\n member t.IsHexadecimal = int (info &&& NLF.IsHexadecimal) <> 0\n member t.IsBinary = int (info &&& NLF.IsBinary) <> 0\n member t.IsOctal = int (info &&& NLF.IsOctal) <> 0\n member t.IsNaN = int (info &&& NLF.IsNaN) <> 0\n member t.IsInfinity = int (info &&& NLF.IsInfinity) <> 0\n\n override t.Equals(other: obj) =\n match other with\n | :? NumberLiteral as other ->\n t.String = other.String\n && t.Info = other.Info\n && t.SuffixChar1 = other.SuffixChar1\n && t.SuffixChar2 = other.SuffixChar2\n && t.SuffixChar3 = other.SuffixChar3\n && t.SuffixChar4 = other.SuffixChar4\n | _ -> false\n\n override t.GetHashCode() =\n if isNotNull string then string.GetHashCode() else 0\n\nlet numberLiteralE (opt: NumberLiteralOptions) (errorInCaseNoLiteralFound: ErrorMessageList) (stream: CharStream<'u>) =\n let index0 = stream.IndexToken\n let stateTag = stream.StateTag\n let mutable c = stream.Peek()\n let mutable error = NoErrorMessages\n let mutable flags = NLF.None\n\n if c = '-' && (opt &&& NLO.AllowMinusSign) <> NLO.None then\n flags <- NLF.HasMinusSign\n c <- stream.SkipAndPeek()\n elif c = '+' && (opt &&& NLO.AllowPlusSign) <> NLO.None then\n flags <- NLF.HasPlusSign\n c <- stream.SkipAndPeek()\n\n let allowStartingPoint = NLO.AllowFraction ||| NLO.AllowFractionWOIntegerPart // for starting point both flags are required\n\n if isDigit c || (c = '.' && (opt &&& allowStartingPoint) = allowStartingPoint) then\n let mutable c1 = '\\u0000'\n if c <> '0'\n || (c1 <- stream.SkipAndPeek();\n c1 <= '9'\n || (opt &&& (NLO.AllowBinary ||| NLO.AllowOctal ||| NLO.AllowHexadecimal)) = NLO.None\n || ((int c1 ||| int ' ') = int 'e'))\n then\n flags <- flags ||| NLF.IsDecimal\n if c <> '.' then\n flags <- flags ||| NLF.HasIntegerPart\n if c <> '0' then\n c <- stream.SkipAndPeek()\n else\n c <- c1\n while isDigit c do\n c <- stream.SkipAndPeek()\n if c = '.' && (opt &&& NLO.AllowFraction) <> NLO.None then\n flags <- flags ||| NLF.HasFraction\n c <- stream.SkipAndPeek()\n if isDigit c then\n c <- stream.SkipAndPeek()\n elif (flags &&& NLF.HasIntegerPart) = NLF.None then\n // at least one digit before or after the . is required\n error <- Errors.ExpectedDecimalDigit\n while isDigit c do\n c <- stream.SkipAndPeek()\n if (int c ||| int ' ') = int 'e' && isNull error && (opt &&& NLO.AllowExponent) <> NLO.None then\n flags <- flags ||| NLF.HasExponent\n c <- stream.SkipAndPeek()\n if c = '-' || c = '+' then\n c <- stream.SkipAndPeek()\n if not (isDigit c) then\n error <- Errors.ExpectedDecimalDigit\n while isDigit c do\n c <- stream.SkipAndPeek()\n else\n match int c1 ||| int ' ' with\n | 0x78 (* 'x' *) when (opt &&& NLO.AllowHexadecimal) <> NLO.None ->\n flags <- flags ||| NLF.IsHexadecimal\n c <- stream.SkipAndPeek()\n if isHex c then\n flags <- flags ||| NLF.HasIntegerPart\n c <- stream.SkipAndPeek()\n elif (opt &&& NLO.AllowFractionWOIntegerPart) = NLO.None then\n // integer part required\n error <- Errors.ExpectedHexadecimalDigit\n while isHex c do\n c <- stream.SkipAndPeek()\n if c = '.' && isNull error && (opt &&& NLO.AllowFraction) <> NLO.None then\n flags <- flags ||| NLF.HasFraction\n c <- stream.SkipAndPeek()\n if isHex c then\n c <- stream.SkipAndPeek()\n elif (flags &&& NLF.HasIntegerPart) = NLF.None then\n // at least one digit before or after the . is required\n error <- Errors.ExpectedHexadecimalDigit\n while isHex c do\n c <- stream.SkipAndPeek()\n elif (flags &&& NLF.HasIntegerPart) = NLF.None then\n // we neither have an integer part nor a fraction\n error <- Errors.ExpectedHexadecimalDigit\n if (int c ||| int ' ') = int 'p' && isNull error && (opt &&& NLO.AllowExponent) <> NLO.None then\n flags <- flags ||| NLF.HasExponent\n c <- stream.SkipAndPeek()\n if c = '-' || c = '+' then\n c <- stream.SkipAndPeek()\n if not (isDigit c) then\n error <- Errors.ExpectedDecimalDigit\n while isDigit c do\n c <- stream.SkipAndPeek()\n | 0x6f (* 'o' *) when (opt &&& NLO.AllowOctal) <> NLO.None ->\n flags <- flags ||| NLF.IsOctal\n c <- stream.SkipAndPeek()\n if isOctal c then\n flags <- flags ||| NLF.HasIntegerPart\n c <- stream.SkipAndPeek()\n else\n error <- Errors.ExpectedOctalDigit\n while isOctal c do\n c <- stream.SkipAndPeek()\n | 0x62 (* 'b' *) when (opt &&& NLO.AllowBinary) <> NLO.None ->\n flags <- flags ||| NLF.IsBinary\n c <- stream.SkipAndPeek()\n if c = '0' || c = '1' then\n flags <- flags ||| NLF.HasIntegerPart\n c <- stream.SkipAndPeek()\n else\n error <- Errors.ExpectedBinaryDigit\n while c = '0' || c = '1' do\n c <- stream.SkipAndPeek()\n | _ ->\n flags <- flags ||| (NLF.IsDecimal ||| NLF.HasIntegerPart)\n c <- c1\n\n if isNull error then\n if (opt &&& NLO.AllowSuffix) = NLO.None || not (isAsciiLetter c) then\n let str = stream.ReadFrom(index0)\n Reply(NumberLiteral(str, flags, EOS, EOS, EOS, EOS))\n else\n let mutable str = if (opt &&& NLO.IncludeSuffixCharsInString) <> NLO.None then null\n else stream.ReadFrom(index0)\n let mutable nSuffix = 1\n let mutable s1 = c\n let mutable s2 = EOS\n let mutable s3 = EOS\n let mutable s4 = EOS\n c <- stream.SkipAndPeek()\n if isAsciiLetter c then\n nSuffix <- 2\n s2 <- c\n c <- stream.SkipAndPeek()\n if isAsciiLetter c then\n nSuffix <- 3\n s3 <- c\n c <- stream.SkipAndPeek()\n if isAsciiLetter c then\n nSuffix <- 4\n s4 <- c\n c <- stream.SkipAndPeek()\n flags <- flags ||| (enum) nSuffix\n if (opt &&& NLO.IncludeSuffixCharsInString) <> NLO.None then\n str <- stream.ReadFrom(index0)\n Reply(NumberLiteral(str, flags, s1, s2, s3, s4))\n else\n Reply(Error, error)\n else\n let cc = int c ||| int ' '\n if\n if cc = int 'i' then\n (opt &&& NLO.AllowInfinity) <> NLO.None\n && stream.SkipCaseFolded(\"inf\") && (flags <- flags ||| NLF.IsInfinity\n stream.SkipCaseFolded(\"inity\") |> ignore\n true)\n elif cc = int 'n' then\n (opt &&& NLO.AllowNaN) <> NLO.None\n && stream.SkipCaseFolded(\"nan\") && (flags <- flags ||| NLF.IsNaN\n true)\n else false\n then\n let str = stream.ReadFrom(index0)\n Reply(NumberLiteral(str, flags, EOS, EOS, EOS, EOS))\n else\n if flags &&& (NLF.HasMinusSign ||| NLF.HasPlusSign) <> NLF.None then\n stream.Seek(index0)\n stream.StateTag <- stateTag\n Reply(Error, errorInCaseNoLiteralFound)\n\nlet numberLiteral opt label = numberLiteralE opt (expected label)\n\nlet pfloat : Parser =\n fun stream ->\n let reply = numberLiteralE NLO.DefaultFloat Errors.ExpectedFloatingPointNumber stream\n if reply.Status = Ok then\n let nl = reply.Result\n try\n let d = if nl.IsDecimal then\n System.Double.Parse(nl.String, System.Globalization.CultureInfo.InvariantCulture)\n elif nl.IsHexadecimal then\n floatOfHexString nl.String\n elif nl.IsInfinity then\n if nl.HasMinusSign then System.Double.NegativeInfinity else System.Double.PositiveInfinity\n else\n System.Double.NaN\n Reply(d)\n with \n | :? System.OverflowException ->\n Reply(if nl.HasMinusSign then System.Double.NegativeInfinity else System.Double.PositiveInfinity)\n | :? System.FormatException ->\n stream.Skip(-nl.String.Length)\n Reply(FatalError, messageError \"The floating-point number has an invalid format (this error is unexpected, please report this error message to fparsec@quanttec.com).\")\n else\n Reply(reply.Status, reply.Error)\n\nlet internal parseUInt64 (c0: char) (stream: CharStream<'u>) (status: ReplyStatus byref) (error: ErrorMessageList byref) =\n Debug.Assert(isDigit c0 && (status = Ok))\n\n // we rely on the compiler eliminating inactive branches\n let opt = NumberLiteralOptions.DefaultUnsignedInteger\n let limit10 = 1844674407370955160UL //(System.UInt64.MaxValue - 9UL)/10UL\n let maxDiv10 = 1844674407370955161UL //System.UInt64.MaxValue/10UL\n let maxMod10 = 5u //System.UInt64.MaxValue%10UL\n\n let limit16 = 1152921504606846975UL //(System.UInt64.MaxValue - 15UL)/16UL\n let maxDiv16 = 1152921504606846975UL //System.UInt64.MaxValue/16UL\n let maxMod16 = 15u //System.UInt64.MaxValue%16UL\n\n let limit8 = 2305843009213693951UL //(System.UInt64.MaxValue - 7UL)/8UL\n let maxDiv8 = 2305843009213693951UL //System.UInt64.MaxValue/8UL\n let maxMod8 = 7u //System.UInt64.MaxValue%8UL\n\n let limit2 = 9223372036854775807UL //(System.UInt64.MaxValue - 1UL)/2UL\n let maxDiv2 = 9223372036854775807UL //System.UInt64.MaxValue/2UL\n let maxMod2 = 1u //System.UInt64.MaxValue%2UL\n\n let mutable n = 0UL\n let mutable c = c0\n let c1 = stream.SkipAndPeek()\n\n if (opt &&& (NLO.AllowBinary ||| NLO.AllowOctal ||| NLO.AllowHexadecimal)) = NLO.None\n || c <> '0' || c1 <= '9'\n then\n n <- uint64 (uint32 c - uint32 '0')\n c <- c1\n while c >= '0' && c <= '9' do\n let nc = uint32 c - uint32 '0'\n if n <= limit10 || (maxMod10 < 9u && n = maxDiv10 && nc <= maxMod10) then\n n <- 10UL*n + uint64 nc\n c <- stream.SkipAndPeek()\n else\n status <- FatalError\n c <- '!' // break\n\n else\n let cc1 = uint32 c1 ||| uint32 ' '\n if (opt &&& NLO.AllowHexadecimal) <> NLO.None && cc1 = uint32 'x' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 0\n if (let cc = uint32 c ||| uint32 ' '\n if c <= '9' then nc <- uint32 c - uint32 '0'; c >= '0'\n else cc <= uint32 'f' && (nc <- cc - 0x57u; cc >= uint32 'a')) // 0x57u = uint32 'a' - 10u\n then\n n <- uint64 nc\n c <- stream.SkipAndPeek()\n while\n (let cc = uint32 c ||| uint32 ' '\n if c <= '9' then nc <- uint32 c - uint32 '0'; c >= '0'\n else cc <= uint32 'f' && (nc <- cc - 0x57u; cc >= uint32 'a'))\n do\n if n <= limit16 || (maxMod16 < 15u && n = maxDiv16 && nc <= maxMod16) then\n n <- 16UL*n + uint64 nc\n c <- stream.SkipAndPeek()\n else\n status <- FatalError\n c <- '!' // break\n else\n status <- Error\n error <- Errors.ExpectedHexadecimalDigit\n\n elif (opt &&& NLO.AllowOctal) <> NLO.None && cc1 = uint32 'o' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 c - uint32 '0'\n if nc = (nc &&& 7u) then\n n <- uint64 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n while nc = (nc &&& 7u) do\n if n <= limit8 || (maxMod8 < 7u && n = maxDiv8 && nc <= maxMod8) then\n n <- 8UL*n + uint64 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n else\n status <- FatalError\n nc <- 11u // break\n else\n status <- Error\n error <- Errors.ExpectedOctalDigit\n\n elif (opt &&& NLO.AllowBinary) <> NLO.None && cc1 = uint32 'b' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 c - uint32 '0'\n if nc = (nc &&& 1u) then\n n <- uint64 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n while nc = (nc &&& 1u) do\n if n <= limit2 || (maxMod2 = 0u && n = maxDiv2 && nc = 0u) then\n n <- 2UL*n + uint64 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n else\n status <- FatalError\n nc <- 11u // break\n else\n status <- Error\n error <- Errors.ExpectedBinaryDigit\n // else c = 0 && not (isDigit c1)\n n\n\nlet internal parseUInt32 (c0: char) (stream: CharStream<'u>) (status: ReplyStatus byref) (error: ErrorMessageList byref) =\n Debug.Assert(isDigit c0 && (status = Ok))\n\n // we rely on the compiler eliminating inactive branches\n let opt = NumberLiteralOptions.DefaultUnsignedInteger\n let limit10 = 429496728u //(System.UInt32.MaxValue - 9u)/10u\n let maxDiv10 = 429496729u //System.UInt32.MaxValue/10u\n let maxMod10 = 5u //System.UInt32.MaxValue%10u\n\n let limit16 = 268435455u //(System.UInt32.MaxValue - 15u)/16u\n let maxDiv16 = 268435455u //System.UInt32.MaxValue/16u\n let maxMod16 = 15u //System.UInt32.MaxValue%16u\n\n let limit8 = 536870911u //(System.UInt32.MaxValue - 7u)/8u\n let maxDiv8 = 536870911u //System.UInt32.MaxValue/8u\n let maxMod8 = 7u //System.UInt32.MaxValue%8u\n\n let limit2 = 2147483647u //(System.UInt32.MaxValue - 1u)/2u\n let maxDiv2 = 2147483647u //System.UInt32.MaxValue/2u\n let maxMod2 = 1u //System.UInt32.MaxValue%2u\n\n let mutable n = 0u\n let mutable c = c0\n let c1 = stream.SkipAndPeek()\n\n if (opt &&& (NLO.AllowBinary ||| NLO.AllowOctal ||| NLO.AllowHexadecimal)) = NLO.None\n || c <> '0' || c1 <= '9'\n then\n n <- uint32 c - uint32 '0'\n c <- c1\n while c >= '0' && c <= '9' do\n let nc = uint32 c - uint32 '0'\n if n <= limit10 || (maxMod10 < 9u && n = maxDiv10 && nc <= maxMod10) then\n n <- 10u*n + nc\n c <- stream.SkipAndPeek()\n else\n status <- FatalError\n c <- '!' // break\n\n else\n let cc1 = uint32 c1 ||| uint32 ' '\n if (opt &&& NLO.AllowHexadecimal) <> NLO.None && cc1 = uint32 'x' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 0\n if (let cc = uint32 c ||| uint32 ' '\n if c <= '9' then nc <- uint32 c - uint32 '0'; c >= '0'\n else cc <= uint32 'f' && (nc <- cc - 0x57u; cc >= uint32 'a')) // 0x57u = uint32 'a' - 10u\n then\n n <- uint32 nc\n c <- stream.SkipAndPeek()\n while\n (let cc = uint32 c ||| uint32 ' '\n if c <= '9' then nc <- uint32 c - uint32 '0'; c >= '0'\n else cc <= uint32 'f' && (nc <- cc - 0x57u; cc >= uint32 'a'))\n do\n if n <= limit16 || (maxMod16 < 15u && n = maxDiv16 && nc <= maxMod16) then\n n <- 16u*n + nc\n c <- stream.SkipAndPeek()\n else\n status <- FatalError\n c <- '!' // break\n else\n status <- Error\n error <- Errors.ExpectedHexadecimalDigit\n\n elif (opt &&& NLO.AllowOctal) <> NLO.None && cc1 = uint32 'o' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 c - uint32 '0'\n if nc = (nc &&& 7u) then\n n <- uint32 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n while nc = (nc &&& 7u) do\n if n <= limit8 || (maxMod8 < 7u && n = maxDiv8 && nc <= maxMod8) then\n n <- 8u*n + nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n else\n status <- FatalError\n nc <- 11u // break\n else\n status <- Error\n error <- Errors.ExpectedOctalDigit\n\n elif (opt &&& NLO.AllowBinary) <> NLO.None && cc1 = uint32 'b' then\n c <- stream.SkipAndPeek()\n let mutable nc = uint32 c - uint32 '0'\n if nc = (nc &&& 1u) then\n n <- uint32 nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n while nc = (nc &&& 1u) do\n if n <= limit2 || (maxMod2 = 0u && n = maxDiv2 && nc = 0u) then\n n <- 2u*n + nc\n c <- stream.SkipAndPeek()\n nc <- uint32 c - uint32 '0'\n else\n status <- FatalError\n nc <- 11u // break\n else\n status <- Error\n error <- Errors.ExpectedBinaryDigit\n // else c = 0 && not (isDigit c1)\n n\n\n[]\nlet internal overflowError message =\n if isNotNull message then messageError message // isNotNull prevents fsc from inlining the function\n else NoErrorMessages\n\nlet inline internal pint (opt: NumberLiteralOptions) (max: 'uint) (uint64_: 'uint -> uint64) (uint: int -> 'uint) (uint_: uint32 -> 'uint) (uint__: uint64 -> 'uint) (int: 'uint -> 'int) (int_: int -> 'int) (errorInCaseNoLiteralFound: ErrorMessageList) (outOfRangeError: ErrorMessageList) (stream: CharStream<'u>) =\n // we rely on the compiler eliminating inactive branches after inlining\n\n let minusIsAllowed = (opt &&& NLO.AllowMinusSign) <> NLO.None\n\n let index = stream.IndexToken\n let stateTag = stream.StateTag\n let mutable c = stream.Peek()\n\n let mutable plusMinus1 = 1\n let mutable signPresent = false\n if minusIsAllowed && c = '-' then\n plusMinus1 <- -1\n signPresent <- true\n c <- stream.SkipAndPeek()\n elif (opt &&& NLO.AllowPlusSign) <> NLO.None && c = '+' then\n signPresent <- true\n c <- stream.SkipAndPeek()\n\n let mutable status = Ok\n let mutable error = NoErrorMessages\n let mutable result = Unchecked.defaultof<_>\n if c >= '0' && c <= '9' then\n let n = if uint64_ max <= uint64 System.UInt32.MaxValue then\n uint_ (parseUInt32 c stream (&status) (&error))\n else\n uint__ (parseUInt64 c stream (&status) (&error))\n let isUInt32Or64 = uint64_ max = uint64 System.UInt32.MaxValue || uint64_ max = System.UInt64.MaxValue\n if status = Ok && (isUInt32Or64 || (n <= max || (minusIsAllowed && plusMinus1 = -1 && n = max + uint 1))) then\n result <- if minusIsAllowed then int_ plusMinus1 * int n else int n\n elif status <> Error then\n status <- FatalError\n stream.Seek(index)\n stream.StateTag <- stateTag\n error <- outOfRangeError\n else\n status <- Error\n error <- errorInCaseNoLiteralFound\n if signPresent then\n stream.Seek(index)\n stream.StateTag <- stateTag\n Reply(status, result, error)\n\nlet pint64 stream = pint NumberLiteralOptions.DefaultInteger (uint64 System.Int64.MaxValue) uint64 uint64 uint64 uint64 int64 int64 Errors.ExpectedInt64 Errors.NumberOutsideOfInt64Range stream\nlet pint32 stream = pint NumberLiteralOptions.DefaultInteger (uint32 System.Int32.MaxValue) uint64 uint32 uint32 uint32 int32 int32 Errors.ExpectedInt32 Errors.NumberOutsideOfInt32Range stream\n // fsc's optimizer seems to have problems with literals of small int types\nlet pint16 stream = pint NumberLiteralOptions.DefaultInteger ((*uint32 System.Int16.MaxValue*)0x7fffu) uint64 uint32 uint32 uint32 int16 int16 Errors.ExpectedInt16 Errors.NumberOutsideOfInt16Range stream\nlet pint8 stream = pint NumberLiteralOptions.DefaultInteger ((*uint32 System.SByte.MaxValue*)0x7fu) uint64 uint32 uint32 uint32 sbyte sbyte Errors.ExpectedInt8 Errors.NumberOutsideOfInt8Range stream\n\nlet puint64 stream = pint NumberLiteralOptions.DefaultUnsignedInteger System.UInt64.MaxValue uint64 uint64 uint64 uint64 uint64 uint64 Errors.ExpectedUInt64 Errors.NumberOutsideOfUInt64Range stream\nlet puint32 stream = pint NumberLiteralOptions.DefaultUnsignedInteger System.UInt32.MaxValue uint64 uint32 uint32 uint32 uint32 uint32 Errors.ExpectedUInt32 Errors.NumberOutsideOfUInt32Range stream\nlet puint16 stream = pint NumberLiteralOptions.DefaultUnsignedInteger 0xffffu uint64 uint32 uint32 uint32 uint16 uint16 Errors.ExpectedUInt16 Errors.NumberOutsideOfUInt16Range stream\nlet puint8 stream = pint NumberLiteralOptions.DefaultUnsignedInteger 0xffu uint64 uint32 uint32 uint32 byte byte Errors.ExpectedUInt8 Errors.NumberOutsideOfUInt8Range stream\n\n\n\n// -------------------\n// Conditional parsing\n// -------------------\n\nlet notFollowedByEof : Parser =\n fun stream ->\n if not (stream.IsEndOfStream) then Reply(())\n else Reply(Error, Errors.UnexpectedEndOfInput)\n\nlet followedByNewline : Parser =\n fun stream ->\n match stream.Peek() with\n |'\\r' | '\\n' -> Reply(())\n | _ -> Reply(Error, Errors.ExpectedNewline)\n\nlet notFollowedByNewline : Parser =\n fun stream ->\n match stream.Peek() with\n |'\\r' | '\\n' -> Reply(Error, Errors.UnexpectedNewline)\n | _ -> Reply(())\n\nlet followedByString (str: string) : Parser =\n checkStringContainsNoNewlineOrEOSChar str \"followedByString\"\n let error = expectedString str\n if str.Length = 1 then\n let chr = str[0]\n fun stream ->\n if stream.Match(chr) then Reply(())\n else Reply(Error, error)\n else\n fun stream ->\n if stream.Match(str) then Reply(())\n else Reply(Error, error)\n\nlet followedByStringCI str : Parser =\n checkStringContainsNoNewlineOrEOSChar str \"followedByStringCI\"\n let error = expectedStringCI str\n if str.Length = 1 then\n let cfChr = Text.FoldCase(str[0])\n fun stream ->\n if stream.MatchCaseFolded(cfChr) then Reply(())\n else Reply(Error, error)\n else\n let cfStr = foldCase str\n fun stream ->\n if stream.MatchCaseFolded(cfStr) then Reply(())\n else Reply(Error, error)\n\nlet notFollowedByString str : Parser =\n checkStringContainsNoNewlineOrEOSChar str \"notFollowedByString\"\n let error = unexpectedString str\n if str.Length = 1 then\n let chr = str[0]\n fun stream ->\n if not (stream.Match(chr)) then Reply(())\n else Reply(Error, error)\n else\n fun stream ->\n if not (stream.Match(str)) then Reply(())\n else Reply(Error, error)\n\nlet notFollowedByStringCI str : Parser =\n checkStringContainsNoNewlineOrEOSChar str \"notFollowedByStringCI\"\n let error = unexpectedStringCI str\n if str.Length = 1 then\n let cfChr = Text.FoldCase(str[0])\n fun stream ->\n if not (stream.MatchCaseFolded(cfChr)) then Reply(())\n else Reply(Error, error)\n else\n let cfStr = foldCase str\n fun stream ->\n if not (stream.MatchCaseFolded(cfStr)) then Reply(())\n else Reply(Error, error)\n\n\nlet inline private charDoesSatisfy f c =\n match c with\n | EOS -> Error\n | _ -> if f (if c <> '\\r' then c else '\\n') then Ok else Error\n\nlet inline private charDoesSatisfyNot f c =\n match c with\n | EOS -> Ok\n | _ -> if not (f (if c <> '\\r' then c else '\\n')) then Ok else Error\n\nlet previousCharSatisfies f : Parser =\n fun stream ->\n let status = charDoesSatisfy f (stream.Peek(-1))\n Reply(status, (), NoErrorMessages)\n\nlet previousCharSatisfiesNot f : Parser =\n fun stream ->\n let status = charDoesSatisfyNot f (stream.Peek(-1))\n Reply(status, (), NoErrorMessages)\n\nlet nextCharSatisfies f : Parser =\n fun stream ->\n let status = charDoesSatisfy f (stream.Peek())\n Reply(status, (), NoErrorMessages)\n\nlet nextCharSatisfiesNot f : Parser =\n fun stream ->\n let status = charDoesSatisfyNot f (stream.Peek())\n Reply(status, (), NoErrorMessages)\n\nlet next2CharsSatisfy f : Parser =\n let optF = OptimizedClosures.FSharpFunc.Adapt(f)\n fun stream ->\n let cs = stream.Peek2()\n let status = match cs.Char0, cs.Char1 with\n | _, EOS\n | EOS, _ -> Error\n | '\\r', '\\n' ->\n match stream.Peek(2u) with\n | EOS -> Error\n | c1 -> if optF.Invoke('\\n', if c1 <> '\\r' then c1 else '\\n')\n then Ok else Error\n | c0, c1 ->\n if optF.Invoke((if c0 <> '\\r' then c0 else '\\n'),\n (if c1 <> '\\r' then c1 else '\\n'))\n then Ok else Error\n Reply(status, (), NoErrorMessages)\n\nlet next2CharsSatisfyNot f : Parser =\n let optF = OptimizedClosures.FSharpFunc.Adapt(f)\n fun stream ->\n let cs = stream.Peek2()\n let status = match cs.Char0, cs.Char1 with\n | _, EOS\n | EOS, _ -> Ok\n | '\\r', '\\n' ->\n match stream.Peek(2u) with\n | EOS -> Ok\n | c1 -> if not (optF.Invoke('\\n', if c1 <> '\\r' then c1 else '\\n'))\n then Ok else Error\n | c0, c1 ->\n if not (optF.Invoke((if c0 <> '\\r' then c0 else '\\n'),\n (if c1 <> '\\r' then c1 else '\\n')))\n then Ok else Error\n Reply(status, (), NoErrorMessages)\n" }, { "path": "FParsec/CharParsers.fsi", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.CharParsers\n\nopen Error\nopen Primitives\n\n// ========================\n// Running parsers on input\n// ========================\n\n/// Values of this type are returned by the runParser functions (not by `Parser<_,_>` functions).\ntype ParserResult<'Result,'UserState> =\n /// Success(result, userState, endPos) holds the result and the user state returned by a successful parser,\n /// together with the position where the parser stopped.\n | Success of 'Result * 'UserState * Position\n /// Failure(errorAsString, error, suserState) holds the parser error and the user state returned by a failing parser,\n /// together with a string representation of the parser error.\n | Failure of string * ParserError * 'UserState\n\n/// `runParserOnString p ustate streamName str` runs the parser `p` directly on the content of the string `str`,\n/// starting with the initial user state `ustate`. The `streamName` is used in error messages to describe\n/// the source of the input (e.g. a file path) and may be empty.\n/// The parser's `Reply` is captured and returned as a `ParserResult` value.\nval runParserOnString: Parser<'a,'u> -> 'u -> streamName: string -> string -> ParserResult<'a,'u>\n\n/// `runParserOnSubstring p ustate streamName str index count` runs the parser `p` directly on the content\n/// of the string `str` between the indices `index` (inclusive) and `index + count` (exclusive),\n/// starting with the initial user state `ustate`. The `streamName` is used in error messages to describe\n/// the source of the input (e.g. a file path) and may be empty.\n/// The parser's `Reply` is captured and returned as a `ParserResult` value.\nval runParserOnSubstring: Parser<'a,'u> -> 'u -> streamName: string -> string -> int -> int -> ParserResult<'a,'u>\n\n/// `runParserOnStream p ustate streamName stream encoding` runs the parser `p` on the content of\n/// the `System.IO.Stream` `stream`, starting with the initial user state `ustate`. The `streamName`\n/// is used in error messages to describe the source of the input (e.g. a file path) and may be empty.\n/// In case no unicode byte order mark is found, the stream data is assumed to be encoded with the given `encoding`.\n/// The parser's `Reply` is captured and returned as a `ParserResult` value.\nval runParserOnStream: Parser<'a,'u> -> 'u -> streamName: string -> System.IO.Stream -> System.Text.Encoding -> ParserResult<'a,'u>\n\n/// `runParserOnFile p ustate path encoding` runs the parser `p` on the content of the file\n/// at the given `path`, starting with the initial user state `ustate`.\n/// In case no unicode byte order mark is found, the file data is assumed to be encoded with the given `encoding`.\n/// The parser's `Reply` is captured and returned as a `ParserResult` value.\nval runParserOnFile: Parser<'a,'u> -> 'u -> path: string -> System.Text.Encoding -> ParserResult<'a,'u>\n\n/// `run parser str` is a convenient abbreviation for `runParserOnString parser () \"\" str`.\nval run: Parser<'Result, unit> -> string -> ParserResult<'Result,unit>\n\n\n// =======\n// Parsers\n// =======\n\n\n// -------------------------------------------------------------\n// Reading the input stream position and handling the user state\n// -------------------------------------------------------------\n\n/// The parser `getPosition` returns the current position in the input Stream.\n/// `getPosition` is equivalent to `fun stream -> Reply(stream.Position)`.\nval getPosition: Parser\n\n/// The parser `getUserState` returns the current user state.\n/// `getUserState` is equivalent to `fun stream -> Reply(stream.UserState)`.\nval getUserState: Parser<'u,'u>\n\n/// The parser `setUserState u` sets the user state to `u`.\n/// `setUserState u` is equivalent to `fun stream -> stream.UserState <- u; Reply(())`.\nval setUserState: 'u -> Parser\n\n/// `updateUserState f` is equivalent to `fun stream -> stream.UserState <- f stream.UserState; Reply(())`.\nval updateUserState: ('u -> 'u) -> Parser\n\n/// The parser `userStateSatisfies f` succeeds if `f` returns `true`\n/// when applied to the current user state, otherwise it fails.\nval userStateSatisfies: ('u -> bool) -> Parser\n\n\n// --------------------\n// Parsing single chars\n// --------------------\n\n/// `pchar c` parses the char `c` and returns `c`.\n/// If `c = '\\r'` or `c = '\\n'` then `pchar c` will parse any one newline (\"\\n\", \"\\r\\n\" or \"\\r\") and return `c`.\nval pchar: char -> Parser\n\n/// `skipChar c` is an optimized implementation of `pchar c |>> ignore`.\nval skipChar: char -> Parser\n\n/// `charReturn c x` is an optimized implementation of `pchar c >>% x`.\nval charReturn: char -> 'a -> Parser<'a,'u>\n\n/// `anyChar` parses any single char or newline (\"\\n\", \"\\r\\n\" or \"\\r\").\n/// Returns the parsed char, or '\\n' in case a newline was parsed.\nval anyChar: Parser\n\n/// `skipAnyChar` is an optimized implementation of `anyChar |>> ignore`.\nval skipAnyChar: Parser\n\n\n/// `satisfy f` parses any one char or newline for which the predicate function `f` returns `true`.\n/// It returns the parsed char.\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'.\n/// Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called\n/// with '\\r' and `satisfy f` will never return the result '\\r'.\nval satisfy: (char -> bool) -> Parser\n\n/// `skipSatisfy f` is an optimized implementation of `satisfy f |>> ignore`.\nval skipSatisfy: (char -> bool) -> Parser\n\n/// `satisfy f label` is an optimized implementation of `satisfy f label`.\nval satisfyL: (char -> bool) -> string -> Parser\n\n/// `skipSatisfyL f label` is an optimized implementation of `skipSatisfy f label`.\nval skipSatisfyL: (char -> bool) -> string -> Parser\n\n\n/// `anyOf str` parses any char contained in the string `str`. It returns the parsed char.\n/// If `str` contains the char '\\n', `anyOf str` parses any newline (\"\\n\", \"\\r\\n\" or \"\\r\")\n/// and returns it as '\\n'. (Note that it does not make a difference whether or not\n/// `str` contains '\\r'; `anyOf str` will never return '\\r'.)\nval anyOf: seq -> Parser\n\n/// `skipAnyOf str` is an optimized implementation of `anyOf str |>> ignore`.\nval skipAnyOf: seq -> Parser\n\n/// `noneOf str` parses any char not contained in the string `str`. It returns the parsed char.\n/// If `str` does not contain the char '\\n', `noneOf str` parses any newline (\"\\n\", \"\\r\\n\" or \"\\r\")\n/// and returns it as as '\\n'. (Note that it does not make a difference whether or not\n/// `str` contains '\\r'; `noneOf str` will never return '\\r'.)\nval noneOf: seq -> Parser\n\n/// `skipNoneOf s` is an optimized implementation of `noneOf s |>> ignore`.\nval skipNoneOf: seq -> Parser\n\n\n/// Parses any char in the range 'A' - 'Z'. Returns the parsed char.\nval asciiUpper: Parser\n\n/// Parses any char in the range 'a' - 'z'. Returns the parsed char.\nval asciiLower: Parser\n\n/// Parses any char in the range 'a' - 'z' and 'A' - 'Z'. Returns the parsed char.\nval asciiLetter: Parser\n\n/// Parses any UTF-16 uppercase letter char identified by `System.Char.IsUpper`.\n/// Returns the parsed char.\nval upper: Parser\n\n/// Parses any UTF-16 lowercase letter char identified by `System.Char.IsLower`.\n/// Returns the parsed char.\nval lower: Parser\n\n/// Parses any UTF-16 letter char identified by `System.Char.IsLetter`.\n/// Returns the parsed char.\nval letter: Parser\n\n/// Parses any char in the range '0' - '9'. Returns the parsed char.\nval digit: Parser\n\n/// Parses any char in the range '0' - '9', 'a' - 'f' and 'A' - 'F'. Returns the parsed char.\nval hex: Parser\n\n/// Parses any char in the range '0' - '7'. Returns the parsed char.\nval octal: Parser\n\n// predicate functions corresponding to the above parsers\n\n/// `isAnyOf str` returns a predicate function.\n/// When this predicate function is applied to a char, it returns `true` if and only if the char is contained in `str`.\nval isAnyOf: seq -> (char -> bool)\n/// `isNoneOf str` returns a predicate function.\n/// When this predicate function is applied to a char, it returns `true` if and only if the char is not contained in `str`.\nval isNoneOf: seq -> (char -> bool)\n/// Returns `true` for any char in the range 'A' - 'Z' and `false` for all other chars.\nval inline isAsciiUpper: char -> bool\n/// Returns `true` for any char in the range 'a' - 'z' and `false` for all other chars.\nval inline isAsciiLower: char -> bool\n/// Returns `true` for any char in the range 'a' - 'z', 'A' - 'Z' and `false` for all other chars.\nval inline isAsciiLetter: char -> bool\n/// `isUpper` is equivalent to `System.Char.IsUpper`.\nval inline isUpper: char -> bool\n/// `isLower` is equivalent to `System.Char.IsLower`.\nval inline isLower: char -> bool\n/// `isLetter` is equivalent to `System.Char.IsLetter`.\nval inline isLetter: char -> bool\n/// Returns `true` for any char in the range '0' - '9' and `false` for all other chars.\nval inline isDigit: char -> bool\n/// Returns `true` for any char in the range '0' - '9', 'a' - 'f', 'A' - 'F' and `false` for all other chars.\nval inline isHex: char -> bool\n/// Returns `true` for any char in the range '0' - '7' and `false` for all other chars.\nval inline isOctal: char -> bool\n\n\n// ------------------\n// Parsing whitespace\n// ------------------\n\n/// Parses the tab char '\\t' and returns '\\t'. Note that a tab char is treated like any other non-newline char:\n/// the column number is incremented by (only) 1.\nval tab: Parser\n\n/// Parses a newline (\"\\n\", \"\\r\\n\" or \"\\r\"). Returns '\\n'.\n/// Is equivalent to `pchar '\\n'`.\nval newline<'u> : Parser\n\n/// `skipNewline` is an optimized implementation of `newline |>> ignore`.\nval skipNewline<'u> : Parser\n\n/// `newlineReturn x` is an optimized implementation of `newline >>% x`.\nval newlineReturn: 'a -> Parser<'a,'u>\n\n/// Parses a unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085\", \"\\u2028\", or \"\\u2029\").\n/// Returns '\\n'. Note that this parser does not accept the formfeed char '\\f' as a newline.\n/// In contrast to most other parsers in FParsec this parser also increments\n/// the internal line count for unicode newline characters other than '\\n' and '\\r'.\nval unicodeNewline<'u> : Parser\n\n/// `skipNewline` is an optimized implementation of `unicodeNewline |>> ignore`.\nval skipUnicodeNewline<'u> : Parser\n\n/// `newlineReturn x` is an optimized implementation of `unicodeNewline >>% x`.\nval unicodeNewlineReturn: 'a -> Parser<'a,'u>\n\n/// Skips over any sequence of *zero* or more whitespaces (space (' '), tab ('\\t')\n/// or newline (\"\\n\", \"\\r\\n\" or \"\\r\")).\nval spaces: Parser\n\n/// Skips over any sequence of *one* or more whitespaces (space (' '), tab('\\t')\n/// or newline (\"\\n\", \"\\r\\n\" or \"\\r\")).\nval spaces1: Parser\n\n/// Skips over any sequence of *zero* or more unicode whitespaces and\n/// registers any unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085, \"\\u000C\",\n/// \"\\u2028\"or \"\\u2029\") as a newline.\nval unicodeSpaces: Parser\n\n/// Skips over any sequence of *one* or more unicode whitespaces and\n/// registers any unicode newline (\"\\n\", \"\\r\\n\", \"\\r\", \"\\u0085, \"\\u000C\",\n/// \"\\u2028\"or \"\\u2029\") as a newline.\nval unicodeSpaces1: Parser\n\n/// The parser `eof` only succeeds at the end of the input. It never consumes input.\nval eof: Parser\n\n\n// ------------------------\n// Parsing strings directly\n// ------------------------\n\n/// `pstring str` parses the string `str` and returns `str`.\n/// It is an atomic parser: either it succeeds or it fails without consuming any input.\n/// `str` may not contain newline chars ('\\n' or '\\r').\nval pstring: string -> Parser\n/// `skipString str` is an optimized implementation of `pstring str |>> ignore`.\nval skipString: string -> Parser\n/// `stringReturn str x` is an optimized implementation of `pstring str >>% x`.\nval stringReturn: string -> 'a -> Parser<'a,'u>\n\n/// `pstringCI str` parses any string that case-insensitively matches the string `str`.\n/// It returns the *parsed* string.\n/// `str` may not contain newline chars ('\\n' or '\\r').\nval pstringCI: string -> Parser\n/// `skipStringCI str` is an optimized implementation of `pstringCI str |>> ignore`.\nval skipStringCI: string -> Parser\n/// `stringCIReturn str x` is an optimized implementation of `pstringCI str >>% x`.\nval stringCIReturn: string -> 'a -> Parser<'a,'u>\n\n/// `anyString n` parses any sequence of `n` chars or newlines (\"\\n\", \"\\r\\n\" or \"\\r\").\n/// It returns the parsed string. In the returned string all newlines are normalized to \"\\n\".\n/// `anyString n` is an atomic parser: either it succeeds or it fails without consuming any input.\nval anyString: int32 -> Parser\n/// `skipAnyString n` is an optimized implementation of `anyString n |>> ignore`.\nval skipAnyString: int32 -> Parser\n\n/// `restOfLine skipNewline` parses any chars before the end of the line\n/// and, if `skipNewline` is `true`, skips to the beginning of the next line (if there is one).\n/// It returns the parsed chars before the end of the line as a string (without a newline).\n/// A line is terminated by a newline (\"\\n\", \"\\r\\n\" or \"\\r\") or the end of the input stream.\nval restOfLine: bool -> Parser\n\n/// `skipRestOfLine skipNewline` is an optimized implementation of `restOfLine skipNewline |>> ignore`.\nval skipRestOfLine: bool -> Parser\n\n/// `charsTillString str skipString maxCount` parses all chars before the first occurance of the string `str` and,\n/// if `skipString` is `true`, skips over `str`. It returns the parsed chars before the string.\n/// If more than `maxCount` chars come before the first occurance of `str`, the parser *fails after consuming* `maxCount` chars.\n/// Newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are counted as single chars and\n/// in the returned string all newlines are normalized to \"\\n\".\n/// `charsTillString str maxCount` throws an `ArgumentOutOfRangeException` if `maxCount` is negative.\nval charsTillString: string -> skipString: bool -> maxCount: int -> Parser\n/// `skipCharsTillString str maxCount` is an optimized implementation of `charsTillString str maxCount |>> ignore`.\nval skipCharsTillString: string -> skipString: bool -> maxCount: int -> Parser\n\n/// `charsTillStringCI str skipString maxCount` parses all chars before the first case-insensitive occurance of the string `str` and,\n/// if `skipString` is `true`, skips over it. It returns the parsed chars before the string.\n/// If more than `maxCount` chars come before the first case-insensitive occurance of `str`,\n/// the parser *fails* after consuming `maxCount` chars.\n/// Newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are counted as single chars and\n/// in the returned string all newlines are normalized to \"\\n\".\n/// `charsTillStringCI str maxCount` throws an `ArgumentOutOfRangeException` if `maxCount` is negative.\nval charsTillStringCI: string -> skipString: bool -> maxCount: int -> Parser\n/// `skipCharsTillStringCI str maxCount` is an optimized implementation of `charsTillStringCI str maxCount |>> ignore`.\nval skipCharsTillStringCI: string -> skipString: bool -> maxCount: int -> Parser\n\n/// `manySatisfy f` parses a sequence of *zero* or more chars that satisfy the predicate function `f`\n/// (i.e. chars for which `f` returns `true`). It returns the parsed chars as a string.\n///\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'.\n/// Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called\n/// with '\\r' and the string returned by `manySatisfy f` will never contain an '\\r'.\nval manySatisfy: (char -> bool) -> Parser\n/// `manySatisfy2 f1 f` behaves like `manySatisfy f`, except that the\n/// first char of the parsed string must satisfy `f1` instead of `f`.\nval manySatisfy2: (char -> bool) -> (char -> bool) -> Parser\n/// `skipManySatisfy f` is an optimized implementation of `manySatisfy f |>> ignore`.\nval skipManySatisfy: (char -> bool) -> Parser\n/// `skipManySatisfy2 f1 f` is an optimized implementation of `manySatisfy2 f1 f |>> ignore`.\nval skipManySatisfy2: (char -> bool) -> (char -> bool) -> Parser\n\n/// `many1Satisfy f` parses a sequence of *one* or more chars that satisfy the predicate function `f`\n/// (i.e. chars for which `f` returns `true`). It returns the parsed chars as a string.\n/// If the first char does not satisfy `f`, this parser fails without consuming input.\n///\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'.\n/// Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called\n/// with '\\r' and the string returned by `many1Satisfy f` will never contain an '\\r'.\nval many1Satisfy: (char -> bool) -> Parser\n/// `many1Satisfy2 f1 f` behaves like `many1Satisfy f`, except that the\n/// first char of the parsed string must satisfy `f1` instead of `f`.\nval many1Satisfy2: (char -> bool) -> (char -> bool) -> Parser\n/// `skipMany1Satisfy f` is an optimized implementation of `many1Satisfy f |>> ignore`.\nval skipMany1Satisfy: (char -> bool) -> Parser\n/// `skipMany1Satisfy2 f1 f` is an optimized implementation of `many1Satisfy2 f1 f |>> ignore`.\nval skipMany1Satisfy2: (char -> bool) -> (char -> bool) -> Parser\n\n/// `many1SatisfyL f label` is an optimized implementation of `many1Satisfy f label`.\nval many1SatisfyL: (char -> bool) -> string -> Parser\n/// `many1Satisfy2L f1 f label` is an optimized implementation of `many1Satisfy2 f1 f label`.\nval many1Satisfy2L: (char -> bool) -> (char -> bool) -> string -> Parser\n/// `skipMany1SatisfyL f label` is an optimized implementation of `skipMany1Satisfy f label`.\nval skipMany1SatisfyL: (char -> bool) -> string -> Parser\n/// `skipMany1Satisfy2L f1 f label` is an optimized implementation of `skipMany1Satisfy2 f1 f label`.\nval skipMany1Satisfy2L: (char -> bool) -> (char -> bool) -> string -> Parser\n\n/// `manyMinMaxSatisfy minCount maxCount f` parses a sequence of `minCount` or more chars that satisfy the\n/// predicate function `f` (i.e. chars for which `f` returns `true`), but not more than `maxCount` chars.\n/// It returns the parsed chars as a string. This parser is atomic, i.e. if the first `minCount` chars\n/// do not all satisfy `f`, the parser fails without consuming any input.\n///\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") is converted to the single char '\\n'.\n/// Thus, to accept a newline `f '\\n'` must return `true`. `f` will never be called with '\\r'\n/// and the string returned by `manyMinMaxSatisfy minCount maxCount f` will never contain an '\\r'.\n///\n/// `manyMinMaxSatisfy` throws an `ArgumentOutOfRangeException` if `maxCount` is negative.\nval manyMinMaxSatisfy: int -> int -> (char -> bool) -> Parser\n/// `manyMinMaxSatisfy2 minCount maxCount f1 f` behaves like `manyMinMaxSatisfy minCount maxCount f`, except that the first char of the parsed string must satisfy `f1` instead of `f`.\nval manyMinMaxSatisfy2: int -> int -> (char -> bool) -> (char -> bool) -> Parser\n/// `skipManyMinMaxSatisfy minCount maxCount f` is an optimized implementation of `manyMinMaxSatisfy minCount maxCount f |>> ignore`.\nval skipManyMinMaxSatisfy: int -> int -> (char -> bool) -> Parser\n/// `skipManyMinMaxSatisfy2 minCount maxCount f1 f` is an optimized implementation of `manyMinMaxSatisfy2 minCount maxCount f1 f |>> ignore`.\nval skipManyMinMaxSatisfy2: int -> int -> (char -> bool) -> (char -> bool) -> Parser\n\n/// `manyMinMaxSatisfyL minCount maxCount f label` is an optimized implementation of `manyMinMaxSatisfy minCount maxCount f label`.\nval manyMinMaxSatisfyL: int -> int -> (char -> bool) -> string -> Parser\n/// `manyMinMaxSatisfy2L minCount maxCount f1 f label` is an optimized implementation of `manyMinMaxSatisfy2 minCount maxCount f1 f label`.\nval manyMinMaxSatisfy2L: int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser\n/// `skipManyMinMaxSatisfyL minCount maxCount f label` is an optimized implementation of `skipManyMinMaxSatisfy minCount maxCount f label`.\nval skipManyMinMaxSatisfyL: int -> int -> (char -> bool) -> string -> Parser\n/// `skipManyMinMaxSatisfy2L minCount maxCount f1 f label` is an optimized implementation of `skipManyMinMaxSatisfy2 minCount maxCount f1 f label`.\nval skipManyMinMaxSatisfy2L: int -> int -> (char -> bool) -> (char -> bool) -> string -> Parser\n\n/// `regex pattern` matches the .NET regular expression given by the string `pattern` on the chars\n/// beginning at the current index in the input stream. It returns the string matched by the regular expression.\n/// If the regular expression does not match, the parser fails without consuming input.\n///\n/// The `System.Text.RegularExpressions.Regex` object that is internally used to match the pattern is constructed\n/// with the `RegexOptions` `MultiLine` and `ExplicitCapture`. In order to ensure that the regular expression\n/// can only match at the beginning of a string, \"\\\\A\" is automatically prepended to the pattern.\n///\n/// Newline chars ('\\r' and '\\n') in the pattern are interpreted literally.\n/// For example, an '\\n' char in the pattern will only match \"\\n\", not \"\\r\" or \"\\r\\n\".\n/// However, in the returned string all newlines (\"\\n\", \"\\r\\n\" or \"\\r\") are normalized to \"\\n\".\n///\n/// For large files the regular expression is *not* applied to a string containing *all* the remaining chars\n/// in the stream. The number of chars that are guaranteed to be visible to the regular expression is specified\n/// during construction of the `CharStream`. If one of the `runParser` function` is used to run the parser,\n/// this number is 43690.\nval regex: string -> Parser\n\n/// `regexL pattern label` is an optimized implementation of `regex pattern label`.\nval regexL: string -> string -> Parser\n\ntype IdentifierOptions =\n new: ?isAsciiIdStart: (char -> bool) *\n ?isAsciiIdContinue: (char -> bool) *\n ?normalization: System.Text.NormalizationForm *\n ?normalizeBeforeValidation: bool *\n ?allowJoinControlChars: bool *\n ?preCheckStart: (char -> bool) *\n ?preCheckContinue: (char -> bool) *\n ?allowAllNonAsciiCharsInPreCheck: bool *\n ?label: string *\n ?invalidCharMessage: string -> IdentifierOptions\n\n/// The `identifier` parser is a configurable parser for the XID identifier syntax\n/// specified in Unicode Standard Annex #31.\nval identifier: IdentifierOptions -> Parser\n\n// ----------------------------------------------\n// Parsing strings with the help of other parsers\n// ----------------------------------------------\n\n/// `manyChars cp` parses a sequence of *zero* or more chars with the char parser `cp`.\n/// It returns the parsed chars as a string.\n///\n/// `manyChars cp` is an optimized implementation of `many (attempt cp)` that returns\n/// the chars as a string instead of a char list. The equivalence to `many (attempt p)`\n/// instead of `many p` implies that `manyChars` never fails.\nval manyChars: Parser -> Parser\n/// `manyChars2 cp1 cp` behaves like `manyChars2 cp`, except that it parses the first char with `cp1` instead of `cp`.\nval manyChars2: Parser -> Parser -> Parser\n\n/// `many1Chars cp` parses a sequence of *one* or more chars with the char parser `cp`.\n/// It returns the parsed chars as a string.\n///\n/// `many1Chars cp` is an optimized implementation of `many1 (attempt cp)` that returns\n/// the chars as a string instead of a char list. The equivalence to `many1 (attempt p)`\n/// instead of `many1 p` implies that `many1Chars` never fails after consuming input.\nval many1Chars: Parser -> Parser\n/// `many1Chars2 cp1 cp` behaves like `many1Chars2 cp`, except that it parses the first char with `cp1` instead of `cp`.\nval many1Chars2: Parser -> Parser -> Parser\n\n/// `manyCharsTill cp endp` parses chars with the char parser `cp` until the parser `endp` succeeds.\n/// It stops after `endp` and returns the parsed chars as a string.\nval manyCharsTill: Parser -> Parser<'b,'u> -> Parser\n/// `manyCharsTill2 cp1 cp endp` behaves like `manyCharsTill cp endp`, except that it parses the first char with `cp1` instead of `cp`.\nval manyCharsTill2: Parser -> Parser -> Parser<'b,'u> -> Parser\n\n/// `manyCharsTillApply cp endp f` parses chars with the char parser `cp` until the parser `endp` succeeds.\n/// It stops after `endp` and returns the result of the function application `f str b`,\n/// where `str` is the parsed string and `b` is result returned by `endp`.\nval manyCharsTillApply: Parser -> Parser<'b,'u> -> (string -> 'b -> 'c) -> Parser<'c,'u>\n/// `manyCharsTillApply2 cp1 cp endp` behaves like `manyCharsTillApply cp endp`, except that it parses the first char with `cp1` instead of `cp`.\nval manyCharsTillApply2: Parser -> Parser -> Parser<'b,'u> -> (string -> 'b -> 'c) -> Parser<'c,'u>\n\n/// `many1CharsTill cp endp` parses one char with the char parser `cp`.\n/// Then it parses more chars with `cp` until the parser `endp` succeeds.\n/// It stops after `endp` and returns the parsed chars as a string.\n///\n/// `many1CharsTill cp endp` is an optimized implementation of `pipe2 cp (manyCharsTill cp endp) (fun c1 str -> c1.ToString() + str)`\nval many1CharsTill: Parser -> Parser<'b,'u> -> Parser\n/// `many1CharsTill2 cp1 cp endp` behaves like `many1CharsTill cp endp`, except that it parses the first char with `cp1` instead of `cp`.\nval many1CharsTill2: Parser -> Parser -> Parser<'b,'u> -> Parser\n\n/// `many1CharsTillApply cp endp` parses one char with the char parser `cp`.\n/// Then it parses more chars with `cp` until the parser `endp` succeeds.\n/// It stops after `endp` and returns the result of the function application `f str b`,\n/// where `str` is the parsed string and `b` is result returned by `endp`.\nval many1CharsTillApply: Parser -> Parser<'b,'u> -> (string -> 'b -> 'c) -> Parser<'c,'u>\n/// `many1CharsTillApply2 cp1 cp endp` behaves like `many1CharsTillApply cp endp`, except that it parses the first char with `cp1` instead of `cp`.\nval many1CharsTillApply2: Parser -> Parser -> Parser<'b,'u> -> (string -> 'b -> 'c) -> Parser<'c,'u>\n\n/// `manyStrings sp` parses a sequence of *zero* or more strings with the string parser `sp`.\n/// It returns the strings in concatenated form.\n/// `manyStrings sp` is an optimized implementation of `manyReduce (+) \"\" sp`.\nval manyStrings: Parser -> Parser\n/// `manyStrings2 sp1 sp` behaves like `manyStrings sp`, except that it parses the first string with `sp1` instead of `sp`.\nval manyStrings2: Parser -> Parser -> Parser\n\n/// `many1Strings sp` parses a sequence of *one* or more strings with the string parser `sp`.\n/// It returns the strings in concatenated form.\n/// Note that `many1Strings sp` does not require the first string to be non-empty.\nval many1Strings: Parser -> Parser\n/// `many1Strings2 sp1 sp` behaves like `many1Strings sp`, except that it parses the first string with `sp1` instead of `sp`.\nval many1Strings2: Parser -> Parser -> Parser\n\n/// `stringsSepBy sp sep` parses *zero* or more occurrences of `sp` separated by `sep`.\n/// It returns the strings parsed by `sp` *and* `sep` in concatenated form.\nval stringsSepBy: Parser -> Parser -> Parser\n\n/// `stringsSepBy1 sp sep` parses *one* or more occurrences of `sp` separated by `sep`.\n/// It returns the strings parsed by `sp` *and* `sep` in concatenated form.\nval stringsSepBy1: Parser -> Parser -> Parser\n\n/// `skipped p` applies the parser `p` and returns the chars skipped over by `p` as a string.\n/// All newlines (\"\\r\\n\", \"\\r\" or \"\\n\") are normalized to \"\\n\".\nval skipped: Parser -> Parser\n\n/// `p |> withSkippedString f` applies the parser `p` and returns the result of `f str x`,\n/// where `str` is the string skipped over by `p` and `x` is the result returned by `p`.\nval withSkippedString: (string -> 'a -> 'b) -> Parser<'a,'u> -> Parser<'b,'u>\n\n\n// ---------------\n// Parsing numbers\n// ---------------\n\n/// Encodes the various options of the `numberLiteral` parser.\n[]\ntype NumberLiteralOptions =\n | None = 0\n | AllowSuffix = 0b000000000001\n | AllowMinusSign = 0b000000000010\n | AllowPlusSign = 0b000000000100\n | AllowFraction = 0b000000001000\n | AllowFractionWOIntegerPart = 0b000000010000\n | AllowExponent = 0b000000100000\n | AllowHexadecimal = 0b000001000000\n | AllowBinary = 0b000010000000\n | AllowOctal = 0b000100000000\n | AllowInfinity = 0b001000000000\n | AllowNaN = 0b010000000000\n\n | IncludeSuffixCharsInString = 0b100000000000\n\n | DefaultInteger = 0b000111000110\n | DefaultUnsignedInteger = 0b000111000000\n | DefaultFloat = 0b011001101110\n\n/// The return type of the `numberLiteral` parser. An instance contains the parsed\n/// number literal and various bits of information about it.\n/// Note that the `String` member contains the string literal without the suffix chars,\n/// except if the `NumberLiteralOptions` passed to the `numberLiteral` parser have the\n/// `IncludeSuffixCharsInString` flag set.\n/// Any parsed suffix chars are always available through the `SuffixChar1` - `4` members.\ntype NumberLiteral =\n new: string:string * info:NumberLiteralResultFlags\n * suffixChar1: char * suffixChar2: char * suffixChar3: char * suffixChar4: char -> NumberLiteral\n\n /// The parsed number literal string. Only includes the parsed suffix chars if the\n /// `NumberLiteralOptions` passed to the `numberLiteral` parser have the `IncludeSuffixCharsInString` flag set.\n member String: string\n /// Eencodes various bits of information on the string literal.\n member Info: NumberLiteralResultFlags\n\n member SuffixLength: int\n /// Returns the first suffix char, or EOS if no suffix char was parsed.\n member SuffixChar1: char\n /// Returns the second suffix char, or EOS if less than two suffix chars were parsed.\n member SuffixChar2: char\n /// Returns the third suffix char, or EOS if less than three suffix chars were parsed\n member SuffixChar3: char\n /// Returns the fourth suffix char, or EOS if less than four suffix chars were parsed\n member SuffixChar4: char\n\n member HasMinusSign: bool\n member HasPlusSign: bool\n member HasIntegerPart: bool\n member HasFraction: bool\n member HasExponent: bool\n member IsInteger: bool\n member IsDecimal: bool\n member IsHexadecimal: bool\n member IsBinary: bool\n member IsOctal: bool\n member IsNaN: bool\n member IsInfinity: bool\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n\nand /// Encodes various bits of information about a parsed number literal.\n []\n NumberLiteralResultFlags =\n | None = 0\n | SuffixLengthMask = 0b0000000000001111\n | HasMinusSign = 0b0000000000010000\n | HasPlusSign = 0b0000000000100000\n | HasIntegerPart = 0b0000000001000000\n | HasFraction = 0b0000000010000000\n | HasExponent = 0b0000000100000000\n | IsDecimal = 0b0000001000000000\n | IsHexadecimal = 0b0000010000000000\n | IsBinary = 0b0000100000000000\n | IsOctal = 0b0001000000000000\n | BaseMask = 0b0001111000000000\n | IsInfinity = 0b0010000000000000\n | IsNaN = 0b0100000000000000\n\n\n/// `numberLiteral options label` parses a number literal and returns the result in form\n/// of a `NumberLiteral` value. The given `NumberLiteralOptions` argument determines the kind\n/// of number literals accepted. The string `label` is used in the `Expected` error message\n/// that is generated when the parser fails without consuming input.\n///\n/// The parser fails without consuming input, if not at least one digit (including the 0 in the\n/// format specifiers \"0x\" etc.) can be parsed. It fails after consuming input, if no decimal\n/// digit comes after an exponent marker or no valid digit comes after a format specifier.\nval numberLiteral: NumberLiteralOptions -> string\n -> Parser\n\n/// `numberLiteralE` is an uncurried version of `numberLiteral` that can be used to\n/// implement number parsers without having to construct a `numberLiteral` closure.\nval numberLiteralE: NumberLiteralOptions -> errorInCaseNoLiteralFound: ErrorMessageList\n -> CharStream<'u> -> Reply\n\n/// Parses a floating-point number in decimal or hexadecimal format.\n/// The special values NaN and Inf(inity)? (case insensitive) are also recognized.\n///\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in \"0x\") can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after \"0x\",\n/// after consuming input, if the value represented by the input string (after rounding) is greater than `System.Double.MaxValue` or less than `System.Double.MinValue`.\nval pfloat: Parser\n\n\n/// Parses an integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.Int64.MaxValue` or less than `System.Int64.MinValue`.\nval pint64: Parser\n\n/// Parses an integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.Int32.MaxValue` or less than `System.Int32.MinValue`.\nval pint32: Parser\n\n/// Parses an integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.Int16.MaxValue` or less than `System.Int16.MinValue`.\nval pint16: Parser\n\n/// Parses an integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than 127 or less than -128.\nval pint8: Parser\n\n/// Parses an unsigned integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.UInt64.MaxValue`.\nval puint64: Parser\n\n/// Parses an unsigned integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.UInt32.MaxValue`.\nval puint32: Parser\n\n/// Parses an unsigned integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than `System.UInt16.MaxValue`.\nval puint16: Parser\n\n/// Parses an unsigned integer in decimal, hexadecimal (\"0x\" prefix), octal (\"0o\") or binary (\"0b\") format.\n/// The parser fails\n/// without consuming input, if not at least one digit (including the '0' in the format specifiers \"0x\" etc.) can be parsed,\n/// after consuming input, if no digit comes after an exponent marker or no hex digit comes after a format specifier,\n/// after consuming input, if the value represented by the input string is greater than 255.\nval puint8: Parser\n\n\n// -------------------\n// Conditional parsing\n// -------------------\n\n/// `notFollowedByEOF` is an optimized implementation of `notFollowedByL eof \"end of input\"`.\nval notFollowedByEof: Parser\n\n/// `followedByNewline` is an optimized implementation of `followedByL newline \"newline\"`.\nval followedByNewline: Parser\n\n/// `notFollowedByNewline` is an optimized implementation of `notFollowedByL newline \"newline\"`.\nval notFollowedByNewline: Parser\n\n/// `followedByString str` is an optimized implementation of `followedByL (pstring str) (\"'\" + str + \"'\")`.\nval followedByString: string -> Parser\n/// `followedByStringCI str` is an optimized implementation of `followedByL (pstringCI str) (\"'\" + str + \"'\")`.\nval followedByStringCI: string -> Parser\n/// `notFollowedByString str` is an optimized implementation of `notFollowedByL (pstring str) (\"'\" + str + \"'\")`.\nval notFollowedByString: string -> Parser\n/// `notFollowedByStringCI str` is an optimized implementation of `notFollowedByL (pstringCI str) (\"'\" + str + \"'\")`.\nval notFollowedByStringCI: string -> Parser\n\n/// `nextCharSatisfies f` is an optimized implementation of `followedBy (satisfy f)`.\nval nextCharSatisfies: (char -> bool) -> Parser\n\n/// `nextCharSatisfiesNot f` is an optimized implementation of `notFollowedBy (satisfy f)`.\nval nextCharSatisfiesNot: (char -> bool) -> Parser\n\n/// `next2CharsSatisfy f` succeeds if the predicate function `f` returns `true`\n/// when applied to the next 2 chars in the input stream, otherwise it fails.\n/// If there aren't 2 chars remaining in the input stream, this parser fails (as opposed to `next2CharsSatisfyNot`).\n/// This parser never changes the parser state.\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n/// If this parser fails, it returns no descriptive error message; hence it should only be\n/// used together with parsers that take care of a potential error.\nval next2CharsSatisfy: (char -> char -> bool) -> Parser\n\n/// `next2CharsSatisfyNot f` succeeds if the predicate function `f` returns `false`\n/// when applied to the next 2 chars in the input stream, otherwise it fails.\n/// If there aren't 2 chars remaining in the input stream, this parser succeeds (as opposed to `next2CharsSatisfy`).\n/// This parser never changes the parser state.\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n/// If this parser fails, it returns no descriptive error message; hence it should only be\n/// used together with parsers that take care of a potential error.\nval next2CharsSatisfyNot: (char -> char -> bool) -> Parser\n\n/// `previousCharSatisfies f` succeeds if the predicate function `f` returns `true`\n/// when applied to the previous char in the stream, otherwise it fails.\n/// If there is no previous char (because the stream is at the beginning),\n/// this parser fails (as opposed to `previousCharSatisfiesNot`).\n/// This parser never changes the parser state.\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n/// If this parser fails, it returns no descriptive error message; hence it should only be\n/// used together with parsers that take care of a potential error.\nval previousCharSatisfies: (char -> bool) -> Parser\n\n/// `previousCharSatisfies f` succeeds if the predicate function `f` returns `false`\n/// when applied to the previous char in the stream, otherwise it fails.\n/// If there is no previous char (because the stream is at the beginning),\n/// this parser succeeds (as opposed to `previousCharSatisfies`).\n/// This parser never changes the parser state.\n/// Any newline (\"\\n\", \"\\r\\n\" or \"\\r\") in the input is interpreted as a single char '\\n'.\n/// If this parser fails, it returns no descriptive error message; hence it should only be\n/// used together with parsers that take care of a potential error.\nval previousCharSatisfiesNot: (char -> bool) -> Parser\n\n\n\n// ================\n// Helper functions\n// ================\n\n/// `EOS` is equal to `CharStream<'u>.EndOfStreamChar`.\n[]\nval EOS: char = '\\uffff';;\n\n/// `foldCase str` returns a case-folded version of `str`\n/// with all chars mappend using the (non-Turkic) Unicode 1-to-1 case folding mappings\n/// for chars below 0x10000. If the argument is `null`, `null` is returned.\nval foldCase: string -> string\n\n/// `normalizeNewlines str` returns a version of `str`\n/// with all occurances of \"\\r\\n\" and \"\\r\" replaced by \"\\n\".\n/// If the argument is `null`, `null` is returned.\nval normalizeNewlines: string -> string\n\n/// Returns a hexadecimal string representation of the `float` argument.\nval floatToHexString: float -> string\n\n/// Returns the `float` value represented by the given string in hexadecimal format.\n/// Raises a `System.FormatException` in case the string representation is invalid.\n/// Raises a `System.OverflowException` if the (absolute) value is too large to be represented by a `float`.\nval floatOfHexString: string -> float\n\n/// Returns a hexadecimal string representation of the `float32` argument.\nval float32ToHexString: float32 -> string\n\n/// Returns the `float32` value represented by the given string in hexadecimal format.\n/// Raises a `System.FormatException` in case the string representation is invalid.\n/// Raises a `System.OverflowException` if the (absolute) value is too large to be represented by a `float32`.\nval float32OfHexString: string -> float32\n" }, { "path": "FParsec/Emit.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule internal FParsec.Emit\n\n#if !LOW_TRUST\n\nopen System.Diagnostics\nopen System.Reflection\nopen System.Reflection.Emit\n\nopen System.Runtime.CompilerServices\nopen Microsoft.FSharp.NativeInterop\n\nopen FParsec.Internals\nopen FParsec.Range\n\n#nowarn \"9\" // \"Uses of this construct may result in the generation of unverifiable .NET IL code.\"\n#nowarn \"51\" // \"The use of native pointers may result in unverifiable .NET IL code\"\n\nlet nativeMemoryAssociatedType = \"NativeMemoryAssociatedType\"\n\nlet moduleBuilder = lazy (\n let assemblyName = AssemblyName(\"FParsec.Emitted\")\n let assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run)\n let moduleBuilder = assemblyBuilder.DefineDynamicModule(\"FParsec.Emitted\")\n // We create a dummy static class that will be used to associate the native memory with this dynamic assembly.\n moduleBuilder\n .DefineType(nativeMemoryAssociatedType, TypeAttributes.Abstract ||| TypeAttributes.Sealed ||| TypeAttributes.Public)\n .CreateType()\n |> ignore\n moduleBuilder\n)\n\n// Allocates memory that can be used by the generated code.\nlet allocTemporaryMem size =\n let associatedType = moduleBuilder.Value.GetType(nativeMemoryAssociatedType, true, false)\n RuntimeHelpers.AllocateTypeAssociatedMemory(associatedType, size)\n\nlet createTypeBuilder name args parent (interfaces : System.Type[]) =\n moduleBuilder.Value.DefineType(\"FParsec.Emitted.\" + name, args, parent, interfaces)\n\nlet loadI4 (ilg: ILGenerator) (i: int32) =\n ilg.Emit(OpCodes.Ldc_I4, i)\n\nlet loadI8 (ilg: ILGenerator) (i: int64) =\n let i4 = int32 i\n if i <> int64 i4 then ilg.Emit(OpCodes.Ldc_I8, i)\n else\n loadI4 ilg i4\n ilg.Emit(OpCodes.Conv_I8)\n\nlet loadU8 (ilg: ILGenerator) (i: uint64) =\n if i > uint64 System.UInt32.MaxValue then ilg.Emit(OpCodes.Ldc_I8, int64 i)\n else\n loadI4 ilg (int32 i)\n ilg.Emit(OpCodes.Conv_U8)\n\nlet loadI (ilg: ILGenerator) (i: nativeint) =\n if sizeof = 4 then\n ilg.Emit(OpCodes.Ldc_I4, int32 i)\n else\n ilg.Emit(OpCodes.Ldc_I8, int64 i)\n ilg.Emit(OpCodes.Conv_I)\n\nlet loadU (ilg: ILGenerator) (i: unativeint) =\n if sizeof = 4 then\n ilg.Emit(OpCodes.Ldc_I4, int32 i)\n else\n ilg.Emit(OpCodes.Ldc_I8, int64 i)\n ilg.Emit(OpCodes.Conv_U)\n\nlet private createLoaderForPrimitiveConstantsImpl (ty: System.Type) (ilg: ILGenerator) : ('T -> unit) =\n let ty = if ty.IsEnum then System.Enum.GetUnderlyingType(ty) else ty\n\n if ty = typeof< int32> then fun x -> loadI4 ilg (box x :?> int32)\n elif ty = typeof then fun x -> loadI4 ilg (int32 (box x :?> uint32))\n elif ty = typeof then fun x -> loadI8 ilg (box x :?> int64)\n elif ty = typeof then fun x -> loadU8 ilg (box x :?> uint64)\n elif ty = typeof< int16> then fun x -> loadI4 ilg (int32 (box x :?> int16))\n elif ty = typeof then fun x -> loadI4 ilg (int32 (box x :?> uint16))\n elif ty = typeof then fun x -> loadI4 ilg (int32 (box x :?> char))\n elif ty = typeof< int8> then fun x -> loadI4 ilg (int32 (box x :?> int8))\n elif ty = typeof then fun x -> loadI4 ilg (int32 (box x :?> uint8))\n elif ty = typeof then\n fun x -> ilg.Emit(if box x :?> bool then OpCodes.Ldc_I4_1 else OpCodes.Ldc_I4_0)\n elif ty = typeof then fun x -> ilg.Emit(OpCodes.Ldc_R8, (box x :?> double))\n elif ty = typeof then fun x -> ilg.Emit(OpCodes.Ldc_R4, (box x :?> float32))\n elif ty = typeof then fun x -> loadI ilg (box x :?> nativeint)\n elif ty = typeof then fun x -> loadU ilg (box x :?> unativeint)\n else invalidArg \"ty\" \"Invalid type argument.\"\n\nlet createLoaderForPrimitiveConstants<'T> ilg : ('T -> unit) =\n createLoaderForPrimitiveConstantsImpl typeof<'T> ilg\n\nlet createLoaderForBoxedPrimitiveConstants (ty: System.Type) ilg : (obj -> unit) =\n createLoaderForPrimitiveConstantsImpl ty ilg\n\n\nlet emitRangeCheck branchIfInRange (ilg: ILGenerator) (label: Label) minValue maxValue (range: Range) =\n Debug.Assert(minValue <= range.Min && range.Max <= maxValue)\n if minValue = range.Min && range.Max = maxValue then\n ilg.Emit(OpCodes.Pop)\n if branchIfInRange then\n ilg.Emit(OpCodes.Br, label)\n elif range.Min = range.Max then\n loadI4 ilg range.Min\n if branchIfInRange then\n ilg.Emit(OpCodes.Beq, label)\n else\n ilg.Emit(OpCodes.Bne_Un, label)\n elif minValue = range.Min then\n // we only have to check the right bound\n loadI4 ilg range.Max\n if branchIfInRange then\n ilg.Emit(OpCodes.Ble, label)\n else\n ilg.Emit(OpCodes.Bgt, label)\n elif range.Max = maxValue then\n // we only have to check the left bound\n loadI4 ilg range.Min\n if branchIfInRange then\n ilg.Emit(OpCodes.Bge, label)\n else\n ilg.Emit(OpCodes.Blt, label)\n else\n // we have to check both bounds\n if range.Min <> 0 then\n loadI4 ilg range.Min\n ilg.Emit(OpCodes.Sub)\n loadI4 ilg (range.Max - range.Min)\n if branchIfInRange then\n ilg.Emit(OpCodes.Ble_Un, label) // unsigned comparison\n else\n ilg.Emit(OpCodes.Bgt_Un, label) // unsigned comparison\n\nlet emitBranchIfOutOfRange ilg label minValue maxValue range =\n emitRangeCheck false ilg label minValue maxValue range\n\nlet emitBranchIfInRange ilg label minValue maxValue range =\n emitRangeCheck true ilg label minValue maxValue range\n\nlet emitRangeTest pushFalseIfInRange (ilg: ILGenerator) minValue maxValue (range: Range) =\n Debug.Assert(minValue <= range.Min && range.Max <= maxValue)\n\n let emitNot() =\n ilg.Emit(OpCodes.Ldc_I4_0)\n ilg.Emit(OpCodes.Ceq)\n\n if minValue = range.Min && range.Max = maxValue then\n ilg.Emit(OpCodes.Pop)\n if pushFalseIfInRange then\n ilg.Emit(OpCodes.Ldc_I4_0)\n else\n ilg.Emit(OpCodes.Ldc_I4_1)\n elif range.Min = range.Max then\n loadI4 ilg range.Min\n ilg.Emit(OpCodes.Ceq)\n if pushFalseIfInRange then emitNot()\n elif minValue = range.Min then\n // we only have to check the right bound\n loadI4 ilg range.Max\n ilg.Emit(OpCodes.Cgt)\n if not pushFalseIfInRange then emitNot()\n elif range.Max = maxValue then\n // we only have to check the left bound\n loadI4 ilg range.Min\n ilg.Emit(OpCodes.Clt)\n if not pushFalseIfInRange then emitNot()\n else\n // we have to check both bounds\n if range.Min <> 0 then\n loadI4 ilg range.Min\n ilg.Emit(OpCodes.Sub)\n loadI4 ilg (range.Max - range.Min)\n ilg.Emit(OpCodes.Cgt_Un) // unsigned comparison\n if not pushFalseIfInRange then emitNot()\n\nlet emitTwoRangeTest (ilg: ILGenerator) (loadVar: ILGenerator -> unit) inverse minValue maxValue (range1: Range) (range2: Range) =\n assert (range1.Max < range2.Min && range1.Max + 1 < range2.Min)\n let needOuterRangeCheck = minValue < range1.Min || range2.Max < maxValue\n let w = sizeof*8\n if needOuterRangeCheck && (maxValue - minValue < w) then\n // use a simple bit vector test:\n // (bits >> (var - off)) & 1\n let off = if minValue > 0 && maxValue < w then 0 else minValue\n let mutable bits = if inverse then unativeint -1n else 0un\n for r in [range1; range2] do\n for i in r.Min .. r.Max do\n let b = i - off\n if inverse then\n bits <- bits ^^^ (1un <<< b)\n else\n bits <- bits ||| (1un <<< b)\n loadU ilg bits\n loadVar ilg\n if off <> 0 then\n loadI4 ilg off\n ilg.Emit(OpCodes.Sub)\n ilg.Emit(OpCodes.Shr_Un)\n ilg.Emit(OpCodes.Ldc_I4_1)\n ilg.Emit(OpCodes.And)\n elif not needOuterRangeCheck\n || (range1.Max + 2 = range2.Min && range1.Min <> range1.Max && range2.Min <> range2.Max)\n then\n if needOuterRangeCheck then\n loadVar ilg\n emitRangeTest inverse ilg minValue maxValue (Range(range1.Min, range2.Max))\n loadVar ilg\n emitRangeTest (not inverse) ilg minValue maxValue (Range(range1.Max + 1, range2.Min - 1))\n if needOuterRangeCheck then\n if inverse then\n ilg.Emit(OpCodes.Or)\n else\n ilg.Emit(OpCodes.And)\n else\n loadVar ilg\n emitRangeTest inverse ilg minValue maxValue range1\n loadVar ilg\n emitRangeTest inverse ilg minValue maxValue range2\n if inverse then\n ilg.Emit(OpCodes.And)\n else\n ilg.Emit(OpCodes.Or)\n\n\ntype TempLocals(ilg: ILGenerator) =\n let mutable intLocal = null\n let mutable boolLocal = null\n\n /// used by emitSetMembershipTest (and indirectly by emitSwitch)\n member t.GetIntLocal() =\n if isNull intLocal then\n intLocal <- ilg.DeclareLocal(typeof)\n intLocal\n\n /// used by emitSwitch\n member t.GetBoolLocal() =\n if isNull boolLocal then\n boolLocal <- ilg.DeclareLocal(typeof)\n boolLocal\n\n/// flag used for testing purposes\nlet mutable noBitVectorTests = false\n\nlet emitSetMembershipTest (ilg: ILGenerator)\n (loadVar: ILGenerator -> unit) (storeResult: ILGenerator -> unit)\n (temps: TempLocals)\n lengthCap densityThreshold\n minValue maxValue\n inverse (ranges: Range[]) =\n\n checkRangesAreValidSortedAndUnconnected ranges\n\n let endLabel = ilg.DefineLabel()\n let outOfRangeLabel = ilg.DefineLabel()\n\n let emitBitVectorTest minValue maxValue iBegin iEnd =\n let first, last = ranges[iBegin].Min, ranges[iEnd - 1].Max\n // set up bit vector in unmanaged memory\n let w = sizeof*8\n // save a subtraction if it doesn't cost too much memory\n let off = if first > 0 && (last < w || (first < 3*w && (last >= first + w))) then 0\n else first\n\n let lastMinusOff = uint32 (last - off)\n if lastMinusOff > uint32 System.Int32.MaxValue then\n raise (System.ArgumentException(\"The ranges span width is too large.\"))\n\n let length = int (lastMinusOff/uint32 w + 1u)\n if uint32 length * uint32 w > uint32 System.Int32.MaxValue then\n raise (System.ArgumentException(\"The ranges span width is too large.\"))\n\n let mutable stackVar = 0un\n let ptr = if length = 1 then NativePtr.ofNativeInt (NativePtr.toNativeInt &&stackVar)\n else NativePtr.ofNativeInt (allocTemporaryMem (length*sizeof))\n\n // fill bit vector ptr[0..length - 1]\n let r = ranges[iBegin]\n let mutable rMin, rMax = r.Min - off, r.Max - off\n let mutable i = iBegin + 1\n if not inverse then\n for j = 0 to length - 1 do\n let mutable n = 0un\n let j1w = (j + 1)*w\n while rMin < j1w do\n n <- n ||| (1un <<< rMin%w)\n if rMin < rMax then rMin <- rMin + 1\n elif i < iEnd then\n let r = ranges[i]\n rMin <- r.Min - off; rMax <- r.Max - off\n i <- i + 1\n else rMin <- System.Int32.MaxValue // break\n NativePtr.set ptr j n\n else\n for j = 0 to length - 1 do\n let mutable n = unativeint -1n\n let j1w = (j + 1)*w\n while rMin < j1w do\n n <- n ^^^ (1un <<< rMin%w)\n if rMin < rMax then rMin <- rMin + 1\n elif i < iEnd then\n let r = ranges[i]\n rMin <- r.Min - off; rMax <- r.Max - off\n i <- i + 1\n else rMin <- System.Int32.MaxValue // break\n NativePtr.set ptr j n\n\n let intTemp = temps.GetIntLocal()\n\n // t = (uint32)(x - off)\n loadVar ilg\n if off <> 0 then\n loadI4 ilg off\n ilg.Emit(OpCodes.Sub)\n ilg.Emit(OpCodes.Stloc, intTemp)\n\n // if (t > (uint32)(last - off)) goto outOfRangeLabel\n if minValue < off || length*w <= maxValue - off then\n ilg.Emit(OpCodes.Ldloc, intTemp)\n loadI4 ilg (last - off)\n ilg.Emit(OpCodes.Bgt_Un, outOfRangeLabel)\n\n if length = 1 then\n // x = *ptr\n loadU ilg stackVar\n else\n // x = *(ptr + t/w)\n loadU ilg (unativeint (NativePtr.toNativeInt ptr))\n ilg.Emit(OpCodes.Ldloc, intTemp)\n loadI4 ilg w\n ilg.Emit(OpCodes.Div_Un)\n loadI4 ilg sizeof\n ilg.Emit(OpCodes.Mul)\n ilg.Emit(OpCodes.Add)\n ilg.Emit(OpCodes.Ldind_I)\n\n // result = (x >> t%w) & 1\n ilg.Emit(OpCodes.Ldloc, intTemp)\n if length > 1 then\n loadI4 ilg w\n ilg.Emit(OpCodes.Rem_Un)\n ilg.Emit(OpCodes.Shr_Un)\n ilg.Emit(OpCodes.Ldc_I4_1)\n ilg.Emit(OpCodes.And)\n storeResult ilg\n ilg.Emit(OpCodes.Br, endLabel)\n\n let emitRangeTest inverse minValue maxValue (range: Range) =\n loadVar ilg\n emitRangeTest inverse ilg minValue maxValue range\n storeResult ilg\n ilg.Emit(OpCodes.Br, endLabel)\n\n let emitTwoRangeTest inverse minValue maxValue range1 range2 =\n emitTwoRangeTest ilg loadVar inverse minValue maxValue range1 range2\n storeResult ilg\n ilg.Emit(OpCodes.Br, endLabel)\n\n let rec emitRegion minValue maxValue iBegin iEnd =\n Debug.Assert(iBegin < iEnd && minValue <= ranges[iBegin].Min && ranges[iEnd - 1].Max <= maxValue)\n\n match iEnd - iBegin with\n | 0 -> failwith \"emitSetMembershipTest.emitRegion\"\n | 1 -> emitRangeTest inverse minValue maxValue ranges[iBegin]\n | 2 -> emitTwoRangeTest inverse minValue maxValue ranges[iBegin] ranges[iBegin + 1]\n | _ -> // at least 3 ranges\n if not noBitVectorTests\n && density lengthCap ranges iBegin iEnd >= densityThreshold\n then\n emitBitVectorTest minValue maxValue iBegin iEnd\n else\n let i, pivotAroundRangeMax = findPivot ranges iBegin iEnd\n let label = ilg.DefineLabel()\n let r = ranges[i]\n loadVar ilg\n if pivotAroundRangeMax then\n loadI4 ilg r.Max\n ilg.Emit(OpCodes.Bgt, label)\n emitRegion minValue r.Max iBegin (i + 1)\n ilg.MarkLabel(label)\n emitRegion (r.Max + 1) maxValue (i + 1) iEnd\n else\n loadI4 ilg r.Min\n ilg.Emit(OpCodes.Blt, label)\n emitRegion r.Min maxValue i iEnd\n ilg.MarkLabel(label)\n emitRegion minValue (r.Min - 1) iBegin i\n\n if ranges.Length <> 0 then\n emitRegion minValue maxValue 0 ranges.Length\n\n ilg.MarkLabel(outOfRangeLabel)\n if inverse then\n ilg.Emit(OpCodes.Ldc_I4_1)\n else\n ilg.Emit(OpCodes.Ldc_I4_0)\n storeResult ilg\n ilg.MarkLabel(endLabel)\n\n\n\nlet emitSwitch (ilg: ILGenerator) (loadVar: ILGenerator -> unit) (temps: TempLocals)\n lengthCap densityThreshold\n minValue maxValue\n (defaultLabel: Label) (ranges: Range[]) (labels: Label[]) =\n Debug.Assert(ranges.Length = labels.Length)\n checkLabelRangesAreValidSortedAndUnconnected ranges labels\n\n let emitJumpTable (* minValue maxValue *) iBegin iEnd =\n // We can't optimize the range check of the switch statement,\n // so we have no use for minValue and maxValue arguments.\n // (In LLVM we could use the 'unreachable' instruction for optimizing the range check.)\n Debug.Assert(iBegin + 2 <= iEnd)\n let first = ranges[iBegin].Min\n let off = first\n let length =\n let last = ranges[iEnd - 1].Max\n let lastMinusOff = last - off\n if uint32 lastMinusOff >= uint32 System.Int32.MaxValue then\n raise (System.ArgumentException(\"The ranges span width is too large.\"))\n lastMinusOff + 1 // length <= Int32.MaxValue\n\n let jt = Array.zeroCreate length\n let mutable j = 0\n for i = iBegin to iEnd - 1 do\n let r = ranges[i]\n let rMin, rMax = r.Min - off, r.Max - off\n while j < rMin do\n jt[j] <- defaultLabel\n j <- j + 1\n let label = labels[i]\n while j <= rMax do\n jt[j] <- label\n j <- j + 1\n\n loadVar ilg\n if off <> 0 then\n loadI4 ilg off\n ilg.Emit(OpCodes.Sub)\n ilg.Emit(OpCodes.Switch, jt)\n ilg.Emit(OpCodes.Br, defaultLabel)\n\n let emitBranchIfInRange2 label (defaultLabel: Label) minValue maxValue (range: Range) =\n if minValue < range.Min || range.Max < maxValue then\n loadVar ilg\n emitBranchIfInRange ilg label minValue maxValue range\n ilg.Emit(OpCodes.Br, defaultLabel)\n else\n ilg.Emit(OpCodes.Br, label)\n\n let emitBranchIfInRange label minValue maxValue (range: Range) =\n loadVar ilg\n emitBranchIfInRange ilg label minValue maxValue range\n\n let emitBranchIfOutOfRange label minValue maxValue (range: Range) =\n if minValue < range.Min || range.Max < maxValue then\n loadVar ilg\n emitBranchIfOutOfRange ilg label minValue maxValue range\n\n let rec emitRegion minValue maxValue iBegin iEnd =\n Debug.Assert(iBegin < iEnd && minValue <= ranges[iBegin].Min && ranges[iEnd - 1].Max <= maxValue)\n\n let pivotAroundRange i pivotAroundRangeMax =\n let label = ilg.DefineLabel()\n let r = ranges[i]\n loadVar ilg\n if pivotAroundRangeMax then\n loadI4 ilg r.Max\n ilg.Emit(OpCodes.Bgt, label)\n emitRegion minValue r.Max iBegin (i + 1)\n ilg.MarkLabel(label)\n emitRegion (r.Max + 1) maxValue (i + 1) iEnd\n else\n loadI4 ilg r.Min\n ilg.Emit(OpCodes.Blt, label)\n emitRegion r.Min maxValue i iEnd\n ilg.MarkLabel(label)\n emitRegion minValue (r.Min - 1) iBegin i\n\n match iEnd - iBegin with\n | 0 ->\n failwith \"emitSwitch.emitRegion\"\n | 1 ->\n emitBranchIfInRange2 labels[iBegin] defaultLabel minValue maxValue ranges[iBegin]\n | 2 ->\n let r1, r2 = ranges[iBegin], ranges[iBegin + 1]\n let l1, l2 = labels[iBegin], labels[iBegin + 1]\n if l1 = l2 then\n Debug.Assert(r1.Max + 1 < r2.Min)\n //emitBranchIfOutOfRange defaultLabel minValue maxValue (Range(r1.Min, r2.Max))\n //emitBranchIfInRange2 defaultLabel l1 r1.Min r2.Max (Range(r1.Max + 1, r2.Min - 1))\n emitTwoRangeTest ilg loadVar false minValue maxValue r1 r2\n ilg.Emit(OpCodes.Brtrue, l1)\n ilg.Emit(OpCodes.Br, defaultLabel)\n else\n let rangesAreConnected = r1.Max + 1 = r2.Min\n let checkLeft, checkRight = minValue < r1.Min, r2.Max < maxValue\n if rangesAreConnected && ((checkLeft && checkRight) || (not checkLeft && not checkRight)) then\n emitBranchIfOutOfRange defaultLabel minValue maxValue (Range(r1.Min, r2.Max))\n // If 64-bit .NET JIT can substitute both of the branches emitted below with\n // the code at the destination, it chooses to substitute the first. Hence,\n // we put the more likely case first (assuming that values are\n // uniformly distributed on {minValue ... maxValue}).\n // (The 32-bit .NET JIT (version 4) doesn't yet seem to seriously attempt\n // a code block reordering optimization.)\n if uint32 (r1.Max - r1.Min) >= uint32 (r2.Max - r2.Min) then\n emitBranchIfInRange2 l1 l2 r1.Min r2.Max r1\n else\n emitBranchIfInRange2 l2 l1 r1.Min r2.Max r2\n else\n if (if rangesAreConnected then checkRight (* not checkLeft *)\n else uint32 (r1.Max - r1.Min) >= uint32 (r2.Max - r2.Min))\n then\n emitBranchIfInRange l1 minValue maxValue r1\n let minRightValue = if checkLeft then minValue else r1.Max + 1\n emitBranchIfInRange2 l2 defaultLabel minRightValue maxValue r2\n else\n emitBranchIfInRange l2 minValue maxValue r2\n let maxLeftValue = if checkRight then maxValue else r2.Min - 1\n emitBranchIfInRange2 l1 defaultLabel minValue maxLeftValue r1\n | _ -> // at least 3 ranges\n let allLabelsAreIdentical =\n let label = labels[iBegin]\n let mutable i = iBegin + 1\n while i < iEnd && label.Equals(labels[i]) do i <- i + 1\n i = iEnd\n if allLabelsAreIdentical then\n let bl = temps.GetBoolLocal()\n // emitSetMembershipTest doesn't use GetBoolLocal itself\n emitSetMembershipTest ilg loadVar (fun ilg -> ilg.Emit(OpCodes.Stloc, bl)) temps\n (lengthCap*8) (densityThreshold/32.)\n minValue maxValue\n false ranges[iBegin..(iEnd - 1)]\n ilg.Emit(OpCodes.Ldloc, bl)\n ilg.Emit(OpCodes.Brtrue, labels[iBegin])\n ilg.Emit(OpCodes.Br, defaultLabel)\n elif density lengthCap ranges iBegin iEnd >= densityThreshold then\n emitJumpTable iBegin iEnd\n else\n let i, pivotAroundRangeMax = findPivot ranges iBegin iEnd\n pivotAroundRange i pivotAroundRangeMax\n\n if ranges.Length <> 0 then\n emitRegion minValue maxValue 0 ranges.Length\n else\n ilg.Emit(OpCodes.Br, defaultLabel)\n\n#endif" }, { "path": "FParsec/Error.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: BSD-style. See accompanying documentation.\n\n[]\nmodule FParsec.Error\n\n//open FParsec\n\nopen System.Diagnostics\nopen System.Globalization\nopen System.IO\nopen FParsec.Internals\n\n// Unfortunately, F# currently doesn't support active patterns with more than 7\n// cases, so we have to use partial patterns.\n\ntype Expected = ErrorMessage.Expected\ntype ExpectedString = ErrorMessage.ExpectedString\ntype ExpectedStringCI = ErrorMessage.ExpectedCaseInsensitiveString\ntype Unexpected = ErrorMessage.Unexpected\ntype UnexpectedString = ErrorMessage.UnexpectedString\ntype UnexpectedStringCI = ErrorMessage.UnexpectedCaseInsensitiveString\ntype Message = ErrorMessage.Message\ntype NestedError = ErrorMessage.NestedError\ntype CompoundError = ErrorMessage.CompoundError\ntype OtherErrorMessage = ErrorMessage.Other\n\nlet (|Expected|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.Expected then Some msg.String else None\n\nlet (|ExpectedString|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.ExpectedString then Some msg.String else None\n\nlet (|ExpectedStringCI|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.ExpectedCaseInsensitiveString then Some msg.String else None\n\nlet (|Unexpected|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.Unexpected then Some msg.String else None\n\nlet (|UnexpectedString|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.UnexpectedString then Some msg.String else None\n\nlet (|UnexpectedStringCI|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.UnexpectedCaseInsensitiveString then Some msg.String else None\n\nlet (|Message|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.Message then Some msg.String else None\n\nlet (|NestedError|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.NestedError then\n let ne = msg :?> ErrorMessage.NestedError\n Some((ne.Position, ne.UserState, ne.Messages))\n else\n None\n\nlet (|CompoundError|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.CompoundError then\n let ce = msg :?> ErrorMessage.CompoundError\n Some((ce.LabelOfCompound, ce.NestedErrorPosition, ce.NestedErrorUserState, ce.NestedErrorMessages))\n else\n None\n\nlet (|OtherErrorMessage|_|) (msg: ErrorMessage) =\n if msg.Type = ErrorMessageType.Other then\n let om = msg :?> ErrorMessage.Other\n Some om.Data\n else\n None\n\n[]\nlet NoErrorMessages = null : ErrorMessageList\n\nlet (|ErrorMessageList|NoErrorMessages|) (error: ErrorMessageList) =\n if isNotNull error then ErrorMessageList(error.Head, error.Tail)\n else NoErrorMessages\n\nlet inline isSingleErrorMessageOfType (ty: ErrorMessageType) (error: ErrorMessageList) =\n isNotNull error && error.Head.Type = ty && isNull error.Tail\n\nlet expected label = ErrorMessageList(ErrorMessage.Expected(label))\nlet expectedString str = ErrorMessageList(ErrorMessage.ExpectedString(str))\nlet expectedStringCI str = ErrorMessageList(ErrorMessage.ExpectedCaseInsensitiveString(str))\nlet unexpected label = ErrorMessageList(ErrorMessage.Unexpected(label))\nlet unexpectedString str = ErrorMessageList(ErrorMessage.UnexpectedString(str))\nlet unexpectedStringCI str = ErrorMessageList(ErrorMessage.UnexpectedCaseInsensitiveString(str))\nlet messageError msg = ErrorMessageList(ErrorMessage.Message(msg))\nlet otherError obj = ErrorMessageList(ErrorMessage.Other(obj : obj))\n\nlet nestedError (stream: CharStream<'u>) (error: ErrorMessageList) =\n (*\n // manually inlined:\n match error with\n | ErrorMessageList(NestedError _, NoErrorMessages) -> error\n | _ -> ErrorMessageList(NestedError(stream.Position, stream.UserState, error), NoErrorMessages)\n *)\n if error |> isSingleErrorMessageOfType ErrorMessageType.NestedError\n then error\n else ErrorMessageList(ErrorMessage.NestedError(stream.Position, stream.UserState, error))\n\nlet compoundError label (stream: CharStream<'u>) (error: ErrorMessageList) =\n // manually inlined:\n (*\n match error with\n | ErrorMessageList(NestedError(pos, ustate, msgs), NoErrorMessages) ->\n ErrorMessageList(CompoundError(label, pos, ustate, msgs), NoErrorMessages)\n | _ -> ErrorMessageList(CompoundError(label, stream.Position, stream.UserState, error), NoErrorMessages)\n *)\n if error |> isSingleErrorMessageOfType ErrorMessageType.NestedError\n then\n let ne = error.Head :?> ErrorMessage.NestedError\n ErrorMessageList(ErrorMessage.CompoundError(label, ne.Position, ne.UserState, ne.Messages))\n else\n ErrorMessageList(ErrorMessage.CompoundError(label, stream.Position, stream.UserState, error))\n\nlet\n#if !NOINLINE\n inline\n#endif\n mergeErrors errorMessages1 errorMessages2 = ErrorMessageList.Merge(errorMessages1, errorMessages2)\n\n/// the default position printer\nlet internal printPosition (tw: System.IO.TextWriter) (p: Position) (indent: string) (columnWidth: int) =\n tw.Write(indent)\n tw.WriteLine(Strings.ErrorPosition(p))\n\nlet internal printErrorPosition (tabSize: int) (lw: LineWrapper) (stream: CharStream<'u>) (p: Position) =\n /// writes the string with all whitespace chars replaced with ' '\n let writeStringWithSimplifiedWhitespace (tw: TextWriter) (s: string) =\n let mutable i0 = 0\n for i = 0 to s.Length - 1 do\n let c = s[i]\n if Text.IsWhitespace(c) then\n if i0 < i then\n tw.Write(s.Substring(i0, i - i0))\n tw.Write(' ')\n i0 <- i + 1\n if i0 < s.Length then\n if i0 = 0 then tw.Write(s)\n else tw.Write(s.Substring(i0, s.Length - i0))\n\n let sn = getLineSnippet stream p (lw.ColumnWidth - lw.Indentation.Length) tabSize lw.WriterIsMultiCharGraphemeSafe\n let str = sn.String\n\n lw.PrintLine(Strings.ErrorPosition(p, sn.UnaccountedNewlines, sn.Column, sn.Utf16Column))\n\n let msgs = ResizeArray<_>()\n if sn.LineContainsTabsBeforeIndex then\n let mutable msg = Strings.ColumnCountAssumesTabStopDistanceOfNChars(tabSize)\n if sn.Column = sn.Utf16Column then\n msg <- msg + Strings.Utf16ColumnCountOnlyCountsEachTabAs1Char\n msgs.Add(msg)\n\n if str.Length > 0 then\n let tw = lw.TextWriter\n tw.Write(lw.Indentation)\n writeStringWithSimplifiedWhitespace tw str\n tw.WriteLine()\n tw.Write(lw.Indentation)\n if sn.TextElementIndex > 0 then\n tw.Write(new string(' ', sn.TextElementIndex))\n tw.Write('^')\n let d = sn.Index - sn.TextElementIndex\n if d <> 0 && not lw.WriterIsMultiCharGraphemeSafe then\n if d > 1 then\n tw.Write(new string('-', d - 1))\n tw.Write('^')\n msgs.Add(Strings.ExactPositionBetweenCaretsDependsOnDisplayUnicodeCapabilities)\n tw.WriteLine()\n\n if sn.Index < str.Length then\n let i = sn.Index\n let c = str[i]\n if System.Char.IsSurrogate(c) then\n if System.Char.IsHighSurrogate(c) then\n if i + 1 < str.Length && System.Char.IsLowSurrogate(str[i + 1]) then\n msgs.Add(Strings.ErrorOccurredAtBeginningOfSurrogatePair(str.Substring(i, 2)))\n else\n msgs.Add(Strings.CharAtErrorPositionIsIsolatedHighSurrogate(c))\n else // low surrogate\n if i > 0 && System.Char.IsHighSurrogate(str[i - 1]) then\n msgs.Add(Strings.ErrorOccurredAtSecondCharInSurrogatePair(str.Substring(i - 1, 2)))\n else\n msgs.Add(Strings.CharAtErrorPositionIsIsolatedLowSurrogate(c))\n elif i > 0 then\n let c1 = str[i - 1]\n if System.Char.IsHighSurrogate(c1) then\n msgs.Add(Strings.CharBeforeErrorPositionIsIsolatedHighSurrogate(c1))\n elif System.Char.IsLowSurrogate(c1) then\n msgs.Add(Strings.CharBeforeErrorPositionIsIsolatedLowSurrogate(c1))\n else\n if p.Index = stream.IndexOfLastCharPlus1 then msgs.Add(Strings.ErrorOccurredAtEndOfInputStream)\n elif str.Length = 0 then msgs.Add(Strings.ErrorOccurredOnAnEmptyLine)\n else msgs.Add(Strings.ErrorOccurredAtEndOfLine)\n\n if sn.LengthOfTextElement > 1 && (sn.LengthOfTextElement > 2 || not (System.Char.IsSurrogate(str[sn.Index]))) then\n let n = sn.Index - sn.IndexOfTextElement + 1\n let te = str.Substring(sn.IndexOfTextElement, sn.LengthOfTextElement)\n msgs.Add(Strings.ErrorOccurredAtNthCharInCombiningCharacterSequence(n, te))\n elif sn.IsBetweenCRAndLF then\n msgs.Add(Strings.ErrorOccurredAtSecondCharInNewline)\n\n if sn.UnaccountedNewlines > 0 then\n let n = sn.UnaccountedNewlines\n msgs.Add(Strings.InputContainsAtLeastNUnaccountedNewlines(n))\n\n if msgs.Count = 1 then lw.PrintLine(Strings.Note, msgs[0])\n elif msgs.Count > 1 then\n let ind = lw.Indentation\n let ind2 = ind + \" \"\n lw.PrintLine(Strings.Note)\n for msg in msgs do\n lw.Print(\"* \")\n lw.Indentation <- ind2\n lw.PrintLine(msg)\n lw.Indentation <- ind\n\n[]\ntype ParserError(position: Position, userState: obj, messages: ErrorMessageList) =\n do if isNull position then nullArg \"pos\"\n\n let defaultColumnWidth = 79\n let defaultIndentation = \"\"\n let defaultIndentationIncrement = \" \"\n let defaultTabSize = 8\n\n member t.Position = position\n member t.UserState = userState\n member T.Messages = messages\n\n override t.ToString() =\n use sw = new System.IO.StringWriter()\n t.WriteTo(sw)\n sw.ToString()\n\n member t.ToString(streamWhereErrorOccurred: CharStream<'u>) =\n use sw = new System.IO.StringWriter()\n t.WriteTo(sw, streamWhereErrorOccurred)\n sw.ToString()\n\n member t.WriteTo(textWriter: System.IO.TextWriter,\n ?positionPrinter: (System.IO.TextWriter -> Position -> string -> int -> unit),\n ?columnWidth: int, ?initialIndentation: string, ?indentationIncrement: string) =\n\n let positionPrinter = defaultArg positionPrinter printPosition\n let columnWidth = defaultArg columnWidth defaultColumnWidth\n let ind = defaultArg initialIndentation defaultIndentation\n let indIncrement = defaultArg indentationIncrement defaultIndentationIncrement\n let lw = new LineWrapper(textWriter, columnWidth, Indentation = ind)\n t.WriteTo(lw, positionPrinter, indIncrement)\n\n member t.WriteTo(textWriter: System.IO.TextWriter,\n streamWhereErrorOccurred: CharStream<'u>,\n ?tabSize: int,\n ?columnWidth: int, ?initialIndentation: string, ?indentationIncrement: string) =\n\n let originalStreamName = t.Position.StreamName\n let getStream = fun (pos: Position) -> if pos.StreamName = originalStreamName then streamWhereErrorOccurred else null\n t.WriteTo(textWriter, getStream, ?tabSize = tabSize, ?columnWidth = columnWidth, ?initialIndentation = initialIndentation, ?indentationIncrement = indentationIncrement)\n\n member t.WriteTo(textWriter: System.IO.TextWriter,\n getStream: (Position -> CharStream<'u>),\n ?tabSize: int,\n ?columnWidth: int, ?initialIndentation: string, ?indentationIncrement: string) =\n\n let columnWidth = defaultArg columnWidth defaultColumnWidth\n let ind = defaultArg initialIndentation defaultIndentation\n let indIncrement = defaultArg indentationIncrement defaultIndentationIncrement\n let tabSize = defaultArg tabSize defaultTabSize\n let lw = new LineWrapper(textWriter, columnWidth, Indentation = ind)\n let positionPrinter =\n fun tw position indent columnWidth ->\n let stream = getStream position\n if isNotNull stream then\n printErrorPosition tabSize lw stream position\n else\n printPosition lw.TextWriter position indent columnWidth\n t.WriteTo(lw, positionPrinter, indIncrement)\n\n member private t.WriteTo(lw: LineWrapper,\n positionPrinter: System.IO.TextWriter -> Position -> string -> int -> unit,\n indentationIncrement: string) =\n\n let rec printMessages (position: Position) (msgs: ErrorMessageList) =\n positionPrinter lw.TextWriter position lw.Indentation lw.ColumnWidth\n let nra() = new ResizeArray<_>()\n let expectedA, unexpectedA, messageA, nestedA, compoundA = nra(), nra(), nra(), nra(), nra()\n let mutable otherCount = 0\n for msg in ErrorMessageList.ToSortedArray(msgs) do\n match msg.Type with\n | ErrorMessageType.Expected -> expectedA.Add(msg.String)\n | ErrorMessageType.ExpectedString -> expectedA.Add(Strings.Quote(msg.String))\n | ErrorMessageType.ExpectedCaseInsensitiveString -> expectedA.Add(Strings.QuoteCaseInsensitive(msg.String))\n | ErrorMessageType.Unexpected -> unexpectedA.Add(msg.String)\n | ErrorMessageType.UnexpectedString -> unexpectedA.Add(Strings.Quote(msg.String))\n | ErrorMessageType.UnexpectedCaseInsensitiveString -> unexpectedA.Add(Strings.QuoteCaseInsensitive(msg.String))\n | ErrorMessageType.Message -> messageA.Add(msg.String)\n | ErrorMessageType.NestedError ->\n let ne = msg :?> ErrorMessage.NestedError\n nestedA.Add((ne.Position, ne.Messages))\n | ErrorMessageType.CompoundError ->\n if not (isNullOrEmpty msg.String) then expectedA.Add(msg.String)\n let ce = msg :?> ErrorMessage.CompoundError\n compoundA.Add((ce.String, ce.NestedErrorPosition, ce.NestedErrorMessages))\n | ErrorMessageType.Other ->\n otherCount <- otherCount + 1\n | _ ->\n failwith \"printMessages\"\n\n let printArray title (a: ResizeArray) (sep: string) =\n lw.Print(title, \" \")\n let n = a.Count\n for i = 0 to n - 3 do\n lw.Print(a[i], \", \")\n if n > 1 then lw.Print(a[n - 2], sep)\n if n > 0 then lw.Print(a[n - 1])\n lw.Newline()\n if expectedA.Count > 0 then\n printArray Strings.Expecting expectedA Strings.Or\n if unexpectedA.Count > 0 then\n printArray Strings.Unexpected unexpectedA Strings.And\n let ind = lw.Indentation\n let indInd = ind + indentationIncrement\n if messageA.Count > 0 then\n if expectedA.Count > 0 || unexpectedA.Count > 0 then\n lw.PrintLine(Strings.OtherErrors)\n lw.Indentation <- indInd\n for m in messageA do\n lw.PrintLine(m)\n if expectedA.Count > 0 || unexpectedA.Count > 0 then\n lw.Indentation <- ind\n for label, pos2, msgs2 in compoundA do\n lw.Newline()\n lw.PrintLine(Strings.CompoundCouldNotBeParsedBecause(label))\n lw.Indentation <- indInd\n printMessages pos2 msgs2\n lw.Indentation <- ind\n for pos2, msgs2 in nestedA do\n lw.Newline()\n lw.PrintLine(Strings.ParserBacktrackedAfter)\n lw.Indentation <- indInd\n printMessages pos2 msgs2\n lw.Indentation <- ind\n if expectedA.Count = 0 && unexpectedA.Count = 0 && messageA.Count = 0\n && compoundA.Count = 0 && nestedA.Count = 0\n then\n lw.PrintLine(Strings.UnknownErrors)\n printMessages position messages\n\n override t.Equals(value: obj) =\n referenceEquals (t :> obj) value\n || match value with\n | null -> false\n | :? ParserError as other ->\n t.Position = other.Position\n && t.Messages = other.Messages\n && t.UserState = other.UserState\n | _ -> false\n\n override t.GetHashCode() = t.Position.GetHashCode() ^^^ hash t.Messages\n\nlet inline internal raiseInfiniteLoopException name stream =\n raise (FParsec.Internal.ParserCombinatorInInfiniteLoopHelper.CreateException(name, stream))\n" }, { "path": "FParsec/Error.fsi", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.Error\n\ntype Expected = ErrorMessage.Expected\ntype ExpectedString = ErrorMessage.ExpectedString\ntype ExpectedStringCI = ErrorMessage.ExpectedCaseInsensitiveString\ntype Unexpected = ErrorMessage.Unexpected\ntype UnexpectedString = ErrorMessage.UnexpectedString\ntype UnexpectedStringCI = ErrorMessage.UnexpectedCaseInsensitiveString\ntype Message = ErrorMessage.Message\ntype NestedError = ErrorMessage.NestedError\ntype CompoundError = ErrorMessage.CompoundError\ntype OtherErrorMessage = ErrorMessage.Other\n\nval (|Expected|_|): ErrorMessage -> string option\nval (|ExpectedString|_|): ErrorMessage -> string option\nval (|ExpectedStringCI|_|): ErrorMessage -> string option\nval (|Unexpected|_|): ErrorMessage -> string option\nval (|UnexpectedString|_|): ErrorMessage -> string option\nval (|UnexpectedStringCI|_|): ErrorMessage -> string option\nval (|Message|_|): ErrorMessage -> string option\nval (|NestedError|_|): ErrorMessage -> (Position * obj * ErrorMessageList) option\nval (|CompoundError|_|): ErrorMessage -> (string * Position * obj * ErrorMessageList) option\nval (|OtherErrorMessage|_|): ErrorMessage -> obj option\n\n[]\nval NoErrorMessages: ErrorMessageList = null;;\nval (|ErrorMessageList|NoErrorMessages|): ErrorMessageList -> Choice\n\nval inline isSingleErrorMessageOfType: ErrorMessageType -> ErrorMessageList -> bool\n\n/// `expectedError label` creates an `ErrorMessageList` with a single `Expected label` message.\nval expected: string -> ErrorMessageList\n/// `expectedStringError str` creates an `ErrorMessageList` with a single `ExpectedString str` message.\nval expectedString: string -> ErrorMessageList\n/// `expectedStringCIError str` creates an `ErrorMessageList` with a single `ExpectedStringCI str` message.\nval expectedStringCI: string -> ErrorMessageList\n\n/// `unexpectedError label` creates an `ErrorMessageList` with a single `Unexpected label` message.\nval unexpected: string -> ErrorMessageList\n/// `unexpectedStringError str` creates an `ErrorMessageList` with a single `UnexpectedString str` message.\nval unexpectedString: string -> ErrorMessageList\n/// `unexpectedStringCIError str` creates an `ErrorMessageList` with a single `UnexpectedStringCI str` message.\nval unexpectedStringCI: string -> ErrorMessageList\n\n/// `messageError msg` creates an `ErrorMessageList` with a single `Message msg` message.\nval messageError: string -> ErrorMessageList\n\n/// `otherError o` creates an `ErrorMessageList` with a single `OtherError o` message.\nval otherError: obj -> ErrorMessageList\n\n/// `backtrackError stream msgs` creates an `ErrorMessageList` with a single `BacktrackPoint stream.Position msgs` message,\n/// except if `msgs` is already an `ErrorMessageList` with a single `BacktrackPoint(_, _)` message,\n/// in which case `msgs` is returned instead.\nval nestedError: CharStream<'u> -> ErrorMessageList -> ErrorMessageList\n\n/// `compoundError label state msgs` creates an `ErrorMessageList` with a single `CompoundError label stream.Position msgs` message,\n/// except if `msgs` is an `ErrorMessageList` with a single `BacktrackPoint(pos2, msgs2)` message,\n/// in which case an `ErrorMessageList` with a single `CompoundError label pos2 msgs2` message is returned instead.\nval compoundError: string -> CharStream<'u> -> ErrorMessageList -> ErrorMessageList\n\n/// `mergeErrors error1 error2` is equivalent to `ErrorMessageList.Merge(error1, error2)`.\nval\n#if !NOINLINE\n inline\n#endif\n mergeErrors: ErrorMessageList -> ErrorMessageList -> ErrorMessageList\n\n/// Represents a simple container type that brings together the position, user state and error messages of a parser error.\n[]\ntype ParserError =\n new: Position * userState:obj * ErrorMessageList -> ParserError\n\n member Position: Position\n member UserState: obj\n member Messages: ErrorMessageList\n\n /// Returns a string representation of the `ParserError`.\n override ToString: unit -> string\n\n /// Returns a string representation of the `ParserError`.\n ///\n /// The given `CharStream` must contain the content of the original `CharStream`\n /// for which this `ParserError` was generated (at the original indices).\n ///\n /// For each error location the printed position information is augmented\n /// with the line of text surrounding the error position, together with a '^'-marker\n /// pointing to the exact location of the error in the input stream.\n member ToString: streamWhereErrorOccurred: CharStream<'u> -> string\n\n /// Writes a string representation of the `ParserError` to the given `TextWriter` value.\n ///\n /// The given `CharStream` must contain the content of the original `CharStream`\n /// for which this `ParserError` was generated (at the original indices).\n ///\n /// For each error location the printed position information is augmented\n /// with the line of text surrounding the error position, together with a '^'-marker\n /// pointing to the exact location of the error in the input stream.\n member WriteTo: textWriter: System.IO.TextWriter\n * streamWhereErrorOccurred: CharStream<'u>\n * ?tabSize: int\n * ?columnWidth: int \n * ?initialIndentation: string * ?indentationIncrement: string\n -> unit\n\n /// Writes a string representation of the `ParserError` to the given `TextWriter` value.\n ///\n /// For each error position `getStreamByName` is called with the `StreamName` of the `Position`.\n /// The returned `CharStream` must be `null` or contain the content of the `CharStream` for which\n /// the error was generated (at the original indices).\n ///\n /// If `getStreamByName` returns a non-null `CharStream`, the printed error position information is\n /// augmented with the line of text surrounding the error position, together with a '^'-marker\n /// pointing to the exact location of the error in the input stream.\n member WriteTo: textWriter: System.IO.TextWriter\n * getStream: (Position -> CharStream<'u>)\n * ?tabSize: int\n * ?columnWidth: int \n * ?initialIndentation: string * ?indentationIncrement: string \n -> unit\n\n /// Writes a string representation of the `ParserError` to the given `TextWriter` value.\n ///\n /// The format of the position information can be customized by specifying the `positionPrinter`\n /// argument. The given function is expected to print a representation of the passed `Position` value\n /// to the passed `TextWriter` value. If possible, it should indent text lines with the passed string\n /// and take into account the maximum column count (including indentation) passed as the last argument.\n member WriteTo: textWriter: System.IO.TextWriter\n * ?positionPrinter: (System.IO.TextWriter -> Position -> string -> int -> unit)\n * ?columnWidth: int \n * ?initialIndentation: string * ?indentationIncrement: string\n -> unit\n\n override Equals: obj -> bool\n override GetHashCode: unit -> int\n\n\nval inline internal raiseInfiniteLoopException: string -> CharStream -> 'a" }, { "path": "FParsec/FParsec-LowTrust.fsproj", "content": "\n\n \n netstandard2.0;netstandard2.1\n \n\n \n\n \n \n \n\n\n" }, { "path": "FParsec/FParsec.fsproj", "content": "\n\n \n net6.0\n \n\n \n\n \n \n \n\n\n" }, { "path": "FParsec/FParsec.targets", "content": "\n\n \n FParsec\n FParsec\n true\n false\n /nooptimizationdata\n \n\n \n FParsec is a parser combinator library for F#.\n $(PackageTags);parser;combinator;f#;fsharp;c#;csharp;parsec;fparsec\n $(Summary)\n\nYou can find comprehensive documentation for FParsec at http://www.quanttec.com/fparsec. The documentation includes a feature list, a tutorial, a user’s guide and an API reference.\n $(Description)\n\nThis package uses the basic “low-trust” configuration of FParsec, which does not use any unverifiable code and is optimized for maximum portability. If you need to parse very large files or if you employ FParsec for performance-critical jobs, consider using the alternate “Big Data Edition” NuGet package (see https://nuget.org/packages/fparsec-big-data-edition).\n $(Description)\n\nThis package uses a configuration of FParsec that supports very large input streams and is optimized for maximum performance in longer running processes. See http://www.quanttec.com/fparsec/download-and-installation.html for more information.\n\nThis version of FParsec is currently not compatible with .NET Core. If you want to use .NET Core, please choose the other FParsec NuGet package instead (see https://nuget.org/packages/fparsec).\n \n\n \n FParsec-Big-Data-Edition\n $(DefineConstants);USE_STATIC_MAPPING_FOR_IS_ANY_OF\n \n\n \n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n \n \n \n\n \n \n $(TargetsForTfmSpecificBuildOutput);IncludeFParsecCSOutput\n \n \n \n \n \n \n \n \n\n \n true\n \n \n \n \n \n\n\n" }, { "path": "FParsec/Internals.fs", "content": "// Copyright (c) Stephan Tolksdorf 2009-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.Internals\n\nopen System.Diagnostics\n\nlet inline referenceEquals<'a when 'a : not struct> (x: 'a) (y: 'a) =\n obj.ReferenceEquals(x, y)\nlet inline isNull<'a when 'a : not struct> (x: 'a) =\n referenceEquals (box x) null\nlet inline isNotNull<'a when 'a : not struct> (x: 'a) =\n not (isNull x)\n\nlet inline isNullOrEmpty (s: string) = System.String.IsNullOrEmpty(s)\n\n// the F# compiler doesn't yet \"fuse\" multiple '+' string concatenations into one, as the C# compiler does\nlet inline concat3 (a: string) (b: string) (c: string) = System.String.Concat(a, b, c)\nlet inline concat4 (a: string) (b: string) (c: string) (d: string) = System.String.Concat(a, b, c, d)\nlet inline concat5 (a: string) (b: string) (c: string) (d: string) (e: string) = System.String.Concat([|a;b;c;d;e|])\nlet inline concat6 (a: string) (b: string) (c: string) (d: string) (e: string) (f: string) = System.String.Concat([|a;b;c;d;e;f|])\nlet inline concat7 (a: string) (b: string) (c: string) (d: string) (e: string) (f: string) (g: string) = System.String.Concat([|a;b;c;d;e;f;g|])\n\nlet findNewlineOrEOSChar = Text.FindNewlineOrEOSChar\n\nlet getSortedUniqueValues (s: seq<_>) =\n let a = Array.ofSeq s\n if a.Length = 0 then a\n else\n Array.sortInPlace a\n let mutable previous = a[0]\n let mutable n = 1\n for i = 1 to a.Length - 1 do\n let c = a[i]\n if c <> previous then n <- n + 1\n previous <- c\n if n = a.Length then a\n else\n let b = Array.zeroCreate n\n let mutable i = 0\n for j = 0 to b.Length - 1 do\n let c = a[i]\n b[j] <- c\n i <- i + 1\n while i < a.Length && a[i] = c do i <- i + 1\n b\n\n/// A primitive pretty printer.\ntype LineWrapper(tw: System.IO.TextWriter, columnWidth: int, writerIsMultiCharGraphemeSafe: bool) =\n do if columnWidth < 1 then invalidArg \"columnWidth\" \"columnWidth must be positive.\"\n\n let mutable indentation = \"\"\n let mutable maxSpace = columnWidth\n let mutable space = columnWidth\n let mutable afterNewline = true\n let mutable afterSpace = false\n\n new (tw: System.IO.TextWriter, columnWidth: int) =\n LineWrapper(tw, columnWidth, not tw.Encoding.IsSingleByte)\n\n member t.TextWriter = tw\n member t.ColumnWidth = columnWidth\n member t.WriterIsMultiCharGraphemeSafe = writerIsMultiCharGraphemeSafe\n\n member t.Indentation\n with get() = indentation\n and set (s: string) =\n let s = if s.Length <= columnWidth - 1 then s\n else s.Substring(0, columnWidth - 1) // guarantee maxSpace >= 1\n indentation <- s\n maxSpace <- columnWidth - s.Length\n if afterNewline then space <- maxSpace\n\n member t.Newline() =\n tw.WriteLine()\n afterNewline <- true\n afterSpace <- false\n space <- maxSpace\n\n member t.Space() =\n afterSpace <- true\n\n member t.Print(s: string) =\n if isNotNull s then\n let mutable start = 0\n for i = 0 to s.Length - 1 do\n let c = s[i]\n if (if c <= ' ' then c = ' ' || (c >= '\\t' && c <= '\\r')\n else c >= '\\u0085' && (c = '\\u0085' || c = '\\u2028' || c = '\\u2029'))\n then // any ' ', tab or newlines\n if start < i then\n t.Write(s.Substring(start, i - start))\n t.Space()\n start <- i + 1\n if start < s.Length then\n if start = 0 then t.Write(s)\n else t.Write(s.Substring(start, s.Length - start))\n\n member t.Print(s1, s2) = t.Print(s1); t.Print(s2)\n member t.Print(s1, s2, s3) = t.Print(s1); t.Print(s2); t.Print(s3)\n member t.PrintLine(s: string) = t.Print(s); t.Newline()\n member t.PrintLine(s1: string, s2: string) = t.Print(s1); t.Print(s2); t.Newline()\n member t.PrintLine(s1: string, s2: string, s3: string) = t.Print(s1); t.Print(s2); t.Print(s3); t.Newline()\n\n member private t.Write(s: string) =\n Debug.Assert(s.Length > 0)\n if afterNewline then\n tw.Write(indentation)\n afterNewline <- false\n let n = if writerIsMultiCharGraphemeSafe then Text.CountTextElements(s) else s.Length\n match afterSpace with\n | true when n + 1 <= space ->\n tw.Write(' ')\n tw.Write(s)\n space <- space - 1 - n\n afterSpace <- false\n | false when n <= space ->\n tw.Write(s)\n space <- space - n\n | _ when s.Length <= maxSpace ->\n tw.WriteLine()\n tw.Write(indentation)\n tw.Write(s)\n space <- maxSpace - n\n afterSpace <- false\n | _ ->\n t.Break(s)\n\n /// breaks a string into multiple lines along text element boundaries.\n member private t.Break(s: string) =\n Debug.Assert(s.Length > 0 && not afterNewline)\n if afterSpace then\n afterSpace <- false\n if space > 1 then\n tw.Write(' ')\n space <- space - 1\n else\n tw.WriteLine()\n tw.Write(indentation)\n space <- maxSpace\n elif space = 0 then\n tw.WriteLine()\n tw.Write(indentation)\n space <- maxSpace\n let te = System.Globalization.StringInfo.GetTextElementEnumerator(s)\n te.MoveNext() |> ignore\n Debug.Assert(te.ElementIndex = 0)\n if writerIsMultiCharGraphemeSafe then\n let mutable startIndex = 0\n while te.MoveNext() do\n space <- space - 1\n if space = 0 then\n let index = te.ElementIndex\n tw.WriteLine(s.Substring(startIndex, index - startIndex))\n tw.Write(indentation)\n space <- maxSpace\n startIndex <- index\n space <- space - 1\n tw.Write(s.Substring(startIndex, s.Length - startIndex))\n else\n // We don't break up text elements, but when we fit string pieces into lines we\n // use UTF-16 lengths instead of text element counts (in order to support displays\n // that have problems with combining character sequences).\n let mutable startIndex = 0\n let mutable lastIndex = 0\n while te.MoveNext() do\n let index = te.ElementIndex\n let count = index - startIndex\n if count < space then\n lastIndex <- index\n elif count = space || lastIndex <= startIndex then\n tw.WriteLine(s.Substring(startIndex, count))\n tw.Write(indentation)\n space <- maxSpace\n startIndex <- index\n else\n tw.WriteLine(s.Substring(startIndex, lastIndex - startIndex))\n tw.Write(indentation)\n space <- maxSpace\n startIndex <- lastIndex\n let index = s.Length\n let count = index - startIndex\n if count <= space then\n tw.Write(s.Substring(startIndex, count))\n space <- space - count\n elif lastIndex <= startIndex then\n tw.WriteLine(s.Substring(startIndex, index - startIndex))\n space <- maxSpace\n afterNewline <- true\n else\n tw.WriteLine(s.Substring(startIndex, lastIndex - startIndex))\n tw.Write(indentation)\n tw.Write(s.Substring(lastIndex, index - lastIndex))\n space <- maxSpace - (index - lastIndex)\n if space < 0 then\n tw.WriteLine()\n space <- maxSpace\n afterNewline <- true\n\n\ntype LineSnippet = {\n String: string\n TextElementIndex: int\n Index: int\n IndexOfTextElement: int\n LengthOfTextElement: int\n UnaccountedNewlines: int\n Column: int64\n Utf16Column: int64 // the UTF16 tabs are only counted as 1 char\n LineContainsTabsBeforeIndex: bool\n IsBetweenCRAndLF: bool\n}\n\nlet getLineSnippet (stream: CharStream<'u>) (p: Position) (space: int) (tabSize: int) multiCharGraphemeSafe =\n Debug.Assert(space > 0 && tabSize > 0)\n Debug.Assert(p.Index >= stream.IndexOfFirstChar && p.Index <= stream.IndexOfLastCharPlus1)\n\n let isCombiningChar (s: string) =\n match System.Globalization.CharUnicodeInfo.GetUnicodeCategory(s, 0) with\n | System.Globalization.UnicodeCategory.NonSpacingMark\n | System.Globalization.UnicodeCategory.SpacingCombiningMark\n | System.Globalization.UnicodeCategory.EnclosingMark\n | System.Globalization.UnicodeCategory.Surrogate\n -> true\n | _ -> false\n\n let isUnicodeNewlineOrEos c =\n match c with\n | '\\n' | '\\r'| '\\u0085'| '\\u2028'| '\\u2029'\n | '\\uffff' -> true\n | _ -> false\n\n // we restrict the maximum column count, so that we don't accidentally\n // completely reread a multi-gigabyte file when it has no newlines\n let maxColForColCount = 1000\n let maxExtraChars = 32\n let colTooLarge = p.Column > int64 maxColForColCount\n\n let oldState = stream.State\n\n let mutable index = p.Index\n stream.Seek(index) // throws if index is too small\n if index <> stream.Index then\n raise (System.ArgumentException(\"The error position lies beyond the end of the stream.\"))\n let isBetweenCRAndLF = stream.Peek() = '\\n' && stream.Peek(-1) = '\\r'\n if isBetweenCRAndLF then\n stream.Skip(-1)\n index <- index - 1L\n else\n let mutable c = stream.Peek()\n let mutable n = 2*space + maxExtraChars\n // skip to end of line, but not over more than n chars\n while not (isUnicodeNewlineOrEos c) && n <> 0 do\n c <- stream.SkipAndPeek()\n n <- n - 1\n if not (isUnicodeNewlineOrEos c) then\n n <- maxExtraChars\n while isCombiningChar (stream.PeekString(2)) && n <> 0 do\n stream.Skip() |> ignore\n n <- n - 1\n let endIndexToken = stream.IndexToken\n\n stream.Seek(index)\n let lineBegin = index - p.Column + 1L\n // use SkipAndPeek instead of Skip, so that we can't move past the beginning of the stream\n stream.SkipAndPeek(if not colTooLarge then -(int32 p.Column - 1) else -(maxColForColCount - 1)) |> ignore\n if colTooLarge then\n let mutable n = if p.Column > int64 System.Int32.MaxValue then maxExtraChars\n else min maxExtraChars (int32 p.Column - maxColForColCount)\n while isCombiningChar (stream.PeekString(2)) && n <> 0 do\n stream.SkipAndPeek(-1) |> ignore\n n <- n - 1\n let mutable beginIndex = stream.Index\n let mutable columnOffset = beginIndex - lineBegin\n let mutable idx = int (index - beginIndex)\n\n let beginIndexToken = stream.IndexToken\n stream.Seek(endIndexToken)\n let mutable str = stream.ReadFrom(beginIndexToken)\n\n // we're done with the stream now\n stream.BacktrackTo(oldState)\n\n let mutable lastLineBeginIdx = 0\n let mutable unaccountedNLs = 0\n let mutable mayContainMultiCharGraphemes = false\n let mutable nTabs = 0\n\n for i = 0 to str.Length - 1 do\n let c = str[i]\n if c >= ' ' then\n if c >= '\\u0300' then\n mayContainMultiCharGraphemes <- true\n elif c = '\\t' then\n nTabs <- nTabs + 1\n elif c = '\\n' || (c = '\\r' && (i + 1 >= str.Length || str[i + 1] <> '\\n')) then\n // there can be no newline after idx\n lastLineBeginIdx <- i + 1\n unaccountedNLs <- unaccountedNLs + 1\n mayContainMultiCharGraphemes <- false\n nTabs <- 0\n\n if unaccountedNLs <> 0 then\n str <- str.Substring(lastLineBeginIdx)\n idx <- idx - lastLineBeginIdx\n columnOffset <- 0L\n\n let utf16Column = columnOffset + int64 (idx + 1)\n let mutable lineContainsTabsBeforeIndex = false\n if nTabs > 0 then // replace tabs with spaces\n let mutable off = if columnOffset = 0L then 0\n else int32 (columnOffset%(int64 tabSize))\n let sb = new System.Text.StringBuilder(str.Length + nTabs*tabSize)\n let mutable i0 = 0\n let mutable idxIncr = 0\n for i = 0 to str.Length - 1 do\n if str[i] = '\\t' then\n if i > i0 then sb.Append(str, i0, i - i0) |> ignore\n let n = tabSize - (off + i)%tabSize\n sb.Append(' ', n) |> ignore\n off <- off + (n - 1)\n if i < idx then\n lineContainsTabsBeforeIndex <- true\n idxIncr <- idxIncr + (n - 1)\n i0 <- i + 1\n if i0 < str.Length then sb.Append(str, i0, str.Length - i0) |> ignore\n str <- sb.ToString()\n idx <- idx + idxIncr\n\n let clip nBefore nAfter =\n let mutable nBefore, nAfter = nBefore, nAfter\n let mutable diff = nBefore + nAfter + 1 - space\n if diff > 0 then\n let d = nBefore - nAfter\n if d > 0 then\n let dd = min diff d\n nBefore <- nBefore - dd\n diff <- diff - dd\n elif d < 0 then\n let dd = min diff -d\n nAfter <- nAfter - dd\n diff <- diff - dd\n if diff <> 0 then\n if diff%2 = 0 then\n nBefore <- nBefore - diff/2\n nAfter <- nAfter - diff/2\n else\n nBefore <- nBefore - diff/2\n nAfter <- nAfter - diff/2 - 1\n nBefore, nAfter\n\n if not mayContainMultiCharGraphemes then\n let nBefore, nAfter = clip idx (if idx < str.Length then str.Length - idx - 1 else 0)\n {String = str.Substring(idx - nBefore, nBefore + nAfter + (if idx < str.Length then 1 else 0))\n Index = nBefore\n TextElementIndex = nBefore\n IndexOfTextElement = nBefore\n LengthOfTextElement = 1\n UnaccountedNewlines = unaccountedNLs\n Column = columnOffset + int64 (idx + 1)\n Utf16Column = utf16Column\n LineContainsTabsBeforeIndex = lineContainsTabsBeforeIndex\n IsBetweenCRAndLF = isBetweenCRAndLF}\n else\n let indices = System.Globalization.StringInfo.ParseCombiningCharacters(str)\n let mutable idxIdx = 0 // the indices index of the text element containing the str char at idx\n while idxIdx < indices.Length && indices[idxIdx] < idx do idxIdx <- idxIdx + 1\n if (if idxIdx < indices.Length then indices[idxIdx] > idx else idx < str.Length) then idxIdx <- idxIdx - 1\n let col = columnOffset + int64 (idxIdx + 1)\n let teIdx = if idxIdx < indices.Length then indices[idxIdx] else str.Length\n let teLength = (if idxIdx + 1 < indices.Length then indices[idxIdx + 1] else str.Length) - teIdx\n let mutable nBefore, nAfter = clip idxIdx (if idxIdx = indices.Length then 0 else indices.Length - idxIdx - 1)\n let mutable strBegin = let ii = idxIdx - nBefore in if ii < indices.Length then indices[ii] else str.Length\n let mutable strEnd = let ii = idxIdx + nAfter + 1 in if ii < indices.Length then indices[ii] else str.Length\n if not multiCharGraphemeSafe then\n while strEnd - strBegin > space && (nBefore > 0 || nAfter > 0) do\n if nBefore > nAfter then\n nBefore <- nBefore - 1\n strBegin <- indices[idxIdx - nBefore]\n else\n nAfter <- nAfter - 1\n strEnd <- indices[idxIdx + nAfter + 1]\n {String = str.Substring(strBegin, strEnd - strBegin)\n Index = idx - strBegin\n TextElementIndex = nBefore\n IndexOfTextElement = teIdx - strBegin\n LengthOfTextElement = teLength\n UnaccountedNewlines = unaccountedNLs\n Column = col\n Utf16Column = utf16Column\n LineContainsTabsBeforeIndex = lineContainsTabsBeforeIndex\n IsBetweenCRAndLF = isBetweenCRAndLF}\n\n\n" }, { "path": "FParsec/Primitives.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.Primitives\n\nopen FParsec.Internals\nopen FParsec.Error\n\n[]\nlet Ok = ReplyStatus.Ok\n[]\nlet Error = ReplyStatus.Error\n[]\nlet FatalError = ReplyStatus.FatalError\n\ntype Parser<'a, 'u> = CharStream<'u> -> Reply<'a>\n\n// The `PrimitiveTests.Reference` module contains simple (but inefficient)\n// reference implementations of most of the functions below.\n\n// =================================\n// Parser primitives and combinators\n// =================================\n\nlet preturn x : Parser<_,_> = fun stream -> Reply(x)\nlet pzero : Parser<_,_> = fun stream -> Reply()\n\n// ---------------------------\n// Chaining and piping parsers\n// ---------------------------\n\nlet (>>=) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) =\n match box f with\n // optimization for uncurried functions\n | :? OptimizedClosures.FSharpFunc<'a, CharStream<'u>, Reply<'b>> as optF ->\n fun stream ->\n let reply1 = p stream\n if reply1.Status = Ok then\n if isNull reply1.Error then\n // in separate branch because the JIT can produce better code for a tail call\n optF.Invoke(reply1.Result, stream)\n else\n let stateTag1 = stream.StateTag\n let mutable reply2 = optF.Invoke(reply1.Result, stream)\n if stateTag1 = stream.StateTag then\n reply2.Error <- mergeErrors reply2.Error reply1.Error\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n | _ ->\n fun stream ->\n let reply1 = p stream\n if reply1.Status = Ok then\n let p2 = f reply1.Result\n if isNull reply1.Error then\n // in separate branch because the JIT can produce better code for a tail call\n p2 stream\n else\n let stateTag1 = stream.StateTag\n let mutable reply2 = p2 stream\n if stateTag1 = stream.StateTag then\n reply2.Error <- mergeErrors reply2.Error reply1.Error\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (>>%) (p: Parser<'a,'u>) x =\n fun stream ->\n let reply = p stream\n Reply(reply.Status, x, reply.Error)\n\nlet (>>.) (p: Parser<'a,'u>) (q: Parser<'b,'u>) =\n fun stream ->\n let mutable reply1 = p stream\n if reply1.Status = Ok then\n if isNull reply1.Error then\n // in separate branch because the JIT can produce better code for a tail call\n q stream\n else\n let stateTag1 = stream.StateTag\n let mutable reply2 = q stream\n if stateTag1 = stream.StateTag then\n reply2.Error <- mergeErrors reply2.Error reply1.Error\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (.>>) (p: Parser<'a,'u>) (q: Parser<'b,'u>) =\n fun stream ->\n let mutable reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = q stream\n let error = if isNull reply1.Error then reply2.Error\n elif stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors reply2.Error reply1.Error\n reply1.Error <- error\n reply1.Status <- reply2.Status\n reply1\n\n\nlet (.>>.) (p: Parser<'a,'u>) (q: Parser<'b,'u>) =\n fun stream ->\n let reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = q stream\n let error = if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors reply1.Error reply2.Error\n let result = if reply2.Status = Ok then (reply1.Result, reply2.Result)\n else Unchecked.defaultof<_>\n Reply(reply2.Status, result, error)\n else\n Reply(reply1.Status, reply1.Error)\n\nlet between (popen: Parser<_,'u>) (pclose: Parser<_,'u>) (p: Parser<_,'u>) =\n fun stream ->\n let reply1 = popen stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let mutable reply2 = p stream\n if reply2.Status = Ok then\n let stateTag2 = stream.StateTag\n let reply3 = pclose stream\n let error = if stateTag2 <> stream.StateTag then reply3.Error\n else\n let error2 = mergeErrors reply2.Error reply3.Error\n if stateTag1 <> stateTag2 then error2\n else mergeErrors reply1.Error error2\n reply2.Error <- error\n reply2.Status <- reply3.Status\n reply2\n else\n let error = if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors reply1.Error reply2.Error\n reply2.Error <- error\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (|>>) (p: Parser<'a,'u>) f =\n fun stream ->\n let reply = p stream\n Reply(reply.Status,\n (if reply.Status = Ok then f reply.Result else Unchecked.defaultof<_>),\n reply.Error)\n\nlet pipe2 (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) f =\n let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n fun stream ->\n let mutable reply = Reply()\n let reply1 = p1 stream\n let mutable error = reply1.Error\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n error <- if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors error reply2.Error\n if reply2.Status = Ok then\n reply.Result <- optF.Invoke(reply1.Result, reply2.Result)\n reply.Status <- Ok\n else reply.Status <- reply2.Status\n else reply.Status <- reply1.Status\n reply.Error <- error\n reply\n\nlet pipe3 (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) (p3: Parser<'c,'u>) f =\n let optF = OptimizedClosures.FSharpFunc<_,_,_,_>.Adapt(f)\n fun stream ->\n let mutable reply = Reply()\n let reply1 = p1 stream\n let mutable error = reply1.Error\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n error <- if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors error reply2.Error\n if reply2.Status = Ok then\n let stateTag2 = stream.StateTag\n let reply3 = p3 stream\n error <- if stateTag2 <> stream.StateTag then reply3.Error\n else mergeErrors error reply3.Error\n if reply3.Status = Ok then\n reply.Result <- optF.Invoke(reply1.Result, reply2.Result, reply3.Result)\n reply.Status <- Ok\n else reply.Status <- reply3.Status\n else reply.Status <- reply2.Status\n else reply.Status <- reply1.Status\n reply.Error <- error\n reply\n\nlet pipe4 (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) (p3: Parser<'c,'u>) (p4: Parser<'d,'u>) f =\n let optF = OptimizedClosures.FSharpFunc<_,_,_,_,_>.Adapt(f)\n fun stream ->\n let mutable reply = Reply()\n let reply1 = p1 stream\n let mutable error = reply1.Error\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n error <- if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors error reply2.Error\n if reply2.Status = Ok then\n let stateTag2 = stream.StateTag\n let reply3 = p3 stream\n error <- if stateTag2 <> stream.StateTag then reply3.Error\n else mergeErrors error reply3.Error\n if reply3.Status = Ok then\n let stateTag3 = stream.StateTag\n let reply4 = p4 stream\n error <- if stateTag3 <> stream.StateTag then reply4.Error\n else mergeErrors error reply4.Error\n if reply4.Status = Ok then\n reply.Result <- optF.Invoke(reply1.Result, reply2.Result, reply3.Result, reply4.Result)\n reply.Status <- Ok\n else reply.Status <- reply4.Status\n else reply.Status <- reply3.Status\n else reply.Status <- reply2.Status\n else reply.Status <- reply1.Status\n reply.Error <- error\n reply\n\nlet pipe5 (p1: Parser<'a,'u>) (p2: Parser<'b,'u>) (p3: Parser<'c,'u>) (p4: Parser<'d,'u>) (p5: Parser<'e,'u>) f =\n let optF = OptimizedClosures.FSharpFunc<_,_,_,_,_,_>.Adapt(f)\n fun stream ->\n let mutable reply = Reply()\n let reply1 = p1 stream\n let mutable error = reply1.Error\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n error <- if stateTag1 <> stream.StateTag then reply2.Error\n else mergeErrors error reply2.Error\n if reply2.Status = Ok then\n let stateTag2 = stream.StateTag\n let reply3 = p3 stream\n error <- if stateTag2 <> stream.StateTag then reply3.Error\n else mergeErrors error reply3.Error\n if reply3.Status = Ok then\n let stateTag3 = stream.StateTag\n let reply4 = p4 stream\n error <- if stateTag3 <> stream.StateTag then reply4.Error\n else mergeErrors error reply4.Error\n if reply4.Status = Ok then\n let stateTag4 = stream.StateTag\n let reply5 = p5 stream\n error <- if stateTag4 <> stream.StateTag then reply5.Error\n else mergeErrors error reply5.Error\n if reply5.Status = Ok then\n reply.Result <- optF.Invoke(reply1.Result, reply2.Result, reply3.Result, reply4.Result, reply5.Result)\n reply.Status <- Ok\n else reply.Status <- reply5.Status\n else reply.Status <- reply4.Status\n else reply.Status <- reply3.Status\n else reply.Status <- reply2.Status\n else reply.Status <- reply1.Status\n reply.Error <- error\n reply\n\n\n// -----------------------------------------------\n// Parsing alternatives and recovering from errors\n// -----------------------------------------------\n\nlet (<|>) (p1: Parser<'a,'u>) (p2: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let mutable stateTag = stream.StateTag\n let mutable reply = p1 stream\n if reply.Status = Error && stateTag = stream.StateTag then\n let error = reply.Error\n reply <- p2 stream\n if stateTag = stream.StateTag then\n reply.Error <- mergeErrors reply.Error error\n reply\n\nlet choice (ps: seq>) =\n match ps with\n | :? (Parser<'a,'u>[]) as ps ->\n if ps.Length = 0 then pzero\n else\n fun stream ->\n let stateTag = stream.StateTag\n let mutable error = NoErrorMessages\n let mutable reply = ps[0] stream\n let mutable i = 1\n while reply.Status = Error && stateTag = stream.StateTag && i < ps.Length do\n error <- mergeErrors error reply.Error\n reply <- ps[i] stream\n i <- i + 1\n if stateTag = stream.StateTag then\n error <- mergeErrors error reply.Error\n reply.Error <- error\n reply\n | :? (Parser<'a,'u> list) as ps ->\n match ps with\n | [] -> pzero\n | hd::tl ->\n fun stream ->\n let stateTag = stream.StateTag\n let mutable error = NoErrorMessages\n let mutable hd, tl = hd, tl\n let mutable reply = hd stream\n while reply.Status = Error && stateTag = stream.StateTag\n && (match tl with\n | h::t -> hd <- h; tl <- t; true\n | _ -> false)\n do\n error <- mergeErrors error reply.Error\n reply <- hd stream\n if stateTag = stream.StateTag then\n error <- mergeErrors error reply.Error\n reply.Error <- error\n reply\n | _ -> fun stream ->\n use iter = ps.GetEnumerator()\n if iter.MoveNext() then\n let stateTag = stream.StateTag\n let mutable error = NoErrorMessages\n let mutable reply = iter.Current stream\n while reply.Status = Error && stateTag = stream.StateTag && iter.MoveNext() do\n error <- mergeErrors error reply.Error\n reply <- iter.Current stream\n if stateTag = stream.StateTag then\n error <- mergeErrors error reply.Error\n reply.Error <- error\n reply\n else\n Reply()\n\n\nlet choiceL (ps: seq>) label : Parser<_,_> =\n let error = expected label\n match ps with\n | :? (Parser<'a,'u>[]) as ps ->\n if ps.Length = 0 then\n fun stream -> Reply(Error, error)\n else\n fun stream ->\n let stateTag = stream.StateTag\n let mutable reply = ps[0] stream\n let mutable i = 1\n while reply.Status = Error && stateTag = stream.StateTag && i < ps.Length do\n reply <- ps[i] stream\n i <- i + 1\n if stateTag = stream.StateTag then\n reply.Error <- error\n reply\n | :? (Parser<'a,'u> list) as ps ->\n match ps with\n | [] -> fun stream -> Reply(Error, error)\n | hd::tl ->\n fun stream ->\n let stateTag = stream.StateTag\n let mutable hd, tl = hd, tl\n let mutable reply = hd stream\n while reply.Status = Error && stateTag = stream.StateTag\n && (match tl with\n | h::t -> hd <- h; tl <- t; true\n | _ -> false)\n do\n reply <- hd stream\n if stateTag = stream.StateTag then\n reply.Error <- error\n reply\n | _ -> fun stream ->\n use iter = ps.GetEnumerator()\n if iter.MoveNext() then\n let stateTag = stream.StateTag\n let mutable reply = iter.Current stream\n while reply.Status = Error && stateTag = stream.StateTag && iter.MoveNext() do\n reply <- iter.Current stream\n if stateTag = stream.StateTag then\n reply.Error <- error\n reply\n else\n Reply(Error, error)\n\nlet (<|>%) (p: Parser<'a,'u>) x : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let mutable reply = p stream\n if reply.Status = Error && stateTag = stream.StateTag then\n reply.Result <- x\n reply.Status <- Ok\n reply\n\nlet opt (p: Parser<'a,'u>) : Parser<'a option,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let reply = p stream\n if reply.Status = Ok then\n Reply(Ok, Some reply.Result, reply.Error)\n else\n // None is represented as null\n let status = if reply.Status = Error && stateTag = stream.StateTag then Ok else reply.Status\n Reply(status, reply.Error)\n\nlet optional (p: Parser<'a,'u>) : Parser =\n fun stream ->\n let stateTag = stream.StateTag\n let reply = p stream\n let status = if reply.Status = Error && stateTag = stream.StateTag then Ok else reply.Status\n Reply(status, (), reply.Error)\n\nlet attempt (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let mutable reply = p stream\n if reply.Status <> Ok then\n if state.Tag <> stream.StateTag then\n reply.Error <- nestedError stream reply.Error\n reply.Status <- Error // turns FatalErrors into Errors\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n elif reply.Status = FatalError then\n reply.Status <- Error\n reply\n\nlet (>>=?) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) : Parser<'b,'u> =\n let optF = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let mutable reply2 = optF.Invoke(reply1.Result, stream)\n if stateTag1 = stream.StateTag then\n let error = mergeErrors reply1.Error reply2.Error\n if reply2.Status <> Error || stateTag1 = state.Tag then\n reply2.Error <- error\n else\n reply2.Error <- nestedError stream error\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (>>?) (p: Parser<'a,'u>) (q: Parser<'b,'u>) : Parser<'b,'u> =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let mutable reply2 = q stream\n if stateTag1 = stream.StateTag then\n let error = mergeErrors reply1.Error reply2.Error\n if reply2.Status <> Error || stateTag1 = state.Tag then\n reply2.Error <- error\n else\n reply2.Error <- nestedError stream error\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n reply2\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (.>>.?) (p: Parser<'a,'u>) (q: Parser<'b,'u>) : Parser<'a*'b,'u> =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let mutable reply2 = q stream\n if stateTag1 = stream.StateTag then\n let error = mergeErrors reply1.Error reply2.Error\n if reply2.Status <> Error || stateTag1 = state.Tag then\n reply2.Error <- error\n else\n reply2.Error <- nestedError stream error\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n let result = if reply2.Status = Ok then (reply1.Result, reply2.Result)\n else Unchecked.defaultof<_>\n Reply(reply2.Status, result, reply2.Error)\n else\n Reply(reply1.Status, reply1.Error)\n\nlet (.>>?) (p: Parser<'a,'u>) (q: Parser<'b,'u>) : Parser<'a,'u> =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let mutable reply1 = p stream\n if reply1.Status = Ok then\n let stateTag1 = stream.StateTag\n let reply2 = q stream\n if stateTag1 = stream.StateTag then\n let error = mergeErrors reply1.Error reply2.Error\n if reply2.Status <> Error || stateTag1 = state.Tag then\n reply1.Error <- error\n reply1.Status <- reply2.Status\n else\n reply1.Error <- nestedError stream error\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n reply1.Status <- Error\n else\n reply1.Error <- reply2.Error\n reply1.Status <- reply2.Status\n reply1\n\n\n// -------------------------------------\n// Conditional parsing and looking ahead\n// -------------------------------------\n\nlet notEmpty (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let mutable reply = p stream\n if stateTag = stream.StateTag && reply.Status = Ok then\n reply.Status <- Error\n reply\n\n// REVIEW: should `followedBy` use the error messages generated by `p`?\n\nlet internal followedByE (p: Parser<'a,'u>) error : Parser =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let reply = p stream\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n if reply.Status = Ok then Reply(())\n else Reply(Error, error)\n\nlet followedBy p = followedByE p NoErrorMessages\nlet followedByL p label = followedByE p (expected label)\n\nlet internal notFollowedByE (p: Parser<'a,'u>) error : Parser =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let reply = p stream\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n if reply.Status <> Ok then Reply(())\n else Reply(Error, error)\n\nlet notFollowedBy p = notFollowedByE p NoErrorMessages\nlet notFollowedByL p label = notFollowedByE p (unexpected label)\n\nlet lookAhead (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let mutable reply = p stream\n if reply.Status = Ok then\n reply.Error <- NoErrorMessages\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state) // passed by ref as a (slight) optimization\n else\n if state.Tag <> stream.StateTag then\n reply.Error <- nestedError stream reply.Error\n stream.BacktrackTo(&state)\n reply.Status <- Error // turn FatalErrors into normal Errors\n reply\n\n\n// --------------------------\n// Customizing error messages\n// --------------------------\n\nlet () (p: Parser<'a,'u>) label : Parser<'a,'u> =\n let error = expected label\n fun stream ->\n let stateTag = stream.StateTag\n let mutable reply = p stream\n if stateTag = stream.StateTag then\n reply.Error <- error\n reply\n\nlet () (p: Parser<'a,'u>) label : Parser<'a,'u> =\n let expErr = expected label\n fun stream ->\n // state is only declared mutable so it can be passed by ref, it won't be mutated\n let mutable state = CharStreamState(stream) // = stream.State (manually inlined)\n let mutable reply = p stream\n if reply.Status = Ok then\n if state.Tag = stream.StateTag then\n reply.Error <- expErr\n else\n if state.Tag = stream.StateTag then\n (*\n // manually inlined:\n let error = match reply.Error with\n | ErrorMessageList(NestedError(pos, userState, msgs), NoErrorMessages)\n -> ErrorMessageList(CompoundError(label, pos, userState, msgs), NoErrorMessages)\n | _ -> expErr\n *)\n let error = if reply.Error |> isSingleErrorMessageOfType ErrorMessageType.NestedError then\n let ne = reply.Error.Head :?> NestedError\n ErrorMessageList(CompoundError(label, ne.Position, ne.UserState, ne.Messages))\n else expErr\n reply.Error <- error\n else\n reply.Error <- compoundError label stream reply.Error\n stream.BacktrackTo(&state) // we backtrack ...\n reply.Status <- FatalError // ... so we need to make sure normal parsing doesn't continue\n reply\n\nlet fail msg : Parser<'a,'u> =\n let error = messageError msg\n fun stream -> Reply(Error, error)\n\nlet failFatally msg : Parser<'a,'u> =\n let error = messageError msg\n fun stream -> Reply(FatalError, error)\n\n// -----------------\n// Parsing sequences\n// -----------------\n\nlet tuple2 p1 p2 = p1 .>>. p2\nlet tuple3 p1 p2 p3 = pipe3 p1 p2 p3 (fun a b c -> (a, b, c))\nlet tuple4 p1 p2 p3 p4 = pipe4 p1 p2 p3 p4 (fun a b c d -> (a, b, c, d))\nlet tuple5 p1 p2 p3 p4 p5 = pipe5 p1 p2 p3 p4 p5 (fun a b c d e -> (a, b, c, d, e))\n\nlet parray n (p: Parser<'a,'u>) =\n if n = 0 then preturn [||]\n else\n fun stream ->\n let mutable reply = p stream\n let mutable error = reply.Error\n let mutable newReply = Reply()\n if reply.Status = Ok then\n let mutable xs = Array.zeroCreate n\n xs[0] <- reply.Result\n let mutable i = 1\n while i < n do\n let mutable stateTag = stream.StateTag\n reply <- p stream\n error <- if stateTag <> stream.StateTag then reply.Error\n else mergeErrors error reply.Error\n if reply.Status = Ok then\n xs[i] <- reply.Result\n i <- i + 1\n else\n i <- n // break\n newReply.Result <- xs // we set the result even if there was an error\n newReply.Error <- error\n newReply.Status <- reply.Status\n newReply\n\nlet skipArray n (p: Parser<'a,'u>) =\n if n = 0 then preturn ()\n else\n fun stream ->\n let mutable reply = p stream\n let mutable error = reply.Error\n let mutable newReply = Reply()\n if reply.Status = Ok then\n let mutable i = 1\n while i < n do\n let mutable stateTag = stream.StateTag\n reply <- p stream\n error <- if stateTag <> stream.StateTag then reply.Error\n else mergeErrors error reply.Error\n if reply.Status = Ok then\n i <- i + 1\n else\n i <- n // break\n // () is represented as null\n newReply.Error <- error\n newReply.Status <- reply.Status\n newReply\n\n[]\ntype Inline =\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n Many(stateFromFirstElement,\n foldState,\n resultFromState,\n elementParser: Parser<_,_>,\n ?firstElementParser: Parser<_,_>,\n ?resultForEmptySequence) : Parser<_,_> =\n fun stream ->\n let mutable stateTag = stream.StateTag\n let firstElementParser = match firstElementParser with Some p -> p | _ -> elementParser\n let mutable reply = firstElementParser stream\n if reply.Status = Ok then\n let mutable xs = stateFromFirstElement reply.Result\n let mutable error = reply.Error\n stateTag <- stream.StateTag\n reply <- elementParser stream\n while reply.Status = Ok do\n if stateTag = stream.StateTag then\n raiseInfiniteLoopException \"many\" stream\n xs <- foldState xs reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n reply <- elementParser stream\n if reply.Status = Error && stateTag = stream.StateTag then\n error <- mergeErrors error reply.Error\n Reply(Ok, resultFromState xs, error)\n else\n error <- if stateTag <> stream.StateTag then reply.Error\n else mergeErrors error reply.Error\n Reply(reply.Status, error)\n else\n match resultForEmptySequence with\n | Some _ (* if we bind f here, fsc won't be able to inline it *)\n when reply.Status = Error && stateTag = stream.StateTag ->\n Reply(Ok, (match resultForEmptySequence with Some f -> f() | _ -> Unchecked.defaultof<_>), reply.Error)\n | _ ->\n Reply(reply.Status, reply.Error)\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n SepBy(stateFromFirstElement,\n foldState,\n resultFromState,\n elementParser: Parser<_,_>,\n separatorParser: Parser<_,_>,\n ?firstElementParser: Parser<_,'u>,\n ?resultForEmptySequence,\n ?separatorMayEndSequence) : Parser<_,'u> =\n fun stream ->\n let mutable stateTag = stream.StateTag\n let firstElementParser = match firstElementParser with Some p -> p | _ -> elementParser\n let mutable reply = firstElementParser stream\n if reply.Status = Ok then\n let mutable xs = stateFromFirstElement reply.Result\n let mutable error = reply.Error\n stateTag <- stream.StateTag\n let mutable sepReply = separatorParser stream\n let mutable sepStateTag = stream.StateTag\n while sepReply.Status = Ok && (reply <- elementParser stream; reply.Status = Ok) do\n xs <- foldState xs sepReply.Result reply.Result\n if sepStateTag <> stream.StateTag then\n error <- reply.Error\n elif stateTag <> sepStateTag then\n error <- mergeErrors sepReply.Error reply.Error\n else\n raiseInfiniteLoopException \"sep(End)By\" stream\n stateTag <- stream.StateTag\n sepReply <- separatorParser stream\n sepStateTag <- stream.StateTag\n if sepReply.Status = Error && stateTag = sepStateTag then\n Reply(Ok, resultFromState xs, mergeErrors error sepReply.Error)\n else\n match separatorMayEndSequence with\n | Some true when reply.Status = Error && sepStateTag = stream.StateTag ->\n error <- mergeErrors (if stateTag <> sepStateTag then sepReply.Error\n else mergeErrors error sepReply.Error) reply.Error\n Reply(Ok, resultFromState xs, error)\n | _ when reply.Status <> Ok ->\n error <- if sepStateTag <> stream.StateTag then reply.Error\n else\n let error2 = mergeErrors sepReply.Error reply.Error\n if stateTag <> sepStateTag then error2\n else mergeErrors error error2\n Reply(reply.Status, error)\n | _ ->\n let error = if stateTag <> sepStateTag then sepReply.Error\n else mergeErrors error sepReply.Error\n Reply(sepReply.Status, error)\n else\n match resultForEmptySequence with\n | Some _ (* if we bind f here, fsc won't be able to inline it *)\n when reply.Status = Error && stateTag = stream.StateTag ->\n Reply(Ok, (match resultForEmptySequence with Some f -> f() | _ -> Unchecked.defaultof<_>), reply.Error)\n | _ ->\n Reply(reply.Status, reply.Error)\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n ManyTill(stateFromFirstElement,\n foldState,\n resultFromStateAndEnd,\n elementParser: Parser<_,_>,\n endParser: Parser<_,_>,\n ?firstElementParser: Parser<_,_>,\n ?resultForEmptySequence) : Parser<_,_> =\n fun stream ->\n // This is really, really ugly, but it does the job,\n // and it does it about as efficient as it can be done here.\n let firstElementParser = match firstElementParser with Some p -> p | _ -> elementParser\n match resultForEmptySequence with\n | None -> // require at least one element\n let mutable reply = firstElementParser stream\n if reply.Status = Ok then\n // ------------------------------------------------------------------\n // the following code is duplicated in the match branch below\n let mutable xs = stateFromFirstElement reply.Result\n let mutable error = reply.Error\n let mutable stateTag = stream.StateTag\n let mutable endReply = endParser stream\n while endReply.Status = Error && stateTag = stream.StateTag do\n endReply.Status <- enum System.Int32.MinValue\n reply <- elementParser stream\n if reply.Status = Ok then\n if stateTag = stream.StateTag then\n raiseInfiniteLoopException \"manyTill\" stream\n xs <- foldState xs reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n endReply <- endParser stream\n if endReply.Status = Ok then\n error <- if stateTag <> stream.StateTag then endReply.Error\n else mergeErrors error endReply.Error\n Reply(Ok, resultFromStateAndEnd xs endReply.Result, error)\n elif endReply.Status = enum System.Int32.MinValue then\n error <- if stateTag <> stream.StateTag then reply.Error\n else mergeErrors (mergeErrors error endReply.Error) reply.Error\n Reply(reply.Status, error)\n else\n error <- if stateTag <> stream.StateTag then endReply.Error\n else mergeErrors error endReply.Error\n Reply(endReply.Status, error)\n // ------------------------------------------------------------------\n else\n Reply(reply.Status, reply.Error)\n | Some _ ->\n let mutable stateTag = stream.StateTag\n let mutable endReply = endParser stream\n if endReply.Status = Error && stateTag = stream.StateTag then\n let mutable reply = firstElementParser stream\n if reply.Status = Ok then\n // ------------------------------------------------------------------\n // the following code is duplicated in the match branch above\n let mutable xs = stateFromFirstElement reply.Result\n let mutable error = reply.Error\n stateTag <- stream.StateTag\n endReply <- endParser stream\n while endReply.Status = Error && stateTag = stream.StateTag do\n endReply.Status <- enum System.Int32.MinValue\n reply <- elementParser stream\n if reply.Status = Ok then\n if stateTag = stream.StateTag then\n raiseInfiniteLoopException \"manyTill\" stream\n xs <- foldState xs reply.Result\n error <- reply.Error\n stateTag <- stream.StateTag\n endReply <- endParser stream\n if endReply.Status = Ok then\n error <- if stateTag <> stream.StateTag then endReply.Error\n else mergeErrors error endReply.Error\n Reply(Ok, resultFromStateAndEnd xs endReply.Result, error)\n elif endReply.Status = enum System.Int32.MinValue then\n error <- if stateTag <> stream.StateTag then reply.Error\n else mergeErrors (mergeErrors error endReply.Error) reply.Error\n Reply(reply.Status, error)\n else\n error <- if stateTag <> stream.StateTag then endReply.Error\n else mergeErrors error endReply.Error\n Reply(endReply.Status, error)\n // ------------------------------------------------------------------\n else\n let error = if stateTag <> stream.StateTag then reply.Error\n else mergeErrors endReply.Error reply.Error\n Reply(reply.Status, error)\n elif endReply.Status = Ok then\n Reply(Ok, (match resultForEmptySequence with Some f -> f endReply.Result | _ -> Unchecked.defaultof<_>), endReply.Error)\n else\n Reply(endReply.Status, endReply.Error)\n\nlet many p = Inline.Many((fun x -> [x]), (fun xs x -> x::xs), List.rev, p, resultForEmptySequence = fun () -> [])\nlet many1 p = Inline.Many((fun x -> [x]), (fun xs x -> x::xs), List.rev, p)\n\nlet skipMany p = Inline.Many((fun _ -> ()), (fun _ _ -> ()), (fun xs -> xs), p, resultForEmptySequence = fun () -> ())\nlet skipMany1 p = Inline.Many((fun _ -> ()), (fun _ _ -> ()), (fun xs -> xs), p)\n\nlet sepBy p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), List.rev, p, sep, resultForEmptySequence = fun () -> [])\nlet sepBy1 p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), List.rev, p, sep)\n\nlet skipSepBy p sep = Inline.SepBy((fun _ -> ()), (fun _ _ _ -> ()), (fun xs -> xs), p, sep, resultForEmptySequence = fun () -> ())\nlet skipSepBy1 p sep = Inline.SepBy((fun _ -> ()), (fun _ _ _ -> ()), (fun xs -> xs), p, sep)\n\nlet sepEndBy p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), List.rev, p, sep, separatorMayEndSequence = true, resultForEmptySequence = fun () -> [])\nlet sepEndBy1 p sep = Inline.SepBy((fun x -> [x]), (fun xs _ x -> x::xs), List.rev, p, sep, separatorMayEndSequence = true)\n\nlet skipSepEndBy p sep = Inline.SepBy((fun _ -> ()), (fun _ _ _ -> ()), (fun xs -> xs), p, sep, separatorMayEndSequence = true, resultForEmptySequence = fun () -> ())\nlet skipSepEndBy1 p sep = Inline.SepBy((fun _ -> ()), (fun _ _ _ -> ()), (fun xs -> xs), p, sep, separatorMayEndSequence = true)\n\nlet manyTill p endp = Inline.ManyTill((fun x -> [x]), (fun xs x -> x::xs), (fun xs _ -> List.rev xs), p, endp, resultForEmptySequence = fun _ -> [])\nlet many1Till p endp = Inline.ManyTill((fun x -> [x]), (fun xs x -> x::xs), (fun xs _ -> List.rev xs), p, endp)\n\nlet skipManyTill p endp = Inline.ManyTill((fun _ -> ()), (fun _ _ -> ()), (fun _ _ -> ()), p, endp, resultForEmptySequence = fun _ -> ())\nlet skipMany1Till p endp = Inline.ManyTill((fun _ -> ()), (fun _ _ -> ()), (fun _ _ -> ()), p, endp)\n\nlet chainl1 p op =\n Inline.SepBy((fun x0 -> x0), (fun x f y -> f x y), (fun x -> x), p, op)\n\nlet chainl p op x = chainl1 p op <|>% x\n\nlet chainr1 p op =\n Inline.SepBy(elementParser = p, separatorParser = op,\n stateFromFirstElement = (fun x0 -> [(Unchecked.defaultof<_>, x0)]),\n foldState = (fun acc op x -> (op, x)::acc),\n resultFromState = function // is called with (op, y) list in reverse order\n | ((op, y)::tl) ->\n let rec calc op y lst =\n match lst with\n | (op2, x)::tl -> calc op2 (op x y) tl\n | [] -> y // op is null\n calc op y tl\n | [] -> // shouldn't happen\n failwith \"chainr1\")\n\n\nlet chainr p op x = chainr1 p op <|>% x\n\n\n// ------------------------------\n// Computation expression syntax\n// ------------------------------\n[]\ntype ParserCombinator() =\n member t.Delay(f:(unit -> Parser<'a,'u>)) = fun stream -> (f()) stream\n member t.Return(x) = preturn x\n member t.Bind(p, f) = p >>= f\n member t.Zero() : Parser<'a,'u> = pzero\n member t.ReturnFrom(p: Parser<'a,'u>) = p\n // no Combine member by purpose\n member t.TryWith(p:Parser<'a,'u>, cf:(exn -> Parser<'a,'u>)) =\n fun stream ->\n (try p stream with e -> (cf e) stream)\n member t.TryFinally(p:Parser<'a,'u>, ff:(unit -> unit)) =\n fun stream ->\n try p stream finally ff ()\n\nlet parse = ParserCombinator()\n\n\n// ----------------------\n// Other helper functions\n// ----------------------\n\nlet createParserForwardedToRef() =\n let dummyParser = fun stream -> failwith \"a parser created with createParserForwardedToRef was not initialized\"\n let r = ref dummyParser\n (fun stream -> r.Value stream), r : Parser<_,'u> * Parser<_,'u> ref\n" }, { "path": "FParsec/Primitives.fsi", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n[]\nmodule FParsec.Primitives\n\nopen FParsec\nopen FParsec.Error\n\n/// The parser succeeded.\n[] val Ok: ReplyStatus = ReplyStatus.Ok;;\n\n/// The parser failed.\n[] val Error: ReplyStatus = ReplyStatus.Error;;\n\n/// The parser failed and no error recovery (except after backtracking) should be tried.\n[] val FatalError: ReplyStatus = ReplyStatus.FatalError;;\n\n/// The type of the parser functions supported by FParsec combinators.\ntype Parser<'Result, 'UserState> = CharStream<'UserState> -> Reply<'Result>\n\n// =================================\n// Parser primitives and combinators\n// =================================\n\n// Two basic primitives that are only seldomly directly used in user code:\n\n/// The parser `preturn x` always succeeds with the result `x` (without changing the parser state).\n/// `preturn x` is defined as `fun stream -> Reply(x)`.\nval preturn: 'a -> Parser<'a,'u>\n\n/// The parser `pzero` always fails with an empty error message list, i.e. an unspecified error.\n/// `pzero x` is defined as `fun stream -> Reply(Error, NoErrorMessages)`.\nval pzero: Parser<'a,'u>\n\n// ---------------------------\n// Chaining and piping parsers\n// ---------------------------\n\n/// The parser `p >>= f` first applies the parser `p` to the input, then applies the function `f`\n/// to the result returned by `p` and finally applies the parser returned by `f` to the input.\nval (>>=): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n\n/// The parser `p >>% x` applies the parser `p` and returns the result `x`.\nval (>>%): Parser<'a,'u> -> 'b -> Parser<'b,'u>\n\n/// The parser `p1 >>. p2` applies the parsers `p1` and `p2` in sequence and returns the result of `p2`.\nval (>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n\n/// The parser `p1 .>> p2` applies the parsers `p1` and `p2` in sequence and returns the result of `p1`.\nval (.>>): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n\n/// The parser `p1 .>>. p2` applies the parsers `p1` and `p2` in sequence and returns the results in a tuple.\nval (.>>.): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n\n/// The parser `between popen pclose p` applies the parsers `pOpen`, `p` and `pEnd` in sequence.\n/// It returns the result of `p`.\nval between: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'c,'u>\n\n/// The parser `p |>> f` applies the parser `p` and\n/// returns the result `f x`, where `x` is the result returned by `p`.\nval (|>>): Parser<'a,'u> -> ('a -> 'b) -> Parser<'b,'u>\n\n/// The parser `pipe2 p1 p2 f` applies the parsers `p1` and `p2` in sequence.\n/// It returns the result `f a b`, where `a` and `b` are the results returned by `p1` and `p2`.\nval pipe2: Parser<'a,'u> -> Parser<'b,'u> -> ('a -> 'b -> 'c) -> Parser<'c,'u>\n\n/// The parser `pipe3 p1 p2 p3 f` applies the parsers `p1`, `p2` and `p3` in sequence.\n/// It returns the result `f a b c`, where `a`, `b` and `c` are the results returned by `p1`, `p2` and `p3`.\nval pipe3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> ('a -> 'b -> 'c -> 'd) -> Parser<'d,'u>\n\n/// The parser `pipe4 p1 p2 p3 p4 f` applies the parsers `p1`, `p2`, `p3` and `p4` in sequence.\n/// It returns the result `f a b c d`, where `a`, `b`, `c` and `d` are the results returned by `p1`, `p2`, `p3` and `p4`.\nval pipe4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u> -> ('a -> 'b -> 'c -> 'd -> 'e) -> Parser<'e,'u>\n\n/// The parser `pipe5 p1 p2 p3 p4 p5 f` applies the parsers `p1`, `p2`, `p3`, `p4` and `p5` in sequence.\n/// It returns the result of the function application `f a b c d e`, where `a`, `b`, `c`, `d` and `e` are the results returned by `p1`, `p2`, `p3`, `p4` and `p5`.\nval pipe5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u> -> Parser<'e,'u> -> ('a -> 'b -> 'c -> 'd -> 'e -> 'f) -> Parser<'f, 'u>\n\n\n// -----------------------------------------------\n// Parsing alternatives and recovering from errors\n// -----------------------------------------------\n\n/// The parser `p1 <|> p2` first applies the parser `p1`.\n/// If `p1` succeeds, the result of `p1` is returned.\n/// If `p1` fails with a non-fatal error and *without changing the parser state*,\n/// the parser `p2` is applied.\n/// Note: The stream position is part of the parser state, so if `p1` fails after consuming input,\n/// `p2` will not be applied.\nval (<|>): Parser<'a,'u> -> Parser<'a,'u> -> Parser<'a,'u>\n\n/// The parser `choice ps` is an optimized implementation of `p1 <|> p2 <|> ... <|> pn`,\n/// where `p1` ... `pn` are the parsers in the sequence `ps`.\nval choice: seq> -> Parser<'a,'u>\n\n/// The parser `choiceL ps label` is an optimized implementation of `choice ps label`.\nval choiceL: seq> -> string -> Parser<'a,'u>\n\n/// The parser `p <|>% x` is an optimized implementation of `p <|> preturn x`.\nval (<|>%): Parser<'a,'u> -> 'a -> Parser<'a,'u>\n\n/// The parser `opt p` parses an optional occurrence of `p` as an option value.\n/// `opt p` is an optimized implementation of `(p |>> Some) <|>% None`.\nval opt: Parser<'a,'u> -> Parser<'a option,'u>\n\n/// The parser `optional p` skips over an optional occurrence of `p`.\n/// `optional p` is an optimized implementation of `(p >>% ()) <|>% ()`.\nval optional: Parser<'a,'u> -> Parser\n\n\n/// The parser `attempt p` applies the parser `p`.\n/// If `p` fails after changing the parser state or with a fatal error,\n/// `attempt p` will backtrack to the original parser state and report a non-fatal error.\nval attempt: Parser<'a,'u> -> Parser<'a,'u>\n\n/// The parser `p >>=? f` behaves like `p >>= f`, except that it will backtrack to the beginning\n/// if the parser returned by `f` fails with a non-fatal error and without changing the parser state,\n/// even if `p` has changed the parser state.\nval (>>=?): Parser<'a,'u> -> ('a -> Parser<'b,'u>) -> Parser<'b,'u>\n\n/// The parser `p1 >>? p2` behaves like `p1 >>. p2`, except that it will backtrack\n/// to the beginning if `p2` fails with a non-fatal error and without changing the parser state,\n/// even if `p1` has changed the parser state.\nval (>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'b,'u>\n\n/// The parser `p1 .>>? p2` behaves like `p1 .>> p2`, except that it will backtrack\n/// to the beginning if `p2` fails with a non-fatal error and without changing the parser state,\n/// even if `p1` has changed the parser state.\nval (.>>?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a,'u>\n\n/// The parser `p1 .>>.? p2` behaves like `p1 .>>. p2`, except that it will backtrack\n/// to the beginning if `p2` fails with a non-fatal error and without changing the parser state,\n/// even if `p1` has changed the parser state.\nval (.>>.?): Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n\n// -------------------------------------\n// Conditional parsing and looking ahead\n// -------------------------------------\n\n/// The parser `notEmpty p` behaves like `p`,\n/// except that it fails when `p` succeeds without consuming input\n/// or changing the parser state in any other way.\nval notEmpty: Parser<'a,'u> -> Parser<'a,'u>\n\n/// The parser `followedBy p` succeeds if the parser `p` succeeds at the current position.\n/// Otherwise it fails with a non-fatal error. This parser never changes the parser state.\n/// If the parser `followedBy p` fails, it returns no descriptive error message.\n/// Hence it should only be used together with other parsers that take care of a potential error.\n/// Alternatively, `followedByL p label` can be used to ensure a more descriptive error message.\nval followedBy: Parser<'a,'u> -> Parser\n\n/// The parser `followedByL p` behaves like `followedBy p`,\n/// except that it returns an `Expected label` error message when the parser `p` fails.\nval followedByL: Parser<'a,'u> -> string -> Parser\n\n/// The parser `notFollowedBy p` succeeds if the parser `p` fails to parse at the current position.\n/// Otherwise it fails with a non-fatal error. This parser never changes the parser state.\n/// If the parser `notFollowedBy p` fails, it returns no descriptive error message.\n/// Hence it should only be used together with other parsers that take care of a potential error.\n/// Alternatively, `notFollowedByL p label` can be used to ensure a more descriptive error message.\nval notFollowedBy: Parser<'a,'u> -> Parser\n\n/// The parser `notFollowedByL p` behaves like `notFollowedBy p`,\n/// except that it returns an `Unexpected label` error message when the parser `p` fails.\nval notFollowedByL: Parser<'a,'u> -> string -> Parser\n\n/// The parser `lookAhead p` parses `p` and restores the original parse state afterwards.\n/// In case `p` fails after changing the parser state, the error messages are wrapped in a `NestedError`.\n/// If it succeeds, any error messages are discarded. Fatal errors are turned into normal errors.\nval lookAhead: Parser<'a,'u> -> Parser<'a,'u>\n\n\n// --------------------------\n// Customizing error messages\n// --------------------------\n\n/// The parser `p label` applies the parser `p`. If `p` does not change the parser state\n/// (usually because `p` failed), the error messages are replaced with `expected label`.\nval (): Parser<'a,'u> -> string -> Parser<'a,'u>\n\n/// The parser `p label` behaves like `p label`, except that when `p` fails\n/// after changing the parser state (for example, because `p` consumes input before it fails),\n/// a `CompoundError` message is generated with both the given string `label` and the\n/// error messages generated by `p`.\nval (): Parser<'a,'u> -> string -> Parser<'a,'u>\n\n/// The parser `fail msg` always fails with a `messageError msg`.\n/// The error message will be displayed together with other error messages generated for\n/// the same input position.\nval fail: string -> Parser<'a,'u>\n\n/// The parser `failFatally msg` always fails with a `messageError msg`. It signals a\n/// FatalError, so that no error recovery is attempted (except via backtracking constructs).\nval failFatally: string -> Parser<'a,'u>\n\n// -----------------\n// Parsing sequences\n// -----------------\n\n/// The parser `tuple2 p1 p2` applies the parsers `p1` and `p2` in sequence and\n/// returns the results in a tuple.\n/// `tuple2 p1 p2` is defined as `p1 .>>. p2`.\nval tuple2: Parser<'a,'u> -> Parser<'b,'u> -> Parser<('a * 'b),'u>\n\n/// The parser `tuple3 p1 p2 p3` applies the parsers `p1`, `p2` and `p3` in sequence and\n/// returns the results in a tuple.\nval tuple3: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<('a * 'b * 'c),'u>\n\n/// The parser `tuple4 p1 p2 p3 p4` applies the parsers `p1`, `p2`, `p3` and `p4` in sequence and\n/// returns the results in a tuple.\nval tuple4: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u> -> Parser<('a * 'b * 'c * 'd),'u>\n\n/// The parser `tuple5 p1 p2 p3 p4 p5` applies the parsers `p1`, `p2`, `p3`, `p4` and `p5` in sequence and\n/// returns the results in a tuple.\nval tuple5: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'c,'u> -> Parser<'d,'u> -> Parser<'e,'u> -> Parser<('a * 'b * 'c * 'd * 'e),'u>\n\n\n// p{n}\n\n/// The parser `parray n p` parses `n` occurences of `p` and\n/// returns the returns the results in an array.\n/// For example, `parray 3 p` is equivalent to `pipe3 p p p (fun a b c -> [|a;b;c|])`.\nval parray: int -> Parser<'a,'u> -> Parser<'a[],'u>\n\n/// The parser `skipArray n p` is an optimized implementation of `parray n p |>> ignore`.\nval skipArray: int -> Parser<'a,'u> -> Parser\n\n\n// p*\n\n/// The parser `many p` repeatedly applies the parser `p` until `p` fails.\n/// It returns a list of the results returned by `p`.\n/// At the end of the sequence `p` must fail without changing the parser state and without\n/// signalling a `FatalError`, otherwise `many p` will fail with the error reported by `p`.\n/// `many p` tries to guard against an infinite loop by throwing an exception\n/// if `p` succeeds without changing the parser state.\nval many: Parser<'a,'u> -> Parser<'a list,'u>\n\n/// The parser `skipMany p` is an optimized implementation of `many p |>> ignore`.\nval skipMany: Parser<'a,'u> -> Parser\n\n\n// p+\n\n/// The parser `many1 p` behaves like `many p`, except that it requires `p` to succeed at least one time.\n/// `many1 p` is an optimized implementation of `pipe2 p (many p) (fun hd tl -> hd::tl)`.\nval many1: Parser<'a,'u> -> Parser<'a list,'u>\n\n/// The parser `skipMany1 p` is an optimized implementation of `many1 p |>> ignore`.\nval skipMany1: Parser<'a,'u> -> Parser\n\n\n// (p (sep p)*)?\n\n/// The parser `sepBy p sep` parses *zero* or more occurrences of `p` separated by `sep`\n/// (in EBNF notation: `(p (sep p)*)?`).\nval sepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\n/// The parser `skipSepBy p sep` is an optimized implementation of `sepBy p sep |>> ignore`.\nval skipSepBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n\n// p (sep p)*\n\n/// The parser `sepBy1 p sep` parses *one* or more occurrences of `p` separated by `sep`\n/// (in EBNF notation: `p (sep p)*`).\nval sepBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\n/// The parser `skipSepBy1 p sep` is an optimized implementation of `sepBy1 p sep |>> ignore`.\nval skipSepBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n\n// (p (sep p)* sep?)?\n\n/// The parser `sepEndBy p sep` parses *zero* or more occurrences of `p` separated and\n/// optionally ended by `sep` (in EBNF notation: `(p (sep p)* sep?)?`).\n/// It returns a list of the results returned by `p`.\nval sepEndBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\n/// The parser `skipSepEndBy p sep` is an optimized implementation of `sepEndBy p sep |>> ignore`.\nval skipSepEndBy: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n\n// p (sep p)* sep?\n\n/// The parser `sepEndBy1 p sep` parses *one* or more occurrences of `p` separated and\n/// optionally ended by `sep` (in EBNF notation: `p (sep p)* sep?`).\n/// It returns a list of the results returned by `p`.\nval sepEndBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\n/// The parser `skipSepEndBy1 p sep` is an optimized implementation of `sepEndBy1 p sep |>> ignore`.\nval skipSepEndBy1: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n\n/// The `parser manyTill p endp` repeatedly applies the parser `p` \n/// for as long as `endp` fails (without changing the parser state).\n/// It returns a list of the results returned by `p`.\nval manyTill: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\n/// The parser `skipManyTill p endp` is an optimized implementation of `manyTill p endp |>> ignore`.\nval skipManyTill: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n/// The parser `many1Till p endp` behaves like `manyTill p endp`, except that it requires `p` to succeed at least one time.\n/// `many1Till p endp` is an optimized implementation of `pipe2 p (manyTill p endp) (fun hd tl -> hd::tl)`.\nval many1Till: Parser<'a,'u> -> Parser<'b,'u> -> Parser<'a list,'u>\n\nval skipMany1Till: Parser<'a,'u> -> Parser<'b,'u> -> Parser\n\n\n[]\ntype Inline =\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n Many: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'T -> 'State)\n * resultFromState: ('State -> 'Result)\n * elementParser: Parser<'T,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: (unit -> 'Result)\n -> Parser<'Result,'U>\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n SepBy: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'Separator -> 'T -> 'State)\n * resultFromState: ('State -> 'Result)\n * elementParser: Parser<'T,'U>\n * separatorParser: Parser<'Separator,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: (unit -> 'Result)\n * ?separatorMayEndSequence: bool\n -> Parser<'Result,'U>\n\n#if NOINLINE\n static member\n#else\n []\n static member inline\n#endif\n ManyTill: stateFromFirstElement: ('T -> 'State)\n * foldState: ('State -> 'T -> 'State)\n * resultFromStateAndEnd: ('State -> 'E -> 'Result)\n * elementParser: Parser<'T,'U>\n * endParser: Parser<'E,'U>\n * ?firstElementParser: Parser<'T,'U>\n * ?resultForEmptySequence: ('E -> 'Result)\n -> Parser<'Result,'U>\n\n// (((p op p) op p) ... op p)\n\n/// The parser `chainl1 p op` parses one or more occurrences of `p` separated by `op`\n/// (in EBNF notation: `p (op p)*`).\n/// It returns the value obtained by *left* associative application of all functions\n/// returned by `op` to the results returned by `p`,\n/// i.e. `f_n (... (f_2 (f_1 x_1 x_2) x_3) ...) x_n+1`,\n/// where `f_1` to `f_n` are the functions returned by the parser `op` and\n/// `x_1` to `x_n+1` are the values returned by `p`. If only a single occurance\n/// of `p` and no occurance of `op` is parsed, the result of `p` is returned directly.\nval chainl1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n\n/// The parser `chainl p op defVal` is equivalent to `chainl1 p op <|>% defVal`.\nval chainl: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n\n\n// (p op ... (p op (p op p)))\n\n/// The parser `chainr1 p op` parses one or more occurrences of `p` separated by `op`\n/// (in EBNF notation: `p (op p)*`).\n/// It returns the value obtained by *right* associative application of all functions\n/// returned by `op` to the results returned by `p`,\n/// i.e. `f1 x_1 (f_2 x_2 (... (f_n x_n x_n+1) ...))`,\n/// where `f_1` to `f_n` are the functions returned by the parser `op` and\n/// `x_1` to `x_n+1` are the values returned by `p`. If only a single occurance\n/// of `p` and no occurance of `op` is parsed, the result of `p` is returned directly.\nval chainr1: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> Parser<'a,'u>\n\n/// The parser `chainr p op defVal` is equivalent to `chainr1 p op <|>% defVal`.\nval chainr: Parser<'a,'u> -> Parser<('a -> 'a -> 'a),'u> -> 'a -> Parser<'a,'u>\n\n\n// ------------------------------\n// Computation expression syntax\n// ------------------------------\n\n/// The type of the \"builder object\" that can be used to build parsers with\n/// F#'s \"computation expression\" syntax a.k.a. \"workflow\" syntax.\n[]\ntype ParserCombinator =\n new : unit -> ParserCombinator\n member Delay: f:(unit -> Parser<'a,'u>) -> Parser<'a,'u>\n member Return: 'a -> Parser<'a,'u>\n member Bind: Parser<'a,'u>*('a -> Parser<'b,'u>) -> Parser<'b,'u>\n member Zero: unit -> Parser<'a,'u>\n member ReturnFrom: Parser<'a,'u> -> Parser<'a,'u>\n // no Combine member by purpose\n member TryWith: p:Parser<'a,'u> * cf:(exn -> Parser<'a,'u>) -> Parser<'a,'u>\n member TryFinally: p:Parser<'a,'u>* ff:(unit -> unit) -> Parser<'a,'u>\n\n/// The builder object for building parsers using F#'s computation expression syntax.\nval parse : ParserCombinator\n\n\n// ----------------------\n// Other helper functions\n// ----------------------\n\n// a helper function for defining mutually recursive parser values\n\n/// `let p, pRef = createParserForwardedToRef()` creates a parser `p` that forwards all\n/// calls to the parser in the reference cell `pRef`. Initially, `pRef` holds a reference\n/// to a dummy parser that raises an exception on any invocation.\nval createParserForwardedToRef: unit -> Parser<'a,'u> * Parser<'a,'u> ref\n" }, { "path": "FParsec/Range.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nnamespace FParsec\n\n#if LOW_TRUST\n // we don't need the Range code in LOW_TRUST builds\n#else\n\ntype Range = struct\n val Min: int\n val Max: int\n new (min, max) = assert (min <= max)\n {Min = min; Max = max}\nend\n\n[]\nmodule internal Range =\n open System.Collections.Generic\n open FParsec.Internals\n\n let int32Max = System.Int32.MaxValue\n\n let createInvalidRangeException() =\n System.ArgumentException(\"A range passed as an argument is invalid.\")\n\n let checkRangesAreValidSortedAndUnconnected (ranges: Range[]) =\n if ranges.Length <> 0 then\n let r = ranges[0]\n if r.Min > r.Max then raise (createInvalidRangeException())\n let mutable prevMax = r.Max\n for i = 1 to ranges.Length - 1 do\n let r = ranges[i]\n if r.Min > r.Max then raise (createInvalidRangeException())\n if prevMax = int32Max || prevMax + 1 >= r.Min then\n invalidArg \"ranges\" \"The ranges must be sorted and neither overlapping nor immediately adjacent.\"\n prevMax <- r.Max\n\n let checkLabelRangesAreValidSortedAndUnconnected (ranges: Range[]) (labels: System.Reflection.Emit.Label[]) =\n if ranges.Length <> labels.Length then\n invalidArg \"labels\" \"The range and label arrays must have the same lengths.\"\n if ranges.Length <> 0 then\n let r = ranges[0]\n if r.Min > r.Max then raise (createInvalidRangeException())\n let mutable prevMax = r.Max\n for i = 1 to ranges.Length - 1 do\n let r = ranges[i]\n if r.Min > r.Max then raise (createInvalidRangeException())\n if prevMax = int32Max then\n invalidArg \"ranges\" \"The ranges must be sorted and non-overlapping.\"\n if prevMax + 1 >= r.Min then\n if prevMax + 1 = r.Min then\n if labels[i - 1].Equals(labels[i]) then\n raise (System.ArgumentException(\"Ranges with the same associated label must not be immediately adjacent.\"))\n else\n invalidArg \"ranges\" \"The ranges must be sorted and non-overlapping.\"\n prevMax <- r.Max\n\n let rangeComparer = {new Comparer() with\n member t.Compare(r1, r2) = compare r1.Min r2.Min}\n\n let sortAndMergeRanges allowOverlappingRanges (ranges: Range[]) =\n if ranges.Length = 0 then [||]\n else\n System.Array.Sort(ranges, rangeComparer)\n let mutable connected = 0\n let r = ranges[0]\n if r.Min > r.Max then raise (createInvalidRangeException())\n let mutable prevMax = r.Max\n for i = 1 to ranges.Length - 1 do\n let r = ranges[i]\n if r.Min > r.Max then raise (createInvalidRangeException())\n if prevMax < r.Min then\n if prevMax + 1 = r.Min then\n connected <- connected + 1\n prevMax <- r.Max\n elif allowOverlappingRanges then\n connected <- connected + 1\n if prevMax < r.Max then\n prevMax <- r.Max\n else\n invalidArg \"ranges\" \"The value ranges must be non-overlapping.\"\n\n if connected = 0 then ranges\n else\n let rs = Array.zeroCreate (ranges.Length - connected)\n let mutable j = 0\n for r in ranges do\n if j = 0 || prevMax <> int32Max && prevMax + 1 < r.Min then\n prevMax <- r.Max\n rs[j] <- r\n j <- j + 1\n elif prevMax < r.Max then\n prevMax <- r.Max\n rs[j - 1] <- Range(rs[j - 1].Min, r.Max)\n rs\n\n /// If the comparer is not null, adjacent ranges with the same value are merged.\n let sortAndMergeKeyValueRanges (cmp: EqualityComparer<'T>) (keyValueRanges: seq) =\n // 'T could potentially be a large value type,\n // so we are trying to avoid copying 'T values where possible.\n let rvs = Array.ofSeq keyValueRanges\n if rvs.Length = 0 then [||], [||]\n else\n System.Array.Sort(rvs, {new Comparer() with\n member t.Compare((r1, _), (r2, _)) = compare r1.Min r2.Min})\n let mutable connected = 0\n let (r, _) as rv = rvs[0]\n if r.Min > r.Max then raise (createInvalidRangeException())\n let mutable prevMax = r.Max\n let mutable prevRV = rv\n for i = 1 to rvs.Length - 1 do\n let (r, _) as rv = rvs[i]\n if r.Min > r.Max then raise (createInvalidRangeException())\n if prevMax >= r.Min then\n invalidArg \"keyValueRanges\" \"The ranges must be non-overlapping.\"\n if prevMax + 1 = r.Min && isNotNull cmp && cmp.Equals(snd prevRV, snd rv) then\n connected <- connected + 1\n prevMax <- r.Max\n prevRV <- rv\n let n = rvs.Length - connected\n let rs, vs = Array.zeroCreate n, Array.zeroCreate n\n if connected = 0 then\n for i = 0 to rvs.Length - 1 do\n let rv = rvs[i]\n rs[i] <- fst rv\n vs[i] <- snd rv\n else\n let mutable j = 0\n for ((r, _) as rv) in rvs do\n if j = 0 || not (prevMax + 1 = r.Min && cmp.Equals(snd prevRV, snd rv)) then\n rs[j] <- r\n vs[j] <- snd rv\n j <- j + 1\n else\n rs[j - 1] <- Range(rs[j - 1].Min, r.Max)\n prevMax <- r.Max\n prevRV <- rv\n rs, vs\n\n let mergeSortedKeyLabelRanges (keys: int[]) (labels: System.Reflection.Emit.Label[]) =\n if keys.Length <> labels.Length then\n invalidArg \"keys\" \"The key and label arrays must have the same lengths.\"\n if keys.Length = 0 then [||], [||]\n else\n let mutable prevKey = keys[0]\n let mutable connected = 0\n for i = 1 to keys.Length - 1 do\n let key = keys[i]\n if key <= prevKey then\n invalidArg \"keys\" \"The keys must be sorted and distinct.\"\n if key = prevKey + 1 && labels[i] = labels[i - 1] then\n connected <- connected + 1\n prevKey <- key\n if connected = 0 then\n (keys |> Array.map (fun k -> Range(k, k))), labels\n else\n let ranges = Array.zeroCreate (keys.Length - connected)\n let newLabels = Array.zeroCreate (keys.Length - connected)\n let mutable i = 0\n for j = 0 to ranges.Length - 1 do\n let label = labels[i]\n newLabels[j] <- label\n let first = keys[i]\n let mutable last = first\n i <- i + 1\n while i < keys.Length && keys[i] = last + 1\n && labels[i] = label\n do last <- last + 1\n i <- i + 1\n ranges[j] <- Range(first, last)\n ranges, newLabels\n\n /// Duplicate values are allowed.\n let collectSortAndMergeRanges (values: seq) =\n use iter = values.GetEnumerator()\n if not (iter.MoveNext()) then [||]\n else\n let ranges = ResizeArray<_>()\n let rec loop sorted min max =\n if iter.MoveNext() then\n let k = iter.Current\n if max <> int32Max && max + 1 = k then loop sorted min k\n else\n ranges.Add(Range(min, max))\n loop (sorted && max < k) k k\n else\n ranges.Add(Range(min, max))\n sorted\n let value = iter.Current\n let sorted = loop true value value\n let ranges = ranges.ToArray()\n if sorted then ranges\n else sortAndMergeRanges true ranges\n\n /// ranges, values = collectSortAndMergeKeyValueRanges (cmp: EqualityComparer<'T>) (keyValues: seq)\n /// Duplicate keys are not allowed.\n /// If the comparer is not null, consecutive keys with the same value are combined.\n let collectSortAndMergeKeyValueRanges (cmp: EqualityComparer<'T>) (keyValues: seq) =\n // 'T could potentially be a large value type,\n // so we are trying to avoid copying 'T values where possible.\n let kvs = Array.ofSeq keyValues\n System.Array.Sort(kvs, {new Comparer() with\n member t.Compare((k1, _), (k2,_)) = compare k1 k2})\n if kvs.Length = 0 then [||], [||]\n else\n let mutable prevKey, _ = kvs[0]\n for i = 1 to kvs.Length - 1 do\n let k, _ = kvs[i]\n if k = prevKey then\n invalidArg \"keyValues\" \"The sequence contains a duplicate key.\"\n prevKey <- k\n if isNull cmp then\n let ranges = Array.zeroCreate kvs.Length\n let values = Array.zeroCreate kvs.Length\n for i = 0 to kvs.Length - 1 do\n let k, _ as kv = kvs[i]\n ranges[i] <- Range(k, k)\n values[i] <- snd kv\n ranges, values\n else\n let ranges = ResizeArray<_>()\n let mutable kv = kvs[0]\n let mutable i = 0\n while i < kvs.Length do\n let kv0 = kv\n let mutable k = fst kv\n i <- i + 1\n while i < kvs.Length && (kv <- kvs[i]\n k + 1 = fst kv && cmp.Equals(snd kv0, snd kv))\n do k <- k + 1\n i <- i + 1\n ranges.Add(Range(fst kv0, k))\n let ranges = ranges.ToArray()\n let values = Array.zeroCreate ranges.Length\n let mutable j = 0\n for i = 0 to ranges.Length - 1 do\n let r = ranges[i]\n values[i] <- snd kvs[j]\n j <- j + (r.Max - r.Min + 1)\n ranges, values\n\n /// sumOfLengths (ranges: Range[]) (iBegin: int) (iEnd: int)\n /// precondition: iBegin < iEnd, ranges must be sorted and non-overlapping\n let sumOfLengths (ranges: Range[]) iBegin iEnd =\n assert (iBegin < iEnd)\n // since the ranges are sorted non-overlapping, their sum is <= UInt32.MaxValue + 1\n let mutable n = uint32 (iEnd - iBegin)\n for i = iBegin to iEnd - 1 do\n let r = ranges[i]\n n <- n + uint32 (r.Max - r.Min)\n if n <> 0u then double n\n else double System.UInt32.MaxValue + 1. // n has overflown by exactly 1\n\n /// sumOfCappedLengths (lengthCap: int32) (ranges: Range[]) (iBegin: int) (iEnd: int)\n /// precondition: iBegin < iEnd, ranges must be sorted and non-overlapping\n /// a lengthCap <= 0 is interpreted as a lengthCap of 2^32\n let sumOfCappedLengths lengthCap (ranges: Range[]) iBegin iEnd =\n assert (iBegin < iEnd)\n // since the ranges are sorted non-overlapping, their sum is <= UInt32.MaxValue + 1\n let lengthCapM1 = if lengthCap > 0 then uint32 (lengthCap - 1) else System.UInt32.MaxValue\n let mutable n = uint32 (iEnd - iBegin)\n for i = iBegin to iEnd - 1 do\n let r = ranges[i]\n n <- n + min (uint32 (r.Max - r.Min)) lengthCapM1\n if n <> 0u then double n\n else double System.UInt32.MaxValue + 1. // n has overflown by exactly 1\n\n /// density lengthCap (ranges: Range[]) iBegin iEnd\n /// precondition: iBegin < iEnd, ranges must be sorted and non-overlapping\n let density lengthCap (ranges: Range[]) iBegin iEnd =\n assert (iBegin < iEnd)\n let n = sumOfCappedLengths lengthCap ranges iBegin iEnd\n let d = double ranges[iEnd - 1].Max - double ranges[iBegin].Min + 1.\n n/d\n\n /// rangeIndex, pivotAroundRangeMax = findPivot (ranges: Range[]) iBegin iEnd\n /// precondition: iBegin < iEnd, ranges must be sorted and non-overlapping\n let findPivot (ranges: Range[]) iBegin iEnd =\n assert (iBegin < iEnd)\n // the pivot heuristic is based on Korobeynikov (2007), http://llvm.org/pubs/2007-05-31-Switch-Lowering.pdf\n let mutable first, last = double ranges[iBegin].Min, double ranges[iEnd - 1].Max\n let mutable pivot, pivotAroundPreviousRangeMax = iBegin, false\n let mutable sumLeft, sumRight = 0., sumOfLengths ranges iBegin iEnd\n let sumHalf = sumRight*0.5\n let mutable maxQuality, maxDistanceToMiddle = -1., sumRight\n let r = ranges[iBegin]\n let mutable nextMin, nextMax = double r.Min, double r.Max\n for i = iBegin + 1 to iEnd - 1 do\n let prevMax = nextMax\n let prevLength = nextMax - nextMin + 1.\n sumLeft <- sumLeft + prevLength\n sumRight <- sumRight - prevLength\n let r = ranges[i]\n nextMin <- double r.Min\n nextMax <- double r.Max\n let logDistance = System.Math.Log(nextMin - prevMax)\n let leftDensity = sumLeft/(prevMax - first + 2.) // add 2 instead of 1 to decrease the quality of\n let rightDensity = sumRight/(last - nextMin + 2.) // of the two most extreme possible pivot points\n let quality = (leftDensity + rightDensity)*logDistance\n if quality >= maxQuality then\n let distanceToMiddle = System.Math.Abs(sumLeft - sumHalf);\n if quality > maxQuality || distanceToMiddle < maxDistanceToMiddle then\n maxQuality <- quality\n maxDistanceToMiddle <- distanceToMiddle\n pivot <- i\n pivotAroundPreviousRangeMax <- sumLeft >= sumRight\n if pivotAroundPreviousRangeMax then\n (pivot - 1), true\n else\n pivot, false\n\n let rec findInSortedNonOverlappingRanges (ranges: Range[]) value =\n let rec loop iFirst iLast =\n if iFirst <= iLast then\n let middle = int ((uint32 (iFirst + iLast))/2u)\n let middleRange = ranges[middle]\n if value < middleRange.Min then loop iFirst (middle - 1)\n elif value > middleRange.Max then loop (middle + 1) iLast\n else middle\n else ~~~iFirst\n loop 0 (ranges.Length - 1)\n\n#endif\n\n" }, { "path": "FParsec/StaticMapping.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2012\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.StaticMapping\n\n#if !LOW_TRUST\nopen System.Reflection\nopen System.Reflection.Emit\nopen System.Runtime.Serialization\nopen System.Diagnostics\nopen System.Collections.Generic\nopen System.Threading\n\nopen FParsec\nopen FParsec.Internals\nopen FParsec.Range\nopen FParsec.Emit\n\n/// Unsafe because it doesn't constrain the type argument to reference types.\nlet private UnsafeReferenceEqualityComparer<'T> =\n { new EqualityComparer<'T>() with\n override t.Equals(x, y) = obj.ReferenceEquals(x, y)\n override t.GetHashCode(x) = System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(x)\n }\n\ntype PhysicalEqualityComparer<'T> private () =\n static let instanceOrNull =\n let t = typeof<'T>\n if not t.IsValueType then\n UnsafeReferenceEqualityComparer<'T>\n elif t.IsEnum || typeof>.IsAssignableFrom(t) then\n EqualityComparer<'T>.Default\n else\n null\n\n static member InstanceOrNull = instanceOrNull\n\nlet mutable private staticMappingCounter = 0\n\nlet private createStaticMappingTypeBuilder<'TIn,'TOut>() =\n let name = \"StaticMapping\" + (string (Interlocked.Increment(&staticMappingCounter)))\n let tb = createTypeBuilder\n name\n (TypeAttributes.Public ||| TypeAttributes.Sealed ||| TypeAttributes.Class)\n typeof> null\n let mb = tb.DefineMethod(\"Invoke\",\n MethodAttributes.Public ||| MethodAttributes.HideBySig ||| MethodAttributes.Virtual,\n CallingConventions.HasThis,\n typeof<'TOut>, [|typeof<'TIn>|])\n tb, mb.GetILGenerator()\n\nlet createStaticMappingAssertException() =\n System.Exception(\"An internal assert check in FParsec.StaticMapping failed. Please report this error to fparsec@quanttec.com. (The Data member of the exception object contains the information needed to reproduce the error.)\")\n\nlet internal defaultMappingLengthCap = 32\nlet internal defaultMappingDensityThreshold = 0.4\nlet internal defaultIndicatorLengthCap = 32*8\nlet internal defaultIndicatorDensityThreshold = 0.4/32.\n\nlet internal createStaticIntIndicatorFunctionImpl<'TInt when 'TInt : struct>\n lengthCap densityThreshold minValue maxValue invert ranges : ('TInt -> bool) =\n\n if not (typeof<'TInt> = typeof || typeof<'TInt> = typeof) then\n failwith \"Only char and int are supported as input types.\"\n\n let tb, ilg = createStaticMappingTypeBuilder<'TInt, bool>()\n\n let resultLocal = ilg.DeclareLocal(typeof)\n emitSetMembershipTest ilg\n (fun ilg -> ilg.Emit(OpCodes.Ldarg_1)) // loads var\n (fun ilg -> ilg.Emit(OpCodes.Stloc, resultLocal)) // stores result\n (TempLocals(ilg))\n lengthCap densityThreshold\n minValue maxValue\n invert ranges\n ilg.Emit(OpCodes.Ldloc, resultLocal)\n ilg.Emit(OpCodes.Ret)\n\n let t = tb.CreateType()\n let indicator = FormatterServices.GetUninitializedObject(t) :?> ('TInt -> bool)\n\n#if DEBUG_STATIC_MAPPING\n // saveEmitAssembly \"FParsec.Emitted.dll\"\n\n let raiseException key : unit =\n let e = createStaticMappingAssertException()\n e.Data[\"Argument\"] <- key\n e.Data[\"IsInverted\"] <- invert\n e.Data[\"Ranges\"] <- ranges\n raise e\n\n let findKeyinRanges =\n (if typeof<'TInt> = typeof then\n (box (fun (key: char) -> findInSortedNonOverlappingRanges ranges (int key)))\n else\n (box (findInSortedNonOverlappingRanges ranges))\n ) :?> ('TInt -> int)\n\n fun key ->\n let b1 = indicator key\n let b2_ = findKeyinRanges key >= 0\n let b2 = if invert then not b2_ else b2_\n if b1 <> b2 then raiseException key\n b1\n#else\n indicator\n#endif\n\nlet createStaticCharIndicatorFunction invert (charsInSet: seq) =\n let ranges = collectSortAndMergeRanges (charsInSet |> Seq.map (fun c -> int c))\n createStaticIntIndicatorFunctionImpl\n defaultIndicatorLengthCap defaultIndicatorDensityThreshold\n 0 0xffff\n invert ranges\n\nlet createStaticCharRangeIndicatorFunction invert (rangesInSet: seq) =\n let ranges = sortAndMergeRanges true (Array.ofSeq rangesInSet)\n if ranges.Length <> 0 && ranges[0].Min < 0 || ranges[ranges.Length - 1].Max > 0xffff then\n invalidArg \"charRanges\" \"A range contains values outside the range of valid UTF-16 char values (0 - 0xffff).\"\n createStaticIntIndicatorFunctionImpl\n defaultIndicatorLengthCap defaultIndicatorDensityThreshold\n 0 0xffff\n invert ranges\n\nlet createStaticIntIndicatorFunction invert (valuesInSet: seq) =\n let ranges = collectSortAndMergeRanges valuesInSet\n createStaticIntIndicatorFunctionImpl\n defaultIndicatorLengthCap defaultIndicatorDensityThreshold\n System.Int32.MinValue System.Int32.MaxValue\n invert ranges\n\nlet createStaticIntRangeIndicatorFunction invert (rangesInSet: seq) =\n let ranges = sortAndMergeRanges true (Array.ofSeq rangesInSet)\n createStaticIntIndicatorFunctionImpl\n defaultIndicatorLengthCap defaultIndicatorDensityThreshold\n System.Int32.MinValue System.Int32.MaxValue\n invert ranges\n\n\nlet internal createStaticIntMappingImpl\n lengthCap densityThreshold\n minKey maxKey\n (defaultValue: 'T) (ranges: Range[]) (values: 'T[]) : (int -> 'T) =\n assert (ranges.Length = values.Length)\n\n if ranges.Length = 0 then fun _ -> defaultValue\n else\n let physicalEqualityComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n let T = typeof<'T>\n if T = typeof then\n let values = box values :?> bool[]\n let defaultValue = box defaultValue :?> bool\n box (createStaticIntIndicatorFunctionImpl\n (lengthCap*(defaultIndicatorLengthCap/defaultMappingLengthCap))\n (densityThreshold*(defaultIndicatorDensityThreshold/defaultMappingDensityThreshold))\n minKey maxKey\n defaultValue ranges) :?> (int -> 'T)\n else\n let tb, ilg = createStaticMappingTypeBuilder()\n\n let isPrimitive = T.IsPrimitive || T.IsEnum\n let loadConstant = if isPrimitive then createLoaderForPrimitiveConstants ilg\n else Unchecked.defaultof<_>\n let resultOrIndexLocal = ilg.DeclareLocal(if isPrimitive then T else typeof)\n\n let defaultLabel = ilg.DefineLabel()\n let returnLabel = ilg.DefineLabel()\n\n let labels = Array.zeroCreate ranges.Length\n\n let mutable needToEmit = null\n let mutable needToEmitCount = 0\n let physicalEqualityComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n\n if isNull physicalEqualityComparer then\n for i = 0 to labels.Length - 1 do\n labels[i] <- ilg.DefineLabel()\n else\n // we don't need to emit multiple case handlers for identical values\n needToEmit <- Array.zeroCreate values.Length\n let valueLabels = Dictionary<'T,Label>(values.Length, physicalEqualityComparer)\n for i = 0 to values.Length - 1 do\n let value = values[i]\n let mutable label = Unchecked.defaultof<_>\n if not (valueLabels.TryGetValue(value, &label)) then\n needToEmit[i] <- true\n label <- ilg.DefineLabel()\n valueLabels.Add(value, label)\n labels[i] <- label\n needToEmitCount <- valueLabels.Count\n if needToEmitCount = values.Length then\n needToEmit <- null\n\n emitSwitch ilg\n (fun ilg -> ilg.Emit(OpCodes.Ldarg_1)) // loads key\n (TempLocals(ilg))\n lengthCap densityThreshold\n minKey maxKey\n defaultLabel ranges labels\n\n let returnedValues = if isPrimitive || isNull needToEmit then null\n else Array.zeroCreate needToEmitCount\n let mutable returnedValuesCount = 0\n\n for i = 0 to labels.Length - 1 do\n if isNull needToEmit || needToEmit[i] then\n ilg.MarkLabel(labels[i])\n if isPrimitive then\n loadConstant (values[i])\n else\n if isNotNull returnedValues then\n returnedValues[returnedValuesCount] <- values[i]\n loadI4 ilg returnedValuesCount\n returnedValuesCount <- returnedValuesCount + 1\n ilg.Emit(OpCodes.Stloc, resultOrIndexLocal)\n ilg.Emit(OpCodes.Br, returnLabel)\n\n // return default value\n let defaultValueIsNull = not T.IsValueType && isNull (box defaultValue)\n ilg.MarkLabel(defaultLabel)\n if isPrimitive then\n loadConstant defaultValue\n ilg.Emit(OpCodes.Stloc, resultOrIndexLocal)\n else\n if defaultValueIsNull then\n ilg.Emit(OpCodes.Ldnull)\n else\n ilg.Emit(OpCodes.Ldarg_0)\n ilg.Emit(OpCodes.Ldfld, tb.DefineField(\"DefaultValue\", T, FieldAttributes.Public))\n ilg.Emit(OpCodes.Ret)\n\n // return result\n ilg.MarkLabel(returnLabel)\n if isPrimitive then\n ilg.Emit(OpCodes.Ldloc, resultOrIndexLocal)\n else\n // We could store all the values in individual fields to avoid the bounds check\n // and indirect load, but that probably wouldn't be worth the additional\n // code generation (and garbage collection?) costs (except for tiny mappings).\n ilg.Emit(OpCodes.Ldarg_0)\n ilg.Emit(OpCodes.Ldfld, tb.DefineField(\"Values\", values.GetType(), FieldAttributes.Public))\n ilg.Emit(OpCodes.Ldloc, resultOrIndexLocal)\n ilg.Emit(OpCodes.Ldelem, T)\n ilg.Emit(OpCodes.Ret)\n\n let t = tb.CreateType()\n let mapping = FormatterServices.GetUninitializedObject(t) :?> (int -> 'T)\n if not isPrimitive then\n // we can't use the previously used Fieldbuilders here, because SetValue is not implemented in FieldBuilders\n if not defaultValueIsNull then t.GetField(\"DefaultValue\").SetValue(mapping, defaultValue)\n t.GetField(\"Values\").SetValue(mapping, if isNotNull returnedValues then returnedValues else values)\n\n #if DEBUG_STATIC_MAPPING\n //saveEmitAssembly \"FParsec.Emitted.dll\"\n\n if isNull physicalEqualityComparer then mapping\n else\n let raiseException key : unit =\n let e = createStaticMappingAssertException()\n e.Data[\"Argument\"] <- key\n e.Data[\"Ranges\"] <- ranges\n e.Data[\"Values\"] <- values\n e.Data[\"DefaultValue\"] <- defaultValue\n raise e\n\n fun key ->\n let value = mapping key\n let index = findInSortedNonOverlappingRanges ranges key\n if index >= 0 then\n if not (physicalEqualityComparer.Equals(value, values[index])) then raiseException key\n else\n if not (physicalEqualityComparer.Equals(value, defaultValue)) then raiseException key\n value\n #else\n mapping\n #endif\n\n\nlet internal filterOutDefaultValueRanges (comparer: EqualityComparer<_>) (ranges: Range[]) (values: _[]) defaultValue =\n if isNull comparer then ranges, values\n else\n let mutable n = 0\n for v in values do\n if comparer.Equals(v, defaultValue) then n <- n + 1\n if n = 0 then ranges, values\n else\n let N = values.Length - n\n let newRanges, newValues = Array.zeroCreate N, Array.zeroCreate N\n let mutable j = 0\n for i = 0 to values.Length - 1 do\n let v = values[i]\n if not (comparer.Equals(v, defaultValue)) then\n newValues[j] <- v\n newRanges[j] <- ranges[i]\n j <- j + 1\n newRanges, newValues\n\n// we need to use #seq instead of seq here to prevent the F# compiler\n// from unnecessarily wrapping the returned function value\n\nlet createStaticIntMapping (defaultValue: 'T) (keyValues: #seq) =\n let valueComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n let ranges, values = collectSortAndMergeKeyValueRanges valueComparer keyValues\n let ranges, values = filterOutDefaultValueRanges valueComparer ranges values defaultValue\n createStaticIntMappingImpl\n defaultMappingLengthCap defaultMappingDensityThreshold\n System.Int32.MinValue System.Int32.MaxValue\n defaultValue ranges values\n\nlet createStaticIntRangeMapping (defaultValue: 'T) (keyValues: #seq) =\n let valueComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n let ranges, values = sortAndMergeKeyValueRanges valueComparer keyValues\n let ranges, values = filterOutDefaultValueRanges valueComparer ranges values defaultValue\n createStaticIntMappingImpl\n defaultMappingLengthCap defaultMappingDensityThreshold\n System.Int32.MinValue System.Int32.MaxValue\n defaultValue ranges values\n\ntype private IntType = U2\n | U4\n | U8\n\n[]\ntype Subtree(stringIndex: int, index: int, count: int) = struct\n member t.StringIndex = stringIndex\n member t.Index = index\n member t.Count = count // must be greater 0\nend\n\ntype SubtreeEqualityComparer<'T>(stringValues: (string*'T)[], valueComparer: EqualityComparer<'T>) =\n inherit EqualityComparer()\n\n override t.Equals(subtree1: Subtree, subtree2: Subtree) =\n let aligned = subtree1.StringIndex%2 = subtree2.StringIndex%2 // our string comparison code assumes an identical 4-byte-alignment\n let count = subtree1.Count\n count = subtree2.Count\n && (let mutable i = 0\n while uint32 i < uint32 count do\n let string1, value1 = stringValues[subtree1.Index + i]\n let string2, value2 = stringValues[subtree2.Index + i]\n let remaining = string1.Length - subtree1.StringIndex\n if remaining = string2.Length - subtree2.StringIndex\n && (aligned || remaining <= 1)\n && valueComparer.Equals(value1, value2)\n && System.String.CompareOrdinal(string1, subtree1.StringIndex,\n string2, subtree2.StringIndex, remaining) = 0\n then i <- i + 1\n else i <- System.Int32.MinValue // break\n i = count)\n\n override t.GetHashCode(subtree: Subtree) =\n subtree.Count ^^^ valueComparer.GetHashCode(snd stringValues[subtree.Index])\n\nlet createStaticStringMapping (defaultValue: 'T) (keyValues: #seq) : (string -> 'T) =\n let T = typeof<'T>\n\n let physicalEqualityComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n\n let kvs = Array.ofSeq keyValues\n System.Array.Sort(kvs, {new Comparer() with\n member t.Compare((k1, _), (k2, _)) = System.String.CompareOrdinal(k1, k2)})\n\n let mutable previousKey = null\n for (key, _) in kvs do\n if isNull key then invalidArg \"keyValues\" \"The string keys must not be null.\"\n if key = previousKey then invalidArg \"keyValues\" \"The strings keys must be different.\"\n previousKey <- key\n\n match kvs.Length with\n | 0 -> fun str ->\n let throwIfStringIsNull = str.Length\n defaultValue\n | 1 -> let key, value = kvs[0]\n fun str ->\n let throwIfStringIsNull = str.Length\n if str = key then value else defaultValue\n | _ ->\n let mutable i0 = if fst kvs[0] = \"\" then 1 else 0\n\n let getMinMaxLength iBegin iEnd =\n assert (iBegin < iEnd)\n let firstKey, _ = kvs[iBegin]\n let mutable minLength = firstKey.Length\n let mutable maxLength = minLength\n for i = iBegin + 1 to iEnd - 1 do\n let key, _ = kvs[i]\n let length = key.Length\n minLength <- min length minLength\n maxLength <- max length maxLength\n minLength, maxLength\n\n let minLength, maxLength = getMinMaxLength i0 kvs.Length\n\n let findIndexOfFirstCharAfterCommonPrefix startIndex iBegin iEnd minKeyLength =\n let rec loop index =\n if index = minKeyLength then index\n else\n let c = (fst kvs[iBegin])[index]\n let rec keysEqualAtX i =\n if i = iEnd then true\n elif (fst kvs[i])[index] <> c then false\n else keysEqualAtX (i + 1)\n if not (keysEqualAtX (iBegin + 1)) then index\n else loop (index + 1)\n loop startIndex\n\n let prefixLength = findIndexOfFirstCharAfterCommonPrefix 0 i0 kvs.Length minLength\n\n // sort by first char after common prefix, then by length, then lexicographical\n System.Array.Sort(kvs, {new Comparer() with\n member t.Compare((k1, _), (k2, _)) =\n if k1.Length > prefixLength && k2.Length > prefixLength then\n let d = int k1[prefixLength] - int k2[prefixLength]\n if d <> 0 then d\n else\n let d = k1.Length - k2.Length\n if d <> 0 then d\n else System.String.CompareOrdinal(k1, k2)\n else\n k1.Length - k2.Length})\n\n let tb, ilg = createStaticMappingTypeBuilder()\n\n let isPrimitive = T.IsPrimitive || T.IsEnum\n\n let physicalEqualityComparer = PhysicalEqualityComparer<'T>.InstanceOrNull\n let loadConstant = if isPrimitive then createLoaderForPrimitiveConstants ilg\n else Unchecked.defaultof<_>\n\n let lengthLocal = ilg.DeclareLocal(typeof)\n let loadLength() = ilg.Emit(OpCodes.Ldloc, lengthLocal)\n let storeLength() = ilg.Emit(OpCodes.Stloc, lengthLocal)\n\n let charPointerLocal = ilg.DeclareLocal(typeof>)\n let loadPtr() = ilg.Emit(OpCodes.Ldloc, charPointerLocal)\n let storePtr() = ilg.Emit(OpCodes.Stloc, charPointerLocal)\n\n // Declaring the following local as int instead of char improves\n // code generation on the 64-bit JIT.\n let chLocal = ilg.DeclareLocal(typeof)\n let loadCh = fun (_: ILGenerator) -> ilg.Emit(OpCodes.Ldloc, chLocal)\n let storeCh() = ilg.Emit(OpCodes.Stloc, chLocal)\n\n let resultOrIndexLocal = ilg.DeclareLocal(if isPrimitive then T else typeof)\n let loadResult() = ilg.Emit(OpCodes.Ldloc, resultOrIndexLocal)\n let storeResult() = ilg.Emit(OpCodes.Stloc, resultOrIndexLocal)\n\n let stringLocal = ilg.DeclareLocal(typeof, true) // pinned string\n let storeString() = ilg.Emit(OpCodes.Stloc, stringLocal)\n\n // set up local variables\n ilg.Emit(OpCodes.Ldarg_1) // load string argument\n ilg.Emit(OpCodes.Dup)\n ilg.Emit(OpCodes.Dup)\n storeString() // pins string\n // accessing .Length triggers null reference exception if string is null\n ilg.EmitCall(OpCodes.Call, typeof.GetMethod(\"get_Length\"), null)\n storeLength()\n ilg.Emit(OpCodes.Conv_I)\n ilg.EmitCall(OpCodes.Call, typeof.GetMethod(\"get_OffsetToStringData\"), null)\n ilg.Emit(OpCodes.Add)\n storePtr()\n\n let defaultLabel = ilg.DefineLabel()\n let returnLabel = ilg.DefineLabel()\n\n // some helper functions\n\n let dereferenceAndIncrementPtr intType doIncrement =\n loadPtr()\n if doIncrement then\n ilg.Emit(OpCodes.Dup)\n loadI4 ilg (match intType with\n | U2 -> 1*sizeof\n | U4 -> 2*sizeof\n | U8 -> 4*sizeof)\n ilg.Emit(OpCodes.Add)\n storePtr()\n match intType with\n | U2 -> ilg.Emit(OpCodes.Ldind_U2)\n | U4 -> ilg.Emit(OpCodes.Ldind_U4)\n | U8 -> ilg.Emit(OpCodes.Ldind_I8)\n\n let incrementPtrByNumberOfChars i =\n loadPtr()\n loadI4 ilg (i*sizeof)\n ilg.Emit(OpCodes.Add)\n storePtr()\n\n let returnedValueIndices = if isPrimitive then null else ResizeArray<_>(kvs.Length)\n let returnValue i =\n if isPrimitive then\n loadConstant (snd kvs[i])\n else\n loadI4 ilg (returnedValueIndices.Count)\n returnedValueIndices.Add(i)\n storeResult()\n ilg.Emit(OpCodes.Br, returnLabel)\n\n let mutable longKeyData = new ResizeArray<_>(), null, null, null\n\n /// Emit a call to FParsec.Buffer.Equal helper function to compare\n /// a long segment of the input string.\n let emitLongStringComparison dataIndex dataLength isFinal =\n let data, fieldBuilder, methodInfo, pinnedDataLocal = longKeyData\n let mutable f, m, pdl = fieldBuilder, methodInfo, pinnedDataLocal\n if isNull f then\n f <- tb.DefineField(\"longKeyData\", typeof, FieldAttributes.Public)\n m <- typeof.GetMethod(\"Equals\", [|typeof>; typeof>; typeof|])\n pdl <- ilg.DeclareLocal(typeof, true)\n longKeyData <- (data, f, m, pdl)\n\n ilg.Emit(OpCodes.Ldarg_0)\n ilg.Emit(OpCodes.Ldfld, f)\n ilg.Emit(OpCodes.Dup)\n ilg.Emit(OpCodes.Stloc, pdl) // pin data array\n loadI4 ilg dataIndex\n ilg.Emit(OpCodes.Ldelema, typeof)\n ilg.Emit(OpCodes.Conv_I)\n\n loadPtr()\n if not isFinal then\n incrementPtrByNumberOfChars (dataLength*2)\n loadI4 ilg dataLength\n ilg.Emit(OpCodes.Call, m)\n ilg.Emit(OpCodes.Ldnull)\n ilg.Emit(OpCodes.Stloc, pdl) // unpin data array\n ilg.Emit(OpCodes.Brfalse, defaultLabel)\n\n let emitStringComparison (key: string) idx length isFinal =\n if length > 0 then\n let mutable idx, length = idx, length\n if idx%2 = 1 then\n // align ptr to 4-byte boundary\n // (this assumes that the first char in a string is aligned)\n dereferenceAndIncrementPtr U2 (not isFinal || length > 1)\n loadI4 ilg (int key[idx])\n ilg.Emit(OpCodes.Bne_Un, defaultLabel)\n idx <- idx + 1\n length <- length - 1\n\n if length > sizeof*4 then\n // store string data into longStringData\n let data, _, _, _ = longKeyData\n let dataIndex = data.Count\n while length >= 2 do\n // if necessary we will swap the byte order of the whole data array\n // when we assign it to the longKeyData field\n let v = uint32 key[idx] ||| (uint32 key[idx + 1] <<< 16)\n data.Add(v)\n idx <- idx + 2\n length <- length - 2\n if isFinal && length = 1 then\n data.Add(uint32 key[idx])\n length <- 0\n // emit call to string comparison function\n emitLongStringComparison dataIndex (data.Count - dataIndex) isFinal\n else\n #if UNALIGNED_READS\n if sizeof = 8 then\n while length >= 4 || (isFinal && length = 3) do\n dereferenceAndIncrementPtr U8 (not isFinal || length > 4)\n let v = (uint64 key[idx] )\n ||| (uint64 key[idx + 1] <<< 16)\n ||| (uint64 key[idx + 2] <<< 32)\n ||| (if length > 3 then uint64 key[idx + 3] <<< 48 else 0UL)\n let v = if System.BitConverter.IsLittleEndian then v\n else Buffer.SwapByteOrder(v)\n loadU8 ilg v\n ilg.Emit(OpCodes.Bne_Un, defaultLabel)\n idx <- idx + 4\n length <- length - 4\n #endif\n while length >= 2 || (isFinal && length = 1) do\n dereferenceAndIncrementPtr U4 (not isFinal || length > 2)\n let v = if length = 1 then int key[idx]\n else int key[idx] ||| (int key[idx + 1] <<< 16)\n let v = if System.BitConverter.IsLittleEndian then v\n else int (Buffer.SwapByteOrder(uint32 v))\n loadI4 ilg v\n ilg.Emit(OpCodes.Bne_Un, defaultLabel)\n idx <- idx + 2\n length <- length - 2\n if length > 0 then\n Debug.Assert(not isFinal)\n dereferenceAndIncrementPtr U2 true\n loadI4 ilg (int key[idx])\n ilg.Emit(OpCodes.Bne_Un, defaultLabel)\n\n let subtreeLabels = if isNull physicalEqualityComparer then null\n else System.Collections.Generic.Dictionary(SubtreeEqualityComparer<'T>(kvs, physicalEqualityComparer))\n\n // Partitions the key pairs iBegin..(iEnd - 1) into branches with identical \"branch-key\".\n // Returns [|iBegin, i2, ..., iN, iEnd], [|fst kvs[iBegin], fst kvs[i2], ..., fst kvs[iN]|]\n // where iBegin .. indexN are the indices where the branches start.\n let getBranchIndicesAndKeys (iBegin: int) iEnd getBranchKey =\n let mutable n = 0\n let indices, keys = new ResizeArray(iEnd - iBegin), new ResizeArray(iEnd - iBegin)\n indices.Add(iBegin)\n let mutable prevKey : int = getBranchKey (fst kvs[iBegin])\n keys.Add(prevKey)\n for i = iBegin + 1 to iEnd - 1 do\n let key = getBranchKey (fst kvs[i])\n if key <> prevKey then\n prevKey <- key\n indices.Add(i)\n keys.Add(key)\n indices.Add(iEnd) // the indices array has one element more\n indices.ToArray(), keys.ToArray()\n\n // Returns labels for the subtrees given by the branchIndices and the subtreeStringIndex,\n // and an array with bools indicating whether the respective label was newly created.\n // If the dictionary already contains a label for an equivalent subtree, that label is returned;\n // otherwise, a new label is created.\n let getBranchLabels (subtreeLabels: Dictionary) subtreeStringIndex (branchIndices: int[]) =\n assert (branchIndices.Length >= 2 && branchIndices[0] < branchIndices[1])\n let n = branchIndices.Length - 1\n let isNewLabel = Array.zeroCreate n\n let labels = Array.zeroCreate n\n if isNull subtreeLabels then\n for i = 0 to n - 1 do\n isNewLabel[i] <- true\n labels[i] <- ilg.DefineLabel()\n else\n let mutable iBegin = branchIndices[0]\n for j = 1 to branchIndices.Length - 1 do\n let iEnd = branchIndices[j]\n let subtree = Subtree(subtreeStringIndex, iBegin, iEnd - iBegin)\n iBegin <- iEnd\n\n let b = j - 1\n let mutable label = Unchecked.defaultof<_>\n if subtreeLabels.TryGetValue(subtree, &label) then\n labels[b] <- label\n else\n isNewLabel[b] <- true\n let label = ilg.DefineLabel()\n labels[b] <- label\n subtreeLabels.Add(subtree, label)\n labels, isNewLabel\n\n let tempLocals = new TempLocals(ilg)\n\n // Assumes keys in iBegin..(iEnd - 1) are sorted by the branch-key returned by getBranchKey.\n let switch getBranchKey loadVar minVarValue maxVarValue subtreeLabels iBegin iEnd subtreeStringIndex emitBranchIter =\n let branchIndices, branchKeys = getBranchIndicesAndKeys iBegin iEnd getBranchKey\n let branchLabels, isNewLabel = getBranchLabels subtreeLabels subtreeStringIndex branchIndices\n let switchRanges, switchLabels = mergeSortedKeyLabelRanges branchKeys branchLabels\n emitSwitch ilg loadVar tempLocals\n defaultMappingLengthCap defaultMappingDensityThreshold\n minVarValue maxVarValue\n defaultLabel switchRanges switchLabels\n for i = 0 to isNewLabel.Length - 1 do\n if isNewLabel[i] then\n ilg.MarkLabel(branchLabels[i])\n emitBranchIter branchIndices[i] branchIndices[i + 1]\n\n let subtreeEqualityComparer = if isNull physicalEqualityComparer then Unchecked.defaultof<_>\n else SubtreeEqualityComparer<'T>(kvs, physicalEqualityComparer)\n let subtreeLabels = if isNull physicalEqualityComparer then null\n else Dictionary(subtreeEqualityComparer)\n\n let rec emitSubtree length idx iBegin iEnd =\n assert ( iBegin < iEnd\n && kvs[iBegin..(iEnd - 1)]\n |> Array.map (fun (k,_) -> k.Length)\n |> Array.forall ((=) length))\n let idx1 = findIndexOfFirstCharAfterCommonPrefix idx iBegin iEnd length\n if idx <> idx1 then\n emitStringComparison (fst kvs[iBegin]) idx (idx1 - idx) (idx1 = length)\n if idx1 = length then\n assert (iBegin + 1 = iEnd)\n returnValue iBegin\n else\n let mutable emit = true\n if idx <> idx1 && isNotNull subtreeLabels then\n let subtree = Subtree(idx1, iBegin, iEnd - iBegin)\n let mutable label = Unchecked.defaultof<_>\n if subtreeLabels.TryGetValue(subtree, &label) then\n // an equivalent subtree has already been handled elsewhere\n ilg.Emit(OpCodes.Br, label) // jump to that code\n emit <- false\n else\n let label = ilg.DefineLabel()\n ilg.MarkLabel(label)\n subtreeLabels.Add(subtree, label)\n\n if emit then\n dereferenceAndIncrementPtr U2 (idx1 + 1 < length)\n storeCh()\n switch (fun str -> int str[idx1]) loadCh 0 0xffff\n (if idx1 + 1 < length || isNull subtreeLabels then subtreeLabels // we want to keep the switch branches local\n else Dictionary(subtreeEqualityComparer)) // when they only contain a return statement\n iBegin iEnd\n (idx1 + 1) (emitSubtree length (idx1 + 1))\n\n let emitMaxLengthSubtree stringIndex iBegin iEnd =\n loadLength()\n loadI4 ilg maxLength\n ilg.Emit(OpCodes.Bne_Un, defaultLabel)\n emitSubtree maxLength stringIndex iBegin iEnd\n\n Debug.Assert(i0 < kvs.Length)\n\n if i0 <> 0 then // first key is empty\n let label = ilg.DefineLabel()\n loadLength()\n ilg.Emit(OpCodes.Brtrue, label)\n returnValue 0\n ilg.MarkLabel(label)\n\n if minLength = maxLength then\n emitMaxLengthSubtree 0 i0 kvs.Length\n else // at least two non-empty keys with different lengths\n let checkMinLength() =\n loadLength()\n loadI4 ilg minLength\n ilg.Emit(OpCodes.Blt, defaultLabel)\n\n if prefixLength <> 0 then\n checkMinLength()\n emitStringComparison (fst kvs[i0]) 0 prefixLength false\n if prefixLength = minLength then\n let label = ilg.DefineLabel()\n loadLength()\n loadI4 ilg minLength\n ilg.Emit(OpCodes.Bne_Un, label)\n returnValue i0\n ilg.MarkLabel(label)\n i0 <- i0 + 1\n\n else // prefixLength = 0\n if i0 = 0 && (fst kvs[0])[0] = '\\u0000' then\n // If a key contains a zero as the first char, we can't avoid\n // the following length check (which we otherwise don't need for\n // the switch because of the null termination of strings).\n checkMinLength()\n\n if prefixLength + 1 = maxLength then // prefixLength <> 0\n emitMaxLengthSubtree prefixLength i0 kvs.Length\n else\n let topLevelTreeLabels = if isNull subtreeEqualityComparer then null\n else Dictionary(subtreeEqualityComparer)\n // switch over char after prefix\n dereferenceAndIncrementPtr U2 (prefixLength + 1 < maxLength)\n storeCh()\n switch (fun str -> int str[prefixLength]) loadCh 0 0xffff\n topLevelTreeLabels\n i0 kvs.Length\n (prefixLength + 1)\n (fun iBegin iEnd ->\n // switch over length\n switch (fun str -> str.Length) (fun ilg -> loadLength()) 0 System.Int32.MaxValue\n subtreeLabels\n iBegin iEnd\n (prefixLength + 1)\n (fun iBegin iEnd ->\n emitSubtree (fst kvs[iBegin]).Length (prefixLength + 1) iBegin iEnd))\n\n // return default value\n let defaultValueIsNull = not T.IsValueType && isNull (box defaultValue)\n ilg.MarkLabel(defaultLabel)\n if isPrimitive then\n loadConstant defaultValue\n storeResult()\n else\n if defaultValueIsNull then\n ilg.Emit(OpCodes.Ldnull)\n else\n ilg.Emit(OpCodes.Ldarg_0)\n ilg.Emit(OpCodes.Ldfld, tb.DefineField(\"DefaultValue\", T, FieldAttributes.Public))\n ilg.Emit(OpCodes.Ret)\n\n // return result\n ilg.MarkLabel(returnLabel)\n if isPrimitive then\n loadResult()\n else\n // We could store all the values in individual fields to avoid the bounds check\n // and indirect load, but that probably wouldn't be worth the additional\n // code generation (and garbage collection?) costs (except for tiny mappings).\n ilg.Emit(OpCodes.Ldarg_0)\n ilg.Emit(OpCodes.Ldfld, tb.DefineField(\"Values\", typeof<'T[]>, FieldAttributes.Public))\n loadResult()\n ilg.Emit(OpCodes.Ldelem, T)\n ilg.Emit(OpCodes.Ret)\n\n // compile type\n let t = tb.CreateType()\n // instantiate type\n let mapping = FormatterServices.GetUninitializedObject(t) :?> (string -> 'T)\n if not isPrimitive then\n // we can't use the previously used Fieldbuilders here, because SetValue is not implemented in FieldBuilders\n if not defaultValueIsNull then t.GetField(\"DefaultValue\").SetValue(mapping, defaultValue)\n let values = Array.zeroCreate returnedValueIndices.Count\n let mutable j = 0\n for i in returnedValueIndices do\n values[j] <- snd kvs[i]\n j <- j + 1\n t.GetField(\"Values\").SetValue(mapping, values)\n\n let data, _, _, _ = longKeyData\n if data.Count <> 0 then\n let dataArray = data.ToArray()\n if not (System.BitConverter.IsLittleEndian) then\n FParsec.Buffer.SwapByteOrder(dataArray)\n t.GetField(\"longKeyData\").SetValue(mapping, dataArray)\n\n\n #if DEBUG_STATIC_MAPPING\n // saveEmitAssembly \"FParsec.Emitted.dll\"\n\n if isNull physicalEqualityComparer then mapping\n else\n let dict = new System.Collections.Generic.Dictionary(kvs.Length)\n for k, v in kvs do\n dict.Add(k, v)\n let errorHandler (key: string) : unit =\n let e = new System.Exception(\"An internal assert check in FParsec.StaticMapping.createStringMapping failed. Please report this error to fparsec@quanttec.com. (The Data member of the exception object contains the information needed to reproduce the error.)\")\n e.Data[\"Argument\"] <- key\n e.Data[\"KeysValues\"] <- dict\n e.Data[\"DefaultValue\"] <- defaultValue\n raise e\n\n fun key ->\n let mutable value = Unchecked.defaultof<_>\n if not (dict.TryGetValue(key, &value)) then value <- defaultValue\n let value2 = mapping key\n if not (physicalEqualityComparer.Equals(value, value2)) then errorHandler key\n value\n #else\n mapping\n #endif\n\n#endif" }, { "path": "FParsec/StaticMapping.fsi", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.StaticMapping\n\n#if !LOW_TRUST\n\n/// `createStaticCharIndicatorFunction invert charsInSet`\n/// creates an optimized indicator function for the chars specified by the `charsInSet` sequence.\n/// If `invert` is `false` (`true`), the returned indicator function will return `true` (`false`)\n/// if and only if it is called with a char contained in `charsInSet`.\nval createStaticCharIndicatorFunction:\n invert: bool -> charsInSet: seq -> (char -> bool)\n\n/// `createStaticCharRangeIndicatorFunction invert rangesInSet`\n/// creates an optimized indicator function for the chars in the ranges specified by the `rangesInSet` sequence.\n/// If `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\n/// called with a char contained in at least one of the ranges of `rangesInSet`.\nval createStaticCharRangeIndicatorFunction:\n invert: bool -> rangesInSet: seq -> (char -> bool)\n\n/// `createStaticIntIndicatorFunction invert valuesInSet`\n/// creates an optimized indicator function for the integers specified by the `valuesInSet` sequence.\n/// If `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\n/// called with an integer contained in `valuesInSet`.\nval createStaticIntIndicatorFunction:\n invert: bool -> valuesInSet: seq -> (int -> bool)\n\n/// `createStaticIntRangeIndicatorFunction invert rangesInSet`\n/// creates an optimized indicator function for the integers in the ranges specified by the `rangesInSet` sequence.\n/// If `invert` is `false` (`true`), the returned indicator function will return `true` (`false`) if and only if it is\n/// called with an `int` contained in at least one of the ranges of `rangesInSet`.\nval createStaticIntRangeIndicatorFunction:\n invert: bool -> rangesInSet: seq -> (int -> bool)\n\n/// `createStaticIntMapping defaultValue keyValues`\n/// creates an optimized mapping function that maps integer keys to values.\n/// The `keyValues` sequence specifies the key-value pairs for the mapping.\n/// All keys not specified in `keyValues` are mapped to `defaultValue`.\nval createStaticIntMapping:\n defaultValue: 'T -> keyValues: #seq -> (int -> 'T)\n\n/// `createStaticIntRangeMapping defaultValue keyValues`\n/// creates an optimized mapping function that maps integer key ranges to values.\n/// The `keyValues` sequence specifies the range-value pairs for the mapping.\n/// All keys not contained in one of the ranges in `keyValues` are mapped to `defaultValue`.\nval createStaticIntRangeMapping:\n defaultValue: 'T -> keyValues: #seq -> (int -> 'T)\n\n/// `createStaticStringMapping defaultValue keyValues`\n/// creates an optimized mapping function that maps string keys to values.\n/// The `keyValues` sequence specifies the key-value pairs for the mapping.\n/// All keys not specified in `keyValues` are mapped to `defaultValue`. A `null` key is not supported.\nval createStaticStringMapping:\n defaultValue: 'T -> keyValues: #seq -> (string -> 'T)\n\n\nval internal filterOutDefaultValueRanges:\n comparer: System.Collections.Generic.EqualityComparer<'T>\n -> ranges: Range[]\n -> values: 'T[]\n -> defaultValue: 'T\n -> Range[]*'T[]\n\nval internal createStaticIntIndicatorFunctionImpl<'TInt when 'TInt : struct> :\n lengthCap: int -> densityThreshold: double\n -> minValue: int -> maxValue: int\n -> invert: bool -> ranges: Range[]\n -> ('TInt -> bool)\n\nval internal createStaticIntMappingImpl:\n lengthCap: int -> densityThreshold: double\n -> minKey: int -> maxKey: int\n -> defaultValue: 'T -> ranges: Range[] -> values: 'T[]\n -> (int -> 'T)\n\n#endif" }, { "path": "FParsec-LowTrust.sln", "content": "Microsoft Visual Studio Solution File, Format Version 12.00\r\n# Visual Studio Version 16\r\nVisualStudioVersion = 16.0.28621.142\r\nMinimumVisualStudioVersion = 10.0.40219.1\r\nProject(\"{9A19103F-16F7-4668-BE54-9A1E7A4F7556}\") = \"FParsecCS\", \"FParsecCS\\FParsecCS-LowTrust.csproj\", \"{8521556A-F853-4456-8D20-96C42F97E15A}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"FParsec\", \"FParsec\\FParsec-LowTrust.fsproj\", \"{019F9A66-F105-43C7-841D-E4D312659B61}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"Samples\", \"Samples\", \"{01D1CDB5-2645-4929-865F-79B755DBC5B8}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"Calculator\", \"Samples\\Calculator\\Calculator-LowTrust.fsproj\", \"{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"FSharpParsingSample\", \"FSharpParsingSample\", \"{20AE5602-B9B6-434D-A41D-CB988AC49E79}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"InterpFParsec\", \"Samples\\FSharpParsingSample\\FParsecVersion\\InterpFParsec-LowTrust.fsproj\", \"{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"JsonParser\", \"Samples\\JSON\\JsonParser-LowTrust.fsproj\", \"{3889AFB4-60BC-46CB-9747-4BD2F413B351}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"PegParser\", \"Samples\\PEG\\PegParser-LowTrust.fsproj\", \"{2E8F33E4-77F0-4954-9486-239D7124EB86}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"Tutorial\", \"Samples\\Tutorial\\Tutorial-LowTrust.fsproj\", \"{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}\"\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"Test\", \"Test\\Test-LowTrust.fsproj\", \"{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"Build\", \"Build\", \"{1C2DDDBD-BF95-4F55-8651-6EAD5D984BD9}\"\r\n\tProjectSection(SolutionItems) = preProject\r\n\t\t.travis.yml = .travis.yml\r\n\t\tappveyor.yml = appveyor.yml\r\n\t\tDirectory.Build.props = Directory.Build.props\r\n\t\tBuild\\FParsec.Common.targets = Build\\FParsec.Common.targets\r\n\t\tglobal.json = global.json\r\n\t\tpack.ps1 = pack.ps1\r\n\t\treadme.md = readme.md\r\n\tEndProjectSection\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"InterpLexYacc\", \"Samples\\FSharpParsingSample\\LexYaccVersion\\InterpLexYacc.fsproj\", \"{C0616007-EAC1-4648-9124-727B4539EEB4}\"\r\nEndProject\r\nGlobal\r\n\tGlobalSection(SolutionConfigurationPlatforms) = preSolution\r\n\t\tDebug-LowTrust|AnyCPU = Debug-LowTrust|AnyCPU\r\n\t\tDebug-LowTrust|x64 = Debug-LowTrust|x64\r\n\t\tDebug-LowTrust|x86 = Debug-LowTrust|x86\r\n\t\tRelease-LowTrust|AnyCPU = Release-LowTrust|AnyCPU\r\n\t\tRelease-LowTrust|x64 = Release-LowTrust|x64\r\n\t\tRelease-LowTrust|x86 = Release-LowTrust|x86\r\n\tEndGlobalSection\r\n\tGlobalSection(ProjectConfigurationPlatforms) = postSolution\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\tEndGlobalSection\r\n\tGlobalSection(SolutionProperties) = preSolution\r\n\t\tHideSolutionNode = FALSE\r\n\tEndGlobalSection\r\n\tGlobalSection(NestedProjects) = preSolution\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{20AE5602-B9B6-434D-A41D-CB988AC49E79} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80} = {20AE5602-B9B6-434D-A41D-CB988AC49E79}\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4} = {20AE5602-B9B6-434D-A41D-CB988AC49E79}\r\n\tEndGlobalSection\r\n\tGlobalSection(ExtensibilityGlobals) = postSolution\r\n\t\tSolutionGuid = {EA9DAE45-7810-49BB-BDDA-C57E105B79BC}\r\n\tEndGlobalSection\r\nEndGlobal\r\n" }, { "path": "FParsec.sln", "content": "Microsoft Visual Studio Solution File, Format Version 12.00\r\n# Visual Studio Version 16\r\nVisualStudioVersion = 16.0.28621.142\r\nMinimumVisualStudioVersion = 10.0.40219.1\r\nProject(\"{9A19103F-16F7-4668-BE54-9A1E7A4F7556}\") = \"FParsecCS\", \"FParsecCS\\FParsecCS.csproj\", \"{8521556A-F853-4456-8D20-96C42F97E15A}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"FParsec\", \"FParsec\\FParsec.fsproj\", \"{019F9A66-F105-43C7-841D-E4D312659B61}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"Samples\", \"Samples\", \"{01D1CDB5-2645-4929-865F-79B755DBC5B8}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"Calculator\", \"Samples\\Calculator\\Calculator.fsproj\", \"{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"FSharpParsingSample\", \"FSharpParsingSample\", \"{20AE5602-B9B6-434D-A41D-CB988AC49E79}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"InterpFParsec\", \"Samples\\FSharpParsingSample\\FParsecVersion\\InterpFParsec.fsproj\", \"{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"JsonParser\", \"Samples\\JSON\\JsonParser.fsproj\", \"{3889AFB4-60BC-46CB-9747-4BD2F413B351}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"PegParser\", \"Samples\\PEG\\PegParser.fsproj\", \"{2E8F33E4-77F0-4954-9486-239D7124EB86}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"Tutorial\", \"Samples\\Tutorial\\Tutorial.fsproj\", \"{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}\"\r\nEndProject\r\nProject(\"{F2A71F9B-5D33-465A-A702-920D77279786}\") = \"Test\", \"Test\\Test.fsproj\", \"{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}\"\r\nEndProject\r\nProject(\"{2150E333-8FDC-42A3-9474-1A3956D46DE8}\") = \"Build\", \"Build\", \"{1C2DDDBD-BF95-4F55-8651-6EAD5D984BD9}\"\r\n\tProjectSection(SolutionItems) = preProject\r\n\t\t.travis.yml = .travis.yml\r\n\t\tappveyor.yml = appveyor.yml\r\n\t\tDirectory.Build.props = Directory.Build.props\r\n\t\tBuild\\FParsec.Common.targets = Build\\FParsec.Common.targets\r\n\t\tglobal.json = global.json\r\n\t\tpack.ps1 = pack.ps1\r\n\t\treadme.md = readme.md\r\n\tEndProjectSection\r\nEndProject\r\nProject(\"{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}\") = \"InterpLexYacc\", \"Samples\\FSharpParsingSample\\LexYaccVersion\\InterpLexYacc.fsproj\", \"{C0616007-EAC1-4648-9124-727B4539EEB4}\"\r\nEndProject\r\nGlobal\r\n\tGlobalSection(SolutionConfigurationPlatforms) = preSolution\r\n\t\tDebug|AnyCPU = Debug|AnyCPU\r\n\t\tDebug|x64 = Debug|x64\r\n\t\tDebug|x86 = Debug|x86\r\n\t\tDebug-LowTrust|AnyCPU = Debug-LowTrust|AnyCPU\r\n\t\tDebug-LowTrust|x64 = Debug-LowTrust|x64\r\n\t\tDebug-LowTrust|x86 = Debug-LowTrust|x86\r\n\t\tRelease|AnyCPU = Release|AnyCPU\r\n\t\tRelease|x64 = Release|x64\r\n\t\tRelease|x86 = Release|x86\r\n\t\tRelease-LowTrust|AnyCPU = Release-LowTrust|AnyCPU\r\n\t\tRelease-LowTrust|x64 = Release-LowTrust|x64\r\n\t\tRelease-LowTrust|x86 = Release-LowTrust|x86\r\n\tEndGlobalSection\r\n\tGlobalSection(ProjectConfigurationPlatforms) = postSolution\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{8521556A-F853-4456-8D20-96C42F97E15A}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{019F9A66-F105-43C7-841D-E4D312659B61}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{5AE7C1E6-A511-41A9-9A9F-E8A0944319A2}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|AnyCPU.ActiveCfg = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|AnyCPU.Build.0 = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|x64.ActiveCfg = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|x64.Build.0 = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|x86.ActiveCfg = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug|x86.Build.0 = Debug|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|AnyCPU.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|AnyCPU.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x64.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x64.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x86.ActiveCfg = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Debug-LowTrust|x86.Build.0 = Debug-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|AnyCPU.ActiveCfg = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|AnyCPU.Build.0 = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|x64.ActiveCfg = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|x64.Build.0 = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|x86.ActiveCfg = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release|x86.Build.0 = Release|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|AnyCPU.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|AnyCPU.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x64.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x64.Build.0 = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x86.ActiveCfg = Release-LowTrust|AnyCPU\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4}.Release-LowTrust|x86.Build.0 = Release-LowTrust|AnyCPU\r\n\tEndGlobalSection\r\n\tGlobalSection(SolutionProperties) = preSolution\r\n\t\tHideSolutionNode = FALSE\r\n\tEndGlobalSection\r\n\tGlobalSection(NestedProjects) = preSolution\r\n\t\t{A9B15BCE-C37B-4BA6-BC72-6E8A438F205D} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{20AE5602-B9B6-434D-A41D-CB988AC49E79} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{B56A16AD-5BFE-4D99-932C-9073CAFF3D80} = {20AE5602-B9B6-434D-A41D-CB988AC49E79}\r\n\t\t{3889AFB4-60BC-46CB-9747-4BD2F413B351} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{2E8F33E4-77F0-4954-9486-239D7124EB86} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{CEDA985E-30D3-400E-9869-4A22D5F9ADA5} = {01D1CDB5-2645-4929-865F-79B755DBC5B8}\r\n\t\t{C0616007-EAC1-4648-9124-727B4539EEB4} = {20AE5602-B9B6-434D-A41D-CB988AC49E79}\r\n\tEndGlobalSection\r\n\tGlobalSection(ExtensibilityGlobals) = postSolution\r\n\t\tSolutionGuid = {EA9DAE45-7810-49BB-BDDA-C57E105B79BC}\r\n\tEndGlobalSection\r\nEndGlobal\r\n" }, { "path": "FParsecCS/Buffer.cs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Buffers.Binary;\nusing System.Diagnostics;\nusing System.Runtime.CompilerServices;\nusing System.Runtime.InteropServices;\n\nnamespace FParsec {\n\npublic static class Buffer {\n\n#if !LOW_TRUST\n\n/// Calculates: end - begin.
\n/// Precondition: 2^31 > end - begin >= 0.\ninternal static unsafe uint PositiveDistance(char* begin, char* end) {\n return (uint)((byte*)end - (byte*)begin)/2;\n}\n\n/// Calculates: end - begin.
\n/// Precondition: end - begin >= 0.\ninternal static unsafe long PositiveDistance64(char* begin, char* end) {\n return (long)((ulong)((byte*)end - (byte*)begin)/2);\n}\n\n/// Copies size bytes from src to dst. Correctly handles overlapped memory blocks.\nstatic internal unsafe void Copy(byte* dst, byte* src, int size) {\n if (size < 0) throw new ArgumentOutOfRangeException(\"size\", \"The size must be non-negative.\");\n\n System.Buffer.MemoryCopy(src, dst, size, size);\n}\n\n#endif\n\ninternal static uint SwapByteOrder(uint value) => BinaryPrimitives.ReverseEndianness(value);\n\ninternal static ulong SwapByteOrder(ulong value) => BinaryPrimitives.ReverseEndianness(value);\n\ninternal static void SwapByteOrder(Span array) {\n for (int i = 0; i < array.Length; ++i) {\n array[i] = BinaryPrimitives.ReverseEndianness(array[i]);\n }\n}\n\n#if LOW_TRUST\ninternal static uint[] CopyUIntsStoredInLittleEndianByteArray(ReadOnlySpan src, int srcIndex, int srcLength) {\n Debug.Assert(srcLength%sizeof(uint) == 0);\n var subArray = new uint[srcLength/sizeof(uint)];\n src.Slice(srcIndex, srcLength).CopyTo(MemoryMarshal.AsBytes(new Span(subArray)));\n if (!BitConverter.IsLittleEndian) SwapByteOrder(subArray);\n return subArray;\n}\n#endif\n\n#if !LOW_TRUST\n// used by StaticMapping.createStaticStringMapping\npublic static unsafe bool Equals(uint* ptr1, uint* ptr2, int length) =>\n new ReadOnlySpan(ptr1, length).SequenceEqual(new ReadOnlySpan(ptr2, length));\n\ninternal static unsafe uint* LoadLittleEndianUInt32Data(byte* data, int offset, int length)\n{\n if (BitConverter.IsLittleEndian)\n {\n return (uint*)(data + offset);\n }\n\n void* buffer = (void*)RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(Buffer), length);\n \n new ReadOnlySpan(data + offset, length).CopyTo(new Span(buffer, length));\n Buffer.SwapByteOrder(new Span(buffer, length / sizeof(uint)));\n return (uint*)buffer;\n}\n#endif\n\n}\n\n}" }, { "path": "FParsecCS/CaseFoldTable.cs", "content": "// Copyright (c) Stephan Tolksdorf 2009-2012\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Diagnostics;\nusing System.Runtime.CompilerServices;\n\nnamespace FParsec {\n\ninternal static class CaseFoldTable {\n#if LOW_TRUST\n public static readonly char[] FoldedChars = CreateFoldedCharsArray();\n\n private static char[] CreateFoldedCharsArray() {\n Debug.Assert(oneToOneMappings.Length%2 == 0);\n var table = new char[0x10000];\n for (int i = 0; i < table.Length; ++i)\n table[i] = (char)i;\n for (int i = oneToOneMappings.Length - 2; i >= 0; i -= 2)\n table[oneToOneMappings[i]] = oneToOneMappings[i + 1];\n return table;\n }\n#else\n public static readonly unsafe char* FoldedChars = Initialize();\n\n private static unsafe char* Initialize() {\n // initialize FoldedCharsArray\n int n = oneToOneMappings.Length;\n Debug.Assert(n%2 == 0);\n char* chars = (char*)RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(CaseFoldTable), 0x10000 * sizeof(char));\n fixed (char* mappings = oneToOneMappings) {\n var uints = (uint*) chars;\n uint c0 = BitConverter.IsLittleEndian ? 0x10000u : 0x1u;\n for (int i = 0; i < 0x10000/2; i += 4) {\n uints[i ] = c0;\n uints[i + 1] = c0 + 0x20002u;\n uints[i + 2] = c0 + 0x40004u;\n uints[i + 3] = c0 + 0x60006u;\n c0 = unchecked(c0 + 0x80008u);\n }\n for (int i = n - 2; i >= 0; i -= 2)\n chars[mappings[i]] = mappings[i + 1];\n }\n\n return chars;\n }\n#endif\n\n private const string oneToOneMappings = \"\\u0041\\u0061\\u0042\\u0062\\u0043\\u0063\\u0044\\u0064\\u0045\\u0065\\u0046\\u0066\\u0047\\u0067\\u0048\\u0068\\u0049\\u0069\\u004A\\u006A\\u004B\\u006B\\u004C\\u006C\\u004D\\u006D\\u004E\\u006E\\u004F\\u006F\\u0050\\u0070\\u0051\\u0071\\u0052\\u0072\\u0053\\u0073\\u0054\\u0074\\u0055\\u0075\\u0056\\u0076\\u0057\\u0077\\u0058\\u0078\\u0059\\u0079\\u005A\\u007A\\u00B5\\u03BC\\u00C0\\u00E0\\u00C1\\u00E1\\u00C2\\u00E2\\u00C3\\u00E3\\u00C4\\u00E4\\u00C5\\u00E5\\u00C6\\u00E6\\u00C7\\u00E7\\u00C8\\u00E8\\u00C9\\u00E9\\u00CA\\u00EA\\u00CB\\u00EB\\u00CC\\u00EC\\u00CD\\u00ED\\u00CE\\u00EE\\u00CF\\u00EF\\u00D0\\u00F0\\u00D1\\u00F1\\u00D2\\u00F2\\u00D3\\u00F3\\u00D4\\u00F4\\u00D5\\u00F5\\u00D6\\u00F6\\u00D8\\u00F8\\u00D9\\u00F9\\u00DA\\u00FA\\u00DB\\u00FB\\u00DC\\u00FC\\u00DD\\u00FD\\u00DE\\u00FE\\u0100\\u0101\\u0102\\u0103\\u0104\\u0105\\u0106\\u0107\\u0108\\u0109\\u010A\\u010B\\u010C\\u010D\\u010E\\u010F\\u0110\\u0111\\u0112\\u0113\\u0114\\u0115\\u0116\\u0117\\u0118\\u0119\\u011A\\u011B\\u011C\\u011D\\u011E\\u011F\\u0120\\u0121\\u0122\\u0123\\u0124\\u0125\\u0126\\u0127\\u0128\\u0129\\u012A\\u012B\\u012C\\u012D\\u012E\\u012F\\u0132\\u0133\\u0134\\u0135\\u0136\\u0137\\u0139\\u013A\\u013B\\u013C\\u013D\\u013E\\u013F\\u0140\\u0141\\u0142\\u0143\\u0144\\u0145\\u0146\\u0147\\u0148\\u014A\\u014B\\u014C\\u014D\\u014E\\u014F\\u0150\\u0151\\u0152\\u0153\\u0154\\u0155\\u0156\\u0157\\u0158\\u0159\\u015A\\u015B\\u015C\\u015D\\u015E\\u015F\\u0160\\u0161\\u0162\\u0163\\u0164\\u0165\\u0166\\u0167\\u0168\\u0169\\u016A\\u016B\\u016C\\u016D\\u016E\\u016F\\u0170\\u0171\\u0172\\u0173\\u0174\\u0175\\u0176\\u0177\\u0178\\u00FF\\u0179\\u017A\\u017B\\u017C\\u017D\\u017E\\u017F\\u0073\\u0181\\u0253\\u0182\\u0183\\u0184\\u0185\\u0186\\u0254\\u0187\\u0188\\u0189\\u0256\\u018A\\u0257\\u018B\\u018C\\u018E\\u01DD\\u018F\\u0259\\u0190\\u025B\\u0191\\u0192\\u0193\\u0260\\u0194\\u0263\\u0196\\u0269\\u0197\\u0268\\u0198\\u0199\\u019C\\u026F\\u019D\\u0272\\u019F\\u0275\\u01A0\\u01A1\\u01A2\\u01A3\\u01A4\\u01A5\\u01A6\\u0280\\u01A7\\u01A8\\u01A9\\u0283\\u01AC\\u01AD\\u01AE\\u0288\\u01AF\\u01B0\\u01B1\\u028A\\u01B2\\u028B\\u01B3\\u01B4\\u01B5\\u01B6\\u01B7\\u0292\\u01B8\\u01B9\\u01BC\\u01BD\\u01C4\\u01C6\\u01C5\\u01C6\\u01C7\\u01C9\\u01C8\\u01C9\\u01CA\\u01CC\\u01CB\\u01CC\\u01CD\\u01CE\\u01CF\\u01D0\\u01D1\\u01D2\\u01D3\\u01D4\\u01D5\\u01D6\\u01D7\\u01D8\\u01D9\\u01DA\\u01DB\\u01DC\\u01DE\\u01DF\\u01E0\\u01E1\\u01E2\\u01E3\\u01E4\\u01E5\\u01E6\\u01E7\\u01E8\\u01E9\\u01EA\\u01EB\\u01EC\\u01ED\\u01EE\\u01EF\\u01F1\\u01F3\\u01F2\\u01F3\\u01F4\\u01F5\\u01F6\\u0195\\u01F7\\u01BF\\u01F8\\u01F9\\u01FA\\u01FB\\u01FC\\u01FD\\u01FE\\u01FF\\u0200\\u0201\\u0202\\u0203\\u0204\\u0205\\u0206\\u0207\\u0208\\u0209\\u020A\\u020B\\u020C\\u020D\\u020E\\u020F\\u0210\\u0211\\u0212\\u0213\\u0214\\u0215\\u0216\\u0217\\u0218\\u0219\\u021A\\u021B\\u021C\\u021D\\u021E\\u021F\\u0220\\u019E\\u0222\\u0223\\u0224\\u0225\\u0226\\u0227\\u0228\\u0229\\u022A\\u022B\\u022C\\u022D\\u022E\\u022F\\u0230\\u0231\\u0232\\u0233\\u023A\\u2C65\\u023B\\u023C\\u023D\\u019A\\u023E\\u2C66\\u0241\\u0242\\u0243\\u0180\\u0244\\u0289\\u0245\\u028C\\u0246\\u0247\\u0248\\u0249\\u024A\\u024B\\u024C\\u024D\\u024E\\u024F\\u0345\\u03B9\\u0370\\u0371\\u0372\\u0373\\u0376\\u0377\\u037F\\u03F3\\u0386\\u03AC\\u0388\\u03AD\\u0389\\u03AE\\u038A\\u03AF\\u038C\\u03CC\\u038E\\u03CD\\u038F\\u03CE\\u0391\\u03B1\\u0392\\u03B2\\u0393\\u03B3\\u0394\\u03B4\\u0395\\u03B5\\u0396\\u03B6\\u0397\\u03B7\\u0398\\u03B8\\u0399\\u03B9\\u039A\\u03BA\\u039B\\u03BB\\u039C\\u03BC\\u039D\\u03BD\\u039E\\u03BE\\u039F\\u03BF\\u03A0\\u03C0\\u03A1\\u03C1\\u03A3\\u03C3\\u03A4\\u03C4\\u03A5\\u03C5\\u03A6\\u03C6\\u03A7\\u03C7\\u03A8\\u03C8\\u03A9\\u03C9\\u03AA\\u03CA\\u03AB\\u03CB\\u03C2\\u03C3\\u03CF\\u03D7\\u03D0\\u03B2\\u03D1\\u03B8\\u03D5\\u03C6\\u03D6\\u03C0\\u03D8\\u03D9\\u03DA\\u03DB\\u03DC\\u03DD\\u03DE\\u03DF\\u03E0\\u03E1\\u03E2\\u03E3\\u03E4\\u03E5\\u03E6\\u03E7\\u03E8\\u03E9\\u03EA\\u03EB\\u03EC\\u03ED\\u03EE\\u03EF\\u03F0\\u03BA\\u03F1\\u03C1\\u03F4\\u03B8\\u03F5\\u03B5\\u03F7\\u03F8\\u03F9\\u03F2\\u03FA\\u03FB\\u03FD\\u037B\\u03FE\\u037C\\u03FF\\u037D\\u0400\\u0450\\u0401\\u0451\\u0402\\u0452\\u0403\\u0453\\u0404\\u0454\\u0405\\u0455\\u0406\\u0456\\u0407\\u0457\\u0408\\u0458\\u0409\\u0459\\u040A\\u045A\\u040B\\u045B\\u040C\\u045C\\u040D\\u045D\\u040E\\u045E\\u040F\\u045F\\u0410\\u0430\\u0411\\u0431\\u0412\\u0432\\u0413\\u0433\\u0414\\u0434\\u0415\\u0435\\u0416\\u0436\\u0417\\u0437\\u0418\\u0438\\u0419\\u0439\\u041A\\u043A\\u041B\\u043B\\u041C\\u043C\\u041D\\u043D\\u041E\\u043E\\u041F\\u043F\\u0420\\u0440\\u0421\\u0441\\u0422\\u0442\\u0423\\u0443\\u0424\\u0444\\u0425\\u0445\\u0426\\u0446\\u0427\\u0447\\u0428\\u0448\\u0429\\u0449\\u042A\\u044A\\u042B\\u044B\\u042C\\u044C\\u042D\\u044D\\u042E\\u044E\\u042F\\u044F\\u0460\\u0461\\u0462\\u0463\\u0464\\u0465\\u0466\\u0467\\u0468\\u0469\\u046A\\u046B\\u046C\\u046D\\u046E\\u046F\\u0470\\u0471\\u0472\\u0473\\u0474\\u0475\\u0476\\u0477\\u0478\\u0479\\u047A\\u047B\\u047C\\u047D\\u047E\\u047F\\u0480\\u0481\\u048A\\u048B\\u048C\\u048D\\u048E\\u048F\\u0490\\u0491\\u0492\\u0493\\u0494\\u0495\\u0496\\u0497\\u0498\\u0499\\u049A\\u049B\\u049C\\u049D\\u049E\\u049F\\u04A0\\u04A1\\u04A2\\u04A3\\u04A4\\u04A5\\u04A6\\u04A7\\u04A8\\u04A9\\u04AA\\u04AB\\u04AC\\u04AD\\u04AE\\u04AF\\u04B0\\u04B1\\u04B2\\u04B3\\u04B4\\u04B5\\u04B6\\u04B7\\u04B8\\u04B9\\u04BA\\u04BB\\u04BC\\u04BD\\u04BE\\u04BF\\u04C0\\u04CF\\u04C1\\u04C2\\u04C3\\u04C4\\u04C5\\u04C6\\u04C7\\u04C8\\u04C9\\u04CA\\u04CB\\u04CC\\u04CD\\u04CE\\u04D0\\u04D1\\u04D2\\u04D3\\u04D4\\u04D5\\u04D6\\u04D7\\u04D8\\u04D9\\u04DA\\u04DB\\u04DC\\u04DD\\u04DE\\u04DF\\u04E0\\u04E1\\u04E2\\u04E3\\u04E4\\u04E5\\u04E6\\u04E7\\u04E8\\u04E9\\u04EA\\u04EB\\u04EC\\u04ED\\u04EE\\u04EF\\u04F0\\u04F1\\u04F2\\u04F3\\u04F4\\u04F5\\u04F6\\u04F7\\u04F8\\u04F9\\u04FA\\u04FB\\u04FC\\u04FD\\u04FE\\u04FF\\u0500\\u0501\\u0502\\u0503\\u0504\\u0505\\u0506\\u0507\\u0508\\u0509\\u050A\\u050B\\u050C\\u050D\\u050E\\u050F\\u0510\\u0511\\u0512\\u0513\\u0514\\u0515\\u0516\\u0517\\u0518\\u0519\\u051A\\u051B\\u051C\\u051D\\u051E\\u051F\\u0520\\u0521\\u0522\\u0523\\u0524\\u0525\\u0526\\u0527\\u0528\\u0529\\u052A\\u052B\\u052C\\u052D\\u052E\\u052F\\u0531\\u0561\\u0532\\u0562\\u0533\\u0563\\u0534\\u0564\\u0535\\u0565\\u0536\\u0566\\u0537\\u0567\\u0538\\u0568\\u0539\\u0569\\u053A\\u056A\\u053B\\u056B\\u053C\\u056C\\u053D\\u056D\\u053E\\u056E\\u053F\\u056F\\u0540\\u0570\\u0541\\u0571\\u0542\\u0572\\u0543\\u0573\\u0544\\u0574\\u0545\\u0575\\u0546\\u0576\\u0547\\u0577\\u0548\\u0578\\u0549\\u0579\\u054A\\u057A\\u054B\\u057B\\u054C\\u057C\\u054D\\u057D\\u054E\\u057E\\u054F\\u057F\\u0550\\u0580\\u0551\\u0581\\u0552\\u0582\\u0553\\u0583\\u0554\\u0584\\u0555\\u0585\\u0556\\u0586\\u10A0\\u2D00\\u10A1\\u2D01\\u10A2\\u2D02\\u10A3\\u2D03\\u10A4\\u2D04\\u10A5\\u2D05\\u10A6\\u2D06\\u10A7\\u2D07\\u10A8\\u2D08\\u10A9\\u2D09\\u10AA\\u2D0A\\u10AB\\u2D0B\\u10AC\\u2D0C\\u10AD\\u2D0D\\u10AE\\u2D0E\\u10AF\\u2D0F\\u10B0\\u2D10\\u10B1\\u2D11\\u10B2\\u2D12\\u10B3\\u2D13\\u10B4\\u2D14\\u10B5\\u2D15\\u10B6\\u2D16\\u10B7\\u2D17\\u10B8\\u2D18\\u10B9\\u2D19\\u10BA\\u2D1A\\u10BB\\u2D1B\\u10BC\\u2D1C\\u10BD\\u2D1D\\u10BE\\u2D1E\\u10BF\\u2D1F\\u10C0\\u2D20\\u10C1\\u2D21\\u10C2\\u2D22\\u10C3\\u2D23\\u10C4\\u2D24\\u10C5\\u2D25\\u10C7\\u2D27\\u10CD\\u2D2D\\u13F8\\u13F0\\u13F9\\u13F1\\u13FA\\u13F2\\u13FB\\u13F3\\u13FC\\u13F4\\u13FD\\u13F5\\u1E00\\u1E01\\u1E02\\u1E03\\u1E04\\u1E05\\u1E06\\u1E07\\u1E08\\u1E09\\u1E0A\\u1E0B\\u1E0C\\u1E0D\\u1E0E\\u1E0F\\u1E10\\u1E11\\u1E12\\u1E13\\u1E14\\u1E15\\u1E16\\u1E17\\u1E18\\u1E19\\u1E1A\\u1E1B\\u1E1C\\u1E1D\\u1E1E\\u1E1F\\u1E20\\u1E21\\u1E22\\u1E23\\u1E24\\u1E25\\u1E26\\u1E27\\u1E28\\u1E29\\u1E2A\\u1E2B\\u1E2C\\u1E2D\\u1E2E\\u1E2F\\u1E30\\u1E31\\u1E32\\u1E33\\u1E34\\u1E35\\u1E36\\u1E37\\u1E38\\u1E39\\u1E3A\\u1E3B\\u1E3C\\u1E3D\\u1E3E\\u1E3F\\u1E40\\u1E41\\u1E42\\u1E43\\u1E44\\u1E45\\u1E46\\u1E47\\u1E48\\u1E49\\u1E4A\\u1E4B\\u1E4C\\u1E4D\\u1E4E\\u1E4F\\u1E50\\u1E51\\u1E52\\u1E53\\u1E54\\u1E55\\u1E56\\u1E57\\u1E58\\u1E59\\u1E5A\\u1E5B\\u1E5C\\u1E5D\\u1E5E\\u1E5F\\u1E60\\u1E61\\u1E62\\u1E63\\u1E64\\u1E65\\u1E66\\u1E67\\u1E68\\u1E69\\u1E6A\\u1E6B\\u1E6C\\u1E6D\\u1E6E\\u1E6F\\u1E70\\u1E71\\u1E72\\u1E73\\u1E74\\u1E75\\u1E76\\u1E77\\u1E78\\u1E79\\u1E7A\\u1E7B\\u1E7C\\u1E7D\\u1E7E\\u1E7F\\u1E80\\u1E81\\u1E82\\u1E83\\u1E84\\u1E85\\u1E86\\u1E87\\u1E88\\u1E89\\u1E8A\\u1E8B\\u1E8C\\u1E8D\\u1E8E\\u1E8F\\u1E90\\u1E91\\u1E92\\u1E93\\u1E94\\u1E95\\u1E9B\\u1E61\\u1E9E\\u00DF\\u1EA0\\u1EA1\\u1EA2\\u1EA3\\u1EA4\\u1EA5\\u1EA6\\u1EA7\\u1EA8\\u1EA9\\u1EAA\\u1EAB\\u1EAC\\u1EAD\\u1EAE\\u1EAF\\u1EB0\\u1EB1\\u1EB2\\u1EB3\\u1EB4\\u1EB5\\u1EB6\\u1EB7\\u1EB8\\u1EB9\\u1EBA\\u1EBB\\u1EBC\\u1EBD\\u1EBE\\u1EBF\\u1EC0\\u1EC1\\u1EC2\\u1EC3\\u1EC4\\u1EC5\\u1EC6\\u1EC7\\u1EC8\\u1EC9\\u1ECA\\u1ECB\\u1ECC\\u1ECD\\u1ECE\\u1ECF\\u1ED0\\u1ED1\\u1ED2\\u1ED3\\u1ED4\\u1ED5\\u1ED6\\u1ED7\\u1ED8\\u1ED9\\u1EDA\\u1EDB\\u1EDC\\u1EDD\\u1EDE\\u1EDF\\u1EE0\\u1EE1\\u1EE2\\u1EE3\\u1EE4\\u1EE5\\u1EE6\\u1EE7\\u1EE8\\u1EE9\\u1EEA\\u1EEB\\u1EEC\\u1EED\\u1EEE\\u1EEF\\u1EF0\\u1EF1\\u1EF2\\u1EF3\\u1EF4\\u1EF5\\u1EF6\\u1EF7\\u1EF8\\u1EF9\\u1EFA\\u1EFB\\u1EFC\\u1EFD\\u1EFE\\u1EFF\\u1F08\\u1F00\\u1F09\\u1F01\\u1F0A\\u1F02\\u1F0B\\u1F03\\u1F0C\\u1F04\\u1F0D\\u1F05\\u1F0E\\u1F06\\u1F0F\\u1F07\\u1F18\\u1F10\\u1F19\\u1F11\\u1F1A\\u1F12\\u1F1B\\u1F13\\u1F1C\\u1F14\\u1F1D\\u1F15\\u1F28\\u1F20\\u1F29\\u1F21\\u1F2A\\u1F22\\u1F2B\\u1F23\\u1F2C\\u1F24\\u1F2D\\u1F25\\u1F2E\\u1F26\\u1F2F\\u1F27\\u1F38\\u1F30\\u1F39\\u1F31\\u1F3A\\u1F32\\u1F3B\\u1F33\\u1F3C\\u1F34\\u1F3D\\u1F35\\u1F3E\\u1F36\\u1F3F\\u1F37\\u1F48\\u1F40\\u1F49\\u1F41\\u1F4A\\u1F42\\u1F4B\\u1F43\\u1F4C\\u1F44\\u1F4D\\u1F45\\u1F59\\u1F51\\u1F5B\\u1F53\\u1F5D\\u1F55\\u1F5F\\u1F57\\u1F68\\u1F60\\u1F69\\u1F61\\u1F6A\\u1F62\\u1F6B\\u1F63\\u1F6C\\u1F64\\u1F6D\\u1F65\\u1F6E\\u1F66\\u1F6F\\u1F67\\u1F88\\u1F80\\u1F89\\u1F81\\u1F8A\\u1F82\\u1F8B\\u1F83\\u1F8C\\u1F84\\u1F8D\\u1F85\\u1F8E\\u1F86\\u1F8F\\u1F87\\u1F98\\u1F90\\u1F99\\u1F91\\u1F9A\\u1F92\\u1F9B\\u1F93\\u1F9C\\u1F94\\u1F9D\\u1F95\\u1F9E\\u1F96\\u1F9F\\u1F97\\u1FA8\\u1FA0\\u1FA9\\u1FA1\\u1FAA\\u1FA2\\u1FAB\\u1FA3\\u1FAC\\u1FA4\\u1FAD\\u1FA5\\u1FAE\\u1FA6\\u1FAF\\u1FA7\\u1FB8\\u1FB0\\u1FB9\\u1FB1\\u1FBA\\u1F70\\u1FBB\\u1F71\\u1FBC\\u1FB3\\u1FBE\\u03B9\\u1FC8\\u1F72\\u1FC9\\u1F73\\u1FCA\\u1F74\\u1FCB\\u1F75\\u1FCC\\u1FC3\\u1FD8\\u1FD0\\u1FD9\\u1FD1\\u1FDA\\u1F76\\u1FDB\\u1F77\\u1FE8\\u1FE0\\u1FE9\\u1FE1\\u1FEA\\u1F7A\\u1FEB\\u1F7B\\u1FEC\\u1FE5\\u1FF8\\u1F78\\u1FF9\\u1F79\\u1FFA\\u1F7C\\u1FFB\\u1F7D\\u1FFC\\u1FF3\\u2126\\u03C9\\u212A\\u006B\\u212B\\u00E5\\u2132\\u214E\\u2160\\u2170\\u2161\\u2171\\u2162\\u2172\\u2163\\u2173\\u2164\\u2174\\u2165\\u2175\\u2166\\u2176\\u2167\\u2177\\u2168\\u2178\\u2169\\u2179\\u216A\\u217A\\u216B\\u217B\\u216C\\u217C\\u216D\\u217D\\u216E\\u217E\\u216F\\u217F\\u2183\\u2184\\u24B6\\u24D0\\u24B7\\u24D1\\u24B8\\u24D2\\u24B9\\u24D3\\u24BA\\u24D4\\u24BB\\u24D5\\u24BC\\u24D6\\u24BD\\u24D7\\u24BE\\u24D8\\u24BF\\u24D9\\u24C0\\u24DA\\u24C1\\u24DB\\u24C2\\u24DC\\u24C3\\u24DD\\u24C4\\u24DE\\u24C5\\u24DF\\u24C6\\u24E0\\u24C7\\u24E1\\u24C8\\u24E2\\u24C9\\u24E3\\u24CA\\u24E4\\u24CB\\u24E5\\u24CC\\u24E6\\u24CD\\u24E7\\u24CE\\u24E8\\u24CF\\u24E9\\u2C00\\u2C30\\u2C01\\u2C31\\u2C02\\u2C32\\u2C03\\u2C33\\u2C04\\u2C34\\u2C05\\u2C35\\u2C06\\u2C36\\u2C07\\u2C37\\u2C08\\u2C38\\u2C09\\u2C39\\u2C0A\\u2C3A\\u2C0B\\u2C3B\\u2C0C\\u2C3C\\u2C0D\\u2C3D\\u2C0E\\u2C3E\\u2C0F\\u2C3F\\u2C10\\u2C40\\u2C11\\u2C41\\u2C12\\u2C42\\u2C13\\u2C43\\u2C14\\u2C44\\u2C15\\u2C45\\u2C16\\u2C46\\u2C17\\u2C47\\u2C18\\u2C48\\u2C19\\u2C49\\u2C1A\\u2C4A\\u2C1B\\u2C4B\\u2C1C\\u2C4C\\u2C1D\\u2C4D\\u2C1E\\u2C4E\\u2C1F\\u2C4F\\u2C20\\u2C50\\u2C21\\u2C51\\u2C22\\u2C52\\u2C23\\u2C53\\u2C24\\u2C54\\u2C25\\u2C55\\u2C26\\u2C56\\u2C27\\u2C57\\u2C28\\u2C58\\u2C29\\u2C59\\u2C2A\\u2C5A\\u2C2B\\u2C5B\\u2C2C\\u2C5C\\u2C2D\\u2C5D\\u2C2E\\u2C5E\\u2C60\\u2C61\\u2C62\\u026B\\u2C63\\u1D7D\\u2C64\\u027D\\u2C67\\u2C68\\u2C69\\u2C6A\\u2C6B\\u2C6C\\u2C6D\\u0251\\u2C6E\\u0271\\u2C6F\\u0250\\u2C70\\u0252\\u2C72\\u2C73\\u2C75\\u2C76\\u2C7E\\u023F\\u2C7F\\u0240\\u2C80\\u2C81\\u2C82\\u2C83\\u2C84\\u2C85\\u2C86\\u2C87\\u2C88\\u2C89\\u2C8A\\u2C8B\\u2C8C\\u2C8D\\u2C8E\\u2C8F\\u2C90\\u2C91\\u2C92\\u2C93\\u2C94\\u2C95\\u2C96\\u2C97\\u2C98\\u2C99\\u2C9A\\u2C9B\\u2C9C\\u2C9D\\u2C9E\\u2C9F\\u2CA0\\u2CA1\\u2CA2\\u2CA3\\u2CA4\\u2CA5\\u2CA6\\u2CA7\\u2CA8\\u2CA9\\u2CAA\\u2CAB\\u2CAC\\u2CAD\\u2CAE\\u2CAF\\u2CB0\\u2CB1\\u2CB2\\u2CB3\\u2CB4\\u2CB5\\u2CB6\\u2CB7\\u2CB8\\u2CB9\\u2CBA\\u2CBB\\u2CBC\\u2CBD\\u2CBE\\u2CBF\\u2CC0\\u2CC1\\u2CC2\\u2CC3\\u2CC4\\u2CC5\\u2CC6\\u2CC7\\u2CC8\\u2CC9\\u2CCA\\u2CCB\\u2CCC\\u2CCD\\u2CCE\\u2CCF\\u2CD0\\u2CD1\\u2CD2\\u2CD3\\u2CD4\\u2CD5\\u2CD6\\u2CD7\\u2CD8\\u2CD9\\u2CDA\\u2CDB\\u2CDC\\u2CDD\\u2CDE\\u2CDF\\u2CE0\\u2CE1\\u2CE2\\u2CE3\\u2CEB\\u2CEC\\u2CED\\u2CEE\\u2CF2\\u2CF3\\uA640\\uA641\\uA642\\uA643\\uA644\\uA645\\uA646\\uA647\\uA648\\uA649\\uA64A\\uA64B\\uA64C\\uA64D\\uA64E\\uA64F\\uA650\\uA651\\uA652\\uA653\\uA654\\uA655\\uA656\\uA657\\uA658\\uA659\\uA65A\\uA65B\\uA65C\\uA65D\\uA65E\\uA65F\\uA660\\uA661\\uA662\\uA663\\uA664\\uA665\\uA666\\uA667\\uA668\\uA669\\uA66A\\uA66B\\uA66C\\uA66D\\uA680\\uA681\\uA682\\uA683\\uA684\\uA685\\uA686\\uA687\\uA688\\uA689\\uA68A\\uA68B\\uA68C\\uA68D\\uA68E\\uA68F\\uA690\\uA691\\uA692\\uA693\\uA694\\uA695\\uA696\\uA697\\uA698\\uA699\\uA69A\\uA69B\\uA722\\uA723\\uA724\\uA725\\uA726\\uA727\\uA728\\uA729\\uA72A\\uA72B\\uA72C\\uA72D\\uA72E\\uA72F\\uA732\\uA733\\uA734\\uA735\\uA736\\uA737\\uA738\\uA739\\uA73A\\uA73B\\uA73C\\uA73D\\uA73E\\uA73F\\uA740\\uA741\\uA742\\uA743\\uA744\\uA745\\uA746\\uA747\\uA748\\uA749\\uA74A\\uA74B\\uA74C\\uA74D\\uA74E\\uA74F\\uA750\\uA751\\uA752\\uA753\\uA754\\uA755\\uA756\\uA757\\uA758\\uA759\\uA75A\\uA75B\\uA75C\\uA75D\\uA75E\\uA75F\\uA760\\uA761\\uA762\\uA763\\uA764\\uA765\\uA766\\uA767\\uA768\\uA769\\uA76A\\uA76B\\uA76C\\uA76D\\uA76E\\uA76F\\uA779\\uA77A\\uA77B\\uA77C\\uA77D\\u1D79\\uA77E\\uA77F\\uA780\\uA781\\uA782\\uA783\\uA784\\uA785\\uA786\\uA787\\uA78B\\uA78C\\uA78D\\u0265\\uA790\\uA791\\uA792\\uA793\\uA796\\uA797\\uA798\\uA799\\uA79A\\uA79B\\uA79C\\uA79D\\uA79E\\uA79F\\uA7A0\\uA7A1\\uA7A2\\uA7A3\\uA7A4\\uA7A5\\uA7A6\\uA7A7\\uA7A8\\uA7A9\\uA7AA\\u0266\\uA7AB\\u025C\\uA7AC\\u0261\\uA7AD\\u026C\\uA7B0\\u029E\\uA7B1\\u0287\\uA7B2\\u029D\\uA7B3\\uAB53\\uA7B4\\uA7B5\\uA7B6\\uA7B7\\uAB70\\u13A0\\uAB71\\u13A1\\uAB72\\u13A2\\uAB73\\u13A3\\uAB74\\u13A4\\uAB75\\u13A5\\uAB76\\u13A6\\uAB77\\u13A7\\uAB78\\u13A8\\uAB79\\u13A9\\uAB7A\\u13AA\\uAB7B\\u13AB\\uAB7C\\u13AC\\uAB7D\\u13AD\\uAB7E\\u13AE\\uAB7F\\u13AF\\uAB80\\u13B0\\uAB81\\u13B1\\uAB82\\u13B2\\uAB83\\u13B3\\uAB84\\u13B4\\uAB85\\u13B5\\uAB86\\u13B6\\uAB87\\u13B7\\uAB88\\u13B8\\uAB89\\u13B9\\uAB8A\\u13BA\\uAB8B\\u13BB\\uAB8C\\u13BC\\uAB8D\\u13BD\\uAB8E\\u13BE\\uAB8F\\u13BF\\uAB90\\u13C0\\uAB91\\u13C1\\uAB92\\u13C2\\uAB93\\u13C3\\uAB94\\u13C4\\uAB95\\u13C5\\uAB96\\u13C6\\uAB97\\u13C7\\uAB98\\u13C8\\uAB99\\u13C9\\uAB9A\\u13CA\\uAB9B\\u13CB\\uAB9C\\u13CC\\uAB9D\\u13CD\\uAB9E\\u13CE\\uAB9F\\u13CF\\uABA0\\u13D0\\uABA1\\u13D1\\uABA2\\u13D2\\uABA3\\u13D3\\uABA4\\u13D4\\uABA5\\u13D5\\uABA6\\u13D6\\uABA7\\u13D7\\uABA8\\u13D8\\uABA9\\u13D9\\uABAA\\u13DA\\uABAB\\u13DB\\uABAC\\u13DC\\uABAD\\u13DD\\uABAE\\u13DE\\uABAF\\u13DF\\uABB0\\u13E0\\uABB1\\u13E1\\uABB2\\u13E2\\uABB3\\u13E3\\uABB4\\u13E4\\uABB5\\u13E5\\uABB6\\u13E6\\uABB7\\u13E7\\uABB8\\u13E8\\uABB9\\u13E9\\uABBA\\u13EA\\uABBB\\u13EB\\uABBC\\u13EC\\uABBD\\u13ED\\uABBE\\u13EE\\uABBF\\u13EF\\uFF21\\uFF41\\uFF22\\uFF42\\uFF23\\uFF43\\uFF24\\uFF44\\uFF25\\uFF45\\uFF26\\uFF46\\uFF27\\uFF47\\uFF28\\uFF48\\uFF29\\uFF49\\uFF2A\\uFF4A\\uFF2B\\uFF4B\\uFF2C\\uFF4C\\uFF2D\\uFF4D\\uFF2E\\uFF4E\\uFF2F\\uFF4F\\uFF30\\uFF50\\uFF31\\uFF51\\uFF32\\uFF52\\uFF33\\uFF53\\uFF34\\uFF54\\uFF35\\uFF55\\uFF36\\uFF56\\uFF37\\uFF57\\uFF38\\uFF58\\uFF39\\uFF59\\uFF3A\\uFF5A\";\n} // class CaseFoldTable\n\n}\n\n/*\n// The oneToOneMappings string has been generated with the following F# program, which\n// extracts the (non-Turkic) 1-to-1 case folding mappings for chars below 0x10000 from\n// http://www.unicode.org/Public/8.8.0/ucd/CaseFolding.txt\n\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\n(*\n# CaseFolding-8.0.0.txt\n# Date: 2015-01-13, 18:16:36 GMT [MD]\n#\n# Unicode Character Database\n# Copyright (c) 1991-2015 Unicode, Inc.\n# For terms of use, see http://www.unicode.org/terms_of_use.html\n# For documentation, see http://www.unicode.org/reports/tr44/\n#\n# Case Folding Properties\n#\n# This file is a supplement to the UnicodeData file.\n# It provides a case folding mapping generated from the Unicode Character Database.\n# If all characters are mapped according to the full mapping below, then\n# case differences (according to UnicodeData.txt and SpecialCasing.txt)\n# are eliminated.\n#\n# The data supports both implementations that require simple case foldings\n# (where string lengths don't change), and implementations that allow full case folding\n# (where string lengths may grow). Note that where they can be supported, the\n# full case foldings are superior: for example, they allow \"MASSE\" and \"Maße\" to match.\n#\n# All code points not listed in this file map to themselves.\n#\n# NOTE: case folding does not preserve normalization formats!\n#\n# For information on case folding, including how to have case folding\n# preserve normalization formats, see Section 3.13 Default Case Algorithms in\n# The Unicode Standard.\n#\n# ================================================================================\n# Format\n# ================================================================================\n# The entries in this file are in the following machine-readable format:\n#\n# ; ; ; # \n#\n# The status field is:\n# C: common case folding, common mappings shared by both simple and full mappings.\n# F: full case folding, mappings that cause strings to grow in length. Multiple characters are separated by spaces.\n# S: simple case folding, mappings to single characters where different from F.\n# T: special case for uppercase I and dotted uppercase I\n# - For non-Turkic languages, this mapping is normally not used.\n# - For Turkic languages (tr, az), this mapping can be used instead of the normal mapping for these characters.\n# Note that the Turkic mappings do not maintain canonical equivalence without additional processing.\n# See the discussions of case mapping in the Unicode Standard for more information.\n#\n# Usage:\n# A. To do a simple case folding, use the mappings with status C + S.\n# B. To do a full case folding, use the mappings with status C + F.\n#\n# The mappings with status T can be used or omitted depending on the desired case-folding\n# behavior. (The default option is to exclude them.)\n#\n# =================================================================\n\n# Property: Case_Folding\n\n# All code points not explicitly listed for Case_Folding\n# have the value C for the status field, and the code point itself for the mapping field.\n\n# =================================================================\n*)\n// continue txt file as string\nlet datastr = @\"0041; C; 0061; # LATIN CAPITAL LETTER A\n0042; C; 0062; # LATIN CAPITAL LETTER B\n0043; C; 0063; # LATIN CAPITAL LETTER C\n0044; C; 0064; # LATIN CAPITAL LETTER D\n0045; C; 0065; # LATIN CAPITAL LETTER E\n0046; C; 0066; # LATIN CAPITAL LETTER F\n0047; C; 0067; # LATIN CAPITAL LETTER G\n0048; C; 0068; # LATIN CAPITAL LETTER H\n0049; C; 0069; # LATIN CAPITAL LETTER I\n0049; T; 0131; # LATIN CAPITAL LETTER I\n004A; C; 006A; # LATIN CAPITAL LETTER J\n004B; C; 006B; # LATIN CAPITAL LETTER K\n004C; C; 006C; # LATIN CAPITAL LETTER L\n004D; C; 006D; # LATIN CAPITAL LETTER M\n004E; C; 006E; # LATIN CAPITAL LETTER N\n004F; C; 006F; # LATIN CAPITAL LETTER O\n0050; C; 0070; # LATIN CAPITAL LETTER P\n0051; C; 0071; # LATIN CAPITAL LETTER Q\n0052; C; 0072; # LATIN CAPITAL LETTER R\n0053; C; 0073; # LATIN CAPITAL LETTER S\n0054; C; 0074; # LATIN CAPITAL LETTER T\n0055; C; 0075; # LATIN CAPITAL LETTER U\n0056; C; 0076; # LATIN CAPITAL LETTER V\n0057; C; 0077; # LATIN CAPITAL LETTER W\n0058; C; 0078; # LATIN CAPITAL LETTER X\n0059; C; 0079; # LATIN CAPITAL LETTER Y\n005A; C; 007A; # LATIN CAPITAL LETTER Z\n00B5; C; 03BC; # MICRO SIGN\n00C0; C; 00E0; # LATIN CAPITAL LETTER A WITH GRAVE\n00C1; C; 00E1; # LATIN CAPITAL LETTER A WITH ACUTE\n00C2; C; 00E2; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX\n00C3; C; 00E3; # LATIN CAPITAL LETTER A WITH TILDE\n00C4; C; 00E4; # LATIN CAPITAL LETTER A WITH DIAERESIS\n00C5; C; 00E5; # LATIN CAPITAL LETTER A WITH RING ABOVE\n00C6; C; 00E6; # LATIN CAPITAL LETTER AE\n00C7; C; 00E7; # LATIN CAPITAL LETTER C WITH CEDILLA\n00C8; C; 00E8; # LATIN CAPITAL LETTER E WITH GRAVE\n00C9; C; 00E9; # LATIN CAPITAL LETTER E WITH ACUTE\n00CA; C; 00EA; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX\n00CB; C; 00EB; # LATIN CAPITAL LETTER E WITH DIAERESIS\n00CC; C; 00EC; # LATIN CAPITAL LETTER I WITH GRAVE\n00CD; C; 00ED; # LATIN CAPITAL LETTER I WITH ACUTE\n00CE; C; 00EE; # LATIN CAPITAL LETTER I WITH CIRCUMFLEX\n00CF; C; 00EF; # LATIN CAPITAL LETTER I WITH DIAERESIS\n00D0; C; 00F0; # LATIN CAPITAL LETTER ETH\n00D1; C; 00F1; # LATIN CAPITAL LETTER N WITH TILDE\n00D2; C; 00F2; # LATIN CAPITAL LETTER O WITH GRAVE\n00D3; C; 00F3; # LATIN CAPITAL LETTER O WITH ACUTE\n00D4; C; 00F4; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX\n00D5; C; 00F5; # LATIN CAPITAL LETTER O WITH TILDE\n00D6; C; 00F6; # LATIN CAPITAL LETTER O WITH DIAERESIS\n00D8; C; 00F8; # LATIN CAPITAL LETTER O WITH STROKE\n00D9; C; 00F9; # LATIN CAPITAL LETTER U WITH GRAVE\n00DA; C; 00FA; # LATIN CAPITAL LETTER U WITH ACUTE\n00DB; C; 00FB; # LATIN CAPITAL LETTER U WITH CIRCUMFLEX\n00DC; C; 00FC; # LATIN CAPITAL LETTER U WITH DIAERESIS\n00DD; C; 00FD; # LATIN CAPITAL LETTER Y WITH ACUTE\n00DE; C; 00FE; # LATIN CAPITAL LETTER THORN\n00DF; F; 0073 0073; # LATIN SMALL LETTER SHARP S\n0100; C; 0101; # LATIN CAPITAL LETTER A WITH MACRON\n0102; C; 0103; # LATIN CAPITAL LETTER A WITH BREVE\n0104; C; 0105; # LATIN CAPITAL LETTER A WITH OGONEK\n0106; C; 0107; # LATIN CAPITAL LETTER C WITH ACUTE\n0108; C; 0109; # LATIN CAPITAL LETTER C WITH CIRCUMFLEX\n010A; C; 010B; # LATIN CAPITAL LETTER C WITH DOT ABOVE\n010C; C; 010D; # LATIN CAPITAL LETTER C WITH CARON\n010E; C; 010F; # LATIN CAPITAL LETTER D WITH CARON\n0110; C; 0111; # LATIN CAPITAL LETTER D WITH STROKE\n0112; C; 0113; # LATIN CAPITAL LETTER E WITH MACRON\n0114; C; 0115; # LATIN CAPITAL LETTER E WITH BREVE\n0116; C; 0117; # LATIN CAPITAL LETTER E WITH DOT ABOVE\n0118; C; 0119; # LATIN CAPITAL LETTER E WITH OGONEK\n011A; C; 011B; # LATIN CAPITAL LETTER E WITH CARON\n011C; C; 011D; # LATIN CAPITAL LETTER G WITH CIRCUMFLEX\n011E; C; 011F; # LATIN CAPITAL LETTER G WITH BREVE\n0120; C; 0121; # LATIN CAPITAL LETTER G WITH DOT ABOVE\n0122; C; 0123; # LATIN CAPITAL LETTER G WITH CEDILLA\n0124; C; 0125; # LATIN CAPITAL LETTER H WITH CIRCUMFLEX\n0126; C; 0127; # LATIN CAPITAL LETTER H WITH STROKE\n0128; C; 0129; # LATIN CAPITAL LETTER I WITH TILDE\n012A; C; 012B; # LATIN CAPITAL LETTER I WITH MACRON\n012C; C; 012D; # LATIN CAPITAL LETTER I WITH BREVE\n012E; C; 012F; # LATIN CAPITAL LETTER I WITH OGONEK\n0130; F; 0069 0307; # LATIN CAPITAL LETTER I WITH DOT ABOVE\n0130; T; 0069; # LATIN CAPITAL LETTER I WITH DOT ABOVE\n0132; C; 0133; # LATIN CAPITAL LIGATURE IJ\n0134; C; 0135; # LATIN CAPITAL LETTER J WITH CIRCUMFLEX\n0136; C; 0137; # LATIN CAPITAL LETTER K WITH CEDILLA\n0139; C; 013A; # LATIN CAPITAL LETTER L WITH ACUTE\n013B; C; 013C; # LATIN CAPITAL LETTER L WITH CEDILLA\n013D; C; 013E; # LATIN CAPITAL LETTER L WITH CARON\n013F; C; 0140; # LATIN CAPITAL LETTER L WITH MIDDLE DOT\n0141; C; 0142; # LATIN CAPITAL LETTER L WITH STROKE\n0143; C; 0144; # LATIN CAPITAL LETTER N WITH ACUTE\n0145; C; 0146; # LATIN CAPITAL LETTER N WITH CEDILLA\n0147; C; 0148; # LATIN CAPITAL LETTER N WITH CARON\n0149; F; 02BC 006E; # LATIN SMALL LETTER N PRECEDED BY APOSTROPHE\n014A; C; 014B; # LATIN CAPITAL LETTER ENG\n014C; C; 014D; # LATIN CAPITAL LETTER O WITH MACRON\n014E; C; 014F; # LATIN CAPITAL LETTER O WITH BREVE\n0150; C; 0151; # LATIN CAPITAL LETTER O WITH DOUBLE ACUTE\n0152; C; 0153; # LATIN CAPITAL LIGATURE OE\n0154; C; 0155; # LATIN CAPITAL LETTER R WITH ACUTE\n0156; C; 0157; # LATIN CAPITAL LETTER R WITH CEDILLA\n0158; C; 0159; # LATIN CAPITAL LETTER R WITH CARON\n015A; C; 015B; # LATIN CAPITAL LETTER S WITH ACUTE\n015C; C; 015D; # LATIN CAPITAL LETTER S WITH CIRCUMFLEX\n015E; C; 015F; # LATIN CAPITAL LETTER S WITH CEDILLA\n0160; C; 0161; # LATIN CAPITAL LETTER S WITH CARON\n0162; C; 0163; # LATIN CAPITAL LETTER T WITH CEDILLA\n0164; C; 0165; # LATIN CAPITAL LETTER T WITH CARON\n0166; C; 0167; # LATIN CAPITAL LETTER T WITH STROKE\n0168; C; 0169; # LATIN CAPITAL LETTER U WITH TILDE\n016A; C; 016B; # LATIN CAPITAL LETTER U WITH MACRON\n016C; C; 016D; # LATIN CAPITAL LETTER U WITH BREVE\n016E; C; 016F; # LATIN CAPITAL LETTER U WITH RING ABOVE\n0170; C; 0171; # LATIN CAPITAL LETTER U WITH DOUBLE ACUTE\n0172; C; 0173; # LATIN CAPITAL LETTER U WITH OGONEK\n0174; C; 0175; # LATIN CAPITAL LETTER W WITH CIRCUMFLEX\n0176; C; 0177; # LATIN CAPITAL LETTER Y WITH CIRCUMFLEX\n0178; C; 00FF; # LATIN CAPITAL LETTER Y WITH DIAERESIS\n0179; C; 017A; # LATIN CAPITAL LETTER Z WITH ACUTE\n017B; C; 017C; # LATIN CAPITAL LETTER Z WITH DOT ABOVE\n017D; C; 017E; # LATIN CAPITAL LETTER Z WITH CARON\n017F; C; 0073; # LATIN SMALL LETTER LONG S\n0181; C; 0253; # LATIN CAPITAL LETTER B WITH HOOK\n0182; C; 0183; # LATIN CAPITAL LETTER B WITH TOPBAR\n0184; C; 0185; # LATIN CAPITAL LETTER TONE SIX\n0186; C; 0254; # LATIN CAPITAL LETTER OPEN O\n0187; C; 0188; # LATIN CAPITAL LETTER C WITH HOOK\n0189; C; 0256; # LATIN CAPITAL LETTER AFRICAN D\n018A; C; 0257; # LATIN CAPITAL LETTER D WITH HOOK\n018B; C; 018C; # LATIN CAPITAL LETTER D WITH TOPBAR\n018E; C; 01DD; # LATIN CAPITAL LETTER REVERSED E\n018F; C; 0259; # LATIN CAPITAL LETTER SCHWA\n0190; C; 025B; # LATIN CAPITAL LETTER OPEN E\n0191; C; 0192; # LATIN CAPITAL LETTER F WITH HOOK\n0193; C; 0260; # LATIN CAPITAL LETTER G WITH HOOK\n0194; C; 0263; # LATIN CAPITAL LETTER GAMMA\n0196; C; 0269; # LATIN CAPITAL LETTER IOTA\n0197; C; 0268; # LATIN CAPITAL LETTER I WITH STROKE\n0198; C; 0199; # LATIN CAPITAL LETTER K WITH HOOK\n019C; C; 026F; # LATIN CAPITAL LETTER TURNED M\n019D; C; 0272; # LATIN CAPITAL LETTER N WITH LEFT HOOK\n019F; C; 0275; # LATIN CAPITAL LETTER O WITH MIDDLE TILDE\n01A0; C; 01A1; # LATIN CAPITAL LETTER O WITH HORN\n01A2; C; 01A3; # LATIN CAPITAL LETTER OI\n01A4; C; 01A5; # LATIN CAPITAL LETTER P WITH HOOK\n01A6; C; 0280; # LATIN LETTER YR\n01A7; C; 01A8; # LATIN CAPITAL LETTER TONE TWO\n01A9; C; 0283; # LATIN CAPITAL LETTER ESH\n01AC; C; 01AD; # LATIN CAPITAL LETTER T WITH HOOK\n01AE; C; 0288; # LATIN CAPITAL LETTER T WITH RETROFLEX HOOK\n01AF; C; 01B0; # LATIN CAPITAL LETTER U WITH HORN\n01B1; C; 028A; # LATIN CAPITAL LETTER UPSILON\n01B2; C; 028B; # LATIN CAPITAL LETTER V WITH HOOK\n01B3; C; 01B4; # LATIN CAPITAL LETTER Y WITH HOOK\n01B5; C; 01B6; # LATIN CAPITAL LETTER Z WITH STROKE\n01B7; C; 0292; # LATIN CAPITAL LETTER EZH\n01B8; C; 01B9; # LATIN CAPITAL LETTER EZH REVERSED\n01BC; C; 01BD; # LATIN CAPITAL LETTER TONE FIVE\n01C4; C; 01C6; # LATIN CAPITAL LETTER DZ WITH CARON\n01C5; C; 01C6; # LATIN CAPITAL LETTER D WITH SMALL LETTER Z WITH CARON\n01C7; C; 01C9; # LATIN CAPITAL LETTER LJ\n01C8; C; 01C9; # LATIN CAPITAL LETTER L WITH SMALL LETTER J\n01CA; C; 01CC; # LATIN CAPITAL LETTER NJ\n01CB; C; 01CC; # LATIN CAPITAL LETTER N WITH SMALL LETTER J\n01CD; C; 01CE; # LATIN CAPITAL LETTER A WITH CARON\n01CF; C; 01D0; # LATIN CAPITAL LETTER I WITH CARON\n01D1; C; 01D2; # LATIN CAPITAL LETTER O WITH CARON\n01D3; C; 01D4; # LATIN CAPITAL LETTER U WITH CARON\n01D5; C; 01D6; # LATIN CAPITAL LETTER U WITH DIAERESIS AND MACRON\n01D7; C; 01D8; # LATIN CAPITAL LETTER U WITH DIAERESIS AND ACUTE\n01D9; C; 01DA; # LATIN CAPITAL LETTER U WITH DIAERESIS AND CARON\n01DB; C; 01DC; # LATIN CAPITAL LETTER U WITH DIAERESIS AND GRAVE\n01DE; C; 01DF; # LATIN CAPITAL LETTER A WITH DIAERESIS AND MACRON\n01E0; C; 01E1; # LATIN CAPITAL LETTER A WITH DOT ABOVE AND MACRON\n01E2; C; 01E3; # LATIN CAPITAL LETTER AE WITH MACRON\n01E4; C; 01E5; # LATIN CAPITAL LETTER G WITH STROKE\n01E6; C; 01E7; # LATIN CAPITAL LETTER G WITH CARON\n01E8; C; 01E9; # LATIN CAPITAL LETTER K WITH CARON\n01EA; C; 01EB; # LATIN CAPITAL LETTER O WITH OGONEK\n01EC; C; 01ED; # LATIN CAPITAL LETTER O WITH OGONEK AND MACRON\n01EE; C; 01EF; # LATIN CAPITAL LETTER EZH WITH CARON\n01F0; F; 006A 030C; # LATIN SMALL LETTER J WITH CARON\n01F1; C; 01F3; # LATIN CAPITAL LETTER DZ\n01F2; C; 01F3; # LATIN CAPITAL LETTER D WITH SMALL LETTER Z\n01F4; C; 01F5; # LATIN CAPITAL LETTER G WITH ACUTE\n01F6; C; 0195; # LATIN CAPITAL LETTER HWAIR\n01F7; C; 01BF; # LATIN CAPITAL LETTER WYNN\n01F8; C; 01F9; # LATIN CAPITAL LETTER N WITH GRAVE\n01FA; C; 01FB; # LATIN CAPITAL LETTER A WITH RING ABOVE AND ACUTE\n01FC; C; 01FD; # LATIN CAPITAL LETTER AE WITH ACUTE\n01FE; C; 01FF; # LATIN CAPITAL LETTER O WITH STROKE AND ACUTE\n0200; C; 0201; # LATIN CAPITAL LETTER A WITH DOUBLE GRAVE\n0202; C; 0203; # LATIN CAPITAL LETTER A WITH INVERTED BREVE\n0204; C; 0205; # LATIN CAPITAL LETTER E WITH DOUBLE GRAVE\n0206; C; 0207; # LATIN CAPITAL LETTER E WITH INVERTED BREVE\n0208; C; 0209; # LATIN CAPITAL LETTER I WITH DOUBLE GRAVE\n020A; C; 020B; # LATIN CAPITAL LETTER I WITH INVERTED BREVE\n020C; C; 020D; # LATIN CAPITAL LETTER O WITH DOUBLE GRAVE\n020E; C; 020F; # LATIN CAPITAL LETTER O WITH INVERTED BREVE\n0210; C; 0211; # LATIN CAPITAL LETTER R WITH DOUBLE GRAVE\n0212; C; 0213; # LATIN CAPITAL LETTER R WITH INVERTED BREVE\n0214; C; 0215; # LATIN CAPITAL LETTER U WITH DOUBLE GRAVE\n0216; C; 0217; # LATIN CAPITAL LETTER U WITH INVERTED BREVE\n0218; C; 0219; # LATIN CAPITAL LETTER S WITH COMMA BELOW\n021A; C; 021B; # LATIN CAPITAL LETTER T WITH COMMA BELOW\n021C; C; 021D; # LATIN CAPITAL LETTER YOGH\n021E; C; 021F; # LATIN CAPITAL LETTER H WITH CARON\n0220; C; 019E; # LATIN CAPITAL LETTER N WITH LONG RIGHT LEG\n0222; C; 0223; # LATIN CAPITAL LETTER OU\n0224; C; 0225; # LATIN CAPITAL LETTER Z WITH HOOK\n0226; C; 0227; # LATIN CAPITAL LETTER A WITH DOT ABOVE\n0228; C; 0229; # LATIN CAPITAL LETTER E WITH CEDILLA\n022A; C; 022B; # LATIN CAPITAL LETTER O WITH DIAERESIS AND MACRON\n022C; C; 022D; # LATIN CAPITAL LETTER O WITH TILDE AND MACRON\n022E; C; 022F; # LATIN CAPITAL LETTER O WITH DOT ABOVE\n0230; C; 0231; # LATIN CAPITAL LETTER O WITH DOT ABOVE AND MACRON\n0232; C; 0233; # LATIN CAPITAL LETTER Y WITH MACRON\n023A; C; 2C65; # LATIN CAPITAL LETTER A WITH STROKE\n023B; C; 023C; # LATIN CAPITAL LETTER C WITH STROKE\n023D; C; 019A; # LATIN CAPITAL LETTER L WITH BAR\n023E; C; 2C66; # LATIN CAPITAL LETTER T WITH DIAGONAL STROKE\n0241; C; 0242; # LATIN CAPITAL LETTER GLOTTAL STOP\n0243; C; 0180; # LATIN CAPITAL LETTER B WITH STROKE\n0244; C; 0289; # LATIN CAPITAL LETTER U BAR\n0245; C; 028C; # LATIN CAPITAL LETTER TURNED V\n0246; C; 0247; # LATIN CAPITAL LETTER E WITH STROKE\n0248; C; 0249; # LATIN CAPITAL LETTER J WITH STROKE\n024A; C; 024B; # LATIN CAPITAL LETTER SMALL Q WITH HOOK TAIL\n024C; C; 024D; # LATIN CAPITAL LETTER R WITH STROKE\n024E; C; 024F; # LATIN CAPITAL LETTER Y WITH STROKE\n0345; C; 03B9; # COMBINING GREEK YPOGEGRAMMENI\n0370; C; 0371; # GREEK CAPITAL LETTER HETA\n0372; C; 0373; # GREEK CAPITAL LETTER ARCHAIC SAMPI\n0376; C; 0377; # GREEK CAPITAL LETTER PAMPHYLIAN DIGAMMA\n037F; C; 03F3; # GREEK CAPITAL LETTER YOT\n0386; C; 03AC; # GREEK CAPITAL LETTER ALPHA WITH TONOS\n0388; C; 03AD; # GREEK CAPITAL LETTER EPSILON WITH TONOS\n0389; C; 03AE; # GREEK CAPITAL LETTER ETA WITH TONOS\n038A; C; 03AF; # GREEK CAPITAL LETTER IOTA WITH TONOS\n038C; C; 03CC; # GREEK CAPITAL LETTER OMICRON WITH TONOS\n038E; C; 03CD; # GREEK CAPITAL LETTER UPSILON WITH TONOS\n038F; C; 03CE; # GREEK CAPITAL LETTER OMEGA WITH TONOS\n0390; F; 03B9 0308 0301; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS\n0391; C; 03B1; # GREEK CAPITAL LETTER ALPHA\n0392; C; 03B2; # GREEK CAPITAL LETTER BETA\n0393; C; 03B3; # GREEK CAPITAL LETTER GAMMA\n0394; C; 03B4; # GREEK CAPITAL LETTER DELTA\n0395; C; 03B5; # GREEK CAPITAL LETTER EPSILON\n0396; C; 03B6; # GREEK CAPITAL LETTER ZETA\n0397; C; 03B7; # GREEK CAPITAL LETTER ETA\n0398; C; 03B8; # GREEK CAPITAL LETTER THETA\n0399; C; 03B9; # GREEK CAPITAL LETTER IOTA\n039A; C; 03BA; # GREEK CAPITAL LETTER KAPPA\n039B; C; 03BB; # GREEK CAPITAL LETTER LAMDA\n039C; C; 03BC; # GREEK CAPITAL LETTER MU\n039D; C; 03BD; # GREEK CAPITAL LETTER NU\n039E; C; 03BE; # GREEK CAPITAL LETTER XI\n039F; C; 03BF; # GREEK CAPITAL LETTER OMICRON\n03A0; C; 03C0; # GREEK CAPITAL LETTER PI\n03A1; C; 03C1; # GREEK CAPITAL LETTER RHO\n03A3; C; 03C3; # GREEK CAPITAL LETTER SIGMA\n03A4; C; 03C4; # GREEK CAPITAL LETTER TAU\n03A5; C; 03C5; # GREEK CAPITAL LETTER UPSILON\n03A6; C; 03C6; # GREEK CAPITAL LETTER PHI\n03A7; C; 03C7; # GREEK CAPITAL LETTER CHI\n03A8; C; 03C8; # GREEK CAPITAL LETTER PSI\n03A9; C; 03C9; # GREEK CAPITAL LETTER OMEGA\n03AA; C; 03CA; # GREEK CAPITAL LETTER IOTA WITH DIALYTIKA\n03AB; C; 03CB; # GREEK CAPITAL LETTER UPSILON WITH DIALYTIKA\n03B0; F; 03C5 0308 0301; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS\n03C2; C; 03C3; # GREEK SMALL LETTER FINAL SIGMA\n03CF; C; 03D7; # GREEK CAPITAL KAI SYMBOL\n03D0; C; 03B2; # GREEK BETA SYMBOL\n03D1; C; 03B8; # GREEK THETA SYMBOL\n03D5; C; 03C6; # GREEK PHI SYMBOL\n03D6; C; 03C0; # GREEK PI SYMBOL\n03D8; C; 03D9; # GREEK LETTER ARCHAIC KOPPA\n03DA; C; 03DB; # GREEK LETTER STIGMA\n03DC; C; 03DD; # GREEK LETTER DIGAMMA\n03DE; C; 03DF; # GREEK LETTER KOPPA\n03E0; C; 03E1; # GREEK LETTER SAMPI\n03E2; C; 03E3; # COPTIC CAPITAL LETTER SHEI\n03E4; C; 03E5; # COPTIC CAPITAL LETTER FEI\n03E6; C; 03E7; # COPTIC CAPITAL LETTER KHEI\n03E8; C; 03E9; # COPTIC CAPITAL LETTER HORI\n03EA; C; 03EB; # COPTIC CAPITAL LETTER GANGIA\n03EC; C; 03ED; # COPTIC CAPITAL LETTER SHIMA\n03EE; C; 03EF; # COPTIC CAPITAL LETTER DEI\n03F0; C; 03BA; # GREEK KAPPA SYMBOL\n03F1; C; 03C1; # GREEK RHO SYMBOL\n03F4; C; 03B8; # GREEK CAPITAL THETA SYMBOL\n03F5; C; 03B5; # GREEK LUNATE EPSILON SYMBOL\n03F7; C; 03F8; # GREEK CAPITAL LETTER SHO\n03F9; C; 03F2; # GREEK CAPITAL LUNATE SIGMA SYMBOL\n03FA; C; 03FB; # GREEK CAPITAL LETTER SAN\n03FD; C; 037B; # GREEK CAPITAL REVERSED LUNATE SIGMA SYMBOL\n03FE; C; 037C; # GREEK CAPITAL DOTTED LUNATE SIGMA SYMBOL\n03FF; C; 037D; # GREEK CAPITAL REVERSED DOTTED LUNATE SIGMA SYMBOL\n0400; C; 0450; # CYRILLIC CAPITAL LETTER IE WITH GRAVE\n0401; C; 0451; # CYRILLIC CAPITAL LETTER IO\n0402; C; 0452; # CYRILLIC CAPITAL LETTER DJE\n0403; C; 0453; # CYRILLIC CAPITAL LETTER GJE\n0404; C; 0454; # CYRILLIC CAPITAL LETTER UKRAINIAN IE\n0405; C; 0455; # CYRILLIC CAPITAL LETTER DZE\n0406; C; 0456; # CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I\n0407; C; 0457; # CYRILLIC CAPITAL LETTER YI\n0408; C; 0458; # CYRILLIC CAPITAL LETTER JE\n0409; C; 0459; # CYRILLIC CAPITAL LETTER LJE\n040A; C; 045A; # CYRILLIC CAPITAL LETTER NJE\n040B; C; 045B; # CYRILLIC CAPITAL LETTER TSHE\n040C; C; 045C; # CYRILLIC CAPITAL LETTER KJE\n040D; C; 045D; # CYRILLIC CAPITAL LETTER I WITH GRAVE\n040E; C; 045E; # CYRILLIC CAPITAL LETTER SHORT U\n040F; C; 045F; # CYRILLIC CAPITAL LETTER DZHE\n0410; C; 0430; # CYRILLIC CAPITAL LETTER A\n0411; C; 0431; # CYRILLIC CAPITAL LETTER BE\n0412; C; 0432; # CYRILLIC CAPITAL LETTER VE\n0413; C; 0433; # CYRILLIC CAPITAL LETTER GHE\n0414; C; 0434; # CYRILLIC CAPITAL LETTER DE\n0415; C; 0435; # CYRILLIC CAPITAL LETTER IE\n0416; C; 0436; # CYRILLIC CAPITAL LETTER ZHE\n0417; C; 0437; # CYRILLIC CAPITAL LETTER ZE\n0418; C; 0438; # CYRILLIC CAPITAL LETTER I\n0419; C; 0439; # CYRILLIC CAPITAL LETTER SHORT I\n041A; C; 043A; # CYRILLIC CAPITAL LETTER KA\n041B; C; 043B; # CYRILLIC CAPITAL LETTER EL\n041C; C; 043C; # CYRILLIC CAPITAL LETTER EM\n041D; C; 043D; # CYRILLIC CAPITAL LETTER EN\n041E; C; 043E; # CYRILLIC CAPITAL LETTER O\n041F; C; 043F; # CYRILLIC CAPITAL LETTER PE\n0420; C; 0440; # CYRILLIC CAPITAL LETTER ER\n0421; C; 0441; # CYRILLIC CAPITAL LETTER ES\n0422; C; 0442; # CYRILLIC CAPITAL LETTER TE\n0423; C; 0443; # CYRILLIC CAPITAL LETTER U\n0424; C; 0444; # CYRILLIC CAPITAL LETTER EF\n0425; C; 0445; # CYRILLIC CAPITAL LETTER HA\n0426; C; 0446; # CYRILLIC CAPITAL LETTER TSE\n0427; C; 0447; # CYRILLIC CAPITAL LETTER CHE\n0428; C; 0448; # CYRILLIC CAPITAL LETTER SHA\n0429; C; 0449; # CYRILLIC CAPITAL LETTER SHCHA\n042A; C; 044A; # CYRILLIC CAPITAL LETTER HARD SIGN\n042B; C; 044B; # CYRILLIC CAPITAL LETTER YERU\n042C; C; 044C; # CYRILLIC CAPITAL LETTER SOFT SIGN\n042D; C; 044D; # CYRILLIC CAPITAL LETTER E\n042E; C; 044E; # CYRILLIC CAPITAL LETTER YU\n042F; C; 044F; # CYRILLIC CAPITAL LETTER YA\n0460; C; 0461; # CYRILLIC CAPITAL LETTER OMEGA\n0462; C; 0463; # CYRILLIC CAPITAL LETTER YAT\n0464; C; 0465; # CYRILLIC CAPITAL LETTER IOTIFIED E\n0466; C; 0467; # CYRILLIC CAPITAL LETTER LITTLE YUS\n0468; C; 0469; # CYRILLIC CAPITAL LETTER IOTIFIED LITTLE YUS\n046A; C; 046B; # CYRILLIC CAPITAL LETTER BIG YUS\n046C; C; 046D; # CYRILLIC CAPITAL LETTER IOTIFIED BIG YUS\n046E; C; 046F; # CYRILLIC CAPITAL LETTER KSI\n0470; C; 0471; # CYRILLIC CAPITAL LETTER PSI\n0472; C; 0473; # CYRILLIC CAPITAL LETTER FITA\n0474; C; 0475; # CYRILLIC CAPITAL LETTER IZHITSA\n0476; C; 0477; # CYRILLIC CAPITAL LETTER IZHITSA WITH DOUBLE GRAVE ACCENT\n0478; C; 0479; # CYRILLIC CAPITAL LETTER UK\n047A; C; 047B; # CYRILLIC CAPITAL LETTER ROUND OMEGA\n047C; C; 047D; # CYRILLIC CAPITAL LETTER OMEGA WITH TITLO\n047E; C; 047F; # CYRILLIC CAPITAL LETTER OT\n0480; C; 0481; # CYRILLIC CAPITAL LETTER KOPPA\n048A; C; 048B; # CYRILLIC CAPITAL LETTER SHORT I WITH TAIL\n048C; C; 048D; # CYRILLIC CAPITAL LETTER SEMISOFT SIGN\n048E; C; 048F; # CYRILLIC CAPITAL LETTER ER WITH TICK\n0490; C; 0491; # CYRILLIC CAPITAL LETTER GHE WITH UPTURN\n0492; C; 0493; # CYRILLIC CAPITAL LETTER GHE WITH STROKE\n0494; C; 0495; # CYRILLIC CAPITAL LETTER GHE WITH MIDDLE HOOK\n0496; C; 0497; # CYRILLIC CAPITAL LETTER ZHE WITH DESCENDER\n0498; C; 0499; # CYRILLIC CAPITAL LETTER ZE WITH DESCENDER\n049A; C; 049B; # CYRILLIC CAPITAL LETTER KA WITH DESCENDER\n049C; C; 049D; # CYRILLIC CAPITAL LETTER KA WITH VERTICAL STROKE\n049E; C; 049F; # CYRILLIC CAPITAL LETTER KA WITH STROKE\n04A0; C; 04A1; # CYRILLIC CAPITAL LETTER BASHKIR KA\n04A2; C; 04A3; # CYRILLIC CAPITAL LETTER EN WITH DESCENDER\n04A4; C; 04A5; # CYRILLIC CAPITAL LIGATURE EN GHE\n04A6; C; 04A7; # CYRILLIC CAPITAL LETTER PE WITH MIDDLE HOOK\n04A8; C; 04A9; # CYRILLIC CAPITAL LETTER ABKHASIAN HA\n04AA; C; 04AB; # CYRILLIC CAPITAL LETTER ES WITH DESCENDER\n04AC; C; 04AD; # CYRILLIC CAPITAL LETTER TE WITH DESCENDER\n04AE; C; 04AF; # CYRILLIC CAPITAL LETTER STRAIGHT U\n04B0; C; 04B1; # CYRILLIC CAPITAL LETTER STRAIGHT U WITH STROKE\n04B2; C; 04B3; # CYRILLIC CAPITAL LETTER HA WITH DESCENDER\n04B4; C; 04B5; # CYRILLIC CAPITAL LIGATURE TE TSE\n04B6; C; 04B7; # CYRILLIC CAPITAL LETTER CHE WITH DESCENDER\n04B8; C; 04B9; # CYRILLIC CAPITAL LETTER CHE WITH VERTICAL STROKE\n04BA; C; 04BB; # CYRILLIC CAPITAL LETTER SHHA\n04BC; C; 04BD; # CYRILLIC CAPITAL LETTER ABKHASIAN CHE\n04BE; C; 04BF; # CYRILLIC CAPITAL LETTER ABKHASIAN CHE WITH DESCENDER\n04C0; C; 04CF; # CYRILLIC LETTER PALOCHKA\n04C1; C; 04C2; # CYRILLIC CAPITAL LETTER ZHE WITH BREVE\n04C3; C; 04C4; # CYRILLIC CAPITAL LETTER KA WITH HOOK\n04C5; C; 04C6; # CYRILLIC CAPITAL LETTER EL WITH TAIL\n04C7; C; 04C8; # CYRILLIC CAPITAL LETTER EN WITH HOOK\n04C9; C; 04CA; # CYRILLIC CAPITAL LETTER EN WITH TAIL\n04CB; C; 04CC; # CYRILLIC CAPITAL LETTER KHAKASSIAN CHE\n04CD; C; 04CE; # CYRILLIC CAPITAL LETTER EM WITH TAIL\n04D0; C; 04D1; # CYRILLIC CAPITAL LETTER A WITH BREVE\n04D2; C; 04D3; # CYRILLIC CAPITAL LETTER A WITH DIAERESIS\n04D4; C; 04D5; # CYRILLIC CAPITAL LIGATURE A IE\n04D6; C; 04D7; # CYRILLIC CAPITAL LETTER IE WITH BREVE\n04D8; C; 04D9; # CYRILLIC CAPITAL LETTER SCHWA\n04DA; C; 04DB; # CYRILLIC CAPITAL LETTER SCHWA WITH DIAERESIS\n04DC; C; 04DD; # CYRILLIC CAPITAL LETTER ZHE WITH DIAERESIS\n04DE; C; 04DF; # CYRILLIC CAPITAL LETTER ZE WITH DIAERESIS\n04E0; C; 04E1; # CYRILLIC CAPITAL LETTER ABKHASIAN DZE\n04E2; C; 04E3; # CYRILLIC CAPITAL LETTER I WITH MACRON\n04E4; C; 04E5; # CYRILLIC CAPITAL LETTER I WITH DIAERESIS\n04E6; C; 04E7; # CYRILLIC CAPITAL LETTER O WITH DIAERESIS\n04E8; C; 04E9; # CYRILLIC CAPITAL LETTER BARRED O\n04EA; C; 04EB; # CYRILLIC CAPITAL LETTER BARRED O WITH DIAERESIS\n04EC; C; 04ED; # CYRILLIC CAPITAL LETTER E WITH DIAERESIS\n04EE; C; 04EF; # CYRILLIC CAPITAL LETTER U WITH MACRON\n04F0; C; 04F1; # CYRILLIC CAPITAL LETTER U WITH DIAERESIS\n04F2; C; 04F3; # CYRILLIC CAPITAL LETTER U WITH DOUBLE ACUTE\n04F4; C; 04F5; # CYRILLIC CAPITAL LETTER CHE WITH DIAERESIS\n04F6; C; 04F7; # CYRILLIC CAPITAL LETTER GHE WITH DESCENDER\n04F8; C; 04F9; # CYRILLIC CAPITAL LETTER YERU WITH DIAERESIS\n04FA; C; 04FB; # CYRILLIC CAPITAL LETTER GHE WITH STROKE AND HOOK\n04FC; C; 04FD; # CYRILLIC CAPITAL LETTER HA WITH HOOK\n04FE; C; 04FF; # CYRILLIC CAPITAL LETTER HA WITH STROKE\n0500; C; 0501; # CYRILLIC CAPITAL LETTER KOMI DE\n0502; C; 0503; # CYRILLIC CAPITAL LETTER KOMI DJE\n0504; C; 0505; # CYRILLIC CAPITAL LETTER KOMI ZJE\n0506; C; 0507; # CYRILLIC CAPITAL LETTER KOMI DZJE\n0508; C; 0509; # CYRILLIC CAPITAL LETTER KOMI LJE\n050A; C; 050B; # CYRILLIC CAPITAL LETTER KOMI NJE\n050C; C; 050D; # CYRILLIC CAPITAL LETTER KOMI SJE\n050E; C; 050F; # CYRILLIC CAPITAL LETTER KOMI TJE\n0510; C; 0511; # CYRILLIC CAPITAL LETTER REVERSED ZE\n0512; C; 0513; # CYRILLIC CAPITAL LETTER EL WITH HOOK\n0514; C; 0515; # CYRILLIC CAPITAL LETTER LHA\n0516; C; 0517; # CYRILLIC CAPITAL LETTER RHA\n0518; C; 0519; # CYRILLIC CAPITAL LETTER YAE\n051A; C; 051B; # CYRILLIC CAPITAL LETTER QA\n051C; C; 051D; # CYRILLIC CAPITAL LETTER WE\n051E; C; 051F; # CYRILLIC CAPITAL LETTER ALEUT KA\n0520; C; 0521; # CYRILLIC CAPITAL LETTER EL WITH MIDDLE HOOK\n0522; C; 0523; # CYRILLIC CAPITAL LETTER EN WITH MIDDLE HOOK\n0524; C; 0525; # CYRILLIC CAPITAL LETTER PE WITH DESCENDER\n0526; C; 0527; # CYRILLIC CAPITAL LETTER SHHA WITH DESCENDER\n0528; C; 0529; # CYRILLIC CAPITAL LETTER EN WITH LEFT HOOK\n052A; C; 052B; # CYRILLIC CAPITAL LETTER DZZHE\n052C; C; 052D; # CYRILLIC CAPITAL LETTER DCHE\n052E; C; 052F; # CYRILLIC CAPITAL LETTER EL WITH DESCENDER\n0531; C; 0561; # ARMENIAN CAPITAL LETTER AYB\n0532; C; 0562; # ARMENIAN CAPITAL LETTER BEN\n0533; C; 0563; # ARMENIAN CAPITAL LETTER GIM\n0534; C; 0564; # ARMENIAN CAPITAL LETTER DA\n0535; C; 0565; # ARMENIAN CAPITAL LETTER ECH\n0536; C; 0566; # ARMENIAN CAPITAL LETTER ZA\n0537; C; 0567; # ARMENIAN CAPITAL LETTER EH\n0538; C; 0568; # ARMENIAN CAPITAL LETTER ET\n0539; C; 0569; # ARMENIAN CAPITAL LETTER TO\n053A; C; 056A; # ARMENIAN CAPITAL LETTER ZHE\n053B; C; 056B; # ARMENIAN CAPITAL LETTER INI\n053C; C; 056C; # ARMENIAN CAPITAL LETTER LIWN\n053D; C; 056D; # ARMENIAN CAPITAL LETTER XEH\n053E; C; 056E; # ARMENIAN CAPITAL LETTER CA\n053F; C; 056F; # ARMENIAN CAPITAL LETTER KEN\n0540; C; 0570; # ARMENIAN CAPITAL LETTER HO\n0541; C; 0571; # ARMENIAN CAPITAL LETTER JA\n0542; C; 0572; # ARMENIAN CAPITAL LETTER GHAD\n0543; C; 0573; # ARMENIAN CAPITAL LETTER CHEH\n0544; C; 0574; # ARMENIAN CAPITAL LETTER MEN\n0545; C; 0575; # ARMENIAN CAPITAL LETTER YI\n0546; C; 0576; # ARMENIAN CAPITAL LETTER NOW\n0547; C; 0577; # ARMENIAN CAPITAL LETTER SHA\n0548; C; 0578; # ARMENIAN CAPITAL LETTER VO\n0549; C; 0579; # ARMENIAN CAPITAL LETTER CHA\n054A; C; 057A; # ARMENIAN CAPITAL LETTER PEH\n054B; C; 057B; # ARMENIAN CAPITAL LETTER JHEH\n054C; C; 057C; # ARMENIAN CAPITAL LETTER RA\n054D; C; 057D; # ARMENIAN CAPITAL LETTER SEH\n054E; C; 057E; # ARMENIAN CAPITAL LETTER VEW\n054F; C; 057F; # ARMENIAN CAPITAL LETTER TIWN\n0550; C; 0580; # ARMENIAN CAPITAL LETTER REH\n0551; C; 0581; # ARMENIAN CAPITAL LETTER CO\n0552; C; 0582; # ARMENIAN CAPITAL LETTER YIWN\n0553; C; 0583; # ARMENIAN CAPITAL LETTER PIWR\n0554; C; 0584; # ARMENIAN CAPITAL LETTER KEH\n0555; C; 0585; # ARMENIAN CAPITAL LETTER OH\n0556; C; 0586; # ARMENIAN CAPITAL LETTER FEH\n0587; F; 0565 0582; # ARMENIAN SMALL LIGATURE ECH YIWN\n10A0; C; 2D00; # GEORGIAN CAPITAL LETTER AN\n10A1; C; 2D01; # GEORGIAN CAPITAL LETTER BAN\n10A2; C; 2D02; # GEORGIAN CAPITAL LETTER GAN\n10A3; C; 2D03; # GEORGIAN CAPITAL LETTER DON\n10A4; C; 2D04; # GEORGIAN CAPITAL LETTER EN\n10A5; C; 2D05; # GEORGIAN CAPITAL LETTER VIN\n10A6; C; 2D06; # GEORGIAN CAPITAL LETTER ZEN\n10A7; C; 2D07; # GEORGIAN CAPITAL LETTER TAN\n10A8; C; 2D08; # GEORGIAN CAPITAL LETTER IN\n10A9; C; 2D09; # GEORGIAN CAPITAL LETTER KAN\n10AA; C; 2D0A; # GEORGIAN CAPITAL LETTER LAS\n10AB; C; 2D0B; # GEORGIAN CAPITAL LETTER MAN\n10AC; C; 2D0C; # GEORGIAN CAPITAL LETTER NAR\n10AD; C; 2D0D; # GEORGIAN CAPITAL LETTER ON\n10AE; C; 2D0E; # GEORGIAN CAPITAL LETTER PAR\n10AF; C; 2D0F; # GEORGIAN CAPITAL LETTER ZHAR\n10B0; C; 2D10; # GEORGIAN CAPITAL LETTER RAE\n10B1; C; 2D11; # GEORGIAN CAPITAL LETTER SAN\n10B2; C; 2D12; # GEORGIAN CAPITAL LETTER TAR\n10B3; C; 2D13; # GEORGIAN CAPITAL LETTER UN\n10B4; C; 2D14; # GEORGIAN CAPITAL LETTER PHAR\n10B5; C; 2D15; # GEORGIAN CAPITAL LETTER KHAR\n10B6; C; 2D16; # GEORGIAN CAPITAL LETTER GHAN\n10B7; C; 2D17; # GEORGIAN CAPITAL LETTER QAR\n10B8; C; 2D18; # GEORGIAN CAPITAL LETTER SHIN\n10B9; C; 2D19; # GEORGIAN CAPITAL LETTER CHIN\n10BA; C; 2D1A; # GEORGIAN CAPITAL LETTER CAN\n10BB; C; 2D1B; # GEORGIAN CAPITAL LETTER JIL\n10BC; C; 2D1C; # GEORGIAN CAPITAL LETTER CIL\n10BD; C; 2D1D; # GEORGIAN CAPITAL LETTER CHAR\n10BE; C; 2D1E; # GEORGIAN CAPITAL LETTER XAN\n10BF; C; 2D1F; # GEORGIAN CAPITAL LETTER JHAN\n10C0; C; 2D20; # GEORGIAN CAPITAL LETTER HAE\n10C1; C; 2D21; # GEORGIAN CAPITAL LETTER HE\n10C2; C; 2D22; # GEORGIAN CAPITAL LETTER HIE\n10C3; C; 2D23; # GEORGIAN CAPITAL LETTER WE\n10C4; C; 2D24; # GEORGIAN CAPITAL LETTER HAR\n10C5; C; 2D25; # GEORGIAN CAPITAL LETTER HOE\n10C7; C; 2D27; # GEORGIAN CAPITAL LETTER YN\n10CD; C; 2D2D; # GEORGIAN CAPITAL LETTER AEN\n13F8; C; 13F0; # CHEROKEE SMALL LETTER YE\n13F9; C; 13F1; # CHEROKEE SMALL LETTER YI\n13FA; C; 13F2; # CHEROKEE SMALL LETTER YO\n13FB; C; 13F3; # CHEROKEE SMALL LETTER YU\n13FC; C; 13F4; # CHEROKEE SMALL LETTER YV\n13FD; C; 13F5; # CHEROKEE SMALL LETTER MV\n1E00; C; 1E01; # LATIN CAPITAL LETTER A WITH RING BELOW\n1E02; C; 1E03; # LATIN CAPITAL LETTER B WITH DOT ABOVE\n1E04; C; 1E05; # LATIN CAPITAL LETTER B WITH DOT BELOW\n1E06; C; 1E07; # LATIN CAPITAL LETTER B WITH LINE BELOW\n1E08; C; 1E09; # LATIN CAPITAL LETTER C WITH CEDILLA AND ACUTE\n1E0A; C; 1E0B; # LATIN CAPITAL LETTER D WITH DOT ABOVE\n1E0C; C; 1E0D; # LATIN CAPITAL LETTER D WITH DOT BELOW\n1E0E; C; 1E0F; # LATIN CAPITAL LETTER D WITH LINE BELOW\n1E10; C; 1E11; # LATIN CAPITAL LETTER D WITH CEDILLA\n1E12; C; 1E13; # LATIN CAPITAL LETTER D WITH CIRCUMFLEX BELOW\n1E14; C; 1E15; # LATIN CAPITAL LETTER E WITH MACRON AND GRAVE\n1E16; C; 1E17; # LATIN CAPITAL LETTER E WITH MACRON AND ACUTE\n1E18; C; 1E19; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX BELOW\n1E1A; C; 1E1B; # LATIN CAPITAL LETTER E WITH TILDE BELOW\n1E1C; C; 1E1D; # LATIN CAPITAL LETTER E WITH CEDILLA AND BREVE\n1E1E; C; 1E1F; # LATIN CAPITAL LETTER F WITH DOT ABOVE\n1E20; C; 1E21; # LATIN CAPITAL LETTER G WITH MACRON\n1E22; C; 1E23; # LATIN CAPITAL LETTER H WITH DOT ABOVE\n1E24; C; 1E25; # LATIN CAPITAL LETTER H WITH DOT BELOW\n1E26; C; 1E27; # LATIN CAPITAL LETTER H WITH DIAERESIS\n1E28; C; 1E29; # LATIN CAPITAL LETTER H WITH CEDILLA\n1E2A; C; 1E2B; # LATIN CAPITAL LETTER H WITH BREVE BELOW\n1E2C; C; 1E2D; # LATIN CAPITAL LETTER I WITH TILDE BELOW\n1E2E; C; 1E2F; # LATIN CAPITAL LETTER I WITH DIAERESIS AND ACUTE\n1E30; C; 1E31; # LATIN CAPITAL LETTER K WITH ACUTE\n1E32; C; 1E33; # LATIN CAPITAL LETTER K WITH DOT BELOW\n1E34; C; 1E35; # LATIN CAPITAL LETTER K WITH LINE BELOW\n1E36; C; 1E37; # LATIN CAPITAL LETTER L WITH DOT BELOW\n1E38; C; 1E39; # LATIN CAPITAL LETTER L WITH DOT BELOW AND MACRON\n1E3A; C; 1E3B; # LATIN CAPITAL LETTER L WITH LINE BELOW\n1E3C; C; 1E3D; # LATIN CAPITAL LETTER L WITH CIRCUMFLEX BELOW\n1E3E; C; 1E3F; # LATIN CAPITAL LETTER M WITH ACUTE\n1E40; C; 1E41; # LATIN CAPITAL LETTER M WITH DOT ABOVE\n1E42; C; 1E43; # LATIN CAPITAL LETTER M WITH DOT BELOW\n1E44; C; 1E45; # LATIN CAPITAL LETTER N WITH DOT ABOVE\n1E46; C; 1E47; # LATIN CAPITAL LETTER N WITH DOT BELOW\n1E48; C; 1E49; # LATIN CAPITAL LETTER N WITH LINE BELOW\n1E4A; C; 1E4B; # LATIN CAPITAL LETTER N WITH CIRCUMFLEX BELOW\n1E4C; C; 1E4D; # LATIN CAPITAL LETTER O WITH TILDE AND ACUTE\n1E4E; C; 1E4F; # LATIN CAPITAL LETTER O WITH TILDE AND DIAERESIS\n1E50; C; 1E51; # LATIN CAPITAL LETTER O WITH MACRON AND GRAVE\n1E52; C; 1E53; # LATIN CAPITAL LETTER O WITH MACRON AND ACUTE\n1E54; C; 1E55; # LATIN CAPITAL LETTER P WITH ACUTE\n1E56; C; 1E57; # LATIN CAPITAL LETTER P WITH DOT ABOVE\n1E58; C; 1E59; # LATIN CAPITAL LETTER R WITH DOT ABOVE\n1E5A; C; 1E5B; # LATIN CAPITAL LETTER R WITH DOT BELOW\n1E5C; C; 1E5D; # LATIN CAPITAL LETTER R WITH DOT BELOW AND MACRON\n1E5E; C; 1E5F; # LATIN CAPITAL LETTER R WITH LINE BELOW\n1E60; C; 1E61; # LATIN CAPITAL LETTER S WITH DOT ABOVE\n1E62; C; 1E63; # LATIN CAPITAL LETTER S WITH DOT BELOW\n1E64; C; 1E65; # LATIN CAPITAL LETTER S WITH ACUTE AND DOT ABOVE\n1E66; C; 1E67; # LATIN CAPITAL LETTER S WITH CARON AND DOT ABOVE\n1E68; C; 1E69; # LATIN CAPITAL LETTER S WITH DOT BELOW AND DOT ABOVE\n1E6A; C; 1E6B; # LATIN CAPITAL LETTER T WITH DOT ABOVE\n1E6C; C; 1E6D; # LATIN CAPITAL LETTER T WITH DOT BELOW\n1E6E; C; 1E6F; # LATIN CAPITAL LETTER T WITH LINE BELOW\n1E70; C; 1E71; # LATIN CAPITAL LETTER T WITH CIRCUMFLEX BELOW\n1E72; C; 1E73; # LATIN CAPITAL LETTER U WITH DIAERESIS BELOW\n1E74; C; 1E75; # LATIN CAPITAL LETTER U WITH TILDE BELOW\n1E76; C; 1E77; # LATIN CAPITAL LETTER U WITH CIRCUMFLEX BELOW\n1E78; C; 1E79; # LATIN CAPITAL LETTER U WITH TILDE AND ACUTE\n1E7A; C; 1E7B; # LATIN CAPITAL LETTER U WITH MACRON AND DIAERESIS\n1E7C; C; 1E7D; # LATIN CAPITAL LETTER V WITH TILDE\n1E7E; C; 1E7F; # LATIN CAPITAL LETTER V WITH DOT BELOW\n1E80; C; 1E81; # LATIN CAPITAL LETTER W WITH GRAVE\n1E82; C; 1E83; # LATIN CAPITAL LETTER W WITH ACUTE\n1E84; C; 1E85; # LATIN CAPITAL LETTER W WITH DIAERESIS\n1E86; C; 1E87; # LATIN CAPITAL LETTER W WITH DOT ABOVE\n1E88; C; 1E89; # LATIN CAPITAL LETTER W WITH DOT BELOW\n1E8A; C; 1E8B; # LATIN CAPITAL LETTER X WITH DOT ABOVE\n1E8C; C; 1E8D; # LATIN CAPITAL LETTER X WITH DIAERESIS\n1E8E; C; 1E8F; # LATIN CAPITAL LETTER Y WITH DOT ABOVE\n1E90; C; 1E91; # LATIN CAPITAL LETTER Z WITH CIRCUMFLEX\n1E92; C; 1E93; # LATIN CAPITAL LETTER Z WITH DOT BELOW\n1E94; C; 1E95; # LATIN CAPITAL LETTER Z WITH LINE BELOW\n1E96; F; 0068 0331; # LATIN SMALL LETTER H WITH LINE BELOW\n1E97; F; 0074 0308; # LATIN SMALL LETTER T WITH DIAERESIS\n1E98; F; 0077 030A; # LATIN SMALL LETTER W WITH RING ABOVE\n1E99; F; 0079 030A; # LATIN SMALL LETTER Y WITH RING ABOVE\n1E9A; F; 0061 02BE; # LATIN SMALL LETTER A WITH RIGHT HALF RING\n1E9B; C; 1E61; # LATIN SMALL LETTER LONG S WITH DOT ABOVE\n1E9E; F; 0073 0073; # LATIN CAPITAL LETTER SHARP S\n1E9E; S; 00DF; # LATIN CAPITAL LETTER SHARP S\n1EA0; C; 1EA1; # LATIN CAPITAL LETTER A WITH DOT BELOW\n1EA2; C; 1EA3; # LATIN CAPITAL LETTER A WITH HOOK ABOVE\n1EA4; C; 1EA5; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND ACUTE\n1EA6; C; 1EA7; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND GRAVE\n1EA8; C; 1EA9; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND HOOK ABOVE\n1EAA; C; 1EAB; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND TILDE\n1EAC; C; 1EAD; # LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND DOT BELOW\n1EAE; C; 1EAF; # LATIN CAPITAL LETTER A WITH BREVE AND ACUTE\n1EB0; C; 1EB1; # LATIN CAPITAL LETTER A WITH BREVE AND GRAVE\n1EB2; C; 1EB3; # LATIN CAPITAL LETTER A WITH BREVE AND HOOK ABOVE\n1EB4; C; 1EB5; # LATIN CAPITAL LETTER A WITH BREVE AND TILDE\n1EB6; C; 1EB7; # LATIN CAPITAL LETTER A WITH BREVE AND DOT BELOW\n1EB8; C; 1EB9; # LATIN CAPITAL LETTER E WITH DOT BELOW\n1EBA; C; 1EBB; # LATIN CAPITAL LETTER E WITH HOOK ABOVE\n1EBC; C; 1EBD; # LATIN CAPITAL LETTER E WITH TILDE\n1EBE; C; 1EBF; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND ACUTE\n1EC0; C; 1EC1; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND GRAVE\n1EC2; C; 1EC3; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND HOOK ABOVE\n1EC4; C; 1EC5; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND TILDE\n1EC6; C; 1EC7; # LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND DOT BELOW\n1EC8; C; 1EC9; # LATIN CAPITAL LETTER I WITH HOOK ABOVE\n1ECA; C; 1ECB; # LATIN CAPITAL LETTER I WITH DOT BELOW\n1ECC; C; 1ECD; # LATIN CAPITAL LETTER O WITH DOT BELOW\n1ECE; C; 1ECF; # LATIN CAPITAL LETTER O WITH HOOK ABOVE\n1ED0; C; 1ED1; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX AND ACUTE\n1ED2; C; 1ED3; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX AND GRAVE\n1ED4; C; 1ED5; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX AND HOOK ABOVE\n1ED6; C; 1ED7; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX AND TILDE\n1ED8; C; 1ED9; # LATIN CAPITAL LETTER O WITH CIRCUMFLEX AND DOT BELOW\n1EDA; C; 1EDB; # LATIN CAPITAL LETTER O WITH HORN AND ACUTE\n1EDC; C; 1EDD; # LATIN CAPITAL LETTER O WITH HORN AND GRAVE\n1EDE; C; 1EDF; # LATIN CAPITAL LETTER O WITH HORN AND HOOK ABOVE\n1EE0; C; 1EE1; # LATIN CAPITAL LETTER O WITH HORN AND TILDE\n1EE2; C; 1EE3; # LATIN CAPITAL LETTER O WITH HORN AND DOT BELOW\n1EE4; C; 1EE5; # LATIN CAPITAL LETTER U WITH DOT BELOW\n1EE6; C; 1EE7; # LATIN CAPITAL LETTER U WITH HOOK ABOVE\n1EE8; C; 1EE9; # LATIN CAPITAL LETTER U WITH HORN AND ACUTE\n1EEA; C; 1EEB; # LATIN CAPITAL LETTER U WITH HORN AND GRAVE\n1EEC; C; 1EED; # LATIN CAPITAL LETTER U WITH HORN AND HOOK ABOVE\n1EEE; C; 1EEF; # LATIN CAPITAL LETTER U WITH HORN AND TILDE\n1EF0; C; 1EF1; # LATIN CAPITAL LETTER U WITH HORN AND DOT BELOW\n1EF2; C; 1EF3; # LATIN CAPITAL LETTER Y WITH GRAVE\n1EF4; C; 1EF5; # LATIN CAPITAL LETTER Y WITH DOT BELOW\n1EF6; C; 1EF7; # LATIN CAPITAL LETTER Y WITH HOOK ABOVE\n1EF8; C; 1EF9; # LATIN CAPITAL LETTER Y WITH TILDE\n1EFA; C; 1EFB; # LATIN CAPITAL LETTER MIDDLE-WELSH LL\n1EFC; C; 1EFD; # LATIN CAPITAL LETTER MIDDLE-WELSH V\n1EFE; C; 1EFF; # LATIN CAPITAL LETTER Y WITH LOOP\n1F08; C; 1F00; # GREEK CAPITAL LETTER ALPHA WITH PSILI\n1F09; C; 1F01; # GREEK CAPITAL LETTER ALPHA WITH DASIA\n1F0A; C; 1F02; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA\n1F0B; C; 1F03; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA\n1F0C; C; 1F04; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA\n1F0D; C; 1F05; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA\n1F0E; C; 1F06; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI\n1F0F; C; 1F07; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI\n1F18; C; 1F10; # GREEK CAPITAL LETTER EPSILON WITH PSILI\n1F19; C; 1F11; # GREEK CAPITAL LETTER EPSILON WITH DASIA\n1F1A; C; 1F12; # GREEK CAPITAL LETTER EPSILON WITH PSILI AND VARIA\n1F1B; C; 1F13; # GREEK CAPITAL LETTER EPSILON WITH DASIA AND VARIA\n1F1C; C; 1F14; # GREEK CAPITAL LETTER EPSILON WITH PSILI AND OXIA\n1F1D; C; 1F15; # GREEK CAPITAL LETTER EPSILON WITH DASIA AND OXIA\n1F28; C; 1F20; # GREEK CAPITAL LETTER ETA WITH PSILI\n1F29; C; 1F21; # GREEK CAPITAL LETTER ETA WITH DASIA\n1F2A; C; 1F22; # GREEK CAPITAL LETTER ETA WITH PSILI AND VARIA\n1F2B; C; 1F23; # GREEK CAPITAL LETTER ETA WITH DASIA AND VARIA\n1F2C; C; 1F24; # GREEK CAPITAL LETTER ETA WITH PSILI AND OXIA\n1F2D; C; 1F25; # GREEK CAPITAL LETTER ETA WITH DASIA AND OXIA\n1F2E; C; 1F26; # GREEK CAPITAL LETTER ETA WITH PSILI AND PERISPOMENI\n1F2F; C; 1F27; # GREEK CAPITAL LETTER ETA WITH DASIA AND PERISPOMENI\n1F38; C; 1F30; # GREEK CAPITAL LETTER IOTA WITH PSILI\n1F39; C; 1F31; # GREEK CAPITAL LETTER IOTA WITH DASIA\n1F3A; C; 1F32; # GREEK CAPITAL LETTER IOTA WITH PSILI AND VARIA\n1F3B; C; 1F33; # GREEK CAPITAL LETTER IOTA WITH DASIA AND VARIA\n1F3C; C; 1F34; # GREEK CAPITAL LETTER IOTA WITH PSILI AND OXIA\n1F3D; C; 1F35; # GREEK CAPITAL LETTER IOTA WITH DASIA AND OXIA\n1F3E; C; 1F36; # GREEK CAPITAL LETTER IOTA WITH PSILI AND PERISPOMENI\n1F3F; C; 1F37; # GREEK CAPITAL LETTER IOTA WITH DASIA AND PERISPOMENI\n1F48; C; 1F40; # GREEK CAPITAL LETTER OMICRON WITH PSILI\n1F49; C; 1F41; # GREEK CAPITAL LETTER OMICRON WITH DASIA\n1F4A; C; 1F42; # GREEK CAPITAL LETTER OMICRON WITH PSILI AND VARIA\n1F4B; C; 1F43; # GREEK CAPITAL LETTER OMICRON WITH DASIA AND VARIA\n1F4C; C; 1F44; # GREEK CAPITAL LETTER OMICRON WITH PSILI AND OXIA\n1F4D; C; 1F45; # GREEK CAPITAL LETTER OMICRON WITH DASIA AND OXIA\n1F50; F; 03C5 0313; # GREEK SMALL LETTER UPSILON WITH PSILI\n1F52; F; 03C5 0313 0300; # GREEK SMALL LETTER UPSILON WITH PSILI AND VARIA\n1F54; F; 03C5 0313 0301; # GREEK SMALL LETTER UPSILON WITH PSILI AND OXIA\n1F56; F; 03C5 0313 0342; # GREEK SMALL LETTER UPSILON WITH PSILI AND PERISPOMENI\n1F59; C; 1F51; # GREEK CAPITAL LETTER UPSILON WITH DASIA\n1F5B; C; 1F53; # GREEK CAPITAL LETTER UPSILON WITH DASIA AND VARIA\n1F5D; C; 1F55; # GREEK CAPITAL LETTER UPSILON WITH DASIA AND OXIA\n1F5F; C; 1F57; # GREEK CAPITAL LETTER UPSILON WITH DASIA AND PERISPOMENI\n1F68; C; 1F60; # GREEK CAPITAL LETTER OMEGA WITH PSILI\n1F69; C; 1F61; # GREEK CAPITAL LETTER OMEGA WITH DASIA\n1F6A; C; 1F62; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND VARIA\n1F6B; C; 1F63; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND VARIA\n1F6C; C; 1F64; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND OXIA\n1F6D; C; 1F65; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND OXIA\n1F6E; C; 1F66; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI\n1F6F; C; 1F67; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PERISPOMENI\n1F80; F; 1F00 03B9; # GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI\n1F81; F; 1F01 03B9; # GREEK SMALL LETTER ALPHA WITH DASIA AND YPOGEGRAMMENI\n1F82; F; 1F02 03B9; # GREEK SMALL LETTER ALPHA WITH PSILI AND VARIA AND YPOGEGRAMMENI\n1F83; F; 1F03 03B9; # GREEK SMALL LETTER ALPHA WITH DASIA AND VARIA AND YPOGEGRAMMENI\n1F84; F; 1F04 03B9; # GREEK SMALL LETTER ALPHA WITH PSILI AND OXIA AND YPOGEGRAMMENI\n1F85; F; 1F05 03B9; # GREEK SMALL LETTER ALPHA WITH DASIA AND OXIA AND YPOGEGRAMMENI\n1F86; F; 1F06 03B9; # GREEK SMALL LETTER ALPHA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI\n1F87; F; 1F07 03B9; # GREEK SMALL LETTER ALPHA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI\n1F88; F; 1F00 03B9; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PROSGEGRAMMENI\n1F88; S; 1F80; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PROSGEGRAMMENI\n1F89; F; 1F01 03B9; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PROSGEGRAMMENI\n1F89; S; 1F81; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PROSGEGRAMMENI\n1F8A; F; 1F02 03B9; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1F8A; S; 1F82; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1F8B; F; 1F03 03B9; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1F8B; S; 1F83; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1F8C; F; 1F04 03B9; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1F8C; S; 1F84; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1F8D; F; 1F05 03B9; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1F8D; S; 1F85; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1F8E; F; 1F06 03B9; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1F8E; S; 1F86; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1F8F; F; 1F07 03B9; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1F8F; S; 1F87; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1F90; F; 1F20 03B9; # GREEK SMALL LETTER ETA WITH PSILI AND YPOGEGRAMMENI\n1F91; F; 1F21 03B9; # GREEK SMALL LETTER ETA WITH DASIA AND YPOGEGRAMMENI\n1F92; F; 1F22 03B9; # GREEK SMALL LETTER ETA WITH PSILI AND VARIA AND YPOGEGRAMMENI\n1F93; F; 1F23 03B9; # GREEK SMALL LETTER ETA WITH DASIA AND VARIA AND YPOGEGRAMMENI\n1F94; F; 1F24 03B9; # GREEK SMALL LETTER ETA WITH PSILI AND OXIA AND YPOGEGRAMMENI\n1F95; F; 1F25 03B9; # GREEK SMALL LETTER ETA WITH DASIA AND OXIA AND YPOGEGRAMMENI\n1F96; F; 1F26 03B9; # GREEK SMALL LETTER ETA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI\n1F97; F; 1F27 03B9; # GREEK SMALL LETTER ETA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI\n1F98; F; 1F20 03B9; # GREEK CAPITAL LETTER ETA WITH PSILI AND PROSGEGRAMMENI\n1F98; S; 1F90; # GREEK CAPITAL LETTER ETA WITH PSILI AND PROSGEGRAMMENI\n1F99; F; 1F21 03B9; # GREEK CAPITAL LETTER ETA WITH DASIA AND PROSGEGRAMMENI\n1F99; S; 1F91; # GREEK CAPITAL LETTER ETA WITH DASIA AND PROSGEGRAMMENI\n1F9A; F; 1F22 03B9; # GREEK CAPITAL LETTER ETA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1F9A; S; 1F92; # GREEK CAPITAL LETTER ETA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1F9B; F; 1F23 03B9; # GREEK CAPITAL LETTER ETA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1F9B; S; 1F93; # GREEK CAPITAL LETTER ETA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1F9C; F; 1F24 03B9; # GREEK CAPITAL LETTER ETA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1F9C; S; 1F94; # GREEK CAPITAL LETTER ETA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1F9D; F; 1F25 03B9; # GREEK CAPITAL LETTER ETA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1F9D; S; 1F95; # GREEK CAPITAL LETTER ETA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1F9E; F; 1F26 03B9; # GREEK CAPITAL LETTER ETA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1F9E; S; 1F96; # GREEK CAPITAL LETTER ETA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1F9F; F; 1F27 03B9; # GREEK CAPITAL LETTER ETA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1F9F; S; 1F97; # GREEK CAPITAL LETTER ETA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1FA0; F; 1F60 03B9; # GREEK SMALL LETTER OMEGA WITH PSILI AND YPOGEGRAMMENI\n1FA1; F; 1F61 03B9; # GREEK SMALL LETTER OMEGA WITH DASIA AND YPOGEGRAMMENI\n1FA2; F; 1F62 03B9; # GREEK SMALL LETTER OMEGA WITH PSILI AND VARIA AND YPOGEGRAMMENI\n1FA3; F; 1F63 03B9; # GREEK SMALL LETTER OMEGA WITH DASIA AND VARIA AND YPOGEGRAMMENI\n1FA4; F; 1F64 03B9; # GREEK SMALL LETTER OMEGA WITH PSILI AND OXIA AND YPOGEGRAMMENI\n1FA5; F; 1F65 03B9; # GREEK SMALL LETTER OMEGA WITH DASIA AND OXIA AND YPOGEGRAMMENI\n1FA6; F; 1F66 03B9; # GREEK SMALL LETTER OMEGA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI\n1FA7; F; 1F67 03B9; # GREEK SMALL LETTER OMEGA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI\n1FA8; F; 1F60 03B9; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PROSGEGRAMMENI\n1FA8; S; 1FA0; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PROSGEGRAMMENI\n1FA9; F; 1F61 03B9; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PROSGEGRAMMENI\n1FA9; S; 1FA1; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PROSGEGRAMMENI\n1FAA; F; 1F62 03B9; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1FAA; S; 1FA2; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND VARIA AND PROSGEGRAMMENI\n1FAB; F; 1F63 03B9; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1FAB; S; 1FA3; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND VARIA AND PROSGEGRAMMENI\n1FAC; F; 1F64 03B9; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1FAC; S; 1FA4; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND OXIA AND PROSGEGRAMMENI\n1FAD; F; 1F65 03B9; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1FAD; S; 1FA5; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND OXIA AND PROSGEGRAMMENI\n1FAE; F; 1F66 03B9; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1FAE; S; 1FA6; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI\n1FAF; F; 1F67 03B9; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1FAF; S; 1FA7; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI\n1FB2; F; 1F70 03B9; # GREEK SMALL LETTER ALPHA WITH VARIA AND YPOGEGRAMMENI\n1FB3; F; 03B1 03B9; # GREEK SMALL LETTER ALPHA WITH YPOGEGRAMMENI\n1FB4; F; 03AC 03B9; # GREEK SMALL LETTER ALPHA WITH OXIA AND YPOGEGRAMMENI\n1FB6; F; 03B1 0342; # GREEK SMALL LETTER ALPHA WITH PERISPOMENI\n1FB7; F; 03B1 0342 03B9; # GREEK SMALL LETTER ALPHA WITH PERISPOMENI AND YPOGEGRAMMENI\n1FB8; C; 1FB0; # GREEK CAPITAL LETTER ALPHA WITH VRACHY\n1FB9; C; 1FB1; # GREEK CAPITAL LETTER ALPHA WITH MACRON\n1FBA; C; 1F70; # GREEK CAPITAL LETTER ALPHA WITH VARIA\n1FBB; C; 1F71; # GREEK CAPITAL LETTER ALPHA WITH OXIA\n1FBC; F; 03B1 03B9; # GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI\n1FBC; S; 1FB3; # GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI\n1FBE; C; 03B9; # GREEK PROSGEGRAMMENI\n1FC2; F; 1F74 03B9; # GREEK SMALL LETTER ETA WITH VARIA AND YPOGEGRAMMENI\n1FC3; F; 03B7 03B9; # GREEK SMALL LETTER ETA WITH YPOGEGRAMMENI\n1FC4; F; 03AE 03B9; # GREEK SMALL LETTER ETA WITH OXIA AND YPOGEGRAMMENI\n1FC6; F; 03B7 0342; # GREEK SMALL LETTER ETA WITH PERISPOMENI\n1FC7; F; 03B7 0342 03B9; # GREEK SMALL LETTER ETA WITH PERISPOMENI AND YPOGEGRAMMENI\n1FC8; C; 1F72; # GREEK CAPITAL LETTER EPSILON WITH VARIA\n1FC9; C; 1F73; # GREEK CAPITAL LETTER EPSILON WITH OXIA\n1FCA; C; 1F74; # GREEK CAPITAL LETTER ETA WITH VARIA\n1FCB; C; 1F75; # GREEK CAPITAL LETTER ETA WITH OXIA\n1FCC; F; 03B7 03B9; # GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI\n1FCC; S; 1FC3; # GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI\n1FD2; F; 03B9 0308 0300; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND VARIA\n1FD3; F; 03B9 0308 0301; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND OXIA\n1FD6; F; 03B9 0342; # GREEK SMALL LETTER IOTA WITH PERISPOMENI\n1FD7; F; 03B9 0308 0342; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND PERISPOMENI\n1FD8; C; 1FD0; # GREEK CAPITAL LETTER IOTA WITH VRACHY\n1FD9; C; 1FD1; # GREEK CAPITAL LETTER IOTA WITH MACRON\n1FDA; C; 1F76; # GREEK CAPITAL LETTER IOTA WITH VARIA\n1FDB; C; 1F77; # GREEK CAPITAL LETTER IOTA WITH OXIA\n1FE2; F; 03C5 0308 0300; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND VARIA\n1FE3; F; 03C5 0308 0301; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND OXIA\n1FE4; F; 03C1 0313; # GREEK SMALL LETTER RHO WITH PSILI\n1FE6; F; 03C5 0342; # GREEK SMALL LETTER UPSILON WITH PERISPOMENI\n1FE7; F; 03C5 0308 0342; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND PERISPOMENI\n1FE8; C; 1FE0; # GREEK CAPITAL LETTER UPSILON WITH VRACHY\n1FE9; C; 1FE1; # GREEK CAPITAL LETTER UPSILON WITH MACRON\n1FEA; C; 1F7A; # GREEK CAPITAL LETTER UPSILON WITH VARIA\n1FEB; C; 1F7B; # GREEK CAPITAL LETTER UPSILON WITH OXIA\n1FEC; C; 1FE5; # GREEK CAPITAL LETTER RHO WITH DASIA\n1FF2; F; 1F7C 03B9; # GREEK SMALL LETTER OMEGA WITH VARIA AND YPOGEGRAMMENI\n1FF3; F; 03C9 03B9; # GREEK SMALL LETTER OMEGA WITH YPOGEGRAMMENI\n1FF4; F; 03CE 03B9; # GREEK SMALL LETTER OMEGA WITH OXIA AND YPOGEGRAMMENI\n1FF6; F; 03C9 0342; # GREEK SMALL LETTER OMEGA WITH PERISPOMENI\n1FF7; F; 03C9 0342 03B9; # GREEK SMALL LETTER OMEGA WITH PERISPOMENI AND YPOGEGRAMMENI\n1FF8; C; 1F78; # GREEK CAPITAL LETTER OMICRON WITH VARIA\n1FF9; C; 1F79; # GREEK CAPITAL LETTER OMICRON WITH OXIA\n1FFA; C; 1F7C; # GREEK CAPITAL LETTER OMEGA WITH VARIA\n1FFB; C; 1F7D; # GREEK CAPITAL LETTER OMEGA WITH OXIA\n1FFC; F; 03C9 03B9; # GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI\n1FFC; S; 1FF3; # GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI\n2126; C; 03C9; # OHM SIGN\n212A; C; 006B; # KELVIN SIGN\n212B; C; 00E5; # ANGSTROM SIGN\n2132; C; 214E; # TURNED CAPITAL F\n2160; C; 2170; # ROMAN NUMERAL ONE\n2161; C; 2171; # ROMAN NUMERAL TWO\n2162; C; 2172; # ROMAN NUMERAL THREE\n2163; C; 2173; # ROMAN NUMERAL FOUR\n2164; C; 2174; # ROMAN NUMERAL FIVE\n2165; C; 2175; # ROMAN NUMERAL SIX\n2166; C; 2176; # ROMAN NUMERAL SEVEN\n2167; C; 2177; # ROMAN NUMERAL EIGHT\n2168; C; 2178; # ROMAN NUMERAL NINE\n2169; C; 2179; # ROMAN NUMERAL TEN\n216A; C; 217A; # ROMAN NUMERAL ELEVEN\n216B; C; 217B; # ROMAN NUMERAL TWELVE\n216C; C; 217C; # ROMAN NUMERAL FIFTY\n216D; C; 217D; # ROMAN NUMERAL ONE HUNDRED\n216E; C; 217E; # ROMAN NUMERAL FIVE HUNDRED\n216F; C; 217F; # ROMAN NUMERAL ONE THOUSAND\n2183; C; 2184; # ROMAN NUMERAL REVERSED ONE HUNDRED\n24B6; C; 24D0; # CIRCLED LATIN CAPITAL LETTER A\n24B7; C; 24D1; # CIRCLED LATIN CAPITAL LETTER B\n24B8; C; 24D2; # CIRCLED LATIN CAPITAL LETTER C\n24B9; C; 24D3; # CIRCLED LATIN CAPITAL LETTER D\n24BA; C; 24D4; # CIRCLED LATIN CAPITAL LETTER E\n24BB; C; 24D5; # CIRCLED LATIN CAPITAL LETTER F\n24BC; C; 24D6; # CIRCLED LATIN CAPITAL LETTER G\n24BD; C; 24D7; # CIRCLED LATIN CAPITAL LETTER H\n24BE; C; 24D8; # CIRCLED LATIN CAPITAL LETTER I\n24BF; C; 24D9; # CIRCLED LATIN CAPITAL LETTER J\n24C0; C; 24DA; # CIRCLED LATIN CAPITAL LETTER K\n24C1; C; 24DB; # CIRCLED LATIN CAPITAL LETTER L\n24C2; C; 24DC; # CIRCLED LATIN CAPITAL LETTER M\n24C3; C; 24DD; # CIRCLED LATIN CAPITAL LETTER N\n24C4; C; 24DE; # CIRCLED LATIN CAPITAL LETTER O\n24C5; C; 24DF; # CIRCLED LATIN CAPITAL LETTER P\n24C6; C; 24E0; # CIRCLED LATIN CAPITAL LETTER Q\n24C7; C; 24E1; # CIRCLED LATIN CAPITAL LETTER R\n24C8; C; 24E2; # CIRCLED LATIN CAPITAL LETTER S\n24C9; C; 24E3; # CIRCLED LATIN CAPITAL LETTER T\n24CA; C; 24E4; # CIRCLED LATIN CAPITAL LETTER U\n24CB; C; 24E5; # CIRCLED LATIN CAPITAL LETTER V\n24CC; C; 24E6; # CIRCLED LATIN CAPITAL LETTER W\n24CD; C; 24E7; # CIRCLED LATIN CAPITAL LETTER X\n24CE; C; 24E8; # CIRCLED LATIN CAPITAL LETTER Y\n24CF; C; 24E9; # CIRCLED LATIN CAPITAL LETTER Z\n2C00; C; 2C30; # GLAGOLITIC CAPITAL LETTER AZU\n2C01; C; 2C31; # GLAGOLITIC CAPITAL LETTER BUKY\n2C02; C; 2C32; # GLAGOLITIC CAPITAL LETTER VEDE\n2C03; C; 2C33; # GLAGOLITIC CAPITAL LETTER GLAGOLI\n2C04; C; 2C34; # GLAGOLITIC CAPITAL LETTER DOBRO\n2C05; C; 2C35; # GLAGOLITIC CAPITAL LETTER YESTU\n2C06; C; 2C36; # GLAGOLITIC CAPITAL LETTER ZHIVETE\n2C07; C; 2C37; # GLAGOLITIC CAPITAL LETTER DZELO\n2C08; C; 2C38; # GLAGOLITIC CAPITAL LETTER ZEMLJA\n2C09; C; 2C39; # GLAGOLITIC CAPITAL LETTER IZHE\n2C0A; C; 2C3A; # GLAGOLITIC CAPITAL LETTER INITIAL IZHE\n2C0B; C; 2C3B; # GLAGOLITIC CAPITAL LETTER I\n2C0C; C; 2C3C; # GLAGOLITIC CAPITAL LETTER DJERVI\n2C0D; C; 2C3D; # GLAGOLITIC CAPITAL LETTER KAKO\n2C0E; C; 2C3E; # GLAGOLITIC CAPITAL LETTER LJUDIJE\n2C0F; C; 2C3F; # GLAGOLITIC CAPITAL LETTER MYSLITE\n2C10; C; 2C40; # GLAGOLITIC CAPITAL LETTER NASHI\n2C11; C; 2C41; # GLAGOLITIC CAPITAL LETTER ONU\n2C12; C; 2C42; # GLAGOLITIC CAPITAL LETTER POKOJI\n2C13; C; 2C43; # GLAGOLITIC CAPITAL LETTER RITSI\n2C14; C; 2C44; # GLAGOLITIC CAPITAL LETTER SLOVO\n2C15; C; 2C45; # GLAGOLITIC CAPITAL LETTER TVRIDO\n2C16; C; 2C46; # GLAGOLITIC CAPITAL LETTER UKU\n2C17; C; 2C47; # GLAGOLITIC CAPITAL LETTER FRITU\n2C18; C; 2C48; # GLAGOLITIC CAPITAL LETTER HERU\n2C19; C; 2C49; # GLAGOLITIC CAPITAL LETTER OTU\n2C1A; C; 2C4A; # GLAGOLITIC CAPITAL LETTER PE\n2C1B; C; 2C4B; # GLAGOLITIC CAPITAL LETTER SHTA\n2C1C; C; 2C4C; # GLAGOLITIC CAPITAL LETTER TSI\n2C1D; C; 2C4D; # GLAGOLITIC CAPITAL LETTER CHRIVI\n2C1E; C; 2C4E; # GLAGOLITIC CAPITAL LETTER SHA\n2C1F; C; 2C4F; # GLAGOLITIC CAPITAL LETTER YERU\n2C20; C; 2C50; # GLAGOLITIC CAPITAL LETTER YERI\n2C21; C; 2C51; # GLAGOLITIC CAPITAL LETTER YATI\n2C22; C; 2C52; # GLAGOLITIC CAPITAL LETTER SPIDERY HA\n2C23; C; 2C53; # GLAGOLITIC CAPITAL LETTER YU\n2C24; C; 2C54; # GLAGOLITIC CAPITAL LETTER SMALL YUS\n2C25; C; 2C55; # GLAGOLITIC CAPITAL LETTER SMALL YUS WITH TAIL\n2C26; C; 2C56; # GLAGOLITIC CAPITAL LETTER YO\n2C27; C; 2C57; # GLAGOLITIC CAPITAL LETTER IOTATED SMALL YUS\n2C28; C; 2C58; # GLAGOLITIC CAPITAL LETTER BIG YUS\n2C29; C; 2C59; # GLAGOLITIC CAPITAL LETTER IOTATED BIG YUS\n2C2A; C; 2C5A; # GLAGOLITIC CAPITAL LETTER FITA\n2C2B; C; 2C5B; # GLAGOLITIC CAPITAL LETTER IZHITSA\n2C2C; C; 2C5C; # GLAGOLITIC CAPITAL LETTER SHTAPIC\n2C2D; C; 2C5D; # GLAGOLITIC CAPITAL LETTER TROKUTASTI A\n2C2E; C; 2C5E; # GLAGOLITIC CAPITAL LETTER LATINATE MYSLITE\n2C60; C; 2C61; # LATIN CAPITAL LETTER L WITH DOUBLE BAR\n2C62; C; 026B; # LATIN CAPITAL LETTER L WITH MIDDLE TILDE\n2C63; C; 1D7D; # LATIN CAPITAL LETTER P WITH STROKE\n2C64; C; 027D; # LATIN CAPITAL LETTER R WITH TAIL\n2C67; C; 2C68; # LATIN CAPITAL LETTER H WITH DESCENDER\n2C69; C; 2C6A; # LATIN CAPITAL LETTER K WITH DESCENDER\n2C6B; C; 2C6C; # LATIN CAPITAL LETTER Z WITH DESCENDER\n2C6D; C; 0251; # LATIN CAPITAL LETTER ALPHA\n2C6E; C; 0271; # LATIN CAPITAL LETTER M WITH HOOK\n2C6F; C; 0250; # LATIN CAPITAL LETTER TURNED A\n2C70; C; 0252; # LATIN CAPITAL LETTER TURNED ALPHA\n2C72; C; 2C73; # LATIN CAPITAL LETTER W WITH HOOK\n2C75; C; 2C76; # LATIN CAPITAL LETTER HALF H\n2C7E; C; 023F; # LATIN CAPITAL LETTER S WITH SWASH TAIL\n2C7F; C; 0240; # LATIN CAPITAL LETTER Z WITH SWASH TAIL\n2C80; C; 2C81; # COPTIC CAPITAL LETTER ALFA\n2C82; C; 2C83; # COPTIC CAPITAL LETTER VIDA\n2C84; C; 2C85; # COPTIC CAPITAL LETTER GAMMA\n2C86; C; 2C87; # COPTIC CAPITAL LETTER DALDA\n2C88; C; 2C89; # COPTIC CAPITAL LETTER EIE\n2C8A; C; 2C8B; # COPTIC CAPITAL LETTER SOU\n2C8C; C; 2C8D; # COPTIC CAPITAL LETTER ZATA\n2C8E; C; 2C8F; # COPTIC CAPITAL LETTER HATE\n2C90; C; 2C91; # COPTIC CAPITAL LETTER THETHE\n2C92; C; 2C93; # COPTIC CAPITAL LETTER IAUDA\n2C94; C; 2C95; # COPTIC CAPITAL LETTER KAPA\n2C96; C; 2C97; # COPTIC CAPITAL LETTER LAULA\n2C98; C; 2C99; # COPTIC CAPITAL LETTER MI\n2C9A; C; 2C9B; # COPTIC CAPITAL LETTER NI\n2C9C; C; 2C9D; # COPTIC CAPITAL LETTER KSI\n2C9E; C; 2C9F; # COPTIC CAPITAL LETTER O\n2CA0; C; 2CA1; # COPTIC CAPITAL LETTER PI\n2CA2; C; 2CA3; # COPTIC CAPITAL LETTER RO\n2CA4; C; 2CA5; # COPTIC CAPITAL LETTER SIMA\n2CA6; C; 2CA7; # COPTIC CAPITAL LETTER TAU\n2CA8; C; 2CA9; # COPTIC CAPITAL LETTER UA\n2CAA; C; 2CAB; # COPTIC CAPITAL LETTER FI\n2CAC; C; 2CAD; # COPTIC CAPITAL LETTER KHI\n2CAE; C; 2CAF; # COPTIC CAPITAL LETTER PSI\n2CB0; C; 2CB1; # COPTIC CAPITAL LETTER OOU\n2CB2; C; 2CB3; # COPTIC CAPITAL LETTER DIALECT-P ALEF\n2CB4; C; 2CB5; # COPTIC CAPITAL LETTER OLD COPTIC AIN\n2CB6; C; 2CB7; # COPTIC CAPITAL LETTER CRYPTOGRAMMIC EIE\n2CB8; C; 2CB9; # COPTIC CAPITAL LETTER DIALECT-P KAPA\n2CBA; C; 2CBB; # COPTIC CAPITAL LETTER DIALECT-P NI\n2CBC; C; 2CBD; # COPTIC CAPITAL LETTER CRYPTOGRAMMIC NI\n2CBE; C; 2CBF; # COPTIC CAPITAL LETTER OLD COPTIC OOU\n2CC0; C; 2CC1; # COPTIC CAPITAL LETTER SAMPI\n2CC2; C; 2CC3; # COPTIC CAPITAL LETTER CROSSED SHEI\n2CC4; C; 2CC5; # COPTIC CAPITAL LETTER OLD COPTIC SHEI\n2CC6; C; 2CC7; # COPTIC CAPITAL LETTER OLD COPTIC ESH\n2CC8; C; 2CC9; # COPTIC CAPITAL LETTER AKHMIMIC KHEI\n2CCA; C; 2CCB; # COPTIC CAPITAL LETTER DIALECT-P HORI\n2CCC; C; 2CCD; # COPTIC CAPITAL LETTER OLD COPTIC HORI\n2CCE; C; 2CCF; # COPTIC CAPITAL LETTER OLD COPTIC HA\n2CD0; C; 2CD1; # COPTIC CAPITAL LETTER L-SHAPED HA\n2CD2; C; 2CD3; # COPTIC CAPITAL LETTER OLD COPTIC HEI\n2CD4; C; 2CD5; # COPTIC CAPITAL LETTER OLD COPTIC HAT\n2CD6; C; 2CD7; # COPTIC CAPITAL LETTER OLD COPTIC GANGIA\n2CD8; C; 2CD9; # COPTIC CAPITAL LETTER OLD COPTIC DJA\n2CDA; C; 2CDB; # COPTIC CAPITAL LETTER OLD COPTIC SHIMA\n2CDC; C; 2CDD; # COPTIC CAPITAL LETTER OLD NUBIAN SHIMA\n2CDE; C; 2CDF; # COPTIC CAPITAL LETTER OLD NUBIAN NGI\n2CE0; C; 2CE1; # COPTIC CAPITAL LETTER OLD NUBIAN NYI\n2CE2; C; 2CE3; # COPTIC CAPITAL LETTER OLD NUBIAN WAU\n2CEB; C; 2CEC; # COPTIC CAPITAL LETTER CRYPTOGRAMMIC SHEI\n2CED; C; 2CEE; # COPTIC CAPITAL LETTER CRYPTOGRAMMIC GANGIA\n2CF2; C; 2CF3; # COPTIC CAPITAL LETTER BOHAIRIC KHEI\nA640; C; A641; # CYRILLIC CAPITAL LETTER ZEMLYA\nA642; C; A643; # CYRILLIC CAPITAL LETTER DZELO\nA644; C; A645; # CYRILLIC CAPITAL LETTER REVERSED DZE\nA646; C; A647; # CYRILLIC CAPITAL LETTER IOTA\nA648; C; A649; # CYRILLIC CAPITAL LETTER DJERV\nA64A; C; A64B; # CYRILLIC CAPITAL LETTER MONOGRAPH UK\nA64C; C; A64D; # CYRILLIC CAPITAL LETTER BROAD OMEGA\nA64E; C; A64F; # CYRILLIC CAPITAL LETTER NEUTRAL YER\nA650; C; A651; # CYRILLIC CAPITAL LETTER YERU WITH BACK YER\nA652; C; A653; # CYRILLIC CAPITAL LETTER IOTIFIED YAT\nA654; C; A655; # CYRILLIC CAPITAL LETTER REVERSED YU\nA656; C; A657; # CYRILLIC CAPITAL LETTER IOTIFIED A\nA658; C; A659; # CYRILLIC CAPITAL LETTER CLOSED LITTLE YUS\nA65A; C; A65B; # CYRILLIC CAPITAL LETTER BLENDED YUS\nA65C; C; A65D; # CYRILLIC CAPITAL LETTER IOTIFIED CLOSED LITTLE YUS\nA65E; C; A65F; # CYRILLIC CAPITAL LETTER YN\nA660; C; A661; # CYRILLIC CAPITAL LETTER REVERSED TSE\nA662; C; A663; # CYRILLIC CAPITAL LETTER SOFT DE\nA664; C; A665; # CYRILLIC CAPITAL LETTER SOFT EL\nA666; C; A667; # CYRILLIC CAPITAL LETTER SOFT EM\nA668; C; A669; # CYRILLIC CAPITAL LETTER MONOCULAR O\nA66A; C; A66B; # CYRILLIC CAPITAL LETTER BINOCULAR O\nA66C; C; A66D; # CYRILLIC CAPITAL LETTER DOUBLE MONOCULAR O\nA680; C; A681; # CYRILLIC CAPITAL LETTER DWE\nA682; C; A683; # CYRILLIC CAPITAL LETTER DZWE\nA684; C; A685; # CYRILLIC CAPITAL LETTER ZHWE\nA686; C; A687; # CYRILLIC CAPITAL LETTER CCHE\nA688; C; A689; # CYRILLIC CAPITAL LETTER DZZE\nA68A; C; A68B; # CYRILLIC CAPITAL LETTER TE WITH MIDDLE HOOK\nA68C; C; A68D; # CYRILLIC CAPITAL LETTER TWE\nA68E; C; A68F; # CYRILLIC CAPITAL LETTER TSWE\nA690; C; A691; # CYRILLIC CAPITAL LETTER TSSE\nA692; C; A693; # CYRILLIC CAPITAL LETTER TCHE\nA694; C; A695; # CYRILLIC CAPITAL LETTER HWE\nA696; C; A697; # CYRILLIC CAPITAL LETTER SHWE\nA698; C; A699; # CYRILLIC CAPITAL LETTER DOUBLE O\nA69A; C; A69B; # CYRILLIC CAPITAL LETTER CROSSED O\nA722; C; A723; # LATIN CAPITAL LETTER EGYPTOLOGICAL ALEF\nA724; C; A725; # LATIN CAPITAL LETTER EGYPTOLOGICAL AIN\nA726; C; A727; # LATIN CAPITAL LETTER HENG\nA728; C; A729; # LATIN CAPITAL LETTER TZ\nA72A; C; A72B; # LATIN CAPITAL LETTER TRESILLO\nA72C; C; A72D; # LATIN CAPITAL LETTER CUATRILLO\nA72E; C; A72F; # LATIN CAPITAL LETTER CUATRILLO WITH COMMA\nA732; C; A733; # LATIN CAPITAL LETTER AA\nA734; C; A735; # LATIN CAPITAL LETTER AO\nA736; C; A737; # LATIN CAPITAL LETTER AU\nA738; C; A739; # LATIN CAPITAL LETTER AV\nA73A; C; A73B; # LATIN CAPITAL LETTER AV WITH HORIZONTAL BAR\nA73C; C; A73D; # LATIN CAPITAL LETTER AY\nA73E; C; A73F; # LATIN CAPITAL LETTER REVERSED C WITH DOT\nA740; C; A741; # LATIN CAPITAL LETTER K WITH STROKE\nA742; C; A743; # LATIN CAPITAL LETTER K WITH DIAGONAL STROKE\nA744; C; A745; # LATIN CAPITAL LETTER K WITH STROKE AND DIAGONAL STROKE\nA746; C; A747; # LATIN CAPITAL LETTER BROKEN L\nA748; C; A749; # LATIN CAPITAL LETTER L WITH HIGH STROKE\nA74A; C; A74B; # LATIN CAPITAL LETTER O WITH LONG STROKE OVERLAY\nA74C; C; A74D; # LATIN CAPITAL LETTER O WITH LOOP\nA74E; C; A74F; # LATIN CAPITAL LETTER OO\nA750; C; A751; # LATIN CAPITAL LETTER P WITH STROKE THROUGH DESCENDER\nA752; C; A753; # LATIN CAPITAL LETTER P WITH FLOURISH\nA754; C; A755; # LATIN CAPITAL LETTER P WITH SQUIRREL TAIL\nA756; C; A757; # LATIN CAPITAL LETTER Q WITH STROKE THROUGH DESCENDER\nA758; C; A759; # LATIN CAPITAL LETTER Q WITH DIAGONAL STROKE\nA75A; C; A75B; # LATIN CAPITAL LETTER R ROTUNDA\nA75C; C; A75D; # LATIN CAPITAL LETTER RUM ROTUNDA\nA75E; C; A75F; # LATIN CAPITAL LETTER V WITH DIAGONAL STROKE\nA760; C; A761; # LATIN CAPITAL LETTER VY\nA762; C; A763; # LATIN CAPITAL LETTER VISIGOTHIC Z\nA764; C; A765; # LATIN CAPITAL LETTER THORN WITH STROKE\nA766; C; A767; # LATIN CAPITAL LETTER THORN WITH STROKE THROUGH DESCENDER\nA768; C; A769; # LATIN CAPITAL LETTER VEND\nA76A; C; A76B; # LATIN CAPITAL LETTER ET\nA76C; C; A76D; # LATIN CAPITAL LETTER IS\nA76E; C; A76F; # LATIN CAPITAL LETTER CON\nA779; C; A77A; # LATIN CAPITAL LETTER INSULAR D\nA77B; C; A77C; # LATIN CAPITAL LETTER INSULAR F\nA77D; C; 1D79; # LATIN CAPITAL LETTER INSULAR G\nA77E; C; A77F; # LATIN CAPITAL LETTER TURNED INSULAR G\nA780; C; A781; # LATIN CAPITAL LETTER TURNED L\nA782; C; A783; # LATIN CAPITAL LETTER INSULAR R\nA784; C; A785; # LATIN CAPITAL LETTER INSULAR S\nA786; C; A787; # LATIN CAPITAL LETTER INSULAR T\nA78B; C; A78C; # LATIN CAPITAL LETTER SALTILLO\nA78D; C; 0265; # LATIN CAPITAL LETTER TURNED H\nA790; C; A791; # LATIN CAPITAL LETTER N WITH DESCENDER\nA792; C; A793; # LATIN CAPITAL LETTER C WITH BAR\nA796; C; A797; # LATIN CAPITAL LETTER B WITH FLOURISH\nA798; C; A799; # LATIN CAPITAL LETTER F WITH STROKE\nA79A; C; A79B; # LATIN CAPITAL LETTER VOLAPUK AE\nA79C; C; A79D; # LATIN CAPITAL LETTER VOLAPUK OE\nA79E; C; A79F; # LATIN CAPITAL LETTER VOLAPUK UE\nA7A0; C; A7A1; # LATIN CAPITAL LETTER G WITH OBLIQUE STROKE\nA7A2; C; A7A3; # LATIN CAPITAL LETTER K WITH OBLIQUE STROKE\nA7A4; C; A7A5; # LATIN CAPITAL LETTER N WITH OBLIQUE STROKE\nA7A6; C; A7A7; # LATIN CAPITAL LETTER R WITH OBLIQUE STROKE\nA7A8; C; A7A9; # LATIN CAPITAL LETTER S WITH OBLIQUE STROKE\nA7AA; C; 0266; # LATIN CAPITAL LETTER H WITH HOOK\nA7AB; C; 025C; # LATIN CAPITAL LETTER REVERSED OPEN E\nA7AC; C; 0261; # LATIN CAPITAL LETTER SCRIPT G\nA7AD; C; 026C; # LATIN CAPITAL LETTER L WITH BELT\nA7B0; C; 029E; # LATIN CAPITAL LETTER TURNED K\nA7B1; C; 0287; # LATIN CAPITAL LETTER TURNED T\nA7B2; C; 029D; # LATIN CAPITAL LETTER J WITH CROSSED-TAIL\nA7B3; C; AB53; # LATIN CAPITAL LETTER CHI\nA7B4; C; A7B5; # LATIN CAPITAL LETTER BETA\nA7B6; C; A7B7; # LATIN CAPITAL LETTER OMEGA\nAB70; C; 13A0; # CHEROKEE SMALL LETTER A\nAB71; C; 13A1; # CHEROKEE SMALL LETTER E\nAB72; C; 13A2; # CHEROKEE SMALL LETTER I\nAB73; C; 13A3; # CHEROKEE SMALL LETTER O\nAB74; C; 13A4; # CHEROKEE SMALL LETTER U\nAB75; C; 13A5; # CHEROKEE SMALL LETTER V\nAB76; C; 13A6; # CHEROKEE SMALL LETTER GA\nAB77; C; 13A7; # CHEROKEE SMALL LETTER KA\nAB78; C; 13A8; # CHEROKEE SMALL LETTER GE\nAB79; C; 13A9; # CHEROKEE SMALL LETTER GI\nAB7A; C; 13AA; # CHEROKEE SMALL LETTER GO\nAB7B; C; 13AB; # CHEROKEE SMALL LETTER GU\nAB7C; C; 13AC; # CHEROKEE SMALL LETTER GV\nAB7D; C; 13AD; # CHEROKEE SMALL LETTER HA\nAB7E; C; 13AE; # CHEROKEE SMALL LETTER HE\nAB7F; C; 13AF; # CHEROKEE SMALL LETTER HI\nAB80; C; 13B0; # CHEROKEE SMALL LETTER HO\nAB81; C; 13B1; # CHEROKEE SMALL LETTER HU\nAB82; C; 13B2; # CHEROKEE SMALL LETTER HV\nAB83; C; 13B3; # CHEROKEE SMALL LETTER LA\nAB84; C; 13B4; # CHEROKEE SMALL LETTER LE\nAB85; C; 13B5; # CHEROKEE SMALL LETTER LI\nAB86; C; 13B6; # CHEROKEE SMALL LETTER LO\nAB87; C; 13B7; # CHEROKEE SMALL LETTER LU\nAB88; C; 13B8; # CHEROKEE SMALL LETTER LV\nAB89; C; 13B9; # CHEROKEE SMALL LETTER MA\nAB8A; C; 13BA; # CHEROKEE SMALL LETTER ME\nAB8B; C; 13BB; # CHEROKEE SMALL LETTER MI\nAB8C; C; 13BC; # CHEROKEE SMALL LETTER MO\nAB8D; C; 13BD; # CHEROKEE SMALL LETTER MU\nAB8E; C; 13BE; # CHEROKEE SMALL LETTER NA\nAB8F; C; 13BF; # CHEROKEE SMALL LETTER HNA\nAB90; C; 13C0; # CHEROKEE SMALL LETTER NAH\nAB91; C; 13C1; # CHEROKEE SMALL LETTER NE\nAB92; C; 13C2; # CHEROKEE SMALL LETTER NI\nAB93; C; 13C3; # CHEROKEE SMALL LETTER NO\nAB94; C; 13C4; # CHEROKEE SMALL LETTER NU\nAB95; C; 13C5; # CHEROKEE SMALL LETTER NV\nAB96; C; 13C6; # CHEROKEE SMALL LETTER QUA\nAB97; C; 13C7; # CHEROKEE SMALL LETTER QUE\nAB98; C; 13C8; # CHEROKEE SMALL LETTER QUI\nAB99; C; 13C9; # CHEROKEE SMALL LETTER QUO\nAB9A; C; 13CA; # CHEROKEE SMALL LETTER QUU\nAB9B; C; 13CB; # CHEROKEE SMALL LETTER QUV\nAB9C; C; 13CC; # CHEROKEE SMALL LETTER SA\nAB9D; C; 13CD; # CHEROKEE SMALL LETTER S\nAB9E; C; 13CE; # CHEROKEE SMALL LETTER SE\nAB9F; C; 13CF; # CHEROKEE SMALL LETTER SI\nABA0; C; 13D0; # CHEROKEE SMALL LETTER SO\nABA1; C; 13D1; # CHEROKEE SMALL LETTER SU\nABA2; C; 13D2; # CHEROKEE SMALL LETTER SV\nABA3; C; 13D3; # CHEROKEE SMALL LETTER DA\nABA4; C; 13D4; # CHEROKEE SMALL LETTER TA\nABA5; C; 13D5; # CHEROKEE SMALL LETTER DE\nABA6; C; 13D6; # CHEROKEE SMALL LETTER TE\nABA7; C; 13D7; # CHEROKEE SMALL LETTER DI\nABA8; C; 13D8; # CHEROKEE SMALL LETTER TI\nABA9; C; 13D9; # CHEROKEE SMALL LETTER DO\nABAA; C; 13DA; # CHEROKEE SMALL LETTER DU\nABAB; C; 13DB; # CHEROKEE SMALL LETTER DV\nABAC; C; 13DC; # CHEROKEE SMALL LETTER DLA\nABAD; C; 13DD; # CHEROKEE SMALL LETTER TLA\nABAE; C; 13DE; # CHEROKEE SMALL LETTER TLE\nABAF; C; 13DF; # CHEROKEE SMALL LETTER TLI\nABB0; C; 13E0; # CHEROKEE SMALL LETTER TLO\nABB1; C; 13E1; # CHEROKEE SMALL LETTER TLU\nABB2; C; 13E2; # CHEROKEE SMALL LETTER TLV\nABB3; C; 13E3; # CHEROKEE SMALL LETTER TSA\nABB4; C; 13E4; # CHEROKEE SMALL LETTER TSE\nABB5; C; 13E5; # CHEROKEE SMALL LETTER TSI\nABB6; C; 13E6; # CHEROKEE SMALL LETTER TSO\nABB7; C; 13E7; # CHEROKEE SMALL LETTER TSU\nABB8; C; 13E8; # CHEROKEE SMALL LETTER TSV\nABB9; C; 13E9; # CHEROKEE SMALL LETTER WA\nABBA; C; 13EA; # CHEROKEE SMALL LETTER WE\nABBB; C; 13EB; # CHEROKEE SMALL LETTER WI\nABBC; C; 13EC; # CHEROKEE SMALL LETTER WO\nABBD; C; 13ED; # CHEROKEE SMALL LETTER WU\nABBE; C; 13EE; # CHEROKEE SMALL LETTER WV\nABBF; C; 13EF; # CHEROKEE SMALL LETTER YA\nFB00; F; 0066 0066; # LATIN SMALL LIGATURE FF\nFB01; F; 0066 0069; # LATIN SMALL LIGATURE FI\nFB02; F; 0066 006C; # LATIN SMALL LIGATURE FL\nFB03; F; 0066 0066 0069; # LATIN SMALL LIGATURE FFI\nFB04; F; 0066 0066 006C; # LATIN SMALL LIGATURE FFL\nFB05; F; 0073 0074; # LATIN SMALL LIGATURE LONG S T\nFB06; F; 0073 0074; # LATIN SMALL LIGATURE ST\nFB13; F; 0574 0576; # ARMENIAN SMALL LIGATURE MEN NOW\nFB14; F; 0574 0565; # ARMENIAN SMALL LIGATURE MEN ECH\nFB15; F; 0574 056B; # ARMENIAN SMALL LIGATURE MEN INI\nFB16; F; 057E 0576; # ARMENIAN SMALL LIGATURE VEW NOW\nFB17; F; 0574 056D; # ARMENIAN SMALL LIGATURE MEN XEH\nFF21; C; FF41; # FULLWIDTH LATIN CAPITAL LETTER A\nFF22; C; FF42; # FULLWIDTH LATIN CAPITAL LETTER B\nFF23; C; FF43; # FULLWIDTH LATIN CAPITAL LETTER C\nFF24; C; FF44; # FULLWIDTH LATIN CAPITAL LETTER D\nFF25; C; FF45; # FULLWIDTH LATIN CAPITAL LETTER E\nFF26; C; FF46; # FULLWIDTH LATIN CAPITAL LETTER F\nFF27; C; FF47; # FULLWIDTH LATIN CAPITAL LETTER G\nFF28; C; FF48; # FULLWIDTH LATIN CAPITAL LETTER H\nFF29; C; FF49; # FULLWIDTH LATIN CAPITAL LETTER I\nFF2A; C; FF4A; # FULLWIDTH LATIN CAPITAL LETTER J\nFF2B; C; FF4B; # FULLWIDTH LATIN CAPITAL LETTER K\nFF2C; C; FF4C; # FULLWIDTH LATIN CAPITAL LETTER L\nFF2D; C; FF4D; # FULLWIDTH LATIN CAPITAL LETTER M\nFF2E; C; FF4E; # FULLWIDTH LATIN CAPITAL LETTER N\nFF2F; C; FF4F; # FULLWIDTH LATIN CAPITAL LETTER O\nFF30; C; FF50; # FULLWIDTH LATIN CAPITAL LETTER P\nFF31; C; FF51; # FULLWIDTH LATIN CAPITAL LETTER Q\nFF32; C; FF52; # FULLWIDTH LATIN CAPITAL LETTER R\nFF33; C; FF53; # FULLWIDTH LATIN CAPITAL LETTER S\nFF34; C; FF54; # FULLWIDTH LATIN CAPITAL LETTER T\nFF35; C; FF55; # FULLWIDTH LATIN CAPITAL LETTER U\nFF36; C; FF56; # FULLWIDTH LATIN CAPITAL LETTER V\nFF37; C; FF57; # FULLWIDTH LATIN CAPITAL LETTER W\nFF38; C; FF58; # FULLWIDTH LATIN CAPITAL LETTER X\nFF39; C; FF59; # FULLWIDTH LATIN CAPITAL LETTER Y\nFF3A; C; FF5A; # FULLWIDTH LATIN CAPITAL LETTER Z\n10400; C; 10428; # DESERET CAPITAL LETTER LONG I\n10401; C; 10429; # DESERET CAPITAL LETTER LONG E\n10402; C; 1042A; # DESERET CAPITAL LETTER LONG A\n10403; C; 1042B; # DESERET CAPITAL LETTER LONG AH\n10404; C; 1042C; # DESERET CAPITAL LETTER LONG O\n10405; C; 1042D; # DESERET CAPITAL LETTER LONG OO\n10406; C; 1042E; # DESERET CAPITAL LETTER SHORT I\n10407; C; 1042F; # DESERET CAPITAL LETTER SHORT E\n10408; C; 10430; # DESERET CAPITAL LETTER SHORT A\n10409; C; 10431; # DESERET CAPITAL LETTER SHORT AH\n1040A; C; 10432; # DESERET CAPITAL LETTER SHORT O\n1040B; C; 10433; # DESERET CAPITAL LETTER SHORT OO\n1040C; C; 10434; # DESERET CAPITAL LETTER AY\n1040D; C; 10435; # DESERET CAPITAL LETTER OW\n1040E; C; 10436; # DESERET CAPITAL LETTER WU\n1040F; C; 10437; # DESERET CAPITAL LETTER YEE\n10410; C; 10438; # DESERET CAPITAL LETTER H\n10411; C; 10439; # DESERET CAPITAL LETTER PEE\n10412; C; 1043A; # DESERET CAPITAL LETTER BEE\n10413; C; 1043B; # DESERET CAPITAL LETTER TEE\n10414; C; 1043C; # DESERET CAPITAL LETTER DEE\n10415; C; 1043D; # DESERET CAPITAL LETTER CHEE\n10416; C; 1043E; # DESERET CAPITAL LETTER JEE\n10417; C; 1043F; # DESERET CAPITAL LETTER KAY\n10418; C; 10440; # DESERET CAPITAL LETTER GAY\n10419; C; 10441; # DESERET CAPITAL LETTER EF\n1041A; C; 10442; # DESERET CAPITAL LETTER VEE\n1041B; C; 10443; # DESERET CAPITAL LETTER ETH\n1041C; C; 10444; # DESERET CAPITAL LETTER THEE\n1041D; C; 10445; # DESERET CAPITAL LETTER ES\n1041E; C; 10446; # DESERET CAPITAL LETTER ZEE\n1041F; C; 10447; # DESERET CAPITAL LETTER ESH\n10420; C; 10448; # DESERET CAPITAL LETTER ZHEE\n10421; C; 10449; # DESERET CAPITAL LETTER ER\n10422; C; 1044A; # DESERET CAPITAL LETTER EL\n10423; C; 1044B; # DESERET CAPITAL LETTER EM\n10424; C; 1044C; # DESERET CAPITAL LETTER EN\n10425; C; 1044D; # DESERET CAPITAL LETTER ENG\n10426; C; 1044E; # DESERET CAPITAL LETTER OI\n10427; C; 1044F; # DESERET CAPITAL LETTER EW\n10C80; C; 10CC0; # OLD HUNGARIAN CAPITAL LETTER A\n10C81; C; 10CC1; # OLD HUNGARIAN CAPITAL LETTER AA\n10C82; C; 10CC2; # OLD HUNGARIAN CAPITAL LETTER EB\n10C83; C; 10CC3; # OLD HUNGARIAN CAPITAL LETTER AMB\n10C84; C; 10CC4; # OLD HUNGARIAN CAPITAL LETTER EC\n10C85; C; 10CC5; # OLD HUNGARIAN CAPITAL LETTER ENC\n10C86; C; 10CC6; # OLD HUNGARIAN CAPITAL LETTER ECS\n10C87; C; 10CC7; # OLD HUNGARIAN CAPITAL LETTER ED\n10C88; C; 10CC8; # OLD HUNGARIAN CAPITAL LETTER AND\n10C89; C; 10CC9; # OLD HUNGARIAN CAPITAL LETTER E\n10C8A; C; 10CCA; # OLD HUNGARIAN CAPITAL LETTER CLOSE E\n10C8B; C; 10CCB; # OLD HUNGARIAN CAPITAL LETTER EE\n10C8C; C; 10CCC; # OLD HUNGARIAN CAPITAL LETTER EF\n10C8D; C; 10CCD; # OLD HUNGARIAN CAPITAL LETTER EG\n10C8E; C; 10CCE; # OLD HUNGARIAN CAPITAL LETTER EGY\n10C8F; C; 10CCF; # OLD HUNGARIAN CAPITAL LETTER EH\n10C90; C; 10CD0; # OLD HUNGARIAN CAPITAL LETTER I\n10C91; C; 10CD1; # OLD HUNGARIAN CAPITAL LETTER II\n10C92; C; 10CD2; # OLD HUNGARIAN CAPITAL LETTER EJ\n10C93; C; 10CD3; # OLD HUNGARIAN CAPITAL LETTER EK\n10C94; C; 10CD4; # OLD HUNGARIAN CAPITAL LETTER AK\n10C95; C; 10CD5; # OLD HUNGARIAN CAPITAL LETTER UNK\n10C96; C; 10CD6; # OLD HUNGARIAN CAPITAL LETTER EL\n10C97; C; 10CD7; # OLD HUNGARIAN CAPITAL LETTER ELY\n10C98; C; 10CD8; # OLD HUNGARIAN CAPITAL LETTER EM\n10C99; C; 10CD9; # OLD HUNGARIAN CAPITAL LETTER EN\n10C9A; C; 10CDA; # OLD HUNGARIAN CAPITAL LETTER ENY\n10C9B; C; 10CDB; # OLD HUNGARIAN CAPITAL LETTER O\n10C9C; C; 10CDC; # OLD HUNGARIAN CAPITAL LETTER OO\n10C9D; C; 10CDD; # OLD HUNGARIAN CAPITAL LETTER NIKOLSBURG OE\n10C9E; C; 10CDE; # OLD HUNGARIAN CAPITAL LETTER RUDIMENTA OE\n10C9F; C; 10CDF; # OLD HUNGARIAN CAPITAL LETTER OEE\n10CA0; C; 10CE0; # OLD HUNGARIAN CAPITAL LETTER EP\n10CA1; C; 10CE1; # OLD HUNGARIAN CAPITAL LETTER EMP\n10CA2; C; 10CE2; # OLD HUNGARIAN CAPITAL LETTER ER\n10CA3; C; 10CE3; # OLD HUNGARIAN CAPITAL LETTER SHORT ER\n10CA4; C; 10CE4; # OLD HUNGARIAN CAPITAL LETTER ES\n10CA5; C; 10CE5; # OLD HUNGARIAN CAPITAL LETTER ESZ\n10CA6; C; 10CE6; # OLD HUNGARIAN CAPITAL LETTER ET\n10CA7; C; 10CE7; # OLD HUNGARIAN CAPITAL LETTER ENT\n10CA8; C; 10CE8; # OLD HUNGARIAN CAPITAL LETTER ETY\n10CA9; C; 10CE9; # OLD HUNGARIAN CAPITAL LETTER ECH\n10CAA; C; 10CEA; # OLD HUNGARIAN CAPITAL LETTER U\n10CAB; C; 10CEB; # OLD HUNGARIAN CAPITAL LETTER UU\n10CAC; C; 10CEC; # OLD HUNGARIAN CAPITAL LETTER NIKOLSBURG UE\n10CAD; C; 10CED; # OLD HUNGARIAN CAPITAL LETTER RUDIMENTA UE\n10CAE; C; 10CEE; # OLD HUNGARIAN CAPITAL LETTER EV\n10CAF; C; 10CEF; # OLD HUNGARIAN CAPITAL LETTER EZ\n10CB0; C; 10CF0; # OLD HUNGARIAN CAPITAL LETTER EZS\n10CB1; C; 10CF1; # OLD HUNGARIAN CAPITAL LETTER ENT-SHAPED SIGN\n10CB2; C; 10CF2; # OLD HUNGARIAN CAPITAL LETTER US\n118A0; C; 118C0; # WARANG CITI CAPITAL LETTER NGAA\n118A1; C; 118C1; # WARANG CITI CAPITAL LETTER A\n118A2; C; 118C2; # WARANG CITI CAPITAL LETTER WI\n118A3; C; 118C3; # WARANG CITI CAPITAL LETTER YU\n118A4; C; 118C4; # WARANG CITI CAPITAL LETTER YA\n118A5; C; 118C5; # WARANG CITI CAPITAL LETTER YO\n118A6; C; 118C6; # WARANG CITI CAPITAL LETTER II\n118A7; C; 118C7; # WARANG CITI CAPITAL LETTER UU\n118A8; C; 118C8; # WARANG CITI CAPITAL LETTER E\n118A9; C; 118C9; # WARANG CITI CAPITAL LETTER O\n118AA; C; 118CA; # WARANG CITI CAPITAL LETTER ANG\n118AB; C; 118CB; # WARANG CITI CAPITAL LETTER GA\n118AC; C; 118CC; # WARANG CITI CAPITAL LETTER KO\n118AD; C; 118CD; # WARANG CITI CAPITAL LETTER ENY\n118AE; C; 118CE; # WARANG CITI CAPITAL LETTER YUJ\n118AF; C; 118CF; # WARANG CITI CAPITAL LETTER UC\n118B0; C; 118D0; # WARANG CITI CAPITAL LETTER ENN\n118B1; C; 118D1; # WARANG CITI CAPITAL LETTER ODD\n118B2; C; 118D2; # WARANG CITI CAPITAL LETTER TTE\n118B3; C; 118D3; # WARANG CITI CAPITAL LETTER NUNG\n118B4; C; 118D4; # WARANG CITI CAPITAL LETTER DA\n118B5; C; 118D5; # WARANG CITI CAPITAL LETTER AT\n118B6; C; 118D6; # WARANG CITI CAPITAL LETTER AM\n118B7; C; 118D7; # WARANG CITI CAPITAL LETTER BU\n118B8; C; 118D8; # WARANG CITI CAPITAL LETTER PU\n118B9; C; 118D9; # WARANG CITI CAPITAL LETTER HIYO\n118BA; C; 118DA; # WARANG CITI CAPITAL LETTER HOLO\n118BB; C; 118DB; # WARANG CITI CAPITAL LETTER HORR\n118BC; C; 118DC; # WARANG CITI CAPITAL LETTER HAR\n118BD; C; 118DD; # WARANG CITI CAPITAL LETTER SSUU\n118BE; C; 118DE; # WARANG CITI CAPITAL LETTER SII\n118BF; C; 118DF; # WARANG CITI CAPITAL LETTER VIYO\n\"\n\ntype CharMapping =\n | CommonMapping of int\n | SimpleMapping of int\n | TurkishMapping of int\n | FullMapping1 of int\n | FullMapping2 of int*int\n | FullMapping3 of int*int*int\n\nlet hex2int c = (int c &&& 15) + (int c >>> 6)*9 // hex char to int\n\nlet pCodePoint =\n manyMinMaxSatisfyL 4 5 isHex \"codepoint with 4-5 hex digits\"\n |>> fun s ->\n let mutable n = 0\n for i = 0 to s.Length - 1 do\n n <- n*16 + hex2int s.[i]\n n\n\nlet semi = skipString \"; \"\nlet space = skipChar ' '\n\nlet pCharMapping =\n pipe3 pCodePoint (semi >>. anyChar) (semi >>. sepBy pCodePoint space .>> (semi >>. skipRestOfLine true))\n (fun fromChar c toChars ->\n match c with\n | 'C' -> match toChars with\n | [n0] -> (fromChar, CommonMapping n0)\n | 'S' -> match toChars with\n | [n0] -> (fromChar, SimpleMapping n0)\n | 'T' -> match toChars with\n | [n0] -> (fromChar, TurkishMapping n0)\n | 'F' -> match toChars with\n | [n0] -> (fromChar, FullMapping1(n0))\n | [n0; n1] -> (fromChar, FullMapping2(n0, n1))\n | [n0; n1; n2] -> (fromChar, FullMapping3(n0, n1, n2)))\n\nlet pAllMappings = many pCharMapping .>> eof\n\nlet parseMappings() =\n match run pAllMappings datastr with\n | Success(xs, _,_) -> xs\n | Failure(msg,_,_) -> failwith msg\n\nlet getOneToOneMappings() =\n parseMappings()\n |> List.choose (function (src, CommonMapping(dst))\n | (src, SimpleMapping(dst))\n when src < 0xffff\n -> Some (char src, char dst)\n | _ -> None)\n\nlet getOneToOneMappingsAsStrings() =\n let pairs = getOneToOneMappings()\n let sb = new System.Text.StringBuilder()\n for c1, c2 in pairs do\n let c1s, c2s = (int c1).ToString(\"X4\"), (int c2).ToString(\"X4\")\n sb.Append(\"\\u\").Append(c1s).Append(\"\\u\").Append(c2s) |> ignore\n sb.ToString()\n\n\nlet writeOneToOneMappingsToFile(path) =\n use file = new System.IO.StreamWriter(path, false, System.Text.Encoding.UTF8)\n let one2ones = getOneToOneMappingsAsStrings()\n file.WriteLine(one2ones)\n file.Close()\n\nwriteOneToOneMappingsToFile(@\"c:\\temp\\one2onemappings.txt\")\n\n*/\n" }, { "path": "FParsecCS/CharSet.cs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Diagnostics;\n\nnamespace FParsec {\n\n#if !LOW_TRUST\n unsafe\n#endif\ninternal sealed class CharSet {\n private const int WordSize = 32;\n private const int Log2WordSize = 5;\n\n private int Min;\n private int Max;\n private int BitTableMin;\n private int[] BitTable;\n private string CharsNotInBitTable; // We use a string here instead of a char[] because the\n // .NET JITs tend to produce better code for loops involving strings.\n\n public CharSet(string chars) : this(chars, 32) {}\n // because of mandatory bounds checking, we wouldn't get any advantage from a fixed size table\n\n public CharSet(string chars, int maxTableSize) {\n if (chars.Length == 0) {\n BitTableMin = Min = 0x10000;\n Max = -1;\n BitTable = new int[0];\n // charsNotInTable = null;\n return;\n }\n if (maxTableSize < 4) maxTableSize = 4;\n else if (maxTableSize > 0x10000/WordSize) maxTableSize = 0x10000/WordSize;\n int maxTableBits = maxTableSize*WordSize;\n\n char prevChar = chars[0];\n Min = prevChar;\n Max = prevChar;\n BitTableMin = -1;\n int bitTableMax = -1;\n int nCharsNotInTable = 0;\n for (int i = 1; i < chars.Length; ++i) {\n char c = chars[i];\n if (c == prevChar) continue; // filter out repeated chars\n prevChar = c;\n int prevMin = Min;\n if (c < Min) Min = c;\n int prevMax = Max;\n if (c > Max) Max = c;\n if (BitTableMin < 0) {\n // the first time the table range is exceeded the tableMin is set\n if (Max - Min >= maxTableBits) {\n BitTableMin = prevMin; // stays fixed\n bitTableMax = prevMax; // will be updated later\n nCharsNotInTable = 1;\n }\n } else if (c < BitTableMin || c >= BitTableMin + maxTableBits) {\n ++nCharsNotInTable;\n } else {\n bitTableMax = Math.Max(c, bitTableMax);\n }\n }\n if (BitTableMin < 0) {\n BitTableMin = Min;\n bitTableMax = Max;\n }\n int tableSize = bitTableMax - BitTableMin + 1 < maxTableBits\n ? (bitTableMax - BitTableMin + 1)/WordSize + ((bitTableMax - BitTableMin + 1)%WordSize != 0 ? 1 : 0)\n : maxTableSize;\n BitTable = new int[tableSize];\n\n #if LOW_TRUST\n var notInTable = nCharsNotInTable > 0 ? new char[nCharsNotInTable] : null;\n #else\n CharsNotInBitTable = nCharsNotInTable > 0 ? new string('\\u0000', nCharsNotInTable) : \"\";\n fixed (char* notInTable = CharsNotInBitTable) {\n #endif\n prevChar = chars[0] != 'x' ? 'x' : 'y';\n int n = 0;\n for (int i = 0; i < chars.Length; ++i) {\n char c = chars[i];\n if (c == prevChar) continue;\n prevChar = c;\n int off = c - BitTableMin;\n int idx = off >> Log2WordSize;\n if (unchecked((uint)idx) < (uint)BitTable.Length) {\n BitTable[idx] |= 1 << off; // we don't need to mask off because C#'s operator<< does that for us\n } else {\n notInTable[n++] = c;\n }\n }\n Debug.Assert(n == nCharsNotInTable);\n #if !LOW_TRUST\n }\n #else\n if (nCharsNotInTable > 0) CharsNotInBitTable = new string(notInTable);\n #endif\n }\n\n public bool Contains(char value) {\n int off = value - BitTableMin;\n int idx = off >> Log2WordSize;\n if (unchecked((uint)idx) < (uint)BitTable.Length) {\n return ((BitTable[idx] >> off) & 1) != 0; // we don't need to mask off because C#'s operator>> does that for us\n }\n if (CharsNotInBitTable == null) return false;\n if (value >= Min && value <= Max) {\n foreach (char c in CharsNotInBitTable) {\n if (c == value) goto ReturnTrue;\n }\n }\n return false;\n ReturnTrue:\n return true;\n }\n}\n\n}" }, { "path": "FParsecCS/CharStream.cs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2012\n// License: Simplified BSD License. See accompanying documentation.\n\n#if !LOW_TRUST\n\nusing System;\nusing System.IO;\nusing System.Collections.Generic;\nusing System.Text;\nusing System.Text.RegularExpressions;\nusing System.Diagnostics;\nusing System.Reflection;\nusing System.Runtime.Serialization;\nusing System.Runtime.InteropServices;\nusing System.Runtime.CompilerServices;\n\nusing Microsoft.FSharp.Core;\n\nusing FParsec.Cloning;\n\nnamespace FParsec {\n\n/// An opaque representation of a CharStream index.\npublic unsafe struct CharStreamIndexToken {\n#if DEBUG\n internal readonly CharStream CharStream;\n private long Index { get { return GetIndex(CharStream); } }\n#endif\n internal readonly char* Ptr;\n private readonly int BlockPlus1;\n /// Returns -1 if the IndexToken was zero-initialized.\n internal int Block { get { return unchecked(BlockPlus1 - 1); } }\n\n internal CharStreamIndexToken(\n #if DEBUG\n CharStream charStream,\n #endif\n char* ptr,\n int block)\n {\n #if DEBUG\n CharStream = charStream;\n #endif\n Ptr = ptr;\n BlockPlus1 = unchecked(block + 1);\n }\n\n private static void ThrowInvalidIndexToken() {\n throw new InvalidOperationException(\"The CharStreamIndexToken is invalid.\");\n }\n\n public long GetIndex(CharStream charStreamFromWhichIndexTokenWasRetrieved) {\n int block = Block;\n if (block < 0) ThrowInvalidIndexToken(); // tests for a zero-initialized IndexToken\n #if DEBUG\n Debug.Assert(CharStream == charStreamFromWhichIndexTokenWasRetrieved);\n #endif\n return charStreamFromWhichIndexTokenWasRetrieved.GetIndex(Ptr, block);\n }\n}\n\npublic struct TwoChars : IEquatable {\n private uint Chars;\n\n internal TwoChars(uint chars) {\n Chars = chars;\n }\n public TwoChars(char char0, char char1) {\n Chars = ((uint)char1 << 16) | (uint)char0;\n }\n\n public char Char0 { get { return unchecked((char)Chars); } }\n public char Char1 { get { return (char)(Chars >> 16); } }\n\n public override bool Equals(object obj) { return (obj is TwoChars) && Chars == ((TwoChars) obj).Chars; }\n public bool Equals(TwoChars other) { return Chars == other.Chars; }\n public override int GetHashCode() { return unchecked((int)Chars); }\n public static bool operator==(TwoChars left, TwoChars right) { return left.Chars == right.Chars; }\n public static bool operator!=(TwoChars left, TwoChars right) { return left.Chars != right.Chars; }\n}\n\n/// Provides read‐access to a sequence of UTF‐16 chars.\npublic unsafe class CharStream : IDisposable {\n\n // In order to facilitate efficient backtracking we divide the stream into overlapping\n // blocks with equal number of chars. The blocks are overlapping, so that\n // backtracking over short distances at a block boundary doesn't trigger a reread of the\n // previous block.\n //\n // Block 0\n //\n // -----------------|-------- Block 1\n // Overlap\n // --------|--------|-------- Block 2\n // Overlap\n // --------|--------|--------\n // (...)\n // a '-' symbolizes a char, a '|' a block boundary.\n //\n //\n // In general there's no fixed relationship between the number of input bytes and the\n // number of input chars. Worse, the encoding can be stateful, which makes it necessary\n // to persist the decoder state over block boundaries. If we later want to\n // be able to reread a certain block, we therefore need to keep record of various\n // bits of information describing the state of the input stream at the beginning of a block:\n\n private class BlockInfo {\n /// the byte stream index of the first char in the block after the OverhangCharsAtBlockBegin\n public long ByteIndex;\n /// the value of the CharStream's ByteBufferIndex before the block is read\n public int ByteBufferIndex;\n\n /// the number of bytes in the stream from ByteIndex to the first char after the OverhangCharsAfterOverlap\n public int NumberOfBytesInOverlap;\n\n /// the last char in the overlap with the previous block (used for integrity checking)\n public char LastCharInOverlap;\n\n /// chars at the block begin that were already read together with chars of the last block before the overlap\n public string OverhangCharsAtBlockBegin;\n /// chars after the overlap with the previous block that were already read together with the overlap chars\n public string OverhangCharsAfterOverlap;\n\n // Unfortunately the Decoder API has no explicit methods for managing the state,\n // which forces us to use the comparatively inefficient serialization API\n // (via FParsec.Cloning) for this purpose.\n // The absence of explicit state management or at least a cloning method in the\n // Decoder interface is almost as puzzling to me as the absence of such methods\n // in System.Random.\n\n public CloneImage DecoderImageAtBlockBegin;\n public CloneImage DecoderImageAfterOverlap;\n\n public BlockInfo(long byteIndex, int byteBufferIndex,\n int nBytesInOverlapCount, char lastCharInOverlap,\n string overhangCharsAtBlockBegin, CloneImage decoderImageAtBlockBegin,\n string overhangCharsAfterOverlap, CloneImage decoderImageAfterOverlap)\n {\n ByteIndex = byteIndex;\n ByteBufferIndex = byteBufferIndex;\n NumberOfBytesInOverlap = nBytesInOverlapCount;\n LastCharInOverlap = lastCharInOverlap;\n OverhangCharsAtBlockBegin = overhangCharsAtBlockBegin;\n OverhangCharsAfterOverlap = overhangCharsAfterOverlap;\n DecoderImageAtBlockBegin = decoderImageAtBlockBegin;\n DecoderImageAfterOverlap = decoderImageAfterOverlap;\n }\n }\n\n private const int DefaultBlockSize = 3*(1 << 16); // 3*2^16 = 200k\n private const int DefaultByteBufferLength = (1 << 12);\n private static int MinimumByteBufferLength = 128; // must be larger than longest detectable preamble (we can only guess here)\n private const char EOS = '\\uFFFF';\n\n public const char EndOfStreamChar = EOS;\n\n /// Points to the current char in Buffer,\n /// or is null if the end of the stream has been reached.\n internal char* Ptr;\n /// Equals Ptr == null ? null : BufferBegin.\n internal char* PtrBegin;\n /// Equals Ptr == null ? null : BufferEnd.\n internal char* PtrEnd;\n\n /// Begin of the used part of the char buffer. Is constant. Is null if the CharStream is empty.\n internal char* BufferBegin;\n /// End of the used part of the char buffer. Varies for a multi-block stream. Is null if the CharStream is empty.\n internal char* BufferEnd;\n\n /// The block currently loaded in the buffer.\n internal int Block;\n\n /// Any CharStream method or property setter increments this value when it changes the CharStream state.\n /// Backtracking to an old state also restores the old value of the StateTag.\n public\n#if SMALL_STATETAG\n int\n#else\n long\n#endif\n StateTag;\n\n internal long IndexOfFirstCharInBlock;\n\n internal long _IndexOfFirstChar;\n /// The index of the first char in the stream.\n public long IndexOfFirstChar { get { return _IndexOfFirstChar; } }\n\n internal long _Line;\n /// The line number for the next char. (The line count starts with 1.)\n public long Line { get { return _Line; } }\n public void SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(long line) {\n _Line = line;\n }\n\n internal long _LineBegin;\n /// The stream index of the first char of the line that also contains the next char.\n public long LineBegin { get { return _LineBegin; } }\n public void SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(long lineBegin) {\n _LineBegin = lineBegin;\n }\n\n /// The UTF‐16 column number of the next char, i.e. Index ‐ LineBegin + 1.\n public long Column { get { return Index - LineBegin + 1; } }\n\n internal string _Name;\n public string Name {\n get { return _Name; }\n set { _Name = value; ++StateTag; }\n }\n\n /// The Encoding that is used for decoding the underlying byte stream, or\n /// System.Text.UnicodeEncoding in case the stream was directly constructed\n /// from a string or char buffer.\n public Encoding Encoding { get; private set; }\n\n // If the CharStream is constructed from a binary stream, we use a managed string as the char\n // buffer. This allows us to apply regular expressions directly to the input.\n // In the case of multi-block CharStreams we thus have to mutate the buffer string through pointers.\n // This is safe as long as we use a newly constructed string and we don't pass a reference\n // to the internal buffer string to the \"outside world\". (The one instance where we have to pass\n // a reference to the buffer string is regex matching. See the docs for Match(regex) for more info.)\n //\n // Apart from Match(regex) we access the internal buffer only through a pinned pointer.\n // This way we avoid the overhead of redundant bounds checking and can support strings, char arrays\n // and unmanaged char buffers through the same interface.\n //\n // Pinning a string or char array makes life more difficult for the GC. However, as long as\n // the buffer is only short-lived or large enough to be allocated on the large object heap,\n // there shouldn't be a problem. Furthermore, the buffer strings for CharStreams constructed\n // from a binary stream are allocated through the StringBuffer interface and hence always live\n // on the large object heap. Thus, the only scenario to really worry about (and which the\n // documentation explicitly warns about) is when a large number of small CharStreams\n // are constructed directly from strings or char arrays and are used for an extended period of time.\n\n /// The string holding the char buffer, or null if the buffer is not part of a .NET string.\n internal string BufferString;\n /// A pointer to the beginning of BufferString, or null if BufferString is null.\n internal char* BufferStringPointer;\n\n /// Holds the GCHandle for CharStreams directly constructed from strings or char arrays.\n private GCHandle BufferHandle;\n /// Holds the StringBuffer for CharStreams constructed from a binary stream.\n private StringBuffer StringBuffer;\n\n#if DEBUG\n internal FSharpRef SubstreamCount = new FSharpRef(0);\n internal FSharpRef ParentSubstreamCount = null;\n#endif\n\n private MultiBlockData BlockData;\n internal bool IsSingleBlockStream { get { return BlockData == null; } }\n\n /// Contains the data and methods needed in case the input byte stream\n /// is large enough to span multiple blocks of the CharStream.\n private partial class MultiBlockData {\n public CharStream CharStream;\n\n public long IndexOfLastCharPlus1;\n\n /// The index of the last block of the stream, or Int32.MaxValue if the end of stream has not yet been detected.\n public int LastBlock;\n\n public Stream Stream;\n // we keep a separate record of the Stream.Position, so that we don't need to require Stream.CanSeek\n public long StreamPosition;\n // we use StreamLength to avoid calling Read() again on a non-seekable stream after it returned 0 once (see ticket #23)\n public long StreamLength;\n public bool LeaveOpen;\n\n public int MaxCharCountForOneByte;\n public Decoder Decoder;\n public bool DecoderIsSerializable;\n\n public int BlockSize;\n public int BlockOverlap;\n /// BufferBegin + BlockSize - minRegexSpace\n public char* RegexSpaceThreshold;\n\n /// The byte stream index of the first unused byte in the ByteBuffer.\n public long ByteIndex { get { return StreamPosition - (ByteBufferCount - ByteBufferIndex); } }\n\n public List Blocks;\n\n public byte[] ByteBuffer;\n public int ByteBufferIndex;\n public int ByteBufferCount;\n }\n\n public long IndexOfLastCharPlus1 { get {\n return BlockData != null ? BlockData.IndexOfLastCharPlus1\n : IndexOfFirstChar + Buffer.PositiveDistance(BufferBegin, BufferEnd);\n } }\n\n public int BlockOverlap { get {\n return BlockData == null ? 0 : BlockData.BlockOverlap;\n } }\n\n public int MinRegexSpace {\n get {\n return BlockData == null\n ? 0\n : (int)Buffer.PositiveDistance(BlockData.RegexSpaceThreshold,\n BufferBegin + BlockData.BlockSize);\n }\n set {\n if (BlockData != null) {\n if (value < 0 || value > BlockData.BlockOverlap) throw new ArgumentOutOfRangeException(\"value\", \"The MinRegexSpace value must be non-negative and not greater than the BlockOverlap.\");\n BlockData.RegexSpaceThreshold = BufferBegin + BlockData.BlockSize - value;\n }\n }\n }\n\n public bool IsBeginOfStream { get { return Ptr == BufferBegin && Block == 0; } }\n public bool IsEndOfStream { get { return Ptr == null; } }\n\n\n public long Index {\n #if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n #endif\n get {\n if (Ptr != null) {\n Debug.Assert(BufferBegin <= Ptr && Ptr < BufferEnd);\n if (sizeof(System.IntPtr) != 8) // the JIT removes the inactive branch\n return Buffer.PositiveDistance(PtrBegin, Ptr) + IndexOfFirstCharInBlock;\n else\n return Buffer.PositiveDistance64(PtrBegin, Ptr) + IndexOfFirstCharInBlock;\n }\n Debug.Assert(BlockData == null || BlockData.IndexOfLastCharPlus1 != Int64.MaxValue);\n return IndexOfLastCharPlus1;\n }\n }\n\n internal long GetIndex(char* ptr, int block) {\n if (ptr != null) {\n if (block == Block) {\n Debug.Assert(BufferBegin <= ptr && ptr < BufferEnd);\n if (sizeof(System.IntPtr) != 8)\n return Buffer.PositiveDistance(BufferBegin, ptr) + IndexOfFirstCharInBlock;\n else\n return Buffer.PositiveDistance64(BufferBegin, ptr) + IndexOfFirstCharInBlock;\n } else {\n Debug.Assert(BlockData != null && BufferBegin <= ptr && ptr < BufferBegin + BlockData.BlockSize);\n int blockSizeMinusOverlap = BlockData.BlockSize - BlockData.BlockOverlap;\n long indexOfBlockBegin = IndexOfFirstChar + Math.BigMul(block, blockSizeMinusOverlap);\n if (sizeof(System.IntPtr) != 8)\n return Buffer.PositiveDistance(BufferBegin, ptr) + indexOfBlockBegin;\n else\n return Buffer.PositiveDistance64(BufferBegin, ptr) + indexOfBlockBegin;\n }\n }\n Debug.Assert(BlockData == null || BlockData.IndexOfLastCharPlus1 != Int64.MaxValue);\n return IndexOfLastCharPlus1;\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public Position Position { get {\n long index = Index;\n return new Position(_Name, index, Line, index - LineBegin + 1);\n } }\n\n // we don't have a public constructor that only takes a string to avoid potential confusion with a filepath constructor\n internal CharStream(string chars) {\n Debug.Assert(chars != null);\n BufferString = chars;\n BufferHandle = GCHandle.Alloc(chars, GCHandleType.Pinned);\n char* bufferBegin = (char*)BufferHandle.AddrOfPinnedObject();\n BufferStringPointer = bufferBegin;\n CharConstructorContinue(bufferBegin, chars.Length);\n }\n\n public CharStream(string chars, int index, int length) : this(chars, index, length, 0) {}\n public CharStream(string chars, int index, int length, long streamIndexOffset) {\n if (chars == null) throw new ArgumentNullException(\"chars\");\n if (index < 0) throw new ArgumentOutOfRangeException(\"index\", \"index is negative.\");\n if (length < 0 || length > chars.Length - index) throw new ArgumentOutOfRangeException(\"length\", \"index or length is out of range.\");\n if (streamIndexOffset < 0 || streamIndexOffset >= (1L << 60)) throw new ArgumentOutOfRangeException(\"streamIndexOffset\", \"streamIndexOffset must be non-negative and less than 2^60.\");\n IndexOfFirstCharInBlock = streamIndexOffset;\n _IndexOfFirstChar = streamIndexOffset;\n _LineBegin = streamIndexOffset;\n\n BufferString = chars;\n BufferHandle = GCHandle.Alloc(chars, GCHandleType.Pinned);\n char* pBufferString = (char*)BufferHandle.AddrOfPinnedObject();\n BufferStringPointer = pBufferString;\n CharConstructorContinue(pBufferString + index, length);\n }\n\n public CharStream(char[] chars, int index, int length) : this(chars, index, length, 0) { }\n public CharStream(char[] chars, int index, int length, long streamIndexOffset) {\n if (chars == null) throw new ArgumentNullException(\"chars\");\n if (index < 0) throw new ArgumentOutOfRangeException(\"index\", \"index is negative.\");\n if (length < 0 || length > chars.Length - index) throw new ArgumentOutOfRangeException(\"length\", \"index or length is out of range.\");\n if (streamIndexOffset < 0 || streamIndexOffset >= (1L << 60)) throw new ArgumentOutOfRangeException(\"streamIndexOffset\", \"streamIndexOffset must be non-negative and less than 2^60.\");\n IndexOfFirstCharInBlock = streamIndexOffset;\n _IndexOfFirstChar = streamIndexOffset;\n _LineBegin = streamIndexOffset;\n\n BufferHandle = GCHandle.Alloc(chars, GCHandleType.Pinned);\n char* bufferBegin = (char*)BufferHandle.AddrOfPinnedObject() + index;\n if (bufferBegin < unchecked(bufferBegin + length + 1)) { // a pedantic check ...\n CharConstructorContinue(bufferBegin, length);\n } else {\n // ... for a purely theoretic case\n BufferHandle.Free();\n throw new ArgumentOutOfRangeException(\"length\", \"The char array may not be allocated directly below the end of the address space.\");\n }\n\n }\n\n public CharStream(char* chars, int length) : this(chars, length, 0) {}\n public CharStream(char* chars, int length, long streamIndexOffset) {\n if (chars == null) throw new ArgumentNullException(\"chars\");\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n if (chars >= unchecked(chars + length + 1)) // chars + length + 1 must not overflow (the + 1 is needed for some methods below)\n throw new ArgumentOutOfRangeException(\"length\", \"length is too large.\");\n if (streamIndexOffset < 0 || streamIndexOffset >= (1L << 60)) throw new ArgumentOutOfRangeException(\"streamIndexOffset\", \"streamIndexOffset must be non-negative and less than 2^60.\");\n IndexOfFirstCharInBlock = streamIndexOffset;\n _IndexOfFirstChar = streamIndexOffset;\n _LineBegin = streamIndexOffset;\n CharConstructorContinue(chars, length);\n }\n\n private void CharConstructorContinue(char* bufferBegin, int length) {\n Debug.Assert((bufferBegin != null || length == 0) && length >= 0\n && bufferBegin < unchecked(bufferBegin + length + 1)); // the + 1 is needed for some methods below\n\n if (length != 0) {\n BufferBegin = bufferBegin;\n BufferEnd = bufferBegin + length;\n Ptr = bufferBegin;\n PtrBegin = bufferBegin;\n PtrEnd = BufferEnd;\n }\n _Line = 1;\n Encoding = Encoding.Unicode;\n }\n\n internal CharStream(string chars, char* pChars, char* begin, int length) {\n Debug.Assert((chars == null ? pChars == null\n : pChars <= begin && length >= 0 && (int)Buffer.PositiveDistance(pChars, begin) <= chars.Length - length)\n && (begin == null ? length == 0\n : length >= 0 && begin < unchecked(begin + length + 1)));\n\n BufferString = chars;\n BufferStringPointer = pChars;\n if (length != 0) {\n BufferBegin = begin;\n BufferEnd = begin + length;\n Ptr = begin;\n PtrBegin = begin;\n PtrEnd = BufferEnd;\n }\n _Line = 1;\n Encoding = Encoding.Unicode;\n }\n\n public CharStream(string path, Encoding encoding)\n : this(path, encoding, true,\n DefaultBlockSize, DefaultBlockSize/3, DefaultByteBufferLength) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : this(path, encoding, detectEncodingFromByteOrderMarks,\n DefaultBlockSize, DefaultBlockSize/3, DefaultByteBufferLength) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks,\n int blockSize, int blockOverlap, int byteBufferLength)\n {\n if (encoding == null) throw new ArgumentNullException(\"encoding\");\n var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, FileOptions.SequentialScan);\n try {\n StreamConstructorContinue(stream, false, encoding, detectEncodingFromByteOrderMarks,\n blockSize, blockOverlap, byteBufferLength);\n _Name = path;\n } catch {\n stream.Dispose();\n throw;\n }\n }\n\n public CharStream(Stream stream, Encoding encoding)\n : this(stream,\n false, encoding, true,\n DefaultBlockSize, DefaultBlockSize/3, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding)\n : this(stream,\n leaveOpen, encoding, true,\n DefaultBlockSize, DefaultBlockSize/3, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : this(stream,\n leaveOpen, encoding, detectEncodingFromByteOrderMarks,\n DefaultBlockSize, DefaultBlockSize/3, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen,\n Encoding encoding, bool detectEncodingFromByteOrderMarks,\n int blockSize, int blockOverlap, int byteBufferLength)\n {\n if (stream == null) throw new ArgumentNullException(\"stream\");\n if (!stream.CanRead) throw new ArgumentException(\"stream is not readable\");\n if (encoding == null) throw new ArgumentNullException(\"encoding\");\n StreamConstructorContinue(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks,\n blockSize, blockOverlap, byteBufferLength);\n }\n\n /// we modify this flag via reflection in the unit test\n private static bool DoNotRoundUpBlockSizeToSimplifyTesting = false;\n\n private void StreamConstructorContinue(Stream stream, bool leaveOpen,\n Encoding encoding, bool detectEncodingFromByteOrderMarks,\n int blockSize, int blockOverlap, int byteBufferLength)\n {\n if (byteBufferLength < MinimumByteBufferLength) byteBufferLength = MinimumByteBufferLength;\n\n int remainingBytesCount = -1;\n long streamPosition;\n long streamLength;\n if (stream.CanSeek) {\n streamPosition = stream.Position;\n streamLength = stream.Length;\n long remainingBytesCount64 = streamLength - streamPosition;\n if (remainingBytesCount64 <= Int32.MaxValue) {\n remainingBytesCount = (int)remainingBytesCount64;\n if (remainingBytesCount < byteBufferLength) byteBufferLength = remainingBytesCount;\n }\n } else {\n streamPosition = 0;\n streamLength = Int64.MaxValue;\n }\n\n byte[] byteBuffer = new byte[byteBufferLength];\n int byteBufferCount = 0;\n do {\n int n = stream.Read(byteBuffer, byteBufferCount, byteBufferLength - byteBufferCount);\n if (n == 0) {\n remainingBytesCount = byteBufferCount;\n Debug.Assert(!stream.CanSeek || streamPosition + byteBufferCount == streamLength);\n streamLength = streamPosition + byteBufferCount;\n break;\n }\n byteBufferCount += n;\n } while (byteBufferCount < MinimumByteBufferLength);\n streamPosition += byteBufferCount;\n\n int preambleLength = Text.DetectPreamble(byteBuffer, byteBufferCount, ref encoding, detectEncodingFromByteOrderMarks);\n remainingBytesCount -= preambleLength;\n\n _Line = 1;\n Encoding = encoding;\n\n // we allow such small block sizes only to simplify testing\n if (blockSize < 8) blockSize = DefaultBlockSize;\n\n bool allCharsFitIntoOneBlock = false;\n if (remainingBytesCount >= 0 && remainingBytesCount/4 <= blockSize) {\n if (remainingBytesCount != 0) {\n try {\n int maxCharCount = Encoding.GetMaxCharCount(remainingBytesCount); // may throw ArgumentOutOfRangeException\n if (blockSize >= maxCharCount) {\n allCharsFitIntoOneBlock = true;\n blockSize = maxCharCount;\n }\n } catch (ArgumentOutOfRangeException) { }\n } else {\n allCharsFitIntoOneBlock = true;\n blockSize = 0;\n }\n }\n var buffer = StringBuffer.Create(blockSize);\n Debug.Assert(buffer.Length >= blockSize && (blockSize > 0 || buffer.StringPointer == null));\n StringBuffer = buffer;\n BufferString = buffer.String;\n BufferStringPointer = buffer.StringPointer;\n char* bufferBegin = buffer.StringPointer + buffer.Index;\n try {\n Decoder decoder = encoding.GetDecoder();\n if (allCharsFitIntoOneBlock) {\n int bufferCount = preambleLength == byteBufferCount\n ? 0\n : Text.ReadAllRemainingCharsFromStream(bufferBegin, buffer.Length, byteBuffer, preambleLength, byteBufferCount, stream, streamPosition, decoder, streamPosition == streamLength);\n if (!leaveOpen) stream.Close();\n if (bufferCount != 0) {\n BufferBegin = bufferBegin;\n Ptr = bufferBegin;\n PtrBegin = bufferBegin;\n BufferEnd = bufferBegin + bufferCount;\n PtrEnd = BufferEnd;\n }\n Block = 0;\n } else {\n if (!DoNotRoundUpBlockSizeToSimplifyTesting) blockSize = buffer.Length;\n BufferBegin = bufferBegin;\n BufferEnd = bufferBegin;\n var d = new MultiBlockData();\n BlockData = d;\n d.CharStream = this;\n d.Stream = stream;\n d.StreamPosition = streamPosition;\n d.StreamLength = streamLength;\n d.LeaveOpen = leaveOpen;\n d.Decoder = decoder;\n d.DecoderIsSerializable = decoder.GetType().IsSerializable;\n d.ByteBuffer = byteBuffer;\n d.ByteBufferIndex = preambleLength;\n d.ByteBufferCount = byteBufferCount;\n d.MaxCharCountForOneByte = Math.Max(1, Encoding.GetMaxCharCount(1));\n if (d.MaxCharCountForOneByte > 1024) // an arbitrary limit low enough that a char array with this size can be allocated on the stack\n throw new ArgumentException(\"The CharStream class does not support Encodings with GetMaxCharCount(1) > 1024.\");\n if (blockSize < 3*d.MaxCharCountForOneByte) blockSize = 3*d.MaxCharCountForOneByte;\n // MaxCharCountForOneByte == the maximum number of overhang chars\n if( Math.Min(blockOverlap, blockSize - 2*blockOverlap) < d.MaxCharCountForOneByte\n || blockOverlap >= blockSize/2) blockOverlap = blockSize/3;\n d.BlockSize = blockSize;\n d.BlockOverlap = blockOverlap;\n d.RegexSpaceThreshold = bufferBegin + (blockSize - 2*blockOverlap/3);\n d.IndexOfLastCharPlus1 = Int64.MaxValue;\n Block = -2; // special value recognized by ReadBlock\n d.LastBlock = Int32.MaxValue;\n d.Blocks = new List();\n // the first block has no overlap with a previous block\n d.Blocks.Add(new BlockInfo(preambleLength, preambleLength, 0, EOS, null, null, null, null));\n d.ReadBlock(0);\n if (d.LastBlock == 0) {\n if (!d.LeaveOpen) d.Stream.Close();\n BlockData = null;\n }\n }\n } catch {\n buffer.Dispose();\n throw;\n }\n }\n\n public void Dispose() {\n #if DEBUG\n lock (SubstreamCount) {\n if (SubstreamCount.Value != 0)\n throw new InvalidOperationException(\"A CharStream must not be disposed before all of its Substreams have been disposed.\");\n }\n if (ParentSubstreamCount != null) {\n lock (ParentSubstreamCount) --ParentSubstreamCount.Value;\n }\n #endif\n if (BufferHandle.IsAllocated) BufferHandle.Free();\n if (StringBuffer != null) StringBuffer.Dispose();\n if (BlockData != null && !BlockData.LeaveOpen) BlockData.Stream.Close();\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n BufferBegin = null;\n BufferEnd = null;\n }\n\n [System.Diagnostics.CodeAnalysis.SuppressMessage(\"Microsoft.Reliability\", \"CA2000:Dispose objects before losing scope\", Justification=\"The CharStream is manually disposed.\")]\n public static T ParseString(string chars, int index, int length,\n FSharpFunc,T> parser,\n TUserState userState,\n string streamName)\n {\n if (index < 0) throw new ArgumentOutOfRangeException(\"index\", \"index is negative.\");\n if (length < 0 || length > chars.Length - index) throw new ArgumentOutOfRangeException(\"length\", \"length is out of range.\");\n fixed (char* pChars = chars) {\n var stream = new CharStream(chars, pChars, pChars + index, length);\n stream.UserState = userState;\n stream._Name = streamName;\n try {\n return parser.Invoke(stream);\n } finally {\n #if DEBUG\n stream.Dispose();\n #else\n // manually dispose stream\n stream.Ptr = null;\n stream.PtrBegin = null;\n stream.PtrEnd = null;\n stream.BufferBegin = null;\n stream.BufferEnd = null;\n #endif\n }\n }\n }\n\n private partial class MultiBlockData {\n /// Refills the ByteBuffer if no unused byte is remaining.\n /// Returns the number of unused bytes in the (refilled) ByteBuffer.\n private int FillByteBuffer() {\n int n = ByteBufferCount - ByteBufferIndex;\n if (n > 0) return n;\n return ClearAndRefillByteBuffer(0);\n }\n\n /// Refills the ByteBuffer starting at the given index. If the underlying byte\n /// stream contains enough bytes, the ByteBuffer is filled up to the ByteBuffer.Length.\n /// Returns the number of bytes available for consumption in the refilled ByteBuffer.\n private int ClearAndRefillByteBuffer(int byteBufferIndex) {\n Debug.Assert(byteBufferIndex >= 0 && byteBufferIndex <= ByteBuffer.Length);\n // Stream.Read is not guaranteed to use all the provided output buffer, so we need\n // to call it in a loop when we want to rely on the buffer being fully filled\n // (unless we reach the end of the stream). Knowing that the buffer always gets\n // completely filled allows us to calculate the buffer utilization after skipping\n // a certain number of input bytes. For most streams there will be only one loop\n // iteration anyway (or two at the end of the stream).\n int i = byteBufferIndex;\n int m = ByteBuffer.Length - byteBufferIndex;\n while (m != 0 && StreamPosition != StreamLength) { // we check the StreamPosition to avoid calling Read after it returned 0 at the end of the stream (see ticket #23)\n int c = Stream.Read(ByteBuffer, i, m);\n if (c != 0) {\n i += c;\n m -= c;\n StreamPosition += c;\n } else {\n Debug.Assert(!Stream.CanSeek || StreamPosition == StreamLength);\n StreamLength = StreamPosition;\n break;\n }\n }\n int n = i - byteBufferIndex;\n ByteBufferIndex = byteBufferIndex;\n ByteBufferCount = byteBufferIndex + n;\n return n;\n }\n\n /// Reads up to the given maximum number of chars into the given buffer.\n /// If more than the maximum number of chars have to be read from the stream in order to\n /// fill the buffer (due to\tthe way the Decoder API works), the overhang chars are\n /// returned through the output parameter.\n /// Returns a pointer to one char after the last char read.\n private char* ReadCharsFromStream(char* buffer, int maxCount, out string overhangChars) {\n Debug.Assert(maxCount >= 0);\n fixed (byte* byteBuffer = ByteBuffer) {\n overhangChars = null;\n try {\n while (maxCount >= MaxCharCountForOneByte) {// if maxCount < MaxCharCountForOneByte, Convert could throw\n int nBytesInByteBuffer = FillByteBuffer();\n bool flush = nBytesInByteBuffer == 0;\n int bytesUsed, charsUsed; bool completed = false;\n Decoder.Convert(byteBuffer + ByteBufferIndex, nBytesInByteBuffer,\n buffer, maxCount, flush,\n out bytesUsed, out charsUsed, out completed);\n ByteBufferIndex += bytesUsed; // GetChars consumed bytesUsed bytes from the byte buffer\n buffer += charsUsed;\n maxCount -= charsUsed;\n if (flush && completed) return buffer;\n }\n if (maxCount == 0) return buffer;\n\n char* cs = stackalloc char[MaxCharCountForOneByte];\n for (;;) {\n int nBytesInByteBuffer = FillByteBuffer();\n bool flush = nBytesInByteBuffer == 0;\n int bytesUsed, charsUsed; bool completed;\n Decoder.Convert(byteBuffer + ByteBufferIndex, nBytesInByteBuffer,\n cs, MaxCharCountForOneByte, flush,\n out bytesUsed, out charsUsed, out completed);\n ByteBufferIndex += bytesUsed;\n if (charsUsed > 0) {\n int i = 0;\n do {\n *buffer = cs[i];\n ++buffer; ++i;\n if (--maxCount == 0) {\n if (i < charsUsed) overhangChars = new string(cs, i, charsUsed - i);\n return buffer;\n }\n } while (i < charsUsed);\n }\n if (flush && completed) return buffer;\n }\n } catch (DecoderFallbackException e) {\n e.Data.Add(\"Stream.Position\", ByteIndex + e.Index);\n throw;\n }\n }\n }\n\n /// Reads a block of chars (which must be different from the current block)\n /// into the BufferString. If the current CharStream block is block - 1, this method\n /// seeks the CharStream to the first char after the overlap of the two blocks.\n /// Otherwise it seeks the CharStream to the first char in the block. It returns the\n /// CharStream.Ptr value at the new position (which can be null).\n internal char* ReadBlock(int block) {\n int prevBlock = CharStream.Block;\n if (block == prevBlock) throw new InvalidOperationException();\n if (!DecoderIsSerializable && block > 0) {\n if (prevBlock > block)\n throw new NotSupportedException(\"The CharStream does not support seeking backwards over ranges longer than the block overlap because the Encoding's Decoder is not serializable. The decoder has the type: \" + Decoder.GetType().FullName);\n while (prevBlock + 1 < block) ReadBlock(++prevBlock);\n }\n\n BlockInfo bi = Blocks[block]; // will throw if block is out of range\n int blockSizeMinusOverlap = BlockSize - BlockOverlap;\n long charIndex = Math.BigMul(block, blockSizeMinusOverlap);\n char* bufferBegin = CharStream.BufferBegin;\n char* begin, buffer;\n int nCharsToRead;\n\n // fill [0 ... BlockOverlap-1] if block > 0\n if (prevBlock == block - 1) {\n Buffer.Copy((byte*)bufferBegin, (byte*)(bufferBegin + blockSizeMinusOverlap),\n BlockOverlap*sizeof(char));\n Debug.Assert(bufferBegin[BlockOverlap - 1] == bi.LastCharInOverlap);\n begin = buffer = bufferBegin + BlockOverlap;\n } else if (prevBlock >= 0) {\n Stream.Seek(bi.ByteIndex, SeekOrigin.Begin); // will throw if Stream can't seek\n // now that there was no exception, we can change the state...\n StreamPosition = bi.ByteIndex;\n ClearAndRefillByteBuffer(bi.ByteBufferIndex);\n if (block != 0)\n Decoder = (Decoder)bi.DecoderImageAtBlockBegin.CreateClone();\n else\n Decoder.Reset();\n if (prevBlock == block + 1) {\n // move the overlap into [BlockSize - BlockOverlap, BlockSize - 1] before it gets overwritten\n Buffer.Copy((byte*)(bufferBegin + blockSizeMinusOverlap), (byte*)bufferBegin,\n BlockOverlap*sizeof(char));\n }\n begin = buffer = bufferBegin;\n if (block > 0) {\n nCharsToRead = BlockOverlap;\n if (bi.OverhangCharsAtBlockBegin != null) {\n nCharsToRead -= bi.OverhangCharsAtBlockBegin.Length;\n for (int i = 0; i < bi.OverhangCharsAtBlockBegin.Length; ++i)\n *(buffer++) = bi.OverhangCharsAtBlockBegin[i];\n }\n string overhangCharsAfterOverlap;\n buffer = ReadCharsFromStream(buffer, nCharsToRead, out overhangCharsAfterOverlap);\n if ( buffer != bufferBegin + BlockOverlap\n || ByteIndex != bi.ByteIndex + bi.NumberOfBytesInOverlap\n || *(buffer - 1) != bi.LastCharInOverlap\n || overhangCharsAfterOverlap != bi.OverhangCharsAfterOverlap)\n throw new IOException(\"CharStream: stream integrity error\");\n }\n } else { // ReadBlock was called from the constructor\n if (block != 0) throw new InvalidOperationException();\n begin = buffer = bufferBegin;\n }\n\n // fill [0 ... BlockSize-BlockOverlap-1] if block == 0\n // and [BlockOverlap ... BlockSize-BlockOverlap-1] otherwise\n if (block == 0) {\n nCharsToRead = blockSizeMinusOverlap;\n } else {\n nCharsToRead = blockSizeMinusOverlap - BlockOverlap;\n if (bi.OverhangCharsAfterOverlap != null) {\n nCharsToRead -= bi.OverhangCharsAfterOverlap.Length;\n for (int i = 0; i < bi.OverhangCharsAfterOverlap.Length; ++i)\n *(buffer++) = bi.OverhangCharsAfterOverlap[i];\n }\n }\n string overhangCharsAtNextBlockBegin;\n buffer = ReadCharsFromStream(buffer, nCharsToRead, out overhangCharsAtNextBlockBegin);\n\n long byteIndexAtNextBlockBegin = ByteIndex;\n int byteBufferIndexAtNextBlockBegin = ByteBufferIndex;\n\n // fill [BlockSize-BlockOverlap ... BlockSize-1]\n if (block == Blocks.Count - 1) { // next block hasn't yet been read\n Cloner cloner = null;\n CloneImage decoderImageAtNextBlockBegin = null;\n if (DecoderIsSerializable) {\n cloner = Cloner.Create(Decoder.GetType());\n decoderImageAtNextBlockBegin = cloner.CaptureImage(Decoder);\n }\n nCharsToRead = BlockOverlap;\n if (overhangCharsAtNextBlockBegin != null) {\n nCharsToRead -= overhangCharsAtNextBlockBegin.Length;\n for (int i = 0; i < overhangCharsAtNextBlockBegin.Length; ++i)\n *(buffer++) = overhangCharsAtNextBlockBegin[i];\n }\n string overhangCharsAfterOverlapWithNextBlock;\n buffer = ReadCharsFromStream(buffer, nCharsToRead, out overhangCharsAfterOverlapWithNextBlock);\n if (LastBlock == Int32.MaxValue) { // last block hasn't yet been detected\n if (buffer == bufferBegin + BlockSize) {\n var decoderImageAfterOverlapWithNextBlock =\n !DecoderIsSerializable ? null : cloner.CaptureImage(Decoder);\n int nBytesInOverlapWithNextBlock = (int)(ByteIndex - byteIndexAtNextBlockBegin);\n Blocks.Add(new BlockInfo(byteIndexAtNextBlockBegin, byteBufferIndexAtNextBlockBegin,\n nBytesInOverlapWithNextBlock, *(buffer - 1),\n overhangCharsAtNextBlockBegin, decoderImageAtNextBlockBegin,\n overhangCharsAfterOverlapWithNextBlock, decoderImageAfterOverlapWithNextBlock));\n } else { // we reached the end of the stream\n LastBlock = block;\n IndexOfLastCharPlus1 = CharStream.IndexOfFirstChar + charIndex + (buffer - bufferBegin);\n }\n } else if (IndexOfLastCharPlus1 != CharStream.IndexOfFirstChar + charIndex + (buffer - bufferBegin)) {\n throw new IOException(\"CharStream: stream integrity error\");\n }\n } else {\n BlockInfo nbi = Blocks[block + 1];\n if (buffer != bufferBegin + blockSizeMinusOverlap\n || byteIndexAtNextBlockBegin != nbi.ByteIndex\n || byteBufferIndexAtNextBlockBegin != nbi.ByteBufferIndex\n || overhangCharsAtNextBlockBegin != nbi.OverhangCharsAtBlockBegin)\n throw new IOException(\"CharStream: stream integrity error\");\n\n if (prevBlock != block + 1 || (block == 0 && !DecoderIsSerializable)) { // jumping back to block 0 is supported even if the decoder is not serializable\n nCharsToRead = BlockOverlap;\n if (overhangCharsAtNextBlockBegin != null) {\n nCharsToRead -= overhangCharsAtNextBlockBegin.Length;\n for (int i = 0; i < overhangCharsAtNextBlockBegin.Length; ++i)\n *(buffer++) = overhangCharsAtNextBlockBegin[i];\n }\n string overhangCharsAfterOverlapWithNextBlock;\n buffer = ReadCharsFromStream(buffer, nCharsToRead, out overhangCharsAfterOverlapWithNextBlock);\n int nBytesInOverlapWithNextBlock = (int)(ByteIndex - byteIndexAtNextBlockBegin);\n if (buffer != bufferBegin + BlockSize\n || nBytesInOverlapWithNextBlock != nbi.NumberOfBytesInOverlap\n || *(buffer - 1) != nbi.LastCharInOverlap\n || overhangCharsAfterOverlapWithNextBlock != nbi.OverhangCharsAfterOverlap)\n throw new IOException(\"CharStream: stream integrity error\");\n } else {\n Debug.Assert(bufferBegin[BlockSize - 1] == nbi.LastCharInOverlap);\n buffer += BlockOverlap; // we already copied the chars at the beginning of this function\n int off = nbi.NumberOfBytesInOverlap - (ByteBufferCount - ByteBufferIndex);\n if (off > 0) {\n // we wouldn't have gotten here if the Stream didn't support seeking\n Stream.Seek(off, SeekOrigin.Current);\n StreamPosition += off;\n ClearAndRefillByteBuffer(off%ByteBuffer.Length);\n } else {\n ByteBufferIndex += nbi.NumberOfBytesInOverlap;\n }\n Decoder = (Decoder)nbi.DecoderImageAfterOverlap.CreateClone();\n }\n }\n\n CharStream.Block = block;\n //CharStream.CharIndex = charIndex;\n CharStream.IndexOfFirstCharInBlock = CharStream.IndexOfFirstChar + charIndex;\n CharStream.BufferEnd = buffer;\n if (begin != buffer) {\n CharStream.Ptr = begin;\n CharStream.PtrEnd = buffer;\n CharStream.PtrBegin = CharStream.BufferBegin;\n return begin;\n } else {\n CharStream.Ptr = null;\n CharStream.PtrEnd = null;\n CharStream.PtrBegin = null;\n return null;\n }\n }\n } // class MultiBlockData\n\n\n /// Returns an iterator pointing to the given index in the stream,\n /// or to the end of the stream if the indexed position lies beyond the last char in the stream.\n /// The index is negative or less than the BeginIndex.\n /// Accessing the char with the given index requires seeking in the underlying byte stream, but the byte stream does not support seeking or the Encoding's Decoder is not serializable.\n /// An I/O error occured.\n /// The input stream contains invalid bytes and the encoding was constructed with the throwOnInvalidBytes option.\n /// The input stream contains invalid bytes for which the decoder fallback threw this exception.\n /// Can not allocate enough memory for the internal data structure.\n /// Method is called after the stream was disposed.\n public void Seek(long index) {\n ++StateTag;\n // The following comparison is safe in case of an overflow since\n // 0 <= IndexOfFirstCharInBlock < 2^60 + 2^31 * 2^31 and BufferEnd - BufferBegin < 2^31,\n // where 2^31 is an upper bound for both the number of blocks and the number of chars in a block.\n long off = unchecked(index - IndexOfFirstCharInBlock);\n if (0 <= off && off < Buffer.PositiveDistance(BufferBegin, BufferEnd)) {\n Ptr = BufferBegin + (uint)off;\n PtrBegin = BufferBegin;\n PtrEnd = BufferEnd;\n return;\n }\n if (index < IndexOfFirstChar) {\n --StateTag;\n throw (new ArgumentOutOfRangeException(\"index\", \"The index is negative or less than the IndexOfFirstChar.\"));\n }\n if (BlockData == null || index >= BlockData.IndexOfLastCharPlus1) {\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n return;\n }\n // we never get here for streams with only one block\n index -= IndexOfFirstChar;\n int blockSizeMinusOverlap = BlockData.BlockSize - BlockData.BlockOverlap;\n long idx_;\n long block_ = Math.DivRem(index, blockSizeMinusOverlap, out idx_);\n int block = block_ > Int32.MaxValue ? Int32.MaxValue : (int)block_;\n int idx = (int)idx_;\n Seek(block, idx);\n }\n\n private void Seek(int block, int indexInBlock) {\n Debug.Assert(block >= 0 && indexInBlock >= 0 && BlockData != null);\n if (block > Block) {\n if (indexInBlock < BlockData.BlockOverlap) {\n --block;\n indexInBlock += BlockData.BlockSize - BlockData.BlockOverlap;\n }\n } else if (block < Block) {\n int blockSizeMinusOverlap = BlockData.BlockSize - BlockData.BlockOverlap;\n if (indexInBlock >= blockSizeMinusOverlap) {\n ++block;\n indexInBlock -= blockSizeMinusOverlap;\n }\n }\n if (block == Block) {\n Debug.Assert(indexInBlock < Buffer.PositiveDistance(BufferBegin, BufferEnd));\n PtrBegin = BufferBegin;\n PtrEnd = BufferEnd;\n } else {\n int last = BlockData.Blocks.Count - 1;\n if (block >= last) {\n BlockData.ReadBlock(last);\n while (Block < block && Block != BlockData.LastBlock)\n BlockData.ReadBlock(Block + 1);\n if (block != Block || indexInBlock >= Buffer.PositiveDistance(PtrBegin, PtrEnd)) {\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n return;\n }\n } else {\n BlockData.ReadBlock(block);\n Debug.Assert(indexInBlock < Buffer.PositiveDistance(PtrBegin, PtrEnd));\n }\n }\n Ptr = BufferBegin + indexInBlock;\n }\n\n internal void Seek(char* ptr, int block) {\n if (ptr != null) {\n if (block != Block) {\n Debug.Assert(BlockData != null && ptr >= BufferBegin && ptr < BufferBegin + BlockData.BlockSize);\n int indexInBlock = (int)Buffer.PositiveDistance(BufferBegin, ptr);\n Seek(block, indexInBlock);\n } else {\n Debug.Assert(ptr >= BufferBegin && ptr < BufferEnd);\n Ptr = ptr;\n PtrBegin = BufferBegin;\n PtrEnd = BufferEnd;\n }\n } else {\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n }\n }\n\n private void SeekToFirstCharAfterLastCharOfCurrentBlock() {\n if (Ptr != null) {\n if (BlockData != null && Block != BlockData.LastBlock) BlockData.ReadBlock(Block + 1);\n else {\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n }\n }\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public CharStreamIndexToken IndexToken { get {\n return new CharStreamIndexToken(\n #if DEBUG\n this,\n #endif\n Ptr,\n Block\n );\n } }\n\n private static void ThrowInvalidIndexToken() {\n throw new ArgumentException(\"The CharStreamIndexToken is invalid.\");\n }\n\n public void Seek(CharStreamIndexToken indexToken) {\n int block = indexToken.Block;\n if (block < 0) ThrowInvalidIndexToken(); // tests for zero-initialized IndexTokens\n #if DEBUG\n Debug.Assert(this == indexToken.CharStream);\n #endif\n if (Ptr != null && indexToken.Ptr != null && block == Block) {\n Ptr = indexToken.Ptr;\n Debug.Assert(Ptr >= BufferBegin && Ptr < BufferEnd);\n } else {\n Seek(indexToken.Ptr, block);\n }\n ++StateTag;\n }\n\n // Below we split many methods into a default method containing the code\n // for the most frequently used branch and a \"...Continue\" method containing\n // the code for the remaining branches. This allows the JIT to produce\n // faster code for the main branch and in a few cases even to inline it.\n\n public string ReadFrom(CharStreamIndexToken indexOfFirstChar) {\n int block = indexOfFirstChar.Block;\n if (block < 0) ThrowInvalidIndexToken(); // tests for zero-initialized IndexTokens\n #if DEBUG\n Debug.Assert(this == indexOfFirstChar.CharStream);\n #endif\n return ReadFrom(indexOfFirstChar.Ptr, block);\n }\n\n internal string ReadFrom(char* ptr, int block) {\n if (ptr != null && ptr < Ptr && block == Block) {\n Debug.Assert(BufferBegin <= ptr && Ptr < BufferEnd);\n return new string(ptr, 0, (int)Buffer.PositiveDistance(ptr, Ptr));\n }\n return ReadFromContinue(ptr, block);\n }\n private string ReadFromContinue(char* ptr, int block) {\n ulong index1 = (ulong)GetIndex(ptr, block);\n ulong index2 = (ulong)Index;\n\n if (index1 < index2) {\n ulong length_ = index2 - index1;\n // The maximum theoretical string size is Int32.MaxValue,\n // though on .NET it is actually less than 2^30, since the maximum\n // object size is limited to Int32.MaxValue, even on 64-bit systems.\n if (length_ > Int32.MaxValue) {\n // OutOfMemoryException is the exception the .NET string constructor throws\n // if the the string length is larger than the maximum string length,\n // even if enough memory would be available.\n throw new OutOfMemoryException();\n }\n int length = (int)length_;\n var stateTag = StateTag;\n Seek(ptr, block);\n var str = Read((int)length);\n StateTag = stateTag;\n return str;\n } else if (index1 > index2) throw new ArgumentException(\"The current position of the stream must not lie before the position corresponding to the given CharStreamIndexToken/CharStreamState.\");\n return \"\";\n }\n\n public void RegisterNewline() {\n var index = Index;\n Debug.Assert(index != _LineBegin);\n _LineBegin = index;\n ++_Line;\n ++StateTag;\n }\n\n private void RegisterNewlines(char* lineBegin, uint lineOffset) {\n Debug.Assert(BufferBegin <= lineBegin && lineBegin <= BufferEnd && lineOffset > 0);\n _Line += lineOffset;\n long newLineBegin = Buffer.PositiveDistance(BufferBegin, lineBegin) + IndexOfFirstCharInBlock;\n Debug.Assert(newLineBegin != _LineBegin);\n _LineBegin = newLineBegin;\n ++StateTag;\n }\n\n public void RegisterNewlines(int lineOffset, int newColumnMinus1) {\n Debug.Assert(lineOffset != 0 && newColumnMinus1 >= 0);\n _Line += lineOffset;\n Debug.Assert(_Line > 0);\n var newLineBegin = Index - newColumnMinus1;\n Debug.Assert(newLineBegin != _LineBegin);\n _LineBegin = Index - newColumnMinus1;\n ++StateTag;\n }\n\n public void RegisterNewlines(long lineOffset, long newColumnMinus1) {\n Debug.Assert(lineOffset != 0 && newColumnMinus1 >= 0);\n _Line += lineOffset;\n Debug.Assert(_Line > 0);\n var newLineBegin = Index - newColumnMinus1;\n Debug.Assert(newLineBegin != _LineBegin);\n _LineBegin = Index - newColumnMinus1;\n ++StateTag;\n }\n\n public char Peek() {\n char* ptr = Ptr;\n if (ptr != null) return *ptr;\n return EOS;\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public void Skip() {\n char* ptr1 = Ptr + 1;\n if (ptr1 < PtrEnd) {\n Ptr = ptr1;\n ++StateTag;\n return;\n }\n SkipContinue();\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private void SkipContinue() { SkipContinue(1u); }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public char Read() {\n char* ptr = Ptr;\n char* ptr1 = ptr + 1;\n if (ptr1 < PtrEnd) {\n char c = *ptr;\n Ptr = ptr1;\n ++StateTag;\n return c;\n }\n return ReadContinue();\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char ReadContinue() {\n var c = Peek();\n Skip();\n return c;\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public char SkipAndPeek() {\n char* ptr = Ptr + 1;\n if (ptr < PtrEnd) {\n Ptr = ptr;\n ++StateTag;\n return *ptr;\n }\n return SkipAndPeekContinue();\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char SkipAndPeekContinue() { return SkipAndPeekContinue(1u); }\n\n private static readonly bool IsLittleEndian = BitConverter.IsLittleEndian; // improves inlining and dead code elimination, at least with the .NET JIT\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public TwoChars Peek2() {\n char* ptr = Ptr;\n if (ptr + 1 < PtrEnd) {\n #if UNALIGNED_READS\n if (IsLittleEndian) {\n return new TwoChars(*((uint*)ptr));\n } else {\n return new TwoChars(ptr[0], ptr[1]);\n }\n #else\n return new TwoChars(ptr[0], ptr[1]);\n #endif\n }\n return Peek2Continue();\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private TwoChars Peek2Continue() {\n return new TwoChars(Peek(), Peek(1u));\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public char Peek(uint utf16Offset) {\n if (utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd))\n return Ptr[utf16Offset];\n return PeekContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char PeekContinue(uint utf16Offset) {\n if (Ptr == null || BlockData == null || Block == BlockData.LastBlock) return EOS;\n char* ptr = Ptr;\n int block = Block;\n var stateTag = StateTag;\n Seek(Index + utf16Offset);\n char c = Peek();\n Seek(ptr, block); // backtrack\n StateTag = stateTag;\n return c;\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public void Skip(uint utf16Offset) {\n if (utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)) {\n Ptr += utf16Offset;\n ++StateTag;\n return;\n }\n SkipContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private void SkipContinue(uint utf16Offset) {\n if (Ptr == null || utf16Offset == 0) return;\n if (BlockData == null || Block == BlockData.LastBlock) {\n Ptr = null;\n PtrBegin = null;\n PtrEnd = null;\n ++StateTag;\n return;\n }\n Seek(Index + utf16Offset);\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public char SkipAndPeek(uint utf16Offset) {\n if (utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)) {\n char* ptr = Ptr + utf16Offset;\n Ptr = ptr;\n ++StateTag;\n return *ptr;\n }\n return SkipAndPeekContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char SkipAndPeekContinue(uint utf16Offset) {\n SkipContinue(utf16Offset);\n return Peek();\n }\n\n public char Peek(int utf16Offset) { // don't force inlining, because the .NET JIT doesn't optimize after inlining\n if (utf16Offset >= 0\n ? utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)\n : unchecked((uint)-utf16Offset) <= Buffer.PositiveDistance(PtrBegin, Ptr))\n {\n return Ptr[utf16Offset];\n }\n return PeekContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char PeekContinue(int utf16Offset) {\n if (utf16Offset >= 0) return PeekContinue((uint)utf16Offset);\n var newIndex = Index + utf16Offset;\n if (newIndex >= _IndexOfFirstChar) {\n char* ptr = Ptr;\n int block = Block;\n var stateTag = StateTag;\n Seek(Index + utf16Offset);\n char c = Peek();\n Seek(ptr, block);\n StateTag = stateTag;\n return c;\n }\n return EOS;\n }\n\n public void Skip(int utf16Offset) {\n if (utf16Offset >= 0\n ? utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)\n : unchecked((uint)-utf16Offset) <= Buffer.PositiveDistance(PtrBegin, Ptr))\n {\n Ptr = unchecked(Ptr + utf16Offset); // see https://connect.microsoft.com/VisualStudio/feedback/details/522944\n ++StateTag;\n return;\n }\n SkipContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private void SkipContinue(int utf16Offset) {\n if (utf16Offset >= 0) {\n SkipContinue((uint)utf16Offset);\n return;\n }\n Seek(Index + utf16Offset);\n }\n\n public void Skip(long utf16Offset) {\n if (utf16Offset >= 0\n ? utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)\n : unchecked((ulong)-utf16Offset) <= Buffer.PositiveDistance(PtrBegin, Ptr))\n {\n Ptr = unchecked(Ptr + utf16Offset); // see https://connect.microsoft.com/VisualStudio/feedback/details/522944\n ++StateTag;\n return;\n }\n SkipContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private void SkipContinue(long utf16Offset) {\n long index = Index;\n Seek(utf16Offset > Int64.MaxValue - index ? Int64.MaxValue : index + utf16Offset);\n }\n\n public char SkipAndPeek(int utf16Offset) {\n if (utf16Offset >= 0\n ? utf16Offset < Buffer.PositiveDistance(Ptr, PtrEnd)\n : unchecked((uint)-utf16Offset) <= Buffer.PositiveDistance(PtrBegin, Ptr))\n {\n char* ptr = unchecked(Ptr + utf16Offset); // see https://connect.microsoft.com/VisualStudio/feedback/details/522944\n Ptr = ptr;\n ++StateTag;\n return *ptr;\n }\n return SkipAndPeekContinue(utf16Offset);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private char SkipAndPeekContinue(int utf16Offset) {\n if (utf16Offset >= 0) {\n SkipContinue((uint)utf16Offset);\n return Peek();\n }\n var newIndex = Index + utf16Offset;\n if (newIndex >= IndexOfFirstChar) {\n Seek(Index + utf16Offset);\n return Peek();\n } else {\n Seek(_IndexOfFirstChar);\n return EOS;\n }\n }\n\n public string PeekString(int length) {\n if (unchecked((uint)length) <= Buffer.PositiveDistance(Ptr, PtrEnd))\n return new String(Ptr, 0, length);\n return PeekStringContinue(length);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private string PeekStringContinue(int length) {\n return ReadContinue(length, true);\n }\n\n public string Read(int length) {\n char* ptr = Ptr;\n if (unchecked((uint)length) < Buffer.PositiveDistance(ptr, PtrEnd)) {\n Ptr += length;\n ++StateTag;\n return new String(ptr, 0, length);\n }\n return ReadContinue(length);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private string ReadContinue(int length) {\n return ReadContinue(length, false);\n }\n\n private string ReadContinue(int length, bool backtrack) {\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n if (length == 0 || Ptr == null) return \"\";\n if (BlockData == null) {\n int maxLength = (int)Buffer.PositiveDistance(Ptr, PtrEnd);\n if (length > maxLength)\n length = maxLength;\n } else {\n long maxLength = BlockData.IndexOfLastCharPlus1 - Index;\n if (length > maxLength)\n length = (int)maxLength;\n }\n string str = new String('\\u0000', length);\n fixed (char* pStr = str) {\n int cc = ReadContinue(pStr, length, backtrack);\n if (cc == length) return str;\n return new String(pStr, 0, cc);\n }\n }\n\n public int PeekString(char[] buffer, int bufferIndex, int length) {\n return Read(buffer, bufferIndex, length, true);\n }\n public int Read(char[] buffer, int bufferIndex, int length) {\n return Read(buffer, bufferIndex, length, false);\n }\n private int Read(char[] buffer, int bufferIndex, int length, bool backtrack) {\n if (bufferIndex < 0)\n throw new ArgumentOutOfRangeException(\"bufferIndex\", \"bufferIndex is negative.\");\n if (length > buffer.Length - bufferIndex) // throws if buffer is null\n throw new ArgumentOutOfRangeException(\"length\", \"bufferIndex or length is out of range.\");\n // We must exit early for length == 0, because pining an empty array\n // would invoke implementation-defined behaviour.\n if (length <= 0) {\n if (length == 0) return 0;\n throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n }\n fixed (char* pBuffer = buffer)\n return Read(pBuffer + bufferIndex, length, backtrack);\n }\n\n public int PeekString(char* buffer, int length) {\n return Read(buffer, length, true);\n }\n public int Read(char* buffer, int length) {\n return Read(buffer, length, false);\n }\n private int Read(char* buffer, int length, bool backtrack) {\n if (unchecked((uint)length) < Buffer.PositiveDistance(Ptr, PtrEnd)) {\n char* ptr = Ptr;\n int len = length;\n #if UNALIGNED_READS\n if ((unchecked((int)buffer) & 2) != 0 && len != 0) { // align buffer pointer\n *buffer = *ptr;\n ++buffer; ++ptr; --len;\n }\n len -= 8;\n while (len >= 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n ((int*)buffer)[1] = ((int*)ptr)[1];\n ((int*)buffer)[2] = ((int*)ptr)[2];\n ((int*)buffer)[3] = ((int*)ptr)[3];\n buffer += 8; ptr += 8; len -= 8;\n }\n if ((len & 4) != 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n ((int*)buffer)[1] = ((int*)ptr)[1];\n buffer += 4; ptr += 4;\n }\n if ((len & 2) != 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n buffer += 2; ptr += 2;\n }\n #else\n len -= 2;\n while (len >= 0) {\n buffer[0] = ptr[0];\n buffer[1] = ptr[1];\n buffer += 2; ptr += 2; len -= 2;\n }\n #endif\n if ((len & 1) != 0) {\n *buffer = *ptr;\n ++ptr;\n }\n if (!backtrack) {\n Ptr = ptr;\n ++StateTag;\n }\n return length;\n }\n return ReadContinue(buffer, length, backtrack);\n }\n private int ReadContinue(char* buffer, int length, bool backtrack) {\n if (length < 0)\n throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n\n if (length == 0 || Ptr == null) return 0;\n\n int oldLength = length;\n int oldBlock = Block;\n char* oldPtr = Ptr;\n char* ptr = Ptr;\n\n do {\n int len = Math.Min((int)Buffer.PositiveDistance(Ptr, PtrEnd), length);\n Debug.Assert(length > 0 && len > 0);\n length -= len;\n #if UNALIGNED_READS\n if ((unchecked((int)buffer) & 2) != 0) { // align buffer pointer\n *buffer = *ptr;\n ++buffer; ++ptr; --len;\n }\n len -= 8;\n while (len >= 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n ((int*)buffer)[1] = ((int*)ptr)[1];\n ((int*)buffer)[2] = ((int*)ptr)[2];\n ((int*)buffer)[3] = ((int*)ptr)[3];\n buffer += 8; ptr += 8; len -= 8;\n }\n if ((len & 4) != 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n ((int*)buffer)[1] = ((int*)ptr)[1];\n buffer += 4; ptr += 4;\n }\n if ((len & 2) != 0) {\n ((int*)buffer)[0] = ((int*)ptr)[0];\n buffer += 2; ptr += 2;\n }\n #else\n len -= 2;\n while (len >= 0) {\n buffer[0] = ptr[0];\n buffer[1] = ptr[1];\n buffer += 2; ptr += 2; len -= 2;\n }\n #endif\n if ((len & 1) != 0) {\n *buffer = *ptr;\n ++buffer; ++ptr;\n }\n } while (length != 0\n && BlockData != null\n && Block != BlockData.LastBlock\n && (ptr = BlockData.ReadBlock(Block + 1)) != null);\n if (!backtrack) {\n ++StateTag;\n if (ptr != PtrEnd) Ptr = ptr;\n else SeekToFirstCharAfterLastCharOfCurrentBlock();\n } else {\n if (Block != oldBlock) Seek(oldPtr, oldBlock);\n }\n return oldLength - length;\n }\n\n public bool Match(char ch) {\n char* ptr = Ptr;\n return ptr != null && ch == *ptr;\n }\n\n public bool MatchCaseFolded(char caseFoldedChar) {\n char* ptr = Ptr;\n return ptr != null && caseFoldedChar == CaseFoldTable.FoldedChars[*ptr];\n }\n\n public bool Skip(char ch) {\n char* ptr1 = Ptr + 1;\n if (ptr1 < PtrEnd && ch == *Ptr) {\n Ptr = ptr1;\n ++StateTag;\n return true;\n }\n return SkipContinue(ch);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private bool SkipContinue(char ch) {\n if (Match(ch)) {\n Skip();\n return true;\n }\n return false;\n }\n\n public bool SkipCaseFolded(char caseFoldedChar) {\n char* ptr1 = Ptr + 1;\n if (ptr1 < PtrEnd && caseFoldedChar == CaseFoldTable.FoldedChars[*Ptr]) {\n Ptr = ptr1;\n ++StateTag;\n return true;\n }\n return SkipCaseFoldedContinue(caseFoldedChar);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private bool SkipCaseFoldedContinue(char caseFoldedChar) {\n if (MatchCaseFolded(caseFoldedChar)) {\n Skip();\n return true;\n }\n return false;\n }\n\n#if AGGRESSIVE_INLINING\n [MethodImpl(MethodImplOptions.AggressiveInlining)]\n#endif\n public bool Skip(TwoChars twoChars) {\n char* ptr2 = Ptr + 2;\n if (ptr2 < PtrEnd) {\n #if UNALIGNED_READS\n if (IsLittleEndian) {\n if (new TwoChars(*((uint*)Ptr)) == twoChars) {\n Ptr = ptr2;\n ++StateTag;\n return true;\n }\n } else {\n if (twoChars.Char0 == Ptr[0] && twoChars.Char1 == Ptr[1]) {\n Ptr = ptr2;\n ++StateTag;\n return true;\n }\n }\n #else\n if (twoChars.Char0 == Ptr[0] && twoChars.Char1 == Ptr[1]) {\n Ptr = ptr2;\n ++StateTag;\n return true;\n }\n #endif\n return false;\n }\n return SkipContinue(twoChars);\n }\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private bool SkipContinue(TwoChars twoChars) {\n char* cs = stackalloc char[2];\n cs[0] = twoChars.Char0;\n cs[1] = twoChars.Char1;\n return SkipContinue(cs, 2, false);\n }\n\n public bool Match(string chars) {\n if (chars.Length <= Buffer.PositiveDistance(Ptr, PtrEnd)) {\n for (int i = 0; i < chars.Length; ++i) {\n if (Ptr[i] != chars[i]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n return SkipContinue(chars, true);\n }\n\n public bool Skip(string chars) {\n if (chars.Length < Buffer.PositiveDistance(Ptr, PtrEnd)) {\n for (int i = 0; i < chars.Length; ++i) {\n if (Ptr[i] != chars[i]) goto ReturnFalse;\n }\n Ptr += chars.Length;\n ++StateTag;\n return true;\n ReturnFalse:\n return false;\n }\n return SkipContinue(chars, false);\n }\n\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private bool SkipContinue(string chars, bool backtrackEvenIfCharsMatch) {\n fixed (char* pChars = chars)\n return SkipContinue(pChars, chars.Length, backtrackEvenIfCharsMatch);\n }\n\n public bool MatchCaseFolded(string caseFoldedChars) {\n if (caseFoldedChars.Length <= Buffer.PositiveDistance(Ptr, PtrEnd)) {\n for (int i = 0; i < caseFoldedChars.Length; ++i) {\n if (CaseFoldTable.FoldedChars[Ptr[i]] != caseFoldedChars[i]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n return SkipCaseFoldedContinue(caseFoldedChars, true);\n }\n\n public bool SkipCaseFolded(string caseFoldedChars) {\n if (caseFoldedChars.Length < Buffer.PositiveDistance(Ptr, PtrEnd)) {\n for (int i = 0; i < caseFoldedChars.Length; ++i) {\n if (CaseFoldTable.FoldedChars[Ptr[i]] != caseFoldedChars[i]) goto ReturnFalse;\n }\n Ptr += caseFoldedChars.Length;\n ++StateTag;\n return true;\n ReturnFalse:\n return false;\n }\n return SkipCaseFoldedContinue(caseFoldedChars, false);\n }\n\n [MethodImplAttribute(MethodImplOptions.NoInlining)]\n private bool SkipCaseFoldedContinue(string caseFoldedChars, bool backtrackEvenIfCharsMatch) {\n fixed (char* pCaseFoldedChars = caseFoldedChars)\n return SkipCaseFoldedContinue(pCaseFoldedChars, caseFoldedChars.Length, backtrackEvenIfCharsMatch);\n }\n\n public bool Match(char[] chars, int charsIndex, int length) {\n return Skip(chars, charsIndex, length, true);\n }\n public bool Skip(char[] chars, int charsIndex, int length) {\n return Skip(chars, charsIndex, length, false);\n }\n private bool Skip(char[] chars, int charsIndex, int length, bool backtrack) {\n if (charsIndex < 0)\n throw new ArgumentOutOfRangeException(\"charsIndex\", \"charsIndex is negative.\");\n if (length > chars.Length - charsIndex) // throws if chars is null\n throw new ArgumentOutOfRangeException(\"length\", \"length is out of range.\");\n // We must exit early for length == 0, because pining an empty array\n // would invoke implementation-defined behaviour.\n if (length <= 0) {\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n if (!backtrack) ++StateTag;\n return true;\n }\n fixed (char* pChars = chars) return Skip(pChars + charsIndex, length, backtrack);\n }\n\n public bool Match(char* chars, int length) {\n return Skip(chars, length, true);\n }\n public bool Skip(char* chars, int length) {\n return Skip(chars, length, false);\n }\n private bool Skip(char* chars, int length, bool backtrackEvenIfCharsMatch) {\n if (unchecked((uint)length < Buffer.PositiveDistance(Ptr, PtrEnd))) {\n #if UNALIGNED_READS\n char* ptr = Ptr;\n int len = length - 2;\n while (len >= 0) {\n if (*((int*)ptr) != *((int*)chars)) goto ReturnFalse;\n ptr += 2; chars += 2; len -= 2;\n }\n if ((len & 1) != 0) {\n if (*ptr != *chars) goto ReturnFalse;\n ++ptr;\n }\n #else\n char* ptr = Ptr;\n int len = length;\n while (len != 0) {\n if (*ptr != *chars) goto ReturnFalse;\n ++ptr; ++chars; --len;\n }\n #endif\n if (!backtrackEvenIfCharsMatch) {\n Ptr = ptr;\n ++StateTag;\n }\n return true;\n ReturnFalse:\n return false;\n }\n return SkipContinue(chars, length, backtrackEvenIfCharsMatch);\n }\n\n public bool MatchCaseFolded(char* caseFoldedChars, int length) {\n return SkipCaseFolded(caseFoldedChars, length, true);\n }\n public bool SkipCaseFolded(char* caseFoldedChars, int length) {\n return SkipCaseFolded(caseFoldedChars, length, false);\n }\n private bool SkipCaseFolded(char* caseFoldedChars, int length, bool backtrackEvenIfCharsMatch) {\n if (unchecked((uint)length < Buffer.PositiveDistance(Ptr, PtrEnd))) {\n char* ptr = Ptr;\n int len = length;\n while (len != 0) {\n if (CaseFoldTable.FoldedChars[*ptr] != *caseFoldedChars) goto ReturnFalse;\n ++ptr; ++caseFoldedChars; --len;\n }\n if (!backtrackEvenIfCharsMatch) {\n Ptr = ptr;\n ++StateTag;\n }\n return true;\n ReturnFalse:\n return false;\n }\n return SkipCaseFoldedContinue(caseFoldedChars, length, backtrackEvenIfCharsMatch);\n }\n\n private bool SkipContinue(char* chars, int length, bool backtrackEvenIfCharsMatch) {\n if (length <= 0) {\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n return true;\n }\n\n if (Ptr == null) return false;\n\n int oldBlock = Block;\n char* oldPtr = Ptr;\n char* ptr = Ptr;\n\n for (;;) {\n Debug.Assert(length > 0);\n int len = (int)Buffer.PositiveDistance(ptr, PtrEnd);\n if (len < length) {\n if (BlockData == null || Block == BlockData.LastBlock) goto ReturnFalse;\n length -= len;\n } else {\n len = length;\n length = 0;\n }\n Debug.Assert(len > 0);\n #if UNALIGNED_READS\n len -= 2;\n while (len >= 0) {\n if (*((int*)ptr) != *((int*)chars)) goto ReturnFalse;\n ptr += 2; chars += 2; len -= 2;\n }\n if ((len & 1) != 0) {\n if (*ptr != *chars) goto ReturnFalse;\n ++ptr; ++chars;\n }\n #else\n do {\n if (*ptr != *chars) goto ReturnFalse;\n ++ptr; ++chars; --len;\n } while (len != 0);\n #endif\n if (length != 0) {\n Debug.Assert(BlockData != null && Block != BlockData.LastBlock);\n ptr = BlockData.ReadBlock(Block + 1);\n } else {\n if (backtrackEvenIfCharsMatch) {\n if (Block != oldBlock) Seek(oldPtr, oldBlock);\n } else {\n if (ptr != PtrEnd) Ptr = ptr;\n else SeekToFirstCharAfterLastCharOfCurrentBlock();\n ++StateTag;\n }\n return true;\n }\n }\n ReturnFalse:\n if (Block != oldBlock) Seek(oldPtr, oldBlock);\n return false;\n }\n\n private bool SkipCaseFoldedContinue(char* caseFoldedChars, int length, bool backtrackEvenIfCharsMatch) {\n if (length <= 0) {\n if (length == 0) return true;\n throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n }\n\n if (Ptr == null) return false;\n\n int oldBlock = Block;\n char* oldPtr = Ptr;\n char* ptr = Ptr;\n\n for (;;) {\n Debug.Assert(length > 0);\n int len = (int)Buffer.PositiveDistance(ptr, PtrEnd);\n if (len < length) {\n if (BlockData == null || Block == BlockData.LastBlock) goto ReturnFalse;\n length -= len;\n } else {\n len = length;\n length = 0;\n }\n Debug.Assert(len > 0);\n do {\n if (CaseFoldTable.FoldedChars[*ptr] != *caseFoldedChars) goto ReturnFalse;\n ++ptr; ++caseFoldedChars; --len;\n } while (len != 0);\n if (length != 0) {\n Debug.Assert(BlockData != null && Block != BlockData.LastBlock);\n ptr = BlockData.ReadBlock(Block + 1);\n } else {\n if (backtrackEvenIfCharsMatch) {\n if (Block != oldBlock) Seek(oldPtr, oldBlock);\n } else {\n if (ptr != PtrEnd) Ptr = ptr;\n else SeekToFirstCharAfterLastCharOfCurrentBlock();\n ++StateTag;\n }\n return true;\n }\n }\n ReturnFalse:\n if (Block != oldBlock) Seek(oldPtr, oldBlock);\n return false;\n }\n\n public Match Match(Regex regex) {\n if (BufferString == null) throw new NotSupportedException(\"CharStream instances constructed from char arrays or char pointers do not support regular expression matching.\");\n if (Ptr != null) {\n if (BlockData != null && Ptr > BlockData.RegexSpaceThreshold && Block != BlockData.LastBlock) {\n // BlockOverlap > MinRegexSpace\n char c = *Ptr;\n char* ptr = Ptr;\n BlockData.ReadBlock(Block + 1);\n int blockSizeMinusOverlap = BlockData.BlockSize - BlockData.BlockOverlap;\n Ptr = ptr - blockSizeMinusOverlap;\n PtrBegin = BufferBegin; // might have been set to null by ReadBlock\n PtrEnd = BufferEnd;\n Debug.Assert(*Ptr == c && BufferBegin <= Ptr && Ptr < BufferEnd);\n }\n int index = (int)Buffer.PositiveDistance(BufferStringPointer, Ptr);\n int length = (int)Buffer.PositiveDistance(Ptr, BufferEnd);\n return regex.Match(BufferString, index, length);\n }\n return regex.Match(\"\");\n }\n\n public bool SkipWhitespace() {\n char* lineBegin = null;\n uint lineOffset = 0;\n char* ptr = Ptr;\n char* end = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r',\n if (ptr + 1 < PtrEnd) { // PtrEnd might be null\n char c = *ptr;\n ++ptr;\n if (c > ' ') goto ReturnFalse;\n if (c == ' ') {\n if (*ptr > ' ') {\n Ptr = ptr;\n ++StateTag;\n return true;\n }\n goto Loop;\n } else {\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n if (ptr > end) goto Newline;\n }\n } else if (c != '\\n') goto CheckTab;\n if (*ptr > ' ') {\n Ptr = ptr;\n RegisterNewline();\n return true;\n }\n goto Newline;\n CheckTab:\n if (c != '\\t') goto ReturnFalse;\n goto Loop;\n }\n Newline:\n lineBegin = ptr;\n ++lineOffset;\n Loop:\n for (;;) {\n if (ptr >= end) break;\n c = *ptr;\n ++ptr;\n if (c != ' ') {\n if (c != '\\t') {\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n --ptr;\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return true;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return true;\n }\n }\n }\n }\n }\n return SkipWhitespaceContinue(ptr, lineBegin, lineOffset);\n ReturnFalse:\n return false;\n }\n private bool SkipWhitespaceContinue(char* ptr, char* lineBegin, uint lineOffset) {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n if (index == 0) {\n c = Peek();\n if (c == ' ' || c == '\\t') c = SkipAndPeek();\n else if (c != '\\r' && c != '\\n') return false;\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n }\n for (;;) {\n if (c == ' ' || c == '\\t') c = SkipAndPeek();\n else if (c != '\\r' && c != '\\n') {\n StateTag = stateTag + 1;\n return true;\n } else {\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n }\n }\n }\n\n public bool SkipUnicodeWhitespace() {\n char* lineBegin = null;\n uint lineOffset = 0;\n char* end = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r'\n char* ptr = Ptr;\n if (ptr + 1 < PtrEnd) { // PtrEnd might be null\n char c = *ptr;\n ++ptr;\n if (c == ' ') goto Loop;\n if (!Text.IsWhitespace(c)) return false;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n } else if (c != '\\n') goto Loop;\n } else {\n if (c < '\\u2028' ? c != '\\u0085' : c > '\\u2029') goto Loop;\n }\n Newline:\n lineBegin = ptr;\n ++lineOffset;\n Loop:\n for (;;) {\n if (ptr >= end) break;\n c = *ptr;\n ++ptr;\n if (c != ' ') {\n if (Text.IsWhitespace(c)) {\n if (c <= '\\r') {\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n } else if (c < '\\u2028' ? c == '\\u0085' : c <= '\\u2029') goto Newline;\n } else {\n --ptr;\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return true;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return true;\n }\n }\n }\n }\n }\n return SkipUnicodeWhitespaceContinue(ptr, lineBegin, lineOffset);\n }\n private bool SkipUnicodeWhitespaceContinue(char* ptr, char* lineBegin, uint lineOffset) {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n if (index == 0) {\n c = Peek();\n if (!Text.IsWhitespace(c)) return false;\n if (c == ' ' || c == '\\t') c = SkipAndPeek();\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n }\n for (;;) {\n if (c == ' ') c = SkipAndPeek();\n else {\n if (!Text.IsWhitespace(c)) break;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') c = SkipAndPeek();\n } else if (c0 != '\\n') continue;\n } else if (c0 < '\\u2028' ? c0 != '\\u0085' : c0 > '\\u2029') continue;\n RegisterNewline();\n }\n }\n StateTag = stateTag + 1;\n return true;\n }\n\n public bool SkipNewline() {\n var ptr = Ptr;\n if (ptr + 2 < PtrEnd) {\n char c = *ptr;\n ++ptr;\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n } else if (c != '\\n') return false;\n Ptr = ptr;\n RegisterNewline();\n return true;\n } else {\n var stateTag = StateTag;\n char c = Peek();\n if (c == '\\r') {\n c = SkipAndPeek();\n if (c == '\\n') Skip();\n } else {\n if (c != '\\n') return false;\n Skip();\n }\n RegisterNewline();\n StateTag = stateTag + 1;\n return true;\n }\n }\n\n public bool SkipUnicodeNewline() {\n var ptr = Ptr;\n if (ptr + 2 < PtrEnd) {\n char c = *ptr;\n ++ptr;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n } else if (c != '\\n') goto ReturnFalse;\n } else if (c >= '\\u2028' ? c > '\\u2029' : c != '\\u0085') goto ReturnFalse;\n Ptr = ptr;\n RegisterNewline();\n return true;\n } else {\n char c = Peek();\n uint n = 1;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (Peek(1u) == '\\n') n = 2;\n } else if (c != '\\n') goto ReturnFalse;\n } else if (c >= '\\u2028' ? c > '\\u2029' : c != '\\u0085') goto ReturnFalse;\n Skip(n);\n var stateTag = StateTag;\n RegisterNewline();\n StateTag = stateTag;\n return true;\n }\n ReturnFalse:\n return false;\n }\n\n public int SkipNewlineThenWhitespace(int powerOf2TabStopDistance, bool allowFormFeed) {\n int tabStopDistanceMinus1 = unchecked(powerOf2TabStopDistance - 1);\n if (powerOf2TabStopDistance <= 0 || (powerOf2TabStopDistance & tabStopDistanceMinus1) != 0)\n throw new ArgumentOutOfRangeException(\"powerOf2TabStopDistance\", \"powerOf2TabStopDistance must be a positive power of 2.\");\n\n char* lineBegin = null;\n uint lineOffset = 0;\n int ind = -1;\n char* end = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r'\n char* ptr = Ptr;\n if (ptr + 1 < PtrEnd) { // PtrEnd might be null\n char c = *ptr;\n ++ptr;\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n } else if (c != '\\n') {\n return -1;\n }\n Newline:\n lineBegin = ptr;\n ++lineOffset;\n ind = 0;\n for (;;) {\n if (ptr >= end) break;\n c = *ptr;\n ++ptr;\n if (c == ' ') {\n ind = unchecked(ind + 1);\n if (ind >= 0) continue;\n // indentation has overflown, so put back ' ' and return\n ind = unchecked(ind - 1);\n } else if (c <= '\\r') {\n if (c == '\\r') {\n if (*ptr == '\\n') ++ptr;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n if (c == '\\t') {\n // ind = ind + tabStopDistance - ind%tabStopDistance\n int d = tabStopDistanceMinus1 + 1 - (ind & tabStopDistanceMinus1);\n ind = unchecked(ind + d);\n if (ind >= 0) continue;\n // indentation has overflown, so put back '\\t' and return\n ind = unchecked(ind - d);\n } else if (c == '\\f' && allowFormFeed) {\n ind = 0;\n continue;\n }\n }\n --ptr;\n Ptr = ptr;\n RegisterNewlines(lineBegin, lineOffset);\n return ind;\n }\n // end of block\n }\n return SkipNewlineWhitespaceContinue(ptr, lineBegin, lineOffset, ind, tabStopDistanceMinus1, allowFormFeed);\n }\n private int SkipNewlineWhitespaceContinue(char* ptr, char* lineBegin, uint lineOffset, int ind_,\n int tabStopDistanceMinus1, bool allowFormFeed)\n {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n if (index == 0) {\n c = Peek();\n if (!(c == '\\r' || c == '\\n')) return -1;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n }\n int ind = ind_;\n for (;;) {\n if (c == ' ') {\n ind = unchecked(ind + 1);\n if (ind >= 0) c = SkipAndPeek();\n else {\n // indentation has overflown, so put back ' ' and return\n ind = unchecked(ind - 1);\n break;\n }\n } else if (c == '\\r' || c == '\\n') {\n ind = 0;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n } else if (c == '\\t') {\n // ind = ind + tabStopDistance - ind%tabStopDistance\n int d = tabStopDistanceMinus1 + 1 - (ind & tabStopDistanceMinus1);\n ind = unchecked(ind + d);\n if (ind >= 0) c = SkipAndPeek();\n else {\n // indentation has overflown, so put back '\\t' and return\n ind = unchecked(ind - d);\n break;\n }\n } else if (c == '\\f' && allowFormFeed) {\n ind = 0;\n c = SkipAndPeek();\n } else break;\n }\n StateTag = stateTag + 1;\n return ind;\n }\n\n public void SkipRestOfLine(bool skipNewline) {\n char* ptr = Ptr;\n char* end = unchecked(PtrEnd - 2); // - 2, so that we can do (*) without further checking\n if (ptr + 2 < PtrEnd) { // PtrEnd might be null\n for (;;) {\n char c = *ptr;\n if (c > '\\r') {\n if (++ptr == end) break;\n } else if (c != '\\r' && c != '\\n') {\n if (++ptr == end) break;\n } else {\n if (!skipNewline) {\n if (ptr != Ptr) {\n Ptr = ptr;\n ++StateTag;\n }\n return;\n } else {\n ++ptr;\n if (c == '\\r' && *ptr == '\\n') ++ptr;\n Ptr = ptr; // (*)\n RegisterNewline();\n return;\n }\n }\n }\n }\n SkipRestOfLineContinue(ptr, skipNewline);\n }\n private void SkipRestOfLineContinue(char* ptr, bool skipNewline) {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n if (index == 0) {\n c = Peek();\n if (c == EOS || (!skipNewline && (c == '\\r' || c == '\\n'))) return;\n } else {\n c = SkipAndPeek(index);\n }\n while (c != EOS) {\n if (c == '\\r' || c == '\\n') {\n if (skipNewline) SkipNewline();\n break;\n }\n c = SkipAndPeek();\n }\n StateTag = stateTag + 1;\n return;\n }\n\n public string ReadRestOfLine(bool skipNewline) {\n char* ptr = Ptr;\n char* end = unchecked(PtrEnd - 2); // - 2, so that we can do (*) without further checking\n if (ptr + 2 < PtrEnd) { // PtrEnd might be null\n for (;;) {\n char c = *ptr;\n if (c > '\\r') {\n if (++ptr == end) break;\n } else if (c != '\\r' && c != '\\n') {\n if (++ptr == end) break;\n } else {\n char* ptr0 = Ptr;\n if (!skipNewline) {\n if (ptr != ptr0) {\n Ptr = ptr;\n ++StateTag;\n return new string(ptr0, 0, (int)Buffer.PositiveDistance(ptr0, ptr));\n } else {\n return \"\";\n }\n } else {\n var skippedString = ptr == ptr0 ? \"\" : new string(ptr0, 0, (int)Buffer.PositiveDistance(ptr0, ptr));\n ++ptr;\n if (c == '\\r' && *ptr == '\\n') ++ptr;\n Ptr = ptr; // (*)\n RegisterNewline();\n return skippedString;\n }\n }\n }\n }\n return ReadRestOfLineContinue(ptr, skipNewline);\n }\n private string ReadRestOfLineContinue(char* ptr, bool skipNewline) {\n var stateTag = StateTag;\n var indexToken = IndexToken;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n if (index == 0) {\n c = Peek();\n if (c == EOS || (!skipNewline && (c == '\\r' || c == '\\n'))) return \"\";\n } else {\n c = SkipAndPeek(index);\n }\n while (c != EOS) {\n if (c == '\\r' || c == '\\n') {\n var skippedString = ReadFrom(indexToken);\n if (skipNewline) SkipNewline();\n StateTag = stateTag + 1;\n return skippedString;\n }\n c = SkipAndPeek();\n }\n StateTag = stateTag + 1;\n return ReadFrom(indexToken);\n }\n\n public char ReadCharOrNewline() {\n var ptr = Ptr;\n if (ptr + 2 < PtrEnd) {\n char c = *ptr;\n ++ptr;\n if (c != '\\r') {\n if (c != '\\n') {\n Ptr = ptr;\n ++StateTag;\n return c;\n }\n } else if (*ptr == '\\n') ++ptr;\n Ptr = ptr;\n RegisterNewline();\n return '\\n';\n } else {\n char c0 = Peek();\n if (c0 != EOS) {\n char c = SkipAndPeek();\n var stateTag = StateTag;\n if (c0 != '\\r') {\n if (c0 != '\\n') return c0;\n } else if (c == '\\n') Skip();\n RegisterNewline();\n StateTag = stateTag;\n return '\\n';\n }\n return EOS;\n }\n }\n\n public int SkipCharsOrNewlines(int maxCount) {\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd1 = PtrEnd - 1; // - 1 to guarantee the lookahead for '\\r'\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 >= ptr && end2 <= bufferEnd1 ? end2 : bufferEnd1;\n if (ptr < end) {\n for (;;) {\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (ptr == end) break;\n } else {\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < bufferEnd1) ++end;\n }\n } else if (c != '\\n') goto CheckBound;\n lineBegin = ptr;\n ++lineOffset;\n CheckBound:\n if (ptr >= end) break;\n }\n }\n if (end < bufferEnd1) {\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n }\n }\n return SkipCharsOrNewlinesContinue(ptr, lineBegin, lineOffset, nCRLF, maxCount);\n }\n private int SkipCharsOrNewlinesContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF,\n int maxCount)\n {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n if (maxCount == 0 || (c = Peek()) == EOS) return 0;\n count = 0;\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') c = SkipAndPeek();\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n }\n StateTag = unchecked(stateTag + 1);\n return count;\n }\n\n public string ReadCharsOrNewlines(int maxCount, bool normalizeNewlines) {\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* PtrEnd1 = PtrEnd - 1; // - 1 to guarantee the lookahead for '\\r'\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 >= ptr && end2 <= PtrEnd1 ? end2 : PtrEnd1;\n if (ptr < end) {\n for (;;) {\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (ptr == end) break;\n } else {\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < PtrEnd1) ++end;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') goto CheckBound;\n lineBegin = ptr;\n ++lineOffset;\n CheckBound:\n if (ptr >= end) break;\n }\n }\n if (end < PtrEnd1) {\n char* ptr0 = Ptr;\n Ptr = ptr;\n int length = (int)Buffer.PositiveDistance(ptr0, ptr);\n if (lineOffset == 0) {\n ++StateTag;\n return new string(ptr0, 0, length);\n }\n RegisterNewlines(lineBegin, lineOffset);\n return !normalizeNewlines || (nCR | nCRLF) == 0\n ? new string(ptr0, 0, length)\n : Text.CopyWithNormalizedNewlines(ptr0, length, nCRLF, nCR);\n }\n }\n }\n return ReadCharsOrNewlinesContinue(ptr, lineBegin, lineOffset, nCRLF, nCR, maxCount, normalizeNewlines);\n }\n private string ReadCharsOrNewlinesContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF, int nCR,\n int maxCount, bool normalizeNewlines)\n {\n var stateTag = StateTag;\n var indexToken = IndexToken;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n if (maxCount == 0 || (c = Peek()) == EOS) return \"\";\n count = 0;\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n ++nCRLF;\n c = SkipAndPeek();\n } else {\n ++nCR;\n }\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n }\n StateTag = unchecked(stateTag + 1);\n string str = ReadFrom(indexToken);\n if ((nCR | nCRLF) == 0 || !normalizeNewlines) return str;\n fixed (char* pStr = str)\n return Text.CopyWithNormalizedNewlines(pStr, str.Length, nCRLF, nCR);\n }\n\n public int SkipCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc predicate) {\n return SkipCharsOrNewlinesWhile(predicate, predicate);\n }\n public int SkipCharsOrNewlinesWhile(FSharpFunc predicateForFirstChar,\n FSharpFunc predicate)\n {\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n char* ptr = Ptr;\n char* end = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r'\n if (ptr + 1 < PtrEnd) { // PtrEnd might be null\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = ptr;\n lineOffset = 1;\n }\n }\n for (;;) {\n if (ptr >= end) goto EndOfBlock;\n c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicate.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!predicate.Invoke('\\n')) break;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicate.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = ptr;\n ++lineOffset;\n }\n }\n }\n --ptr;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return count;\n }\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n ReturnEmpty:\n return 0;\n }\n EndOfBlock:\n return SkipCharsOrNewlinesWhileContinue(ptr, lineBegin, lineOffset, nCRLF, predicateForFirstChar, predicate);\n }\n private int SkipCharsOrNewlinesWhileContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF,\n FSharpFunc predicateForFirstChar, FSharpFunc predicate)\n {\n var stateTag = StateTag;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n c = Peek();\n char cc = c == '\\r' ? '\\n' : c;\n if (c == EOS || !predicateForFirstChar.Invoke(cc)) return 0;\n count = 1;\n char c0 = c;\n c = SkipAndPeek();\n if (cc == '\\n') {\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n }\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS) break;\n if (c != '\\r' && c != '\\n') {\n if (!predicate.Invoke(c)) break;\n count = unchecked(count + 1);\n if (count >= 0) c = SkipAndPeek();\n else { // overflow\n count = unchecked(count - 1);\n break;\n }\n } else {\n if (!predicate.Invoke('\\n')) break;\n count = unchecked(count + 1);\n if (count >= 0) {\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n } else {\n count = unchecked(count - 1);\n break;\n }\n }\n }\n StateTag = unchecked(stateTag + 1);\n return count;\n }\n\n public string ReadCharsOrNewlinesWhile(FSharpFunc predicate, bool normalizeNewlines) {\n return ReadCharsOrNewlinesWhile(predicate, predicate, normalizeNewlines);\n }\n public string ReadCharsOrNewlinesWhile(\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n bool normalizeNewlines)\n {\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n char* ptr = Ptr;\n char* end = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r'\n if (ptr + 1 < PtrEnd) { // PtrEnd might be null\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n } else {\n ++nCR;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = ptr;\n lineOffset = 1;\n }\n }\n for (;;) {\n if (ptr >= end) goto EndOfBlock;\n c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicate.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!predicate.Invoke('\\n')) break;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n } else {\n ++nCR;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicate.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = ptr;\n ++lineOffset;\n }\n }\n }\n --ptr;\n char* ptr0 = Ptr;\n Ptr = ptr;\n int length = (int)Buffer.PositiveDistance(ptr0, ptr);\n if (lineOffset == 0) {\n ++StateTag;\n return new string(ptr0, 0, length);\n }\n RegisterNewlines(lineBegin, lineOffset);\n return !normalizeNewlines || (nCR | nCRLF) == 0\n ? new string(ptr0, 0, length)\n : Text.CopyWithNormalizedNewlines(ptr0, length, nCRLF, nCR);\n ReturnEmpty:\n return \"\";\n }\n EndOfBlock:\n return ReadCharsOrNewlinesWhileContinue(ptr, lineBegin, lineOffset, nCRLF, nCR, predicateForFirstChar, predicate, normalizeNewlines);\n }\n private string ReadCharsOrNewlinesWhileContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF, int nCR,\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n bool normalizeNewlines)\n {\n var stateTag = StateTag;\n var indexToken = IndexToken;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n c = Peek();\n char cc = c == '\\r' ? '\\n' : c;\n if (c == EOS || !predicateForFirstChar.Invoke(cc)) return \"\";\n count = 1;\n char c0 = c;\n c = SkipAndPeek();\n if (cc == '\\n') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n ++nCRLF;\n c = SkipAndPeek();\n } else {\n ++nCR;\n }\n }\n RegisterNewline();\n }\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS) break;\n if (c != '\\r' && c != '\\n') {\n if (!predicate.Invoke(c)) break;\n count = unchecked(count + 1);\n if (count < 0) break;\n c = SkipAndPeek();\n } else {\n if (!predicate.Invoke('\\n')) break;\n count = unchecked(count + 1);\n if (count < 0) break;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r') {\n if (c == '\\n') {\n ++nCRLF;\n c = SkipAndPeek();\n } else {\n ++nCR;\n }\n }\n RegisterNewline();\n }\n }\n StateTag = unchecked(stateTag + 1);\n string str = ReadFrom(indexToken);\n if ((nCR | nCRLF) == 0 || !normalizeNewlines) return str;\n fixed (char* pStr = str)\n return Text.CopyWithNormalizedNewlines(pStr, str.Length, nCRLF, nCR);\n }\n\n public int SkipCharsOrNewlinesWhile(FSharpFunc predicate, int minCount, int maxCount) {\n return SkipCharsOrNewlinesWhile(predicate, predicate, minCount, maxCount);\n }\n public int SkipCharsOrNewlinesWhile(\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n int minCount, int maxCount)\n {\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd1 = unchecked(PtrEnd - 1); // - 1 to guarantee the lookahead for '\\r'\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 >= ptr && end2 <= bufferEnd1 ? end2 : bufferEnd1;\n if (ptr < end) {\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < bufferEnd1) ++end;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = ptr;\n ++lineOffset;\n }\n }\n for (;;) {\n if (ptr < end) {\n c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicate.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!predicate.Invoke('\\n')) break;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < bufferEnd1) ++end;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicate.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = ptr;\n ++lineOffset;\n }\n }\n } else {\n if (end >= bufferEnd1) goto EndOfBlock;\n goto ReturnCount;\n }\n }\n --ptr;\n ReturnCount:\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n if (count >= minCount) {\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n ReturnEmpty:\n return 0;\n }\n }\n EndOfBlock:\n return SkipCharsOrNewlinesWhileContinue(ptr, lineBegin, lineOffset, nCRLF, predicateForFirstChar, predicate, minCount, maxCount);\n }\n private int SkipCharsOrNewlinesWhileContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF,\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n int minCount, int maxCount)\n {\n var ptr0 = Ptr;\n var block0 = Block;\n var tag0 = StateTag;\n var line0 = _Line;\n var lineBegin0 = _LineBegin;\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n c = Peek();\n if (c == EOS || maxCount == 0) goto ReturnEmpty;\n if (c != '\\r' && c != '\\n') {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n count = 1;\n c = SkipAndPeek();\n } else {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n count = 1;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n }\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS || count == maxCount) break;\n if (c != '\\r' && c != '\\n') {\n if (!predicate.Invoke(c)) break;\n ++count;\n c = SkipAndPeek();\n } else {\n if (!predicate.Invoke('\\n')) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r' && c == '\\n') c = SkipAndPeek();\n RegisterNewline();\n }\n }\n if (count >= minCount) {\n StateTag = unchecked(tag0 + 1);\n return count;\n }\n ReturnEmpty:\n // backtrack\n Seek(ptr0, block0);\n _Line = line0;\n _LineBegin = lineBegin0;\n StateTag = tag0;\n return 0;\n }\n\n public string ReadCharsOrNewlinesWhile(FSharpFunc predicate, int minCount, int maxCount, bool normalizeNewlines) {\n return ReadCharsOrNewlinesWhile(predicate, predicate, minCount, maxCount, normalizeNewlines);\n }\n public string ReadCharsOrNewlinesWhile(\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n int minCount, int maxCount, bool normalizeNewlines)\n {\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd1 = PtrEnd - 1; // - 1 to guarantee the lookahead for '\\r'\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 >= ptr && end2 <= bufferEnd1 ? end2 : bufferEnd1;\n if (ptr < end) {\n char c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < bufferEnd1) ++end;\n } else {\n ++nCR;\n }\n lineBegin = ptr;\n lineOffset = 1;\n } else {\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = ptr;\n lineOffset = 1;\n }\n }\n for (;;) {\n if (ptr < end) {\n c = *ptr;\n ++ptr;\n if (c > '\\r') {\n if (!predicate.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!predicate.Invoke('\\n')) break;\n if (*ptr == '\\n') {\n ++ptr;\n ++nCRLF;\n if (end < bufferEnd1) ++end;\n } else {\n ++nCR;\n }\n lineBegin = ptr;\n ++lineOffset;\n } else {\n if (!predicate.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = ptr;\n ++lineOffset;\n }\n }\n } else {\n if (end >= bufferEnd1) goto EndOfBlock;\n goto ReturnStringInBlock;\n }\n }\n --ptr;\n ReturnStringInBlock:\n {\n char* ptr0 = Ptr;\n int length = (int)Buffer.PositiveDistance(ptr0, ptr);\n if (length - nCRLF >= minCount) {\n Ptr = ptr;\n if (lineOffset == 0) {\n ++StateTag;\n return new string(ptr0, 0, length);\n }\n RegisterNewlines(lineBegin, lineOffset);\n return !normalizeNewlines || (nCR | nCRLF) == 0\n ? new string(ptr0, 0, length)\n : Text.CopyWithNormalizedNewlines(ptr0, length, nCRLF, nCR);\n }\n }\n ReturnEmpty:\n return \"\";\n }\n }\n EndOfBlock:\n return ReadCharsOrNewlinesWhileContinue(ptr, lineBegin, lineOffset, nCRLF, nCR, predicateForFirstChar, predicate, minCount, maxCount, normalizeNewlines);\n }\n private string ReadCharsOrNewlinesWhileContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF, int nCR,\n FSharpFunc predicateForFirstChar, FSharpFunc predicate,\n int minCount, int maxCount, bool normalizeNewlines)\n {\n var ptr0 = Ptr;\n var block0 = Block;\n var tag0 = StateTag;\n var line0 = _Line;\n var lineBegin0 = _LineBegin;\n\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c;\n int count;\n if (index == 0) {\n c = Peek();\n if (c == EOS || maxCount == 0) goto ReturnEmpty;\n if (c != '\\r' && c != '\\n') {\n count = 1;\n if (!predicateForFirstChar.Invoke(c)) goto ReturnEmpty;\n c = SkipAndPeek();\n } else {\n if (!predicateForFirstChar.Invoke('\\n')) goto ReturnEmpty;\n count = 1;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r') {\n if (c == '\\n') {\n ++nCRLF;\n c = SkipAndPeek();\n } else {\n ++nCR;\n }\n }\n RegisterNewline();\n }\n } else {\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n c = SkipAndPeek(index);\n count = (int)index - nCRLF;\n }\n for (;;) {\n if (c == EOS || count == maxCount) break;\n if (c != '\\r' && c != '\\n') {\n if (!predicate.Invoke(c)) break;\n ++count;\n c = SkipAndPeek();\n } else {\n if (!predicate.Invoke('\\n')) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 == '\\r') {\n if (c == '\\n') {\n ++nCRLF;\n c = SkipAndPeek();\n } else {\n ++nCR;\n }\n }\n RegisterNewline();\n }\n }\n if (count >= minCount) {\n StateTag = unchecked(tag0 + 1);\n string str = ReadFrom(ptr0, block0);\n if ((nCR | nCRLF) == 0 || !normalizeNewlines) return str;\n fixed (char* pStr = str)\n return Text.CopyWithNormalizedNewlines(pStr, str.Length, nCRLF, nCR);\n }\n ReturnEmpty:\n // backtrack\n Seek(ptr0, block0);\n _Line = line0;\n _LineBegin = lineBegin0;\n StateTag = tag0;\n return \"\";\n }\n\n private static bool Rest3OfStringEquals(char* str1, char* str2, int length) {\n for (int i = 3; i < length; ++i) {\n if (str1[i] != str2[i]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n\n private static bool Rest3OfStringEqualsCaseFolded(char* str1, char* cfStr2, int length) {\n char* cftable = CaseFoldTable.FoldedChars;\n for (int i = 3; i < length; ++i) {\n if (cftable[str1[i]] != cfStr2[i]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n\n public int SkipCharsOrNewlinesUntilString(string str, int maxCount, out bool foundString) {\n int strLength = str.Length; // throws if str is null\n if (strLength == 0) throw new ArgumentException(\"The string argument is empty.\");\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n fixed (char* pStr = str) {\n uint lineOffset = 0;\n int nCRLF = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd = PtrEnd;\n char* end1 = unchecked(bufferEnd - strLength);\n if (end1 >= ptr && end1 < bufferEnd) {\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 < ptr || end1 <= end2 ? end1 : end2;\n for (;;) {\n char c = *ptr;\n if (c != pStr[0]) {\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n Debug.Assert(ptr + strLength <= PtrEnd);\n if (strLength == 1 || (ptr[1] == pStr[1] &&\n (strLength == 2 || (ptr[2] == pStr[2] &&\n (strLength == 3 || Rest3OfStringEquals(ptr, pStr, strLength))))))\n {\n foundString = true;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n c = *ptr;\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n }\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n lineBegin = ptr;\n ++lineOffset;\n ++nCRLF;\n if (end < end1) ++end;\n else if (ptr > end) break;\n continue;\n }\n } else if (c != '\\n') continue;\n lineBegin = ptr;\n ++lineOffset;\n } // for\n if (ptr < end1) {\n foundString = false;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n }\n }\n return SkipCharsOrNewlinesUntilStringContinue(ptr, lineBegin, lineOffset, nCRLF, pStr, strLength, maxCount, out foundString);\n }\n }\n private int SkipCharsOrNewlinesUntilStringContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF,\n char* pStr, int strLength, int maxCount, out bool foundString)\n {\n var stateTag = StateTag;\n foundString = false;\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char c = SkipAndPeek((uint)index);\n int count = (int)index - nCRLF;\n for (;;) {\n if (c != pStr[0] || !Match(pStr, strLength)) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n c = SkipAndPeek();\n }\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n } else {\n foundString = true;\n break;\n }\n }\n StateTag = count == 0 ? stateTag : unchecked(stateTag + 1);\n return count;\n }\n\n public int SkipCharsOrNewlinesUntilString(\n string str, int maxCount, bool normalizeNewlines,\n out string skippedCharsIfStringFoundOtherwiseNull)\n {\n int strLength = str.Length; // throws if str is null\n if (strLength == 0) throw new ArgumentException(\"The string argument is empty.\");\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n fixed (char* pStr = str) {\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* end1 = unchecked(PtrEnd - strLength);\n if (end1 >= ptr && end1 < PtrEnd) {\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 < ptr || end1 <= end2 ? end1 : end2;\n for (;;) {\n char c = *ptr;\n if (c != pStr[0]) {\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n Debug.Assert(ptr + strLength <= PtrEnd);\n if (strLength == 1 || (ptr[1] == pStr[1] &&\n (strLength == 2 || (ptr[2] == pStr[2] &&\n (strLength == 3 || Rest3OfStringEquals(ptr, pStr, strLength))))))\n {\n char* ptr0 = Ptr;\n if (ptr != ptr0) {\n Ptr = ptr;\n int length = (int)Buffer.PositiveDistance(ptr0, ptr);\n if (lineOffset == 0) {\n if (length != 0) ++StateTag;\n skippedCharsIfStringFoundOtherwiseNull = new string(ptr0, 0, length);\n return length;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n skippedCharsIfStringFoundOtherwiseNull = !normalizeNewlines || (nCR | nCRLF) == 0\n ? new string(ptr0, 0, length)\n : Text.CopyWithNormalizedNewlines(ptr0, length, nCRLF, nCR);\n return length - nCRLF;\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n return 0;\n }\n }\n c = *ptr;\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n }\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n lineBegin = ptr;\n ++lineOffset;\n ++nCRLF;\n if (end < end1) ++end;\n else if (ptr > end) break;\n continue;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') continue;\n lineBegin = ptr;\n ++lineOffset;\n } // for\n if (ptr < end1) {\n skippedCharsIfStringFoundOtherwiseNull = null;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n }\n }\n return SkipCharsOrNewlinesUntilStringContinue(ptr, lineBegin, lineOffset, nCRLF, nCR, pStr, strLength, maxCount, normalizeNewlines, out skippedCharsIfStringFoundOtherwiseNull);\n }\n }\n private int SkipCharsOrNewlinesUntilStringContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF, int nCR,\n char* pStr, int strLength, int maxCount, bool normalizeNewlines, out string skippedCharsIfStringFoundOtherwiseNull)\n {\n var stateTag = StateTag;\n var indexToken = IndexToken;\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n int count = (int)index - nCRLF;\n char c = SkipAndPeek(index);\n for (;;) {\n if (c != pStr[0] || !Match(pStr, strLength)) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = SkipAndPeek();\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n c = SkipAndPeek();\n ++nCRLF;\n } else {\n ++nCR;\n }\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n } else { // found string\n if (count != 0) {\n StateTag = unchecked(stateTag + 1);\n var s = ReadFrom(indexToken);\n if (!normalizeNewlines || (nCR | nCRLF) == 0) {\n skippedCharsIfStringFoundOtherwiseNull = s;\n return count;\n } else {\n fixed (char* ps = s)\n skippedCharsIfStringFoundOtherwiseNull = Text.CopyWithNormalizedNewlines(ps, s.Length, nCRLF, nCR);\n return count;\n }\n } else {\n StateTag = stateTag;\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n return 0;\n }\n }\n }\n StateTag = count == 0 ? stateTag : unchecked(stateTag + 1);\n skippedCharsIfStringFoundOtherwiseNull = null;\n return count;\n }\n\n public int SkipCharsOrNewlinesUntilCaseFoldedString(\n string caseFoldedString, int maxCount,\n out bool foundString)\n {\n int strLength = caseFoldedString.Length; // throws if str is null\n if (strLength == 0) throw new ArgumentException(\"The string argument is empty.\");\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n char* lineBegin = null;\n fixed (char* pStr = caseFoldedString) {\n uint lineOffset = 0;\n int nCRLF = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd = PtrEnd;\n char* end1 = unchecked(bufferEnd - strLength);\n if (end1 >= ptr && end1 < bufferEnd) {\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 < ptr || end1 <= end2 ? end1 : end2;\n\n char* cftable = CaseFoldTable.FoldedChars;\n for (;;) {\n char c = cftable[*ptr];\n if (c != pStr[0]) {\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n Debug.Assert(ptr + strLength <= PtrEnd);\n if (strLength == 1 || (cftable[ptr[1]] == pStr[1] &&\n (strLength == 2 || (cftable[ptr[2]] == pStr[2] &&\n (strLength == 3 || Rest3OfStringEqualsCaseFolded(ptr, pStr, strLength))))))\n {\n foundString = true;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n c = *ptr; // we don't need to casefold here\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n }\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n lineBegin = ptr;\n ++lineOffset;\n ++nCRLF;\n if (end < end1) ++end;\n else if (ptr > end) break;\n continue;\n }\n } else if (c != '\\n') continue;\n lineBegin = ptr;\n ++lineOffset;\n } // for\n if (ptr < end1) {\n foundString = false;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n }\n }\n return SkipCharsOrNewlinesUntilCaseFoldedStringContinue(ptr, lineBegin, lineOffset, nCRLF, pStr, strLength, maxCount, out foundString);\n }\n }\n private int SkipCharsOrNewlinesUntilCaseFoldedStringContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF,\n char* pStr, int strLength, int maxCount, out bool foundString)\n {\n var stateTag = StateTag;\n foundString = false;\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n char* cftable = CaseFoldTable.FoldedChars;\n char c = cftable[SkipAndPeek((uint)index)];\n int count = (int)index - nCRLF;\n for (;;) {\n if (c != pStr[0] || !MatchCaseFolded(pStr, strLength)) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = cftable[SkipAndPeek()];\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n c = cftable[SkipAndPeek()];\n ++nCRLF;\n }\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n } else {\n foundString = true;\n break;\n }\n }\n StateTag = count == 0 ? stateTag : unchecked(stateTag + 1);\n return count;\n }\n\n public int SkipCharsOrNewlinesUntilCaseFoldedString(\n string caseFoldedString, int maxCount, bool normalizeNewlines,\n out string skippedCharsIfStringFoundOtherwiseNull)\n {\n int strLength = caseFoldedString.Length; // throws if str is null\n if (strLength == 0) throw new ArgumentException(\"The string argument is empty.\");\n if (maxCount < 0) throw new ArgumentOutOfRangeException(\"maxCount\", \"maxCount is negative.\");\n fixed (char* pStr = caseFoldedString) {\n char* lineBegin = null;\n uint lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n char* ptr = Ptr;\n if (ptr != null) {\n char* bufferEnd = PtrEnd;\n char* end1 = unchecked(bufferEnd - strLength);\n if (end1 >= ptr && end1 < bufferEnd) {\n char* end2 = unchecked(ptr + maxCount);\n char* end = end2 < ptr || end1 <= end2 ? end1 : end2;\n char* cftable = CaseFoldTable.FoldedChars;\n for (;;) {\n char c = cftable[*ptr];\n if (c != pStr[0]) {\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n Debug.Assert(ptr + strLength <= PtrEnd);\n if (strLength == 1 || (cftable[ptr[1]] == pStr[1] &&\n (strLength == 2 || (cftable[ptr[2]] == pStr[2] &&\n (strLength == 3 || Rest3OfStringEqualsCaseFolded(ptr, pStr, strLength))))))\n {\n char* ptr0 = Ptr;\n if (ptr != ptr0) {\n Ptr = ptr;\n int length = (int)Buffer.PositiveDistance(ptr0, ptr);\n if (lineOffset == 0) {\n if (length != 0) ++StateTag;\n skippedCharsIfStringFoundOtherwiseNull = new string(ptr0, 0, length);\n return length;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n skippedCharsIfStringFoundOtherwiseNull = !normalizeNewlines || (nCR | nCRLF) == 0\n ? new string(ptr0, 0, length)\n : Text.CopyWithNormalizedNewlines(ptr0, length, nCRLF, nCR);\n return length - nCRLF;\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n return 0;\n }\n }\n c = *ptr; // we don't need to casefold here\n if (ptr == end) break;\n ++ptr;\n if (c > '\\r' || c == '\\t') continue;\n }\n if (c == '\\r') {\n if (*ptr == '\\n') {\n ++ptr;\n lineBegin = ptr;\n ++lineOffset;\n ++nCRLF;\n if (end < end1) ++end;\n else if (ptr > end) break;\n continue;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') continue;\n lineBegin = ptr;\n ++lineOffset;\n } // for\n if (ptr < end1) {\n skippedCharsIfStringFoundOtherwiseNull = null;\n int count = (int)Buffer.PositiveDistance(Ptr, ptr) - nCRLF;\n Ptr = ptr;\n if (lineOffset == 0) {\n if (count != 0) ++StateTag;\n return count;\n } else {\n RegisterNewlines(lineBegin, lineOffset);\n return count;\n }\n }\n }\n }\n return SkipCharsOrNewlinesUntilCaseFoldedStringContinue(ptr, lineBegin, lineOffset, nCRLF, nCR, pStr, strLength, maxCount, normalizeNewlines, out skippedCharsIfStringFoundOtherwiseNull);\n }\n }\n private int SkipCharsOrNewlinesUntilCaseFoldedStringContinue(\n char* ptr, char* lineBegin, uint lineOffset, int nCRLF, int nCR,\n char* pStr, int strLength, int maxCount, bool normalizeNewlines, out string skippedCharsIfStringFoundOtherwiseNull)\n {\n var stateTag = StateTag;\n var indexToken = IndexToken;\n if (lineOffset != 0) RegisterNewlines(lineBegin, lineOffset);\n uint index = Buffer.PositiveDistance(Ptr, ptr);\n int count = (int)index - nCRLF;\n char* cftable = CaseFoldTable.FoldedChars;\n char c = cftable[SkipAndPeek(index)];\n for (;;) {\n if (c != pStr[0] || !MatchCaseFolded(pStr, strLength)) {\n if (c == EOS || count == maxCount) break;\n ++count;\n char c0 = c;\n c = cftable[SkipAndPeek()];\n if (c0 <= '\\r') {\n if (c0 == '\\r') {\n if (c == '\\n') {\n c = cftable[SkipAndPeek()];\n ++nCRLF;\n } else {\n ++nCR;\n }\n } else if (c0 != '\\n') continue;\n RegisterNewline();\n }\n } else { // found string\n if (count != 0) {\n StateTag = unchecked(stateTag + 1);\n var s = ReadFrom(indexToken);\n if ((nCR | nCRLF) == 0 || !normalizeNewlines) {\n skippedCharsIfStringFoundOtherwiseNull = s;\n return count;\n } else {\n fixed (char* ps = s)\n skippedCharsIfStringFoundOtherwiseNull = Text.CopyWithNormalizedNewlines(ps, s.Length, nCRLF, nCR);\n return count;\n }\n } else {\n StateTag = stateTag;\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n return 0;\n }\n }\n }\n StateTag = count == 0 ? stateTag : unchecked(stateTag + 1);\n skippedCharsIfStringFoundOtherwiseNull = null;\n return count;\n }\n}\n\npublic unsafe struct CharStreamState {\n#if DEBUG\n internal readonly CharStream CharStream;\n private long Index { get { return GetIndex(CharStream); } }\n#endif\n internal readonly char* Ptr;\n internal readonly int Block;\n#if SMALL_STATETAG\n public readonly int Tag;\n#else\n public readonly long Tag;\n#endif\n public readonly long Line;\n public readonly long LineBegin;\n public readonly TUserState UserState;\n public readonly string Name;\n\n // Public (though undocumented) as long as the .NET JIT doesn't\n // always inline CharStream.State\n public CharStreamState(CharStream charStream) {\n #if DEBUG\n CharStream = charStream;\n #endif\n Ptr = charStream.Ptr;\n Block = charStream.Block;\n Tag = charStream.StateTag;\n Line = charStream._Line;\n LineBegin = charStream._LineBegin;\n UserState = charStream._UserState;\n Name = charStream._Name;\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public CharStreamIndexToken IndexToken { get {\n if (Line <= 0) // tests for a zero-initialized state\n throw new InvalidOperationException(\"The CharStreamState is invalid.\");\n\n return new CharStreamIndexToken(\n #if DEBUG\n CharStream,\n #endif\n Ptr,\n Block);\n } }\n\n // On .NET calling an instance method of a generic struct can be more\n // expensive than calling an instance method of a generic class\n // (when the type parameter value is not statically known at the call\n // site and isn't a value type that makes the .NET JIT specialize\n // the code).\n //\n // Moving the actual implementations of the following methods into\n // the CharStream class allows the .NET JIT to inline them,\n // so that we effectively replace struct method calls with cheaper\n // class method calls.\n\n public long GetIndex(CharStream charStreamFromWhichStateWasRetrieved) {\n return charStreamFromWhichStateWasRetrieved.GetIndex(ref this);\n }\n\n public Position GetPosition(CharStream charStreamFromWhichStateWasRetrieved) {\n return charStreamFromWhichStateWasRetrieved.GetPosition(ref this);\n }\n}\n\n/// Provides read‐access to a sequence of UTF‐16 chars.\npublic unsafe sealed class CharStream : CharStream {\n // we don't have a public constructor that only takes a string to avoid potential confusion with a filepath constructor\n internal CharStream(string chars) : base(chars) { }\n\n public CharStream(string chars, int index, int length) : base(chars, index, length) {}\n\n public CharStream(string chars, int index, int length, long streamIndexOffset)\n : base(chars, index, length, streamIndexOffset) { }\n\n public CharStream(char[] chars, int index, int length) : base(chars, index, length) { }\n\n public CharStream(char[] chars, int index, int length, long streamIndexOffset)\n : base(chars, index, length, streamIndexOffset) { }\n\n public CharStream(char* chars, int length) : base(chars, length) { }\n\n public CharStream(char* chars, int length, long streamIndexOffset)\n : base(chars, length, streamIndexOffset) { }\n\n internal CharStream(string chars, char* pChars, char* begin, int length)\n : base(chars, pChars, begin, length) { }\n\n public CharStream(string path, Encoding encoding)\n : base(path, encoding) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : base(path, encoding, detectEncodingFromByteOrderMarks) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks,\n int blockSize, int blockOverlap, int byteBufferLength)\n : base(path, encoding, detectEncodingFromByteOrderMarks, blockSize, blockOverlap, byteBufferLength) { }\n\n public CharStream(Stream stream, Encoding encoding)\n : base(stream, encoding) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding)\n : base(stream, leaveOpen, encoding) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : base(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks) { }\n\n public CharStream(Stream stream, bool leaveOpen,\n Encoding encoding, bool detectEncodingFromByteOrderMarks,\n int blockSize, int blockOverlap, int byteBufferLength)\n : base(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks,\n blockSize, blockOverlap, byteBufferLength) {}\n\n internal TUserState _UserState;\n public TUserState UserState {\n get { return _UserState; }\n set { _UserState = value; ++StateTag; }\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public CharStreamState State { get {\n return new CharStreamState(this);\n } }\n\n\n // GetIndex and GetPosition are helper methods for CharStreamState\n\n internal long GetIndex(ref CharStreamState state) {\n if (state.Line <= 0) // tests for a zero-initialized state\n throw new InvalidOperationException(\"The CharStreamState is invalid.\");\n #if DEBUG\n Debug.Assert(this == state.CharStream);\n #endif\n return GetIndex(state.Ptr, state.Block);\n }\n\n internal Position GetPosition(ref CharStreamState state) {\n if (state.Line <= 0) // tests for a zero-initialized state\n throw new InvalidOperationException(\"The CharStreamState is invalid.\");\n #if DEBUG\n Debug.Assert(this == state.CharStream);\n #endif\n long index = GetIndex(state.Ptr, state.Block);\n return new Position(state.Name, index, state.Line, index - state.LineBegin + 1);\n }\n\n // Passing a large struct by value is suboptimal, so for optimization purposes\n // we define internal overloads that take ref arguments. Unfortunately, C#/F#\n // doesn't have const-refs, so we can't make these overloads public (at least,\n // not without risking heart attacks within certain user demographics of this library).\n // An alternative would be to move the following methods into the CharStreamState class,\n // but IMHO the resulting API would feel less intuitive and be somewhat less disoverable.\n\n public void BacktrackTo(CharStreamState state) {\n BacktrackTo(ref state);\n }\n internal void BacktrackTo(ref CharStreamState state) {\n if (state.Line <= 0) // tests for zero-initialized states\n throw new ArgumentException(\"The CharStreamState is invalid.\");\n #if DEBUG\n Debug.Assert(this == state.CharStream);\n #endif\n Seek(state.Ptr, state.Block);\n StateTag = state.Tag;\n _Line = state.Line;\n _LineBegin = state.LineBegin;\n _UserState = state.UserState;\n _Name = state.Name;\n }\n\n public string ReadFrom(CharStreamState stateWhereStringBegins, bool normalizeNewlines) {\n return ReadFrom(ref stateWhereStringBegins, normalizeNewlines);\n }\n internal string ReadFrom(ref CharStreamState stateWhereStringBegins, bool normalizeNewlines) {\n if (stateWhereStringBegins.Line <= 0) // tests for zero-initialized states\n throw new ArgumentException(\"The CharStreamState is invalid.\");\n #if DEBUG\n Debug.Assert(this == stateWhereStringBegins.CharStream);\n #endif\n string str = ReadFrom(stateWhereStringBegins.Ptr, stateWhereStringBegins.Block);\n if (!normalizeNewlines || _Line == stateWhereStringBegins.Line) return str;\n return Text.NormalizeNewlines(str);\n }\n\n public CharStream CreateSubstream(CharStreamState stateWhereSubstreamBegins) {\n return CreateSubstream(ref stateWhereSubstreamBegins);\n }\n internal CharStream CreateSubstream(ref CharStreamState stateWhereSubstreamBegins) {\n if (stateWhereSubstreamBegins.Line <= 0) // tests for zero-initialized states\n throw new ArgumentException(\"The CharStreamState is invalid.\");\n #if DEBUG\n Debug.Assert(this == stateWhereSubstreamBegins.CharStream);\n #endif\n CharStream subStream;\n if (IsSingleBlockStream) {\n // the CharStream has only one block, so its safe to\n // construct a new CharStream from a pointer into the original buffer\n char* ptr0 = stateWhereSubstreamBegins.Ptr;\n if (ptr0 == null) ptr0 = BufferEnd;\n char* end = Ptr;\n if (end == null) end = BufferEnd;\n if (end < ptr0) throw new ArgumentException(\"The current position of the stream must not lie before the position corresponding to the given CharStreamState.\");\n int length = (int)Buffer.PositiveDistance(ptr0, end);\n subStream = new CharStream(BufferString, BufferStringPointer, ptr0, length);\n var indexOfFirstChar = Buffer.PositiveDistance(BufferBegin, ptr0) + _IndexOfFirstChar;\n subStream.IndexOfFirstCharInBlock = indexOfFirstChar;\n subStream._IndexOfFirstChar = indexOfFirstChar;\n } else if (Block == stateWhereSubstreamBegins.Block && Ptr != null && stateWhereSubstreamBegins.Ptr != null) {\n char* ptr0 = stateWhereSubstreamBegins.Ptr;\n char* end = Ptr;\n if (end < ptr0) throw new ArgumentException(\"The current position of the stream must not lie before the position corresponding to the given CharStreamState.\");\n int length = (int)Buffer.PositiveDistance(ptr0, end);\n string subString = new String(ptr0, 0, length);\n subStream = new CharStream(subString);\n var indexOfFirstChar = Buffer.PositiveDistance(BufferBegin, ptr0) + _IndexOfFirstChar;\n subStream.IndexOfFirstCharInBlock = indexOfFirstChar;\n subStream._IndexOfFirstChar = indexOfFirstChar;\n } else {\n var subString = ReadFrom(ref stateWhereSubstreamBegins, false);\n subStream = new CharStream(subString);\n var indexOfFirstChar = GetIndex(stateWhereSubstreamBegins.Ptr, stateWhereSubstreamBegins.Block);\n subStream.IndexOfFirstCharInBlock = indexOfFirstChar;\n subStream._IndexOfFirstChar = indexOfFirstChar;\n }\n subStream.StateTag = stateWhereSubstreamBegins.Tag;\n subStream._Line = stateWhereSubstreamBegins.Line;\n subStream._LineBegin = stateWhereSubstreamBegins.LineBegin;\n subStream._Name = stateWhereSubstreamBegins.Name;\n #if DEBUG\n ++SubstreamCount.Value;\n subStream.ParentSubstreamCount = SubstreamCount;\n #endif\n return subStream;\n }\n}\n\n}\n\n#endif // !LOW_TRUST\n" }, { "path": "FParsecCS/CharStreamLT.cs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n#if LOW_TRUST\n\nusing System;\nusing System.IO;\nusing System.Text;\nusing System.Text.RegularExpressions;\nusing System.Diagnostics;\nusing System.Runtime.CompilerServices;\nusing Microsoft.FSharp.Core;\n\nnamespace FParsec {\n\n\n/// An opaque representation of a CharStream index.\npublic struct CharStreamIndexToken {\n#if DEBUG\n internal readonly CharStream CharStream;\n private long Index { get { return GetIndex(CharStream); } }\n#endif\n private readonly int IdxPlus1;\n /// Returns -1 if the IndexToken was zero-initialized.\n internal int Idx { get { return unchecked(IdxPlus1 - 1); } }\n\n internal CharStreamIndexToken(\n #if DEBUG\n CharStream charStream,\n #endif\n int idx)\n {\n #if DEBUG\n CharStream = charStream;\n #endif\n IdxPlus1 = unchecked(idx + 1);\n }\n\n private static void ThrowInvalidIndexToken() {\n throw new InvalidOperationException(\"The CharStreamIndexToken is invalid.\");\n }\n\n public long GetIndex(CharStream charStreamFromWhichIndexTokenWasRetrieved) {\n int idx = Idx;\n if (idx == -1) ThrowInvalidIndexToken(); // tests for a zero-initialized IndexToken\n #if DEBUG\n Debug.Assert(CharStream == charStreamFromWhichIndexTokenWasRetrieved);\n #endif\n return charStreamFromWhichIndexTokenWasRetrieved.GetIndex(idx);\n }\n}\n\npublic struct TwoChars : IEquatable {\n private uint Chars;\n\n internal TwoChars(uint chars) {\n Chars = chars;\n }\n public TwoChars(char char0, char char1) {\n Chars = ((uint)char1 << 16) | (uint)char0;\n }\n\n public char Char0 { get { return unchecked((char)Chars); } }\n public char Char1 { get { return (char)(Chars >> 16); } }\n\n public override bool Equals(object obj) { return (obj is TwoChars) && Chars == ((TwoChars) obj).Chars; }\n public bool Equals(TwoChars other) { return Chars == other.Chars; }\n public override int GetHashCode() { return unchecked((int)Chars); }\n public static bool operator==(TwoChars left, TwoChars right) { return left.Chars == right.Chars; }\n public static bool operator!=(TwoChars left, TwoChars right) { return left.Chars != right.Chars; }\n}\n\n/// Provides read‐access to a sequence of UTF‐16 chars.\npublic class CharStream : IDisposable {\n private const int DefaultByteBufferLength = (1 << 12);\n private static int MinimumByteBufferLength = 128; // must be larger than longest detectable preamble (we can only guess here)\n private const char EOS = '\\uFFFF';\n\n public const char EndOfStreamChar = EOS;\n\n public int BlockOverlap { get { return 0; } }\n\n public int MinRegexSpace { get { return 0; }\n set { } }\n\n internal String String;\n\n /// The current index in the string, or Int32.MinValue if the end of the stream has been reached.\n internal int Idx;\n\n /// Index of the first char in the string belonging to the stream. Is always non-negative.\n internal int IndexBegin;\n /// 1 + index of the last char in the string belonging to the stream. Equals IndexBegin if the stream is empty.\n internal int IndexEnd;\n\n /// Any CharStream method or property setter increments this value when it changes the CharStream state.\n /// Backtracking to an old state also restores the old value of the StateTag.\n public\n#if SMALL_STATETAG\n uint\n#else\n ulong\n#endif\n StateTag;\n\n /// IndexOfFirstChar - IndexBegin\n internal long StringToStreamIndexOffset;\n\n public long IndexOfFirstChar { get { return (uint)IndexBegin + StringToStreamIndexOffset; } }\n public long IndexOfLastCharPlus1 { get { return (uint)IndexEnd + StringToStreamIndexOffset; } }\n\n public long Index { get {\n // return GetIndex(Idx);\n if (Idx >= 0) {\n Debug.Assert(Idx >= IndexBegin && Idx < IndexEnd);\n return (uint)Idx + StringToStreamIndexOffset;\n } else {\n Debug.Assert(Idx == Int32.MinValue);\n return (uint)IndexEnd + StringToStreamIndexOffset;\n }\n } }\n internal long GetIndex(int idx) {\n if (idx >= 0) {\n Debug.Assert(idx >= IndexBegin && idx < IndexEnd);\n return (uint)idx + StringToStreamIndexOffset;\n } else {\n Debug.Assert(idx == Int32.MinValue);\n return (uint)IndexEnd + StringToStreamIndexOffset;\n }\n }\n\n /// Indicates whether the Iterator points to the beginning of the CharStream.\n /// If the CharStream is empty, this property is always true.\n public bool IsBeginOfStream { get {\n return Idx == IndexBegin || (Idx < 0 && IndexBegin == IndexEnd);\n } }\n\n /// Indicates whether the Iterator points to the end of the CharStream,\n /// i.e. whether it points to one char beyond the last char in the CharStream.\n public bool IsEndOfStream { get { return Idx < 0; } }\n\n internal long _Line;\n public long Line { get { return _Line; } }\n public void SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(long line) {\n _Line = line;\n }\n\n internal long _LineBegin;\n public long LineBegin { get { return _LineBegin; } }\n public void SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(long lineBegin) {\n _LineBegin = lineBegin;\n }\n\n /// The UTF‐16 column number of the next char, i.e. Index ‐ LineBegin + 1.\n public long Column { get { return Index - LineBegin + 1; } }\n\n internal string _Name;\n public string Name {\n get { return _Name; }\n set { _Name = value; ++StateTag; }\n }\n\n public Encoding Encoding { get; private set; }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public Position Position { get {\n long index = Index;\n return new Position(_Name, index, Line, index - LineBegin + 1);\n } }\n\n internal CharStream(string chars) {\n Debug.Assert(chars != null);\n String = chars;\n Encoding = Encoding.Unicode;\n _Line = 1;\n var length = chars.Length;\n if (length != 0) {\n // Idx = 0\n IndexEnd = length;\n } else {\n Idx = Int32.MinValue;\n // IndexEnd = 0\n }\n }\n\n public CharStream(string chars, int index, int length) : this(chars, index, length, 0) {}\n\n public CharStream(string chars, int index, int length, long streamBeginIndex) {\n if (chars == null) throw new ArgumentNullException(\"chars\");\n if (index < 0) throw new ArgumentOutOfRangeException(\"index\", \"index is negative.\");\n if (streamBeginIndex < 0 || streamBeginIndex >= (1L << 60)) throw new ArgumentOutOfRangeException(\"streamBeginIndex\", \"streamBeginIndex must be non-negative and less than 2^60.\");\n int indexEnd = unchecked(index + length);\n if (indexEnd < index || indexEnd > chars.Length) throw new ArgumentOutOfRangeException(\"length\", \"index or length is out of range.\");\n String = chars;\n Encoding = Encoding.Unicode;\n _Line = 1;\n Idx = length == 0 ? Int32.MinValue : index;\n IndexBegin = index;\n IndexEnd = indexEnd;\n _LineBegin = streamBeginIndex;\n StringToStreamIndexOffset = streamBeginIndex - index;\n }\n\n public CharStream(string path, Encoding encoding)\n : this(path, encoding, true, DefaultByteBufferLength) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : this(path, encoding, detectEncodingFromByteOrderMarks, DefaultByteBufferLength) { }\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks, int byteBufferLength) {\n if (encoding == null) throw new ArgumentNullException(\"encoding\");\n Stream stream = new FileStream(path, FileMode.Open, FileAccess.Read,\n FileShare.Read, 4096, FileOptions.SequentialScan);\n try {\n StreamConstructorContinue(stream, false, encoding, detectEncodingFromByteOrderMarks, byteBufferLength);\n _Name = path;\n } catch {\n stream.Dispose();\n throw;\n }\n }\n\n public CharStream(Stream stream, Encoding encoding)\n : this(stream, false, encoding, true, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding)\n : this(stream, leaveOpen, encoding, true, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : this(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks, DefaultByteBufferLength) { }\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks, int byteBufferLength) {\n if (stream == null) throw new ArgumentNullException(\"stream\");\n if (!stream.CanRead) throw new ArgumentException(\"stream is not readable\");\n if (encoding == null) throw new ArgumentNullException(\"encoding\");\n StreamConstructorContinue(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks, byteBufferLength);\n }\n\n private void StreamConstructorContinue(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks, int byteBufferLength) {\n // the ByteBuffer must be larger than the longest detectable preamble\n if (byteBufferLength < MinimumByteBufferLength) byteBufferLength = MinimumByteBufferLength;\n\n int remainingBytesCount = -1;\n long streamPosition;\n if (stream.CanSeek) {\n streamPosition = stream.Position;\n long remainingBytesCount64 = stream.Length - streamPosition;\n if (remainingBytesCount64 <= Int32.MaxValue) {\n remainingBytesCount = (int)remainingBytesCount64;\n if (remainingBytesCount < byteBufferLength) byteBufferLength = remainingBytesCount;\n }\n } else {\n streamPosition = 0;\n }\n\n // byteBufferLength should be larger than the longest detectable preamble\n byte[] byteBuffer = new byte[byteBufferLength];\n int byteBufferCount = 0;\n bool flush = false;\n do {\n int n = stream.Read(byteBuffer, byteBufferCount, byteBuffer.Length - byteBufferCount);\n if (n == 0) {\n remainingBytesCount = byteBufferCount;\n flush = true;\n break;\n }\n byteBufferCount += n;\n } while (byteBufferCount < MinimumByteBufferLength);\n streamPosition += byteBufferCount;\n\n int preambleLength = Text.DetectPreamble(byteBuffer, byteBufferCount, ref encoding, detectEncodingFromByteOrderMarks);\n remainingBytesCount -= preambleLength;\n Encoding = encoding;\n _Line = 1;\n if (remainingBytesCount != 0) {\n int charBufferLength = encoding.GetMaxCharCount(byteBufferLength); // might throw\n char[] charBuffer = new char[charBufferLength];\n int stringBufferCapacity = 2*charBufferLength;\n if (remainingBytesCount > 0) {\n try {\n stringBufferCapacity = encoding.GetMaxCharCount(remainingBytesCount); // might throw\n } catch (ArgumentOutOfRangeException) { }\n }\n var sb = new StringBuilder(stringBufferCapacity);\n var decoder = encoding.GetDecoder();\n Debug.Assert(preambleLength < byteBufferCount);\n int byteBufferIndex = preambleLength;\n for (;;) {\n try {\n int charBufferCount = decoder.GetChars(byteBuffer, byteBufferIndex, byteBufferCount - byteBufferIndex, charBuffer, 0, flush);\n sb.Append(charBuffer, 0, charBufferCount);\n } catch (DecoderFallbackException e) {\n e.Data.Add(\"Stream.Position\", streamPosition - (byteBufferCount - byteBufferIndex) + e.Index);\n throw;\n }\n if (flush) break;\n byteBufferIndex = 0;\n byteBufferCount = stream.Read(byteBuffer, 0, byteBuffer.Length);\n streamPosition += byteBufferCount;\n flush = byteBufferCount == 0;\n }\n String = sb.ToString();\n if (!leaveOpen) stream.Dispose();\n } else {\n String = \"\";\n }\n if (String.Length != 0) {\n // Idx = 0\n IndexEnd = String.Length;\n } else {\n Idx = Int32.MinValue;\n // IndexEnd = 0\n }\n }\n\n /// The low trust version of the CharStream class implements the IDisposable\n /// interface only for API compatibility. The Dispose method does not need to be called on\n /// low trust CharStream instances, because the instances hold no resources that need to be disposed.\n public void Dispose() {}\n\n [System.Diagnostics.CodeAnalysis.SuppressMessage(\"Microsoft.Reliability\", \"CA2000:Dispose objects before losing scope\", Justification=\"The CharStream is manually disposed.\")]\n public static T ParseString(string chars, int index, int length,\n FSharpFunc,T> parser,\n TUserState userState,\n string streamName)\n {\n var stream = new CharStream(chars, index, length);\n stream.UserState = userState;\n stream._Name = streamName;\n return parser.Invoke(stream);\n }\n\n public void Seek(long index) {\n long idx = unchecked(index - StringToStreamIndexOffset);\n if (idx >= IndexBegin && idx < IndexEnd) {\n Idx = (int)idx;\n ++StateTag;\n return;\n }\n if (index < IndexOfFirstChar)\n throw (new ArgumentOutOfRangeException(\"index\", \"The index is negative or less than the IndexOfFirstChar.\"));\n ++StateTag;\n Idx = Int32.MinValue;\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public CharStreamIndexToken IndexToken { get {\n return new CharStreamIndexToken(\n #if DEBUG\n this,\n #endif\n Idx\n );\n } }\n\n private static void ThrowInvalidIndexToken() {\n throw new ArgumentException(\"The CharStreamIndexToken is invalid.\");\n }\n\n public void Seek(CharStreamIndexToken indexToken) {\n int idx = indexToken.Idx;\n if (idx == -1) ThrowInvalidIndexToken(); // tests for zero-initialized IndexTokens\n #if DEBUG\n Debug.Assert(this == indexToken.CharStream);\n #endif\n Idx = idx;\n Debug.Assert((Idx >= IndexBegin && Idx < IndexEnd) || Idx == Int32.MinValue);\n ++StateTag;\n }\n\n public string ReadFrom(CharStreamIndexToken indexToken) {\n int idx = indexToken.Idx;\n if (idx == -1) ThrowInvalidIndexToken(); // tests for zero-initialized IndexTokens\n #if DEBUG\n Debug.Assert(this == indexToken.CharStream);\n #endif\n return ReadFrom(idx);\n }\n\n internal string ReadFrom(int idx0) {\n if (idx0 >= 0) {\n Debug.Assert(idx0 >= IndexBegin && idx0 < IndexEnd);\n if (idx0 <= Idx)\n return String.Substring(idx0, Idx - idx0);\n if (Idx < 0)\n return String.Substring(idx0, IndexEnd - idx0);\n } else {\n Debug.Assert(idx0 == Int32.MinValue);\n if (Idx < 0) return \"\";\n }\n throw new ArgumentException(\"The current position of the stream must not lie before the position corresponding to the given CharStreamIndexToken/CharStreamState.\");\n }\n\n public void RegisterNewline() {\n ++_Line;\n var index = Index;\n Debug.Assert(index != _LineBegin);\n _LineBegin = index;\n ++StateTag;\n }\n\n private void RegisterNewLineBegin(int stringLineBegin, int lineOffset) {\n Debug.Assert(lineOffset > 0\n && ((Idx >= stringLineBegin && Idx < IndexEnd) || Idx == Int32.MinValue)\n && stringLineBegin >= IndexBegin && stringLineBegin <= IndexEnd);\n _Line += lineOffset;\n long newLineBegin = (uint)stringLineBegin + StringToStreamIndexOffset;\n Debug.Assert(newLineBegin > _LineBegin);\n _LineBegin = newLineBegin;\n ++StateTag;\n }\n\n public void RegisterNewlines(int lineOffset, int newColumnMinus1) {\n _Line += lineOffset;\n Debug.Assert(_Line > 0 && newColumnMinus1 >= 0);\n var newLineBegin = Index - newColumnMinus1;\n Debug.Assert(lineOffset != 0 && newLineBegin != _LineBegin);\n _LineBegin = Index - newColumnMinus1;\n ++StateTag;\n }\n\n public void RegisterNewlines(long lineOffset, long newColumnMinus1) {\n _Line += lineOffset;\n Debug.Assert(_Line > 0 && newColumnMinus1 >= 0);\n var newLineBegin = Index - newColumnMinus1;\n Debug.Assert(lineOffset != 0 && newLineBegin != _LineBegin);\n _LineBegin = Index - newColumnMinus1;\n ++StateTag;\n }\n\n\n public char Peek() {\n int idx = Idx;\n if (idx >= 0) return String[idx];\n return EOS;\n }\n\n public void Skip() {\n int idx = Idx + 1;\n if (unchecked((uint)idx) < (uint)IndexEnd) {\n Idx = idx;\n ++StateTag;\n } else if (idx == IndexEnd) {\n Idx = Int32.MinValue;\n ++StateTag;\n }\n }\n\n public char Read() {\n int idx = Idx;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n ++StateTag;\n return c;\n }\n return EOS;\n }\n\n public char SkipAndPeek() {\n int idx = Idx + 1;\n if (unchecked((uint)idx) < (uint)IndexEnd) {\n Idx = idx;\n ++StateTag;\n return String[idx];\n } else if (idx == IndexEnd) {\n Idx = Int32.MinValue;\n ++StateTag;\n }\n return EOS;\n }\n\n public TwoChars Peek2() {\n int idx = Idx + 1;\n if (unchecked((uint)idx) < (uint)IndexEnd)\n return new TwoChars(String[idx - 1], String[idx]);\n else if (idx == IndexEnd)\n return new TwoChars(String[idx - 1], EOS);\n else\n return new TwoChars(EOS, EOS);\n }\n\n public char Peek(uint utf16Offset) {\n int n = unchecked((int)utf16Offset);\n if (n >= 0) { // utf16Offset <= Int32.MaxValue\n int idx = unchecked(Idx + n);\n if (unchecked((uint)idx) < (uint)IndexEnd)\n return String[idx];\n }\n return EOS;\n }\n\n public void Skip(uint utf16Offset) {\n ++StateTag;\n int n = unchecked((int)utf16Offset);\n if (n >= 0) { // utf16Offset <= Int32.MaxValue\n int idx = unchecked(Idx + n);\n if (unchecked((uint)idx) < (uint)IndexEnd) {\n Idx = idx;\n return;\n }\n }\n Idx = Int32.MinValue;\n return;\n }\n\n public char SkipAndPeek(uint utf16Offset) {\n ++StateTag;\n int n = unchecked((int)utf16Offset);\n if (n >= 0) { // utf16Offset <= Int32.MaxValue\n int idx = unchecked(Idx + n);\n if (unchecked((uint)idx) < (uint)IndexEnd) {\n Idx = idx;\n return String[idx];\n }\n }\n Idx = Int32.MinValue;\n return EOS;\n }\n\n public char Peek(int utf16Offset) {\n int idx = unchecked(Idx + utf16Offset);\n if (utf16Offset < 0) goto Negative;\n if (unchecked((uint)idx) >= (uint)IndexEnd) goto EndOfStream;\n ReturnChar:\n return String[idx];\n Negative:\n if (Idx >= 0) {\n if (idx >= IndexBegin) goto ReturnChar;\n } else {\n idx = IndexEnd + utf16Offset;\n if (idx >= IndexBegin) goto ReturnChar;\n }\n EndOfStream:\n return EOS;\n }\n\n public void Skip(int utf16Offset) {\n ++StateTag;\n int idx = unchecked(Idx + utf16Offset);\n if (utf16Offset < 0) goto Negative;\n if (unchecked((uint)idx) >= (uint)IndexEnd) goto EndOfStream;\n Return:\n Idx = idx;\n return;\n Negative:\n if (Idx >= 0) {\n if (idx >= IndexBegin) goto Return;\n } else {\n idx = IndexEnd + utf16Offset;\n if (idx >= IndexBegin) goto Return;\n }\n --StateTag;\n throw new ArgumentOutOfRangeException(\"utf16Offset\", \"Index + utf16Offset is negative or less than the index of the first char in the CharStream.\");\n EndOfStream:\n idx = Int32.MinValue;\n goto Return;\n }\n\n public void Skip(long utf16Offset) {\n if (unchecked((int)utf16Offset) == utf16Offset) {\n Skip((int)utf16Offset);\n } else {\n if (utf16Offset < 0) throw new ArgumentOutOfRangeException(\"utf16Offset\", \"Index + utf16Offset is negative or less than the index of the first char in the CharStream.\");\n ++StateTag;\n Idx = Int32.MinValue;\n }\n }\n\n public char SkipAndPeek(int utf16Offset) {\n ++StateTag;\n int idx = unchecked(Idx + utf16Offset);\n if (utf16Offset < 0) goto Negative;\n if (unchecked((uint)idx) >= (uint)IndexEnd) goto EndOfStream;\n ReturnChar:\n Idx = idx;\n return String[idx];\n Negative:\n if (Idx >= 0) {\n if (idx >= IndexBegin) goto ReturnChar;\n } else {\n idx = IndexEnd + utf16Offset;\n if (idx >= IndexBegin) goto ReturnChar;\n if (IndexBegin == IndexEnd) goto EndOfStream;\n }\n Idx = IndexBegin;\n return EOS;\n EndOfStream:\n Idx = Int32.MinValue;\n return EOS;\n }\n\n public string PeekString(int length) {\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n int idx = Idx;\n if (unchecked((uint)idx) + (uint)length <= (uint)IndexEnd)\n return String.Substring(idx, length);\n else\n return idx < 0 ? \"\" : String.Substring(idx, IndexEnd - idx);\n }\n\n public string Read(int length) {\n if (length < 0) throw new ArgumentOutOfRangeException(\"length\", \"length is negative.\");\n ++StateTag;\n var idx = Idx;\n int newIdx = unchecked(idx + length);\n if (unchecked((uint)newIdx) < (uint)IndexEnd) {\n Idx = newIdx;\n return String.Substring(idx, length);\n } else {\n Idx = Int32.MinValue;\n return idx < 0 ? \"\" : String.Substring(idx, IndexEnd - idx);\n }\n }\n\n public int PeekString(char[] buffer, int bufferIndex, int length) {\n return Read(buffer, bufferIndex, length, true);\n }\n public int Read(char[] buffer, int bufferIndex, int length) {\n return Read(buffer, bufferIndex, length, false);\n }\n private int Read(char[] buffer, int bufferIndex, int length, bool backtrack) {\n if (bufferIndex < 0)\n throw new ArgumentOutOfRangeException(\"bufferIndex\", \"bufferIndex is negative.\");\n if (length < 0 || bufferIndex > buffer.Length - length)\n throw new ArgumentOutOfRangeException(\"length\", \"bufferIndex or length is out of range.\");\n if (unchecked((uint)Idx) + (uint)length < (uint)IndexEnd) {\n for (int i = 0; i < length; ++i)\n buffer[bufferIndex + i] = String[Idx + i];\n if (!backtrack) {\n Idx += length;\n ++StateTag;\n }\n return length;\n } else if (Idx >= 0) {\n int n = IndexEnd - Idx;\n for (int i = 0; i < n; ++i)\n buffer[bufferIndex + i] = String[Idx + i];\n if (!backtrack) {\n Idx = Int32.MinValue;\n ++StateTag;\n }\n return n;\n } else {\n return 0;\n }\n }\n\n public bool Match(char ch) {\n return Idx >= 0 && String[Idx] == ch;\n }\n\n public bool MatchCaseFolded(char caseFoldedChar) {\n return Idx >= 0 && CaseFoldTable.FoldedChars[String[Idx]] == caseFoldedChar;\n }\n\n public bool Skip(char ch) {\n int idx = Idx;\n if (idx >= 0 && String[idx] == ch) {\n ++idx;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n ++StateTag;\n return true;\n }\n return false;\n }\n\n public bool SkipCaseFolded(char caseFoldedChar) {\n int idx = Idx;\n if (idx >= 0 && CaseFoldTable.FoldedChars[String[idx]] == caseFoldedChar) {\n ++idx;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n ++StateTag;\n return true;\n }\n return false;\n }\n\n public bool Skip(TwoChars twoChars) {\n int idx2 = unchecked(Idx + 2);\n if (unchecked((uint)idx2) < (uint)IndexEnd) {\n if (String[Idx] == twoChars.Char0 && String[Idx + 1] == twoChars.Char1) {\n ++StateTag;\n Idx = idx2;\n return true;\n }\n } else if (idx2 == IndexEnd && String[Idx] == twoChars.Char0 && String[Idx + 1] == twoChars.Char1) {\n ++StateTag;\n Idx = Int32.MinValue;\n return true;\n }\n return false;\n }\n\n public bool Match(string chars) {\n if (unchecked((uint)Idx) + (uint)chars.Length <= (uint)IndexEnd) {\n for (int i = 0; i < chars.Length; ++i)\n if (chars[i] != String[Idx + i]) goto ReturnFalse;\n return true;\n }\n return chars.Length == 0;\n ReturnFalse:\n return false;\n }\n\n public bool Skip(string chars) {\n int newIdx = unchecked(Idx + chars.Length);\n if (unchecked((uint)newIdx) <= (uint)IndexEnd) {\n for (int i = 0; i < chars.Length; ++i)\n if (chars[i] != String[Idx + i]) goto ReturnFalse;\n if (newIdx == IndexEnd) newIdx = Int32.MinValue;\n Idx = newIdx;\n ++StateTag;\n return true;\n }\n return chars.Length == 0;\n ReturnFalse:\n return false;\n }\n\n public bool MatchCaseFolded(string caseFoldedChars) {\n if (unchecked((uint)Idx) + (uint)caseFoldedChars.Length <= (uint)IndexEnd) {\n for (int i = 0; i < caseFoldedChars.Length; ++i)\n if (caseFoldedChars[i] != CaseFoldTable.FoldedChars[String[Idx + i]]) goto ReturnFalse;\n return true;\n }\n return caseFoldedChars.Length == 0;\n ReturnFalse:\n return false;\n }\n\n public bool SkipCaseFolded(string caseFoldedChars) {\n int newIdx = unchecked(Idx + caseFoldedChars.Length);\n if (unchecked((uint)newIdx) <= (uint)IndexEnd) {\n for (int i = 0; i < caseFoldedChars.Length; ++i)\n if (caseFoldedChars[i] != CaseFoldTable.FoldedChars[String[Idx + i]]) goto ReturnFalse;\n if (newIdx == IndexEnd) newIdx = Int32.MinValue;\n Idx = newIdx;\n ++StateTag;\n return true;\n }\n return caseFoldedChars.Length == 0;\n ReturnFalse:\n return false;\n }\n\n public bool Match(char[] chars, int charsIndex, int length) {\n return Skip(chars, charsIndex, length, true);\n }\n public bool Skip(char[] chars, int charsIndex, int length) {\n return Skip(chars, charsIndex, length, false);\n }\n private bool Skip(char[] chars, int charsIndex, int length, bool backtrackEvenIfCharsMatch) {\n if (charsIndex < 0)\n throw new ArgumentOutOfRangeException(\"charsIndex\", \"charsIndex is negative.\");\n if (length < 0 || charsIndex > chars.Length - length)\n throw new ArgumentOutOfRangeException(\"length\", \"charsIndex or length is out of range.\");\n int newIdx = unchecked(Idx + length);\n if (unchecked((uint)newIdx) <= (uint)IndexEnd) {\n for (int i = 0; i < length; ++i)\n if (chars[charsIndex + i] != String[Idx + i]) goto ReturnFalse;\n if (!backtrackEvenIfCharsMatch) {\n if (newIdx == IndexEnd) newIdx = Int32.MinValue;\n Idx = newIdx;\n ++StateTag;\n return true;\n }\n return true;\n }\n return length == 0;\n ReturnFalse:\n return false;\n }\n\n public Match Match(Regex regex) {\n if (Idx >= 0) return regex.Match(String, Idx, IndexEnd - Idx);\n return regex.Match(\"\");\n }\n\n public bool SkipWhitespace() {\n int lineBegin = 0;\n int lineOffset = 0;\n int idx = Idx;\n int end = IndexEnd;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c > ' ') goto ReturnFalse;\n if (c == ' ') {\n if (idx != end && String[idx] > ' ') {\n Idx = idx;\n ++StateTag;\n return true;\n }\n goto Loop;\n } else {\n if (c == '\\r') {\n if (idx != end && String[idx] == '\\n') ++idx;\n } else if (c != '\\n') goto CheckTab;\n if (idx != end && String[idx] > ' ') {\n Idx = idx;\n RegisterNewline();\n return true;\n }\n goto Newline;\n CheckTab:\n if (c != '\\t') goto ReturnFalse;\n goto Loop;\n }\n Newline:\n lineBegin = idx;\n ++lineOffset;\n Loop:\n for (;;) {\n if (idx != end) {\n c = String[idx];\n ++idx;\n if (c != ' ') {\n if (c != '\\t') {\n if (c == '\\r') {\n if (idx != end && String[idx] == '\\n') ++idx;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n // end of whitespace\n --idx;\n break;\n }\n }\n } else { // end of stream,\n idx = Int32.MinValue;\n break;\n }\n }\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n return true;\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n return true;\n }\n }\n ReturnFalse:\n return false;\n }\n\n public bool SkipUnicodeWhitespace() {\n int lineBegin = 0;\n int lineOffset = 0;\n int idx = Idx;\n int end = IndexEnd;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c == ' ') goto Loop;\n if (!Text.IsWhitespace(c)) goto ReturnFalse;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (idx != end && String[idx] == '\\n') ++idx;\n } else if (c != '\\n') goto Loop;\n } else if (c < '\\u2028' ? c != '\\u0085' : c > '\\u2029') goto Loop;\n Newline:\n lineBegin = idx;\n ++lineOffset;\n Loop:\n for (;;) {\n if (idx != end) {\n c = String[idx];\n ++idx;\n if (c != ' ') {\n if (Text.IsWhitespace(c)) {\n if (c <= '\\r') {\n if (c == '\\r') {\n if (idx != end && String[idx] == '\\n') ++idx;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n } else if (c < '\\u2028' ? c == '\\u0085' : c <= '\\u2029') goto Newline;\n } else { // end of whitespace\n --idx;\n break;\n }\n }\n } else { // end of stream\n idx = Int32.MinValue;\n break;\n }\n }\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n return true;\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n return true;\n }\n }\n ReturnFalse:\n return false;\n }\n\n public bool SkipNewline() {\n int idx = Idx;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') ++idx;\n } else if (c != '\\n') goto ReturnFalse;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n RegisterNewline();\n return true;\n }\nReturnFalse:\n return false;\n }\n\n public bool SkipUnicodeNewline() {\n int idx = Idx;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c < '\\u0085') {\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') ++idx;\n } else if (c != '\\n') goto ReturnFalse;\n } else if (c >= '\\u2028' ? c > '\\u2029' : c != '\\u0085') goto ReturnFalse;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n RegisterNewline();\n return true;\n }\n ReturnFalse:\n return false;\n }\n\n public int SkipNewlineThenWhitespace(int powerOf2TabStopDistance, bool allowFormFeed) {\n int tabStopDistanceMinus1 = unchecked(powerOf2TabStopDistance - 1);\n if (powerOf2TabStopDistance <= 0 || (powerOf2TabStopDistance & tabStopDistanceMinus1) != 0)\n throw new ArgumentOutOfRangeException(\"powerOf2TabStopDistance\", \"powerOf2TabStopDistance must be a positive power of 2.\");\n\n int lineBegin = 0;\n int lineOffset = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n char c = '\\u0000';\n if (idx >= 0) c = String[idx];\n ++idx;\n if (c == '\\r') {\n if (idx != indexEnd && String[idx] == '\\n') ++idx;\n } else if (c != '\\n') {\n return -1;\n }\n Newline:\n lineBegin = idx;\n ++lineOffset;\n int ind = 0;\n for (;;) {\n if (idx != indexEnd) {\n c = String[idx];\n ++idx;\n if (c == ' ') {\n ind = unchecked(ind + 1);\n if (ind >= 0) continue;\n // indentation has overflown, so put back ' ' and return\n ind = unchecked(ind - 1);\n } else if (c <= '\\r') {\n if (c == '\\r') {\n if (idx != indexEnd && String[idx] == '\\n') ++idx;\n goto Newline;\n }\n if (c == '\\n') goto Newline;\n if (c == '\\t') {\n // ind = ind + tabStopDistance - ind%tabStopDistance\n int d = tabStopDistanceMinus1 - (ind & tabStopDistanceMinus1);\n ind = unchecked(ind + d + 1);\n if (ind >= 0) continue;\n // indentation has overflown, so put back '\\t' and return\n ind = unchecked(ind - d - 1);\n } else if (c == '\\f' && allowFormFeed) {\n ind = 0;\n continue;\n }\n }\n // end of indentation\n --idx;\n break;\n } else {\n // end of stream;\n idx = Int32.MinValue;\n break;\n }\n }\n Idx = idx;\n RegisterNewLineBegin(lineBegin, lineOffset);\n return ind;\n }\n\n public void SkipRestOfLine(bool skipNewline) {\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0) {\n for (;;) {\n char c = String[idx];\n if (c > '\\r') {\n if (++idx == indexEnd) break;\n } else if (c != '\\r' && c != '\\n') {\n if (++idx == indexEnd) break;\n } else {\n if (!skipNewline) {\n if (idx != Idx) {\n Idx = idx;\n ++StateTag;\n }\n return;\n } else {\n ++idx;\n if (c == '\\r' && idx != indexEnd && String[idx] == '\\n') ++idx;\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n RegisterNewline();\n return;\n }\n }\n }\n // idx == indexEnd\n {\n Idx = Int32.MinValue;\n ++StateTag;\n }\n }\n }\n\n public string ReadRestOfLine(bool skipNewline) {\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0) {\n for (;;) {\n char c = String[idx];\n if (c > '\\r') {\n if (++idx == indexEnd) break;\n } else if (c != '\\r' && c != '\\n') {\n if (++idx == indexEnd) break;\n } else {\n int idx0 = Idx;\n if (!skipNewline) {\n if (idx != idx0) {\n Idx = idx;\n ++StateTag;\n return String.Substring(idx0, idx - idx0);\n } else {\n return \"\";\n }\n } else {\n var skippedString = idx == idx0 ? \"\" : String.Substring(idx0, idx - idx0);\n ++idx;\n if (c == '\\r' && idx != indexEnd && String[idx] == '\\n') ++idx;\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n RegisterNewline();\n return skippedString;\n }\n }\n }\n // idx == indexEnd\n {\n int idx0 = Idx;\n Idx = Int32.MinValue;\n ++StateTag;\n return String.Substring(idx0, indexEnd - idx0);\n }\n }\n return \"\";\n }\n\n public char ReadCharOrNewline() {\n int idx = Idx;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c != '\\r') {\n if (c != '\\n') {\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n ++StateTag;\n return c;\n }\n } else if (idx != IndexEnd && String[idx] == '\\n') ++idx;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n RegisterNewline();\n return '\\n';\n }\n return EOS;\n }\n\n public int SkipCharsOrNewlines(int maxCharsOrNewlines) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int idx = Idx;\n if (idx >= 0 && maxCharsOrNewlines > 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)IndexEnd ? IndexEnd : (int)end2;\n for (;;) {\n if (idx >= end) break;\n char c = String[idx];\n ++idx;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != IndexEnd) ++end;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n }\n }\n int count = idx - Idx - nCRLF;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n return 0;\n }\n\n public string ReadCharsOrNewlines(int maxCharsOrNewlines, bool normalizeNewlinesInReturnString) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0 && maxCharsOrNewlines > 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)indexEnd ? indexEnd : (int)end2;\n for (;;) {\n if (idx >= end) break;\n char c = String[idx];\n ++idx;\n if (c <= '\\r') {\n if (c == '\\r') {\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n }\n }\n int idx0 = Idx;\n int length = idx - idx0;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n return String.Substring(idx0, length);\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n return !normalizeNewlinesInReturnString || (nCR | nCRLF) == 0\n ? String.Substring(idx0, length)\n : Text.CopyWithNormalizedNewlines(String, idx0, length, nCRLF, nCR);\n }\n }\n return \"\";\n }\n\n public int SkipCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f) {\n return SkipCharsOrNewlinesWhile(f, f);\n }\n public int SkipCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f1, Microsoft.FSharp.Core.FSharpFunc f) {\n int lineOffset = 0;\n int nCRLF = 0;\n int lineBegin = 0;\n int idx = Idx;\n int end = IndexEnd;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!f1.Invoke('\\n')) goto ReturnEmpty;\n if (idx != end && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n for (;;) {\n if (idx == end) goto ReturnCount;\n c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!f.Invoke('\\n')) break;\n if (idx != end && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n }\n --idx;\n ReturnCount:\n int count = idx - Idx - nCRLF;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n ReturnEmpty:\n return 0;\n }\n\n public string ReadCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f, bool normalizeNewlines) {\n return ReadCharsOrNewlinesWhile(f, f, normalizeNewlines);\n }\n public string ReadCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f1, Microsoft.FSharp.Core.FSharpFunc f, bool normalizeNewlinesInReturnString) {\n int lineOffset = 0;\n int nCR = 0;\n int nCRLF = 0;\n int lineBegin = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0) {\n char c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!f1.Invoke('\\n')) goto ReturnEmpty;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n } else {\n ++nCR;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n for (;;) {\n if (idx == indexEnd) goto ReturnString;\n c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!f.Invoke('\\n')) break;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n } else {\n ++nCR;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n }\n --idx;\n ReturnString:\n int idx0 = Idx;\n int length = idx - idx0;\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n return String.Substring(idx0, length);\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n return !normalizeNewlinesInReturnString || (nCR | nCRLF) == 0\n ? String.Substring(idx0, length)\n : Text.CopyWithNormalizedNewlines(String, idx0, length, nCRLF, nCR);\n }\n }\n ReturnEmpty:\n return \"\";\n }\n\n public int SkipCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f, int minCharsOrNewlines, int maxCharsOrNewlines) {\n return SkipCharsOrNewlinesWhile(f, f, minCharsOrNewlines, maxCharsOrNewlines);\n }\n public int SkipCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f1, Microsoft.FSharp.Core.FSharpFunc f, int minCharsOrNewlines, int maxCharsOrNewlines) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0 && maxCharsOrNewlines > 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)indexEnd ? indexEnd : (int)end2;\n char c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!f1.Invoke('\\n')) goto ReturnEmpty;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n for (;;) {\n if (idx >= end) goto ReturnCount;\n c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!f.Invoke('\\n')) break;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n }\n --idx;\n ReturnCount:\n int count = idx - Idx - nCRLF;\n if (count >= minCharsOrNewlines) {\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n }\n ReturnEmpty:\n return 0;\n }\n\n public string ReadCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f, int minCharsOrNewlines, int maxCharsOrNewlines, bool normalizeNewlinesInReturnString) {\n return ReadCharsOrNewlinesWhile(f, f, minCharsOrNewlines, maxCharsOrNewlines, normalizeNewlinesInReturnString);\n }\n public string ReadCharsOrNewlinesWhile(Microsoft.FSharp.Core.FSharpFunc f1, Microsoft.FSharp.Core.FSharpFunc f, int minCharsOrNewlines, int maxCharsOrNewlines, bool normalizeNewlinesInReturnString) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n if (idx >= 0 && maxCharsOrNewlines > 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)indexEnd ? indexEnd : (int)end2;\n char c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n } else if (c == '\\r') {\n if (!f1.Invoke('\\n')) goto ReturnEmpty;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n } else {\n ++nCR;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f1.Invoke(c)) goto ReturnEmpty;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n for (;;) {\n if (idx >= end) goto ReturnString;\n c = String[idx];\n ++idx;\n if (c > '\\r') {\n if (!f.Invoke(c)) break;\n } else if (c == '\\r') {\n if (!f.Invoke('\\n')) break;\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n } else {\n ++nCR;\n }\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (!f.Invoke(c)) break;\n if (c == '\\n') {\n lineBegin = idx;\n ++lineOffset;\n }\n }\n }\n --idx;\n ReturnString:\n int idx0 = Idx;\n int length = idx - idx0;\n if (length - nCRLF >= minCharsOrNewlines) {\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n return String.Substring(idx0, length);\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n return !normalizeNewlinesInReturnString || (nCR | nCRLF) == 0\n ? String.Substring(idx0, length)\n : Text.CopyWithNormalizedNewlines(String, idx0, length, nCRLF, nCR);\n }\n }\n }\n ReturnEmpty:\n return \"\";\n }\n\n private static bool RestOfStringEquals(string str1, int str1Index, string str2) {\n for (int i1 = str1Index + 1, i2 = 1; i2 < str2.Length; ++i1, ++i2) {\n if (str1[i1] != str2[i2]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n\n private static bool RestOfStringEqualsCI(string str1, int str1Index, string cfStr2) {\n char[] cftable = CaseFoldTable.FoldedChars;\n for (int i1 = str1Index + 1, i2 = 1; i2 < cfStr2.Length; ++i1, ++i2) {\n if (cftable[str1[i1]] != cfStr2[i2]) goto ReturnFalse;\n }\n return true;\n ReturnFalse:\n return false;\n }\n\n public int SkipCharsOrNewlinesUntilString(string str, int maxCharsOrNewlines, out bool foundString) {\n if (str.Length == 0) throw new ArgumentException(\"The string argument is empty.\");\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n // The .NET 64-bit JIT emits inefficient code in the loop if we declare first as as char variable.\n int first = str[0];\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int idx = Idx;\n int indexEnd = IndexEnd;\n int end1 = indexEnd - str.Length;\n if (idx >= 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)indexEnd ? indexEnd : (int)end2;\n for (;;) {\n if (idx < end) {\n char c = String[idx];\n if (c != first) {\n ++idx;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n if (idx <= end1 && RestOfStringEquals(String, idx, str)) {\n foundString = true;\n break;\n }\n ++idx;\n if (c > '\\r') continue;\n }\n if (c == '\\r') {\n if (idx != indexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != indexEnd) ++end;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n } else {\n foundString = idx <= end1 && String[idx] == first && RestOfStringEquals(String, idx, str);\n break;\n }\n }\n if (idx != Idx) {\n int count = idx - Idx - nCRLF;\n if (idx == indexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n } else {\n foundString = false;\n }\n return 0;\n }\n\n public int SkipCharsOrNewlinesUntilString(string str, int maxCharsOrNewlines, bool normalizeNewlinesInOutString, out string skippedCharsIfStringFoundOtherwiseNull) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n if (str.Length == 0) throw new ArgumentException(\"The string argument is empty.\");\n // The .NET 64-bit JIT emits inefficient code in the loop if we declare first as as char variable.\n int first = str[0];\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n int idx = Idx;\n int end1 = IndexEnd - str.Length;\n if (idx >= 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)IndexEnd ? IndexEnd : (int)end2;\n for (;;) {\n if (idx < end) {\n char c = String[idx];\n if (c != first) {\n ++idx;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n if (idx <= end1 && RestOfStringEquals(String, idx, str)) break;\n ++idx;\n if (c > '\\r') continue;\n }\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != IndexEnd) ++end;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (idx <= end1 && String[idx] == first && RestOfStringEquals(String, idx, str)) break;\n // string not found\n skippedCharsIfStringFoundOtherwiseNull = null;\n if (idx != Idx) {\n int count = idx - Idx - nCRLF;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n return 0;\n }\n }\n // found string\n int idx0 = Idx;\n int length = idx - idx0;\n if (length != 0) {\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n skippedCharsIfStringFoundOtherwiseNull = String.Substring(idx0, length);\n return length;\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n skippedCharsIfStringFoundOtherwiseNull =\n !normalizeNewlinesInOutString || (nCR | nCRLF) == 0\n ? String.Substring(idx0, length)\n : Text.CopyWithNormalizedNewlines(String, idx0, length, nCRLF, nCR);\n return length - nCRLF;\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = null;\n }\n return 0;\n }\n\n public int SkipCharsOrNewlinesUntilCaseFoldedString(string caseFoldedString, int maxCharsOrNewlines, out bool foundString) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n if (caseFoldedString.Length == 0) throw new ArgumentException(\"The string argument is empty.\");\n // The .NET 64-bit JIT emits inefficient code in the loop if we declare first as as char variable.\n int first = caseFoldedString[0];\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int idx = Idx;\n int end1 = IndexEnd - caseFoldedString.Length;\n char[] cftable = CaseFoldTable.FoldedChars;\n if (idx >= 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)IndexEnd ? IndexEnd : (int)end2;\n for (;;) {\n if (idx < end) {\n char c = cftable[String[idx]];\n if (c != first) {\n ++idx;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n if (idx <= end1 && RestOfStringEqualsCI(String, idx, caseFoldedString)) {\n foundString = true;\n break;\n }\n ++idx;\n if (c > '\\r') continue;\n }\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != IndexEnd) ++end;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n } else {\n foundString = idx <= end1 && cftable[String[idx]] == first && RestOfStringEqualsCI(String, idx, caseFoldedString);\n break;\n }\n }\n if (idx != Idx) {\n int count = idx - Idx - nCRLF;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n } else {\n foundString = false;\n }\n return 0;\n }\n\n public int SkipCharsOrNewlinesUntilCaseFoldedString(string caseFoldedString, int maxCharsOrNewlines, bool normalizeNewlinesInOutString, out string skippedCharsIfStringFoundOtherwiseNull) {\n if (maxCharsOrNewlines < 0) throw new ArgumentOutOfRangeException(\"maxCharsOrNewlines\", \"maxCharsOrNewlines is negative.\");\n if (caseFoldedString.Length == 0) throw new ArgumentException(\"The string argument is empty.\");\n // The .NET 64-bit JIT emits inefficient code in the loop if we declare first as as char variable.\n int first = caseFoldedString[0];\n int lineBegin = 0;\n int lineOffset = 0;\n int nCRLF = 0;\n int nCR = 0;\n int idx = Idx;\n int end1 = IndexEnd - caseFoldedString.Length;\n char[] cftable = CaseFoldTable.FoldedChars;\n if (idx >= 0) {\n uint end2 = (uint)idx + (uint)maxCharsOrNewlines;\n int end = end2 > (uint)IndexEnd ? IndexEnd : (int)end2;\n for (;;) {\n if (idx < end) {\n char c = cftable[String[idx]];\n if (c != first) {\n ++idx;\n if (c > '\\r' || c == '\\t') continue;\n } else {\n if (idx <= end1 && RestOfStringEqualsCI(String, idx, caseFoldedString)) break;\n ++idx;\n if (c > '\\r') continue;\n }\n if (c == '\\r') {\n if (idx != IndexEnd && String[idx] == '\\n') {\n ++idx;\n ++nCRLF;\n if (end != IndexEnd) ++end;\n } else {\n ++nCR;\n }\n } else if (c != '\\n') continue;\n lineBegin = idx;\n ++lineOffset;\n } else {\n if (idx <= end1 && cftable[String[idx]] == first && RestOfStringEqualsCI(String, idx, caseFoldedString)) break;\n // string not found\n skippedCharsIfStringFoundOtherwiseNull = null;\n if (idx != Idx) {\n int count = idx - Idx - nCRLF;\n if (idx == IndexEnd) idx = Int32.MinValue;\n Idx = idx;\n if (lineOffset == 0)\n ++StateTag;\n else\n RegisterNewLineBegin(lineBegin, lineOffset);\n return count;\n }\n return 0;\n }\n }\n // found string\n int idx0 = Idx;\n int length = idx - idx0;\n if (length != 0) {\n Idx = idx;\n if (lineOffset == 0) {\n ++StateTag;\n skippedCharsIfStringFoundOtherwiseNull = String.Substring(idx0, length);\n return length;\n } else {\n RegisterNewLineBegin(lineBegin, lineOffset);\n skippedCharsIfStringFoundOtherwiseNull =\n !normalizeNewlinesInOutString || (nCR | nCRLF) == 0\n ? String.Substring(idx0, length)\n : Text.CopyWithNormalizedNewlines(String, idx0, length, nCRLF, nCR);\n return length - nCRLF;\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = \"\";\n }\n } else {\n skippedCharsIfStringFoundOtherwiseNull = null;\n }\n return 0;\n }\n} // class CharStream\n\n\npublic struct CharStreamState {\n#if DEBUG\n internal readonly CharStream CharStream;\n private long Index { get { return GetIndex(CharStream); } }\n#endif\n internal readonly int Idx;\n#if SMALL_STATETAG\n public readonly uint Tag;\n#else\n public readonly ulong Tag;\n#endif\n public readonly long Line;\n public readonly long LineBegin;\n public readonly TUserState UserState;\n public readonly string Name;\n\n public CharStreamState(CharStream charStream) {\n #if DEBUG\n CharStream = charStream;\n #endif\n Idx = charStream.Idx;\n Tag = charStream.StateTag;\n Line = charStream._Line;\n LineBegin = charStream._LineBegin;\n UserState = charStream._UserState;\n Name = charStream._Name;\n }\n\n private static void ThrowInvalidState() {\n throw new InvalidOperationException(\"The CharStreamState is invalid.\");\n }\n\n public CharStreamIndexToken IndexToken { get {\n if (Line <= 0) ThrowInvalidState(); // tests for a zero-initialized state\n\n return new CharStreamIndexToken(\n #if DEBUG\n CharStream,\n #endif\n Idx);\n } }\n\n public long GetIndex(CharStream charStreamFromWhichStateWasRetrieved) {\n if (Line <= 0) ThrowInvalidState(); // tests for a zero-initialized state\n #if DEBUG\n Debug.Assert(CharStream == charStreamFromWhichStateWasRetrieved);\n #endif\n return charStreamFromWhichStateWasRetrieved.GetIndex(Idx);\n }\n\n public Position GetPosition(CharStream charStreamFromWhichStateWasRetrieved) {\n if (Line <= 0) ThrowInvalidState(); // tests for a zero-initialized state\n #if DEBUG\n Debug.Assert(CharStream == charStreamFromWhichStateWasRetrieved);\n #endif\n long index = charStreamFromWhichStateWasRetrieved.GetIndex(Idx);\n return new Position(Name, index, Line, index - LineBegin + 1);\n }\n}\n\n\n/// Provides read‐access to a sequence of UTF‐16 chars.\npublic sealed class CharStream : CharStream {\n internal CharStream(string chars) : base(chars) {}\n\n public CharStream(string chars, int index, int length) : base(chars, index, length) {}\n\n public CharStream(string chars, int index, int length, long streamBeginIndex)\n : base(chars, index, length, streamBeginIndex) {}\n\n public CharStream(string path, Encoding encoding) : base(path, encoding) {}\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : base(path, encoding, detectEncodingFromByteOrderMarks) {}\n\n public CharStream(string path, Encoding encoding, bool detectEncodingFromByteOrderMarks, int byteBufferLength)\n : base(path, encoding, detectEncodingFromByteOrderMarks, byteBufferLength) {}\n\n public CharStream(Stream stream, Encoding encoding) : base(stream, encoding) {}\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding)\n : base(stream, leaveOpen, encoding) {}\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks)\n : base(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks) {}\n\n public CharStream(Stream stream, bool leaveOpen, Encoding encoding, bool detectEncodingFromByteOrderMarks, int byteBufferLength)\n : base(stream, leaveOpen, encoding, detectEncodingFromByteOrderMarks, byteBufferLength) {}\n\n\n internal TUserState _UserState;\n public TUserState UserState {\n get { return _UserState; }\n set { _UserState = value; ++StateTag; }\n }\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n public CharStreamState State { get {\n return new CharStreamState(this);\n } }\n\n private static void ThrowInvalidState() {\n throw new ArgumentException(\"The CharStreamState is invalid.\");\n }\n\n public void BacktrackTo(CharStreamState state) {\n BacktrackTo(ref state);\n }\n public void BacktrackTo(ref CharStreamState state) {\n if (state.Line <= 0) ThrowInvalidState(); // tests for zero-initialized states\n #if DEBUG\n Debug.Assert(this == state.CharStream);\n #endif\n Idx = state.Idx;\n Debug.Assert((Idx >= IndexBegin && Idx < IndexEnd) || Idx == Int32.MinValue);\n StateTag = state.Tag;\n _Line = state.Line;\n _LineBegin = state.LineBegin;\n _UserState = state.UserState;\n _Name = state.Name;\n }\n\n public string ReadFrom(CharStreamState stateWhereStringBegins, bool normalizeNewlines) {\n return ReadFrom(ref stateWhereStringBegins, normalizeNewlines);\n }\n public string ReadFrom(ref CharStreamState state, bool normalizeNewlines) {\n if (state.Line <= 0) ThrowInvalidState(); // tests for zero-initialized states\n #if DEBUG\n Debug.Assert(this == state.CharStream);\n #endif\n var str = ReadFrom(state.Idx);\n if (!normalizeNewlines || state.Line == _Line) return str;\n return Text.NormalizeNewlines(str);\n }\n\n public CharStream CreateSubstream(CharStreamState stateWhereSubstreamBegins) {\n return CreateSubstream(ref stateWhereSubstreamBegins);\n }\n public CharStream CreateSubstream(ref CharStreamState stateWhereSubstreamBegins) {\n if (stateWhereSubstreamBegins.Line <= 0) ThrowInvalidState(); // tests for zero-initialized states\n #if DEBUG\n Debug.Assert(this == stateWhereSubstreamBegins.CharStream);\n #endif\n int idx0 = stateWhereSubstreamBegins.Idx;\n if (unchecked((uint)idx0 > (uint)Idx))\n throw new ArgumentException(\"The current position of the stream must not lie before the position corresponding to the given CharStreamState.\");\n var subStream = new CharStream(String);\n subStream._Name = stateWhereSubstreamBegins.Name;\n subStream.Idx = idx0 == Idx ? Int32.MinValue : idx0;\n subStream.IndexBegin = idx0 < 0 ? IndexEnd : idx0;\n subStream.IndexEnd = Idx < 0 ? IndexEnd : Idx;\n subStream.StringToStreamIndexOffset = StringToStreamIndexOffset;\n subStream._Line = stateWhereSubstreamBegins.Line;\n subStream._LineBegin = stateWhereSubstreamBegins.LineBegin;\n return subStream;\n }\n}\n\n}\n\n#endif\n" }, { "path": "FParsecCS/Cloning.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n#if !LOW_TRUST\n\nusing System;\nusing System.Collections.Generic;\nusing System.Reflection;\nusing System.Runtime.Serialization;\nusing System.Diagnostics;\nusing System.Reflection.Emit;\n\nnamespace FParsec.Cloning {\n\n// The classes in this namespace provide a cloning service based on the serialization API.\n\n// Capturing the state of an object and/or cloning it with this API is often at least an\n// order of magnitude faster than doing the same with the BinaryFormatter and a MemoryStream\n// (ignoring some initial setup costs and the JITing time).\n\n// Some implementation details:\n//\n// The serialization API of the BCL (as supported by the BinaryFormatter) spans several\n// interfaces, classes and attributes under the System.Runtime.Serialization namespace.\n// Unfortunately the publicly available API documentation provided by Microsoft does not\n// sufficiently cover certain important details of the serialization API behaviour.\n//\n// For example, the documentation only vaguely discusses whether serialization events\n// like OnDeserialized are invoked in a certain order on multiple objects in a graph.\n// It seems that many API users intuitively expect a certain ordering, at least in\n// simple cases, but it is also clear that an ordering can not be guaranteed in all\n// cases (e.g. in the case of a cyclic object graph).\n//\n// The .NET BinaryFormatter seems to attempt to invoke the deserialization events\n// on dependent objects first, but its behaviour is inconsistent and in some situations\n// arguably buggy. The following bug report discusses these issues in more detail:\n// https://connect.microsoft.com/VisualStudio/feedback/details/549277\n//\n// For the sake of compatibility with the .NET BinaryFormatter we try to mimic the\n// basic principles of its behaviour. However, we certainly do not try to copy every\n// bug or inconsistency.\n//\n// In order to be on the safe side we sort the serialized object graph topologically\n// if it contains objects implementing an OnDeserialized handler or the ISerializable\n// or IObjectReferenence interfaces. Since the object graph can contain cycles, we\n// first identify strongly connected components using a variant of Tarjan's algorithm.\n// Any OnDeserializing handler, deserialization constructor, OnDeserialized handler or\n// IObjectReference.GetRealObject method (in that order) is then invoked in the\n// topological order starting with the most dependent objects. Objects in a strongly\n// connected component (with more than 1 object) are processed in the reverse order\n// in which the objects in the component where discovered during a depth-first search\n// of the serialized object graph. In a first pass the OnDeserializing handlers and\n// deserialization constructors of the objects in the component are invoked. Any\n// OnDeserialized handlers are then invoked in a second pass.\n// OnSerializing handlers are invoked immediately before an object is serialized.\n// OnSerialized handlers are invoked in an undefined order at the end of the\n// serialization job (not immediately after an object's subgraph has been serialized).\n//\n// We only allow an object implementing IObjectReference in a cycle of the serialized\n// object graph under the following conditions (which are more restrictive than\n// what the .NET BinaryFormatter enforces):\n// - There may only be 1 object implementing IObjectReference in a cycle.\n// - The type implementing IObjectReference must not be a value type.\n// - All objects containing references to the IObjectReference object must have reference types.\n// - The type implementing IObjectReference must not have any OnDeserialized handler.\n// - There must not be any other object in the cycle implementing ISerializable.\n//\n// Similar to the .NET BinaryFormatter we delay all IDeserializationCallbacks until\n// the end of the deserialization of the complete object graph (not just the relevant\n// subgraph). As explained in the referenced Connect bug report this behaviour has some\n// severe consequences for the usefulness of IDeserializationCallbacks and the\n// composability of the whole serialization API. However, for compatibility we really\n// have to stick to Microsoft's design, even if in our case it would actually\n// be simpler to invoke the callbacks in topological order as soon as an object's\n// subgraph (and its strongly connected component) is completely deserialized.\n//\n// If the serialized object graph contains unboxed value type instances, any event\n// handlers are invoked on boxed copies as follows:\n// OnSerializing and OnSerialized handlers are not called on the original value type\n// instance (which can be a field or an array element), but on a boxed copy of the\n// instance. Thus, if the handler mutates the instance, the changes do not show up in the\n// object graph that was serialized, though changes made by OnSerializing (but not\n// OnSerialized) will show up in the deserialized object graph. This behaviour\n// simplifies the implementation and is in accordance with the behaviour of the\n// .NET BinaryFormatter.\n// OnDeserializing and OnDeserialized handlers are invoked on a boxed value type instance\n// too, but this time any changes show up in the deserialized object graph, because\n// the boxed instance is copied into the deserialized object graph after the\n// OnDeserialized event (at least if the instance is not part of an object cycle).\n// This deviates from the BinaryFormatter behaviour in that\n// the BinaryFormatter seems to copy the boxed instance into the deserialized object\n// graph before the OnDeserialized event. However, since mutating the instance\n// in an OnDeserialized handler has no effect when using the BinaryFormatter,\n// hopefully no one causes an incompatibility with this implementation by actually trying\n// to mutate the instance. (Note that mutable structs with serialization event handlers\n// are extremely rare anyway).\n// An IDeserializationCallback.OnDeserialization handler is invoked on the boxed instance\n// after it has been copied into the deserialized object graph (and then is not copied\n// again), so any changes won't show up in the deserialized unboxed value type instance\n// (this holds for both the .NET BinaryFormatter and this implementation).\n\n\n/// Contains the serialized state of on object.\npublic abstract class CloneImage {\n /// Deserializes the object state into a new object.\n public abstract object CreateClone();\n\n internal CloneImage() {}\n}\n\npublic abstract class Cloner {\n // public interface\n public readonly Type Type;\n\n /// Returns a cloner for the given run-time type.\n /// The run-time type of the objects to clone. The type must be serializable.\n public static Cloner Create(Type type) {\n lock (Cache) return CreateWithoutLock(type);\n }\n\n /// Copies the given object using the serialization API.\n /// The object to clone. instance.GetType() must equal the Type the Cloner was created for.\n public object Clone(object instance) {\n return CaptureImage(instance, false).CreateClone();\n }\n\n /// Returns an image of the given object instance.\n /// The object to capture an image of.\n public CloneImage CaptureImage(object instance) {\n return CaptureImage(instance, true);\n }\n\n // internal/protected interface\n\n private readonly CloneEventHandlers EventHandlers;\n\n private Cloner(Type type, CloneEventHandlers eventHandlers) { Type = type; EventHandlers = eventHandlers; }\n\n internal abstract State CaptureShallowStateAndEnqueueNestedState(object value, CaptureContext captureContext);\n\n internal sealed class CaptureContext {\n public readonly bool IsReturnedToUser;\n\n public CaptureContext(bool stateIsReturnedToUser) {\n IsReturnedToUser = stateIsReturnedToUser;\n }\n\n // currently uses a static queue, but could easily be rewritten to use an instance queue\n public int GetObjectIndex(object instance, Cloner cloner) {\n Debug.Assert(instance.GetType() == cloner.Type);\n int objectIndex;\n if (!ObjectIndices.TryGetValue(instance, out objectIndex)) {\n objectIndex = ObjectIndices.Count;\n ObjectIndices.Add(instance, objectIndex);\n var item = new WorkItem{Cloner = cloner, Instance = instance};\n WorkQueue.Enqueue(item);\n }\n return objectIndex;\n }\n }\n\n // internal interface\n\n internal abstract class State {\n /// May be null.\n public readonly CloneEventHandlers EventHandlers;\n\n /// Indices of nested objects in the object graph. May be null.\n public readonly int[] ObjectIndices;\n\n /// May be null.\n public int[] StronglyConnectedComponent;\n\n public abstract Type Type { get; }\n public abstract object CreateUninitializedObject();\n public abstract void WriteToUninitializedObject(object instance, object[] objectGraph);\n\n public State(CloneEventHandlers eventHandlers, int[] objectIndices) {\n EventHandlers = eventHandlers;\n ObjectIndices = objectIndices;\n }\n\n private State() {}\n public static readonly State Dummy = new DummyState();\n private sealed class DummyState : State {\n public override Type Type { get {\n throw new NotImplementedException();\n } }\n\n public override object CreateUninitializedObject() {\n throw new NotImplementedException();\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n throw new NotImplementedException();\n }\n }\n }\n\n private static readonly StreamingContext StreamingContext = new StreamingContext(StreamingContextStates.Clone);\n private static readonly FormatterConverter FormatterConverter = new FormatterConverter();\n\n private static readonly Func CloneMemberwise = CreateMemberwiseCloneDelegate();\n private static Func CreateMemberwiseCloneDelegate() {\n var dynamicMethod = new DynamicMethod(\"InvokeMemberwiseClone\", typeof(object), new Type[]{typeof(object)}, true);\n var ilg = dynamicMethod.GetILGenerator();\n ilg.Emit(OpCodes.Ldarg_0);\n var method = typeof(object).GetMethod(\"MemberwiseClone\", BindingFlags.NonPublic | BindingFlags.Instance);\n ilg.EmitCall(OpCodes.Call, method, null); // non-virtual call\n ilg.Emit(OpCodes.Ret);\n return (Func)dynamicMethod.CreateDelegate(typeof(Func));\n }\n\n // private data and methods\n\n // Cache serves as the synchronization root for the Create and CaptureImage methods\n private static readonly Dictionary Cache = new Dictionary();\n\n private static Cloner CreateWithoutLock(Type type) {\n Cloner cloner;\n if (Cache.TryGetValue(type, out cloner)) return cloner;\n\n if (!type.IsSerializable)\n throw new SerializationException(\"The type '\" + type.ToString() + \"' is not marked as serializable.\");\n\n if (!type.IsArray) {\n var eventHandlers = CloneEventHandlers.Create(type);\n if (eventHandlers != null && (eventHandlers.Events & CloneEvents.ISerializable) != 0) {\n cloner = new CustomSerializationCloner(type, eventHandlers);\n } else {\n bool typeIsBlittable;\n var fields = GetSerializedFields(type, out typeIsBlittable);\n if (typeIsBlittable && (eventHandlers == null || (eventHandlers.Events & CloneEvents.OnDeserializing) == 0))\n cloner = new BlittableCloner(type, eventHandlers, fields);\n else\n cloner = new NativeSerializationCloner(type, eventHandlers, fields);\n }\n } else { // array\n var elementType = type.GetElementType();\n if (elementType.IsPrimitive || elementType == typeof(string)) {\n cloner = new BlittableCloner(type, null, new FieldInfo[0]);\n } else {\n var elementCloner = CreateWithoutLock(elementType);\n if (elementType.IsValueType && elementCloner is BlittableCloner)\n cloner = new BlittableCloner(type, null, new FieldInfo[0]);\n else if (type.GetArrayRank() == 1)\n cloner = new Rank1ArrayCloner(type, elementCloner);\n else\n cloner = new RankNArrayCloner(type, elementCloner);\n }\n }\n\n Cache.Add(type, cloner);\n return cloner;\n }\n\n // for optimization purposes CaptureImage uses some static queues\n\n private sealed class PhyiscalEqualityObjectComparer : System.Collections.Generic.EqualityComparer {\n public override bool Equals(object x, object y) { return x == y; }\n public override int GetHashCode(object obj) {\n return System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(obj);\n }\n }\n\n private static readonly Dictionary ObjectIndices = new Dictionary(new PhyiscalEqualityObjectComparer());\n private static readonly List States = new List();\n private static readonly Queue WorkQueue = new Queue();\n private static readonly List OnSerializedList = new List();\n private static readonly List ObjectReferenceList = new List();\n\n private struct WorkItem {\n public Cloner Cloner;\n public object Instance;\n\n public WorkItem(Cloner cloner, object instance) {\n Cloner = cloner;\n Instance = instance;\n }\n }\n\n private struct OnSerializedListItem {\n public CloneEventHandlers EventHandlers;\n public object Instance;\n\n public OnSerializedListItem(CloneEventHandlers cloneEventHandlers, object instance) {\n EventHandlers = cloneEventHandlers;\n Instance = instance;\n }\n }\n\n private static bool Contains(int[] arrayOrNull, int element) {\n if (arrayOrNull != null) {\n foreach (var e in arrayOrNull)\n if (e == element) return true;\n }\n return false;\n }\n\n private CloneImage CaptureImage(object instance, bool imageIsReturnedToUser) {\n if (instance.GetType() != Type)\n throw new ArgumentException(\"The object instance does not have the run-time type the Cloner was created for.\");\n lock (Cache) {\n try {\n bool needSort = false;\n\n // reserve 0-index spot\n ObjectIndices.Add(State.Dummy, 0);\n States.Add(null);\n\n var captureInfo = new CaptureContext(imageIsReturnedToUser);\n\n ObjectIndices.Add(instance, 1);\n WorkQueue.Enqueue(new WorkItem(this, instance));\n int deserializationCallbackCount = 0;\n do {\n var item = WorkQueue.Dequeue();\n var cloner = item.Cloner;\n if (cloner.EventHandlers == null) {\n States.Add(item.Cloner.CaptureShallowStateAndEnqueueNestedState(item.Instance, captureInfo));\n } else if (cloner.EventHandlers.Events == CloneEvents.ISerializable) {\n States.Add(item.Cloner.CaptureShallowStateAndEnqueueNestedState(item.Instance, captureInfo));\n needSort = true;\n } else {\n var eventHandlers = cloner.EventHandlers;\n if ((eventHandlers.Events & CloneEvents.OnSerializing) != 0)\n eventHandlers.InvokeOnSerializing(item.Instance, StreamingContext);\n if ((eventHandlers.Events & CloneEvents.OnSerialized) != 0)\n OnSerializedList.Add(new OnSerializedListItem(eventHandlers, item.Instance));\n var state = item.Cloner.CaptureShallowStateAndEnqueueNestedState(item.Instance, captureInfo);\n States.Add(state);\n eventHandlers = state.EventHandlers; // may be different from cloner.EventHandlers (for CustomSerializationState)\n if ((eventHandlers.Events & ( CloneEvents.ISerializable\n | CloneEvents.OnDeserialized\n | CloneEvents.IObjectReference)) != 0) //\n {\n needSort = true;\n if ((eventHandlers.Events & CloneEvents.IObjectReference) != 0)\n ObjectReferenceList.Add(States.Count - 1);\n }\n // unfortunately the BinaryFormatter doesn't guarantee any order for IDeserializationCallbacks\n if ((eventHandlers.Events & (CloneEvents.IDeserializationCallback)) != 0)\n ++deserializationCallbackCount;\n }\n } while (WorkQueue.Count != 0);\n var states = States.ToArray();\n\n if (OnSerializedList.Count != 0) {\n foreach (var item in OnSerializedList)\n item.EventHandlers.InvokeOnSerialized(item.Instance, StreamingContext);\n }\n\n if (!needSort)\n return new SimpleImage(states, deserializationCallbackCount);\n\n int[] order = ComputeTopologicalOrder(states);\n if (ObjectReferenceList.Count != 0) {\n foreach (var index1 in ObjectReferenceList) {\n var state1 = states[index1];\n var scc = state1.StronglyConnectedComponent;\n if (scc == null) continue;\n var type1 = state1.Type;\n if (type1.IsValueType)\n throw new SerializationException(\"The serialized object graph contains a cycle that includes a value type object (type: \"+ type1.FullName +\") implementing IObjectReference.\");\n if ((state1.EventHandlers.Events & CloneEvents.OnDeserialized) != 0)\n throw new SerializationException(\"The serialized object graph contains a cycle that includes an object (type: \"+ type1.FullName +\") implementing IObjectReference and also exposing an OnDeserialized handler.\");\n foreach (var index2 in scc) {\n if (index2 == index1) continue;\n var state2 = states[index2];\n var type2 = state2.Type;\n if (state2.EventHandlers != null && (state2.EventHandlers.Events & (CloneEvents.ISerializable | CloneEvents.IObjectReference)) != 0) {\n var msg = String.Format(\"The serialized object graph contains a cycle that includes an object (type: {0}) implementing IObjectReference and another object (type: {1}) implementing ISerializable and/or IObjectReference .\", type1.FullName, type2.FullName);\n throw new SerializationException(msg);\n }\n if (type2.IsValueType && Contains(state2.ObjectIndices, index1)) {\n var msg = String.Format(\"The serialized object graph contains a cycle that includes a value type object (type: {0}) referencing an IObjectReference object (type: {1}) in the same cycle.\", type2.FullName, type1.FullName);\n throw new SerializationException(msg);\n }\n }\n }\n }\n return new OrderedImage(states, order, deserializationCallbackCount);\n } finally {\n States.Clear();\n ObjectIndices.Clear();\n if (WorkQueue.Count != 0) WorkQueue.Clear();\n if (OnSerializedList.Count != 0) OnSerializedList.Clear();\n if (ObjectReferenceList.Count != 0) ObjectReferenceList.Clear();\n }\n }\n }\n\n private sealed class BlittableCloner : Cloner {\n internal readonly FieldInfo[] SerializedFields;\n\n public BlittableCloner(Type type, CloneEventHandlers eventHandlers, FieldInfo[] serializedFields) : base(type, eventHandlers) {\n Debug.Assert(serializedFields != null);\n SerializedFields = serializedFields;\n }\n\n internal override State CaptureShallowStateAndEnqueueNestedState(object instance, CaptureContext captureContext) {\n Debug.Assert(Type == instance.GetType());\n if (captureContext.IsReturnedToUser) {\n return new BlittableState(EventHandlers, CloneMemberwise(instance));\n } else {\n return new BlittableState(EventHandlers, instance);\n }\n }\n }\n\n private sealed class BlittableState : State {\n private object Value;\n\n public BlittableState(CloneEventHandlers eventHandlers, object value) : base(eventHandlers, null) {\n Value = value;\n }\n\n public override Type Type { get { return Value.GetType(); } }\n\n public override object CreateUninitializedObject() {\n return Cloner.CloneMemberwise(Value);\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) { }\n }\n\n private sealed class Rank1ArrayCloner : Cloner {\n Cloner PreviousElementCloner;\n\n public Rank1ArrayCloner(Type type, Cloner elementCloner) : base(type, null) {\n PreviousElementCloner = elementCloner;\n }\n\n internal override State CaptureShallowStateAndEnqueueNestedState(object instance, CaptureContext captureContext) {\n Debug.Assert(Type == instance.GetType());\n var array = (Array)instance;\n var lowerBound = array.GetLowerBound(0);\n var length = array.Length; // should throw an exception if length > Int32.MaxValue\n if (length == 0) return new BlittableState(null, instance);\n var throwExceptionOnOverflow = checked(lowerBound + length);\n var objectIndices = new int[length];\n var cloner = PreviousElementCloner;\n var previousType = cloner.Type;\n for (int i = 0; i < length; ++i) {\n var value = array.GetValue(lowerBound + i);\n if (value != null) {\n var type = value.GetType();\n if (type != previousType) {\n cloner = CreateWithoutLock(type);\n previousType = type;\n }\n objectIndices[i] = captureContext.GetObjectIndex(value, cloner);\n }\n }\n PreviousElementCloner = cloner;\n return new Rank1ArrayState(Type.GetElementType(), lowerBound, objectIndices);\n }\n }\n\n private sealed class Rank1ArrayState : State {\n private readonly Type ElementType;\n private readonly int LowerBound;\n\n public Rank1ArrayState(Type elementType, int lowerBound, int[] objectIndices) : base(null, objectIndices) {\n Debug.Assert(objectIndices != null);\n ElementType = elementType;\n LowerBound = lowerBound;\n }\n\n public override Type Type { get { return ElementType.MakeArrayType(); } }\n\n public override object CreateUninitializedObject() {\n if (LowerBound == 0)\n return Array.CreateInstance(ElementType, ObjectIndices.Length);\n else\n return Array.CreateInstance(ElementType, new int[]{ObjectIndices.Length}, new int[]{LowerBound});\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n var array = (Array)instance;\n var objectIndices = ObjectIndices;\n for (int i = 0; i < objectIndices.Length; ++i) {\n var objectIndex = objectIndices[i];\n if (objectIndex == 0) continue;\n array.SetValue(objectGraph[objectIndex], LowerBound + i);\n }\n }\n }\n\n private sealed class RankNArrayCloner : Cloner {\n Cloner PreviousElementCloner;\n\n public RankNArrayCloner(Type type, Cloner elementCloner) : base(type, null) {\n PreviousElementCloner = elementCloner;\n }\n\n internal override State CaptureShallowStateAndEnqueueNestedState(object instance, CaptureContext captureContext) {\n Debug.Assert(Type == instance.GetType());\n var array = (Array)instance;\n var rank = array.Rank;\n var lowerBounds = new int[rank];\n var lengths = new int[rank];\n var ends = new int[rank];\n var numberOfElements = 1;\n for (int d = 0; d < rank; ++d) {\n var lowerBound = array.GetLowerBound(d);\n lowerBounds[d] = lowerBound;\n var length = array.GetLength(d);\n lengths[d] = length;\n ends[d] = checked(lowerBound + length);\n numberOfElements = checked(numberOfElements * length);\n }\n var objectIndices = new int[numberOfElements];\n var cloner = PreviousElementCloner;\n var previousType = cloner.Type;\n var indices = (int[])lowerBounds.Clone();\n for (int i = 0; i < numberOfElements; ++i) {\n var value = array.GetValue(indices);\n if (value != null) {\n var type = value.GetType();\n if (type != previousType) {\n cloner = CreateWithoutLock(type);\n previousType = type;\n }\n objectIndices[i] = captureContext.GetObjectIndex(value, cloner);\n }\n // increment multi-dimensional index\n var d = rank - 1;\n do {\n if (++indices[d] < ends[d]) break;\n indices[d] = lowerBounds[d];\n } while (--d >= 0);\n }\n PreviousElementCloner = cloner;\n return new RankNArrayState(Type.GetElementType(), lengths, lowerBounds, ends, objectIndices);\n }\n }\n\n private sealed class RankNArrayState : State {\n private readonly Type ElementType;\n private readonly int[] Lengths;\n private readonly int[] LowerBounds;\n private readonly int[] Ends;\n\n public RankNArrayState(Type elementType, int[] lengths, int[] lowerBounds, int[] ends, int[] objectIndices)\n : base(null, objectIndices)\n {\n Debug.Assert(lengths != null && lengths.Length == lowerBounds.Length && lengths.Length == ends.Length && objectIndices != null);\n ElementType = elementType;\n Lengths = lengths;\n LowerBounds = lowerBounds;\n Ends = ends;\n }\n\n public override Type Type { get { return ElementType.MakeArrayType(Lengths.Length); } }\n\n public override object CreateUninitializedObject() {\n return Array.CreateInstance(ElementType, Lengths, LowerBounds);\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n var array = (Array)instance;\n var indices = (int[])LowerBounds.Clone();\n foreach (var objectIndex in ObjectIndices) {\n if (objectIndex != 0)\n array.SetValue(objectGraph[objectIndex], indices);\n // increment multi-dimensional index\n var d = LowerBounds.Length - 1;\n do {\n if (++indices[d] < Ends[d]) break;\n indices[d] = LowerBounds[d];\n } while (--d >= 0);\n }\n }\n }\n\n private sealed class NativeSerializationCloner : Cloner {\n internal readonly FieldInfo[] SerializedFields;\n private readonly Cloner[] Cloners;\n\n private Func FieldValuesGetter; // lazily initialized\n internal Action FieldValuesSetter; // lazily initialized\n\n public NativeSerializationCloner(Type type, CloneEventHandlers eventHandlers, FieldInfo[] serializedFields) : base(type, eventHandlers) {\n SerializedFields = serializedFields;\n Cloners = new Cloner[serializedFields.Length];\n }\n\n internal override State CaptureShallowStateAndEnqueueNestedState(object instance, CaptureContext captureContext) {\n Debug.Assert(Type == instance.GetType());\n if (SerializedFields.Length == 0)\n return new NativeSerializationState(this);\n var getter = FieldValuesGetter;\n if (getter == null)\n FieldValuesGetter = getter = CreateFieldValuesGetter(Type, SerializedFields);\n var values = getter(instance); // GetFieldValues(instance, SerializedFields);\n int[] objectIndices = new int[values.Length];\n for (int i = 0; i < values.Length; ++i) {\n var value = values[i];\n if (value == null) continue;\n var type = value.GetType();\n if (type.IsPrimitive || type == typeof(string)) continue;\n values[i] = null;\n var cloner = Cloners[i];\n if (cloner == null || type != cloner.Type) {\n cloner = CreateWithoutLock(type);\n Cloners[i] = cloner;\n }\n objectIndices[i] = captureContext.GetObjectIndex(value, cloner);\n }\n return new NativeSerializationState(this, values, objectIndices);\n }\n }\n\n private sealed class NativeSerializationState : State {\n private readonly NativeSerializationCloner Cloner;\n private readonly object[] Values; // maybe null if object has no fields\n\n public NativeSerializationState(NativeSerializationCloner cloner)\n : base(cloner.EventHandlers, null)\n {\n Cloner = cloner;\n }\n\n public NativeSerializationState(NativeSerializationCloner cloner, object[] values, int[] objectIndices)\n : base(cloner.EventHandlers, objectIndices)\n {\n Debug.Assert(cloner != null && values.Length != 0 && values.Length == objectIndices.Length);\n Cloner = cloner;\n Values = values;\n }\n\n public override Type Type { get { return Cloner.Type; } }\n\n public override object CreateUninitializedObject() {\n return FormatterServices.GetUninitializedObject(Cloner.Type);\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n if (ObjectIndices == null) return;\n var setter = Cloner.FieldValuesSetter;\n if (setter == null)\n Cloner.FieldValuesSetter = setter = CreateFieldValuesSetter(Cloner.Type, Cloner.SerializedFields);\n setter(instance, Values, ObjectIndices, objectGraph);\n }\n }\n\n // NativeSerializationProxyState is used by CustomSerializationCloner to store the state of\n // proxy objects which don't implement ISerializable.\n private sealed class NativeSerializationProxyState : State {\n private readonly Type Type_;\n private readonly FieldInfo[] Fields;\n private readonly object[] Values;\n\n public NativeSerializationProxyState(Type type, CloneEventHandlers eventHandlers)\n : base(eventHandlers, null)\n {\n Type_ = type;\n }\n\n public NativeSerializationProxyState(Type type, CloneEventHandlers eventHandlers, FieldInfo[] fields, object[] values, int[] objectIndices)\n : base(eventHandlers, objectIndices)\n {\n Debug.Assert(fields.Length == values.Length && values.Length == objectIndices.Length);\n Type_ = type;\n Fields = fields;\n Values = values;\n }\n\n public override Type Type { get { return Type_; } }\n\n public override object CreateUninitializedObject() {\n return FormatterServices.GetUninitializedObject(Type_);\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n if (ObjectIndices == null) return;\n // We can't use a NativeSerializationCloner.FieldValuesSetter here\n // because some primitive values might have a type different from the type of the field\n // they are assigned to. FieldInfo.SetValue does some automatic conversions in those\n // cases that the FieldValuesSetter doesn't (e.g. integer type widening).\n for (int i = 0; i < ObjectIndices.Length; ++i) {\n var objectIndex = ObjectIndices[i];\n if (objectIndex == 0) {\n var value = Values[i];\n if (value != null) Fields[i].SetValue(instance, value);\n } else {\n Fields[i].SetValue(instance, objectGraph[objectIndex]);\n }\n }\n }\n }\n\n private struct CustomSerializationMemberInfo {\n public string Name;\n public Type Type;\n public object Value;\n }\n\n private sealed class CustomSerializationCloner : Cloner {\n internal readonly ConstructorInfo Constructor;\n internal Action ConstructorCaller; // lazily initalized\n private Cloner PreviousProxyCloner;\n private Cloner[] Cloners;\n\n private static Type[] SerializableConstructorArgumentTypes = new Type[] {typeof(SerializationInfo), typeof(StreamingContext)};\n\n public CustomSerializationCloner(Type type,\n CloneEventHandlers eventHandlers)\n : base(type, eventHandlers)\n {\n Constructor = type.GetConstructor(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance, null, SerializableConstructorArgumentTypes, null);\n PreviousProxyCloner = this;\n }\n\n internal override State CaptureShallowStateAndEnqueueNestedState(object instance, CaptureContext captureContext) {\n Debug.Assert(Type == instance.GetType());\n var info = new SerializationInfo(Type, FormatterConverter);\n ((ISerializable)instance).GetObjectData(info, StreamingContext);\n var n = info.MemberCount;\n var members = new CustomSerializationMemberInfo[n];\n var objectIndices = new int[n];\n if (Cloners == null || Cloners.Length != n) Cloners = new Cloner[n];\n var iter = info.GetEnumerator();\n for (int i = 0; iter.MoveNext(); ++i) {\n var entry = iter.Current;\n members[i].Name = entry.Name;\n members[i].Type = entry.ObjectType;\n var value = entry.Value;\n if (value == null) continue;\n Type type = value.GetType();\n if (type.IsPrimitive || type == typeof(string)) {\n members[i].Value = value;\n continue;\n }\n var cloner = Cloners[i];\n if (cloner == null || type != cloner.Type) {\n cloner = CreateWithoutLock(type);\n Cloners[i] = cloner;\n }\n objectIndices[i] = captureContext.GetObjectIndex(value, cloner);\n }\n\n Type proxyType;\n\n if (!info.IsFullTypeNameSetExplicit && !info.IsAssemblyNameSetExplicit) {\n proxyType = info.ObjectType;\n } else {\n try {\n var assembly = Assembly.Load(info.AssemblyName);\n proxyType = assembly.GetType(info.FullTypeName, true);\n } catch (Exception e) {\n var msg = \"Can not load the type '\" + info.FullTypeName + \"' in the assembly '\" + info.AssemblyName + \"'.\";\n throw new SerializationException(msg, e);\n }\n }\n\n if (proxyType == Type) {\n if (Constructor == null) throw new SerializationException(\"The ISerializable type '\" + Type.ToString() + \"' does not define a proper deserialization constructor.\");\n return new CustomSerializationState(this, members, objectIndices);\n }\n\n Cloner proxyCloner;\n if (proxyType == PreviousProxyCloner.Type) {\n proxyCloner = PreviousProxyCloner;\n } else {\n proxyCloner = CreateWithoutLock(proxyType);\n PreviousProxyCloner = proxyCloner;\n }\n\n if (proxyType.IsArray) {\n // On .NET a NullReferenceException is thrown on deserialization of an array type proxy.\n throw new SerializationException(\"The type '\" + Type.ToString() + \"' uses an array type ('\" + proxyType.ToString() + \"') as its serialization proxy type.\");\n }\n\n CustomSerializationCloner csc = proxyCloner as CustomSerializationCloner;\n if (csc != null) {\n if (csc.Constructor == null) throw new SerializationException(\"The ISerializable type '\" + csc.Type.ToString() + \"' does not define a proper deserialization constructor.\");\n return new CustomSerializationState(csc, members, objectIndices);\n }\n\n if (n == 0) return new NativeSerializationProxyState(proxyType, proxyCloner.EventHandlers);\n\n FieldInfo[] proxyFields;\n {\n var nsc = proxyCloner as NativeSerializationCloner;\n if (nsc != null) {\n proxyFields = nsc.SerializedFields;\n } else {\n var bc = proxyCloner as BlittableCloner;\n Debug.Assert(bc != null);\n proxyFields = bc.SerializedFields;\n }\n }\n\n // The BinaryFormatter on .NET simply assigns the values in the SerializationInfo\n // to the field with the same name (of the most derived class) in the proxy object.\n // There are no checks whether all fields are assigned values or whether the target has\n // multiple fields with the same name. The types are only checked once the values are\n // assigned to the proxy object fields. Integer types are automatically widened and\n // types are cast to base or interface types if necessary.\n\n var proxyValues = new object[proxyFields.Length];\n var proxyObjectIndices = new int[proxyFields.Length];\n for (int i = 0; i < n; ++i) {\n var name = members[i].Name;\n for (int j = 0; j < proxyFields.Length; ++j) {\n if (name == proxyFields[j].Name) {\n proxyValues[j] = members[i].Value;\n proxyObjectIndices[j] = objectIndices[i];\n break;\n }\n }\n }\n return new NativeSerializationProxyState(proxyType, proxyCloner.EventHandlers, proxyFields, proxyValues, proxyObjectIndices);\n }\n }\n\n private sealed class CustomSerializationState : State {\n private readonly CustomSerializationCloner Cloner;\n private readonly CustomSerializationMemberInfo[] Members;\n\n public CustomSerializationState(CustomSerializationCloner cloner,\n CustomSerializationMemberInfo[] members,\n int[] objectIndices)\n : base(cloner.EventHandlers, objectIndices)\n {\n Cloner = cloner;\n Members = members;\n }\n\n public override Type Type { get { return Cloner.Type; } }\n\n public override object CreateUninitializedObject() {\n return FormatterServices.GetUninitializedObject(Cloner.Type);\n }\n\n public override void WriteToUninitializedObject(object instance, object[] objectGraph) {\n var info = new SerializationInfo(Cloner.Type, FormatterConverter);\n for (int i = 0; i < Members.Length; ++i) {\n var member = Members[i];\n var index = ObjectIndices[i];\n var value = index == 0 ? member.Value : objectGraph[index];\n info.AddValue(member.Name, value, member.Type);\n }\n var constructorCaller = Cloner.ConstructorCaller;\n if (constructorCaller == null)\n Cloner.ConstructorCaller = constructorCaller = CreateISerializableConstructorCaller(Cloner.Constructor);\n constructorCaller(instance, info, StreamingContext);\n }\n }\n\n private sealed class SimpleImage : CloneImage {\n private readonly Cloner.State[] States;\n private readonly int DeserializationCallbackCount;\n\n internal SimpleImage(Cloner.State[] states, int deserializationCallbackCount) {\n Debug.Assert(states.Length > 1 && states[0] == null);\n States = states;\n DeserializationCallbackCount = deserializationCallbackCount;\n }\n\n public override object CreateClone() {\n int callbackIndicesIndex = DeserializationCallbackCount;\n int[] callbackIndices =\n DeserializationCallbackCount == 0 ? null : new int[DeserializationCallbackCount];\n var objects = new object[States.Length];\n // States[0] is null\n for (int i = 1; i < States.Length; ++i)\n objects[i] = States[i].CreateUninitializedObject();\n for (int index = States.Length - 1; index != 0; --index) {\n var state = States[index];\n var instance = objects[index];\n var eventHandlers = state.EventHandlers;\n if (eventHandlers == null) {\n state.WriteToUninitializedObject(objects[index], objects);\n } else {\n var events = eventHandlers.Events;\n Debug.Assert((events & ( CloneEvents.ISerializable\n | CloneEvents.OnDeserialized\n | CloneEvents.IObjectReference)) == 0);\n if ((events & CloneEvents.OnDeserializing) != 0)\n eventHandlers.InvokeOnDeserializing(instance, Cloner.StreamingContext);\n if ((events & CloneEvents.IDeserializationCallback) != 0)\n callbackIndices[--callbackIndicesIndex] = index;\n state.WriteToUninitializedObject(instance, objects);\n }\n }\n if (callbackIndices != null) {\n Debug.Assert(callbackIndicesIndex == 0);\n foreach (var index in callbackIndices)\n ((IDeserializationCallback)objects[index]).OnDeserialization(null);\n }\n return objects[1];\n }\n }\n\n private sealed class OrderedImage : CloneImage {\n private readonly Cloner.State[] States;\n private readonly int[] Order;\n private readonly int DeserializationCallbackCount;\n\n internal OrderedImage(Cloner.State[] states, int[] order, int deserializationCallbackCount) {\n Debug.Assert(states.Length > 1 && states.Length == order.Length && states[0] == null);\n States = states;\n Order = order;\n DeserializationCallbackCount = deserializationCallbackCount;\n }\n\n public static object GetRealObject(object instance) {\n var or = (IObjectReference)instance;\n instance = or.GetRealObject(Cloner.StreamingContext);\n if (instance != or) {\n or = instance as IObjectReference;\n int i = 0;\n while (or != null) {\n if (++i == 100) throw new SerializationException(\"An object's implementation of the IObjectReference interface returned too many nested references to other objects that implement IObjectReference.\");\n instance = or.GetRealObject(Cloner.StreamingContext);\n if (instance == or) break;\n or = instance as IObjectReference;\n }\n if (instance == null) throw new SerializationException(\"An object's IObjectReference.GetRealObject implementation returned null.\");\n }\n return instance;\n }\n\n public override object CreateClone() {\n int callbackIndicesIndex = DeserializationCallbackCount;\n object[] callbackObjects =\n DeserializationCallbackCount == 0 ? null : new object[DeserializationCallbackCount];\n var objects = new object[States.Length];\n for (int i = 1; i < States.Length; ++i)\n objects[i] = States[i].CreateUninitializedObject();\n var delayedOnDeserializedEvents = new List();\n int objectReferenceIndex = 0;\n object objectReference = null;\n int[] lastScc = null;\n for (int i = Order.Length - 1; i != 0; --i) {\n var index = Order[i];\n var state = States[index];\n var scc = state.StronglyConnectedComponent;\n if (scc != lastScc) {\n lastScc = scc;\n if (objectReference != null) {\n ReplaceObjectReferenceInSCCWithRealObject(objectReference, objectReferenceIndex, objects);\n objectReferenceIndex = 0;\n objectReference = null;\n }\n if (delayedOnDeserializedEvents.Count != 0)\n InvokeDelayedOnDeserializedEvents(delayedOnDeserializedEvents, objects); // also clears delayedOnDeserializedEvents\n if (scc != null) {\n foreach (var idx in scc) {\n var handlers = States[idx].EventHandlers;\n if (handlers != null && (handlers.Events & CloneEvents.IObjectReference) != 0) {\n objectReferenceIndex = idx;\n objectReference = objects[idx];\n objects[idx] = null; // set to null until we call ReplaceObjectReferenceInSCCWithRealObject\n }\n }\n }\n }\n var instance = objects[index];\n var eventHandlers = state.EventHandlers;\n if (eventHandlers == null) {\n state.WriteToUninitializedObject(instance, objects);\n } else {\n var events = eventHandlers.Events;\n if (instance != null) {\n if ((events & CloneEvents.OnDeserializing) != 0)\n eventHandlers.InvokeOnDeserializing(instance, Cloner.StreamingContext);\n state.WriteToUninitializedObject(instance, objects);\n if ((events & CloneEvents.OnDeserialized) != 0) {\n if (scc == null) eventHandlers.InvokeOnDeserialized(instance, Cloner.StreamingContext);\n else delayedOnDeserializedEvents.Add(index);\n }\n if ((events & CloneEvents.IObjectReference) != 0) {\n Debug.Assert(state.StronglyConnectedComponent == null);\n objects[index] = GetRealObject(instance);\n }\n } else {\n Debug.Assert(index == objectReferenceIndex);\n }\n // It's a pity we have to process the IDeserializationCallback separately\n // from OnDeserialized events to stay compatible with the .NET BinaryFormatter.\n if ((events & CloneEvents.IDeserializationCallback) != 0)\n callbackObjects[--callbackIndicesIndex] = instance ?? objectReference;\n }\n }\n if (objectReference != null)\n ReplaceObjectReferenceInSCCWithRealObject(objectReference, objectReferenceIndex, objects);\n if (delayedOnDeserializedEvents.Count != 0)\n InvokeDelayedOnDeserializedEvents(delayedOnDeserializedEvents, objects);\n if (callbackObjects != null) {\n Debug.Assert(callbackIndicesIndex == 0);\n // We call the callback in in the reverse topological order at the end of\n // deserialization, which is similar to what the BinaryFormatter does, unfortunately.\n foreach (var obj in callbackObjects)\n ((IDeserializationCallback)obj).OnDeserialization(null);\n }\n return objects[1];\n }\n\n private void InvokeDelayedOnDeserializedEvents(List indices, object[] objects) {\n foreach (var index in indices) {\n var handlers = States[index].EventHandlers;\n handlers.InvokeOnDeserialized(objects[index], Cloner.StreamingContext);\n }\n indices.Clear();\n }\n\n private void ReplaceObjectReferenceInSCCWithRealObject(object objectReference, int objectReferenceIndex, object[] objects) {\n var state = States[objectReferenceIndex];\n var eventHandlers = state.EventHandlers;\n var events = eventHandlers.Events;\n if ((events & CloneEvents.OnDeserializing) != 0)\n eventHandlers.InvokeOnDeserializing(objectReference, Cloner.StreamingContext);\n state.WriteToUninitializedObject(objectReference, objects);\n Debug.Assert((events & CloneEvents.OnDeserialized) == 0);\n objects[objectReferenceIndex] = GetRealObject(objectReference);\n // set all references to real object\n foreach (var index2 in state.StronglyConnectedComponent) {\n if (index2 == objectReferenceIndex) continue;\n var state2 = States[index2];\n Debug.Assert(state2.EventHandlers == null || (state2.EventHandlers.Events & (CloneEvents.ISerializable | CloneEvents.IObjectReference)) == 0);\n if (Cloner.Contains(state2.ObjectIndices, objectReferenceIndex)) {\n Debug.Assert(!state2.Type.IsValueType);\n state2.WriteToUninitializedObject(objects[index2], objects); // overwrite all fields\n }\n }\n }\n }\n\n /// Returns the public and non-public fields of the type (and its base types),\n /// except fields with the NonSerialized attribute. In the returned array fields from\n /// derived types come before fields from base types.\n internal static FieldInfo[] GetSerializedFields(Type type, out bool typeIsBlittable) {\n Debug.Assert(type.IsSerializable && !type.IsInterface);\n // We need the fields of the most derived type first, but GetFields returns the\n // field in an undefined order, so we have to climb the type hierarchy.\n var fields = type.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly);\n bool isBlittable = true;\n int nonSerialized = 0;\n foreach (var f in fields) {\n if (f.IsNotSerialized) ++nonSerialized;\n var ft = f.FieldType;\n if (!ft.IsPrimitive && ft != typeof(string)) {\n if (!ft.IsValueType) isBlittable = false;\n else {\n bool fIsBlittable;\n GetSerializedFields(ft, out fIsBlittable);\n isBlittable &= fIsBlittable;\n }\n }\n }\n int numberOfBases = 0;\n var bt = type.BaseType;\n while (bt != null && bt != typeof(object)) {\n if (!bt.IsSerializable)\n throw new SerializationException(BaseTypeNotSerializableMessage(bt, type));\n ++numberOfBases;\n bt = bt.BaseType;\n }\n if (numberOfBases == 0) {\n if (nonSerialized == 0) {\n typeIsBlittable = isBlittable;\n return fields;\n } else {\n typeIsBlittable = false;\n var serializedFields = new FieldInfo[fields.Length - nonSerialized];\n int i = 0;\n foreach (var f in fields) if (!f.IsNotSerialized) serializedFields[i++] = f;\n return serializedFields;\n }\n } else {\n var baseFieldArrays = new FieldInfo[numberOfBases][];\n bt = type.BaseType;\n for (int i = 0; i < numberOfBases; ++i, bt = bt.BaseType) {\n var baseFields = bt.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly);\n foreach (var bf in baseFields) {\n if (bf.IsNotSerialized) ++nonSerialized;\n var bft = bf.FieldType;\n if (!bft.IsPrimitive && bft != typeof(string)) {\n if (!bft.IsValueType) isBlittable = false;\n else {\n bool bfIsBlittable;\n GetSerializedFields(bft, out bfIsBlittable);\n isBlittable &= bfIsBlittable;\n }\n }\n }\n baseFieldArrays[i] = baseFields;\n }\n\n typeIsBlittable = nonSerialized == 0 & isBlittable;\n\n var numberOfSerializedFields = fields.Length - nonSerialized;\n foreach (var baseFields in baseFieldArrays) numberOfSerializedFields += baseFields.Length;\n\n if (nonSerialized == 0 && numberOfSerializedFields == fields.Length) return fields;\n\n var combinedFields = new FieldInfo[numberOfSerializedFields];\n if (nonSerialized == 0) {\n int i = 0;\n foreach (var f in fields) combinedFields[i++] = f;\n foreach (var baseFields in baseFieldArrays)\n foreach (var bf in baseFields) combinedFields[i++] = bf;\n } else {\n int i = 0;\n foreach (var f in fields)\n if (!f.IsNotSerialized) combinedFields[i++] = f;\n foreach (var baseFields in baseFieldArrays)\n foreach (var bf in baseFields)\n if (!bf.IsNotSerialized) combinedFields[i++] = bf;\n }\n return combinedFields;\n }\n }\n\n internal static string BaseTypeNotSerializableMessage(Type baseType, Type childType) {\n return \"The serializable type '\" + childType.ToString() + \"' has a base type '\" + baseType.ToString() + \"' that is not serializable.\";\n }\n\n /*\n private static object[] GetFieldValues(object instance, FieldInfo[] fields) {\n var values = new object[fields.Length];\n for (int i = 0; i < fields.Length; ++i) {\n var f = fields[i];\n values[i] = f.GetValue(instance);\n }\n return values;\n }\n */\n internal static Func CreateFieldValuesGetter(Type type, FieldInfo[] fields) {\n if (fields.Length == 0) throw new ArgumentException(\"The fields array must be non-empty.\");\n\n var dynamicMethod = new DynamicMethod(\"FieldValuesGetter\",\n MethodAttributes.Public | MethodAttributes.Static,\n CallingConventions.Standard,\n typeof(object[]), new Type[]{typeof(object), typeof(object)},\n type, true);\n var ilg = dynamicMethod.GetILGenerator();\n var isValueType = type.IsValueType;\n\n // arg 0: dummy argument (makes delegate invocation faster)\n // arg 1: (boxed) object instance\n\n ilg.DeclareLocal(typeof(object[])); // local 0: the returned values array\n ilg.DeclareLocal(typeof(object)); // local 1: temporary object value\n\n // create the values array\n ilg.Emit(OpCodes.Ldc_I4, fields.Length);\n ilg.Emit(OpCodes.Newarr, typeof(object));\n ilg.Emit(OpCodes.Stloc_0);\n\n // cast/unbox the object instace\n ilg.Emit(OpCodes.Ldarg_1);\n if (!isValueType)\n ilg.Emit(OpCodes.Castclass, type);\n else\n ilg.Emit(OpCodes.Unbox, type);\n\n // The unbox IL construction doesn't return a normal managed pointer\n // but a \"controlled-mutability\" managed pointer. Since there's no way\n // to declare a controlled-mutability managed pointer local and one\n // can't convert such a pointer into a normal managed pointer, we can't\n // store away the pointer for later field accesses. Instead we use\n // OpCodes.Dup to keep the pointer around. Alternatively we could copy\n // the value type instance onto the stack, but that can be costly for\n // large value types.\n\n for (int i = 0; i < fields.Length; ++i) {\n if (i + 1 != fields.Length) ilg.Emit(OpCodes.Dup);\n\n var field = fields[i];\n ilg.Emit(OpCodes.Ldfld, field);\n if (field.FieldType.IsValueType)\n ilg.Emit(OpCodes.Box, field.FieldType);\n ilg.Emit(OpCodes.Stloc_1);\n\n // store object into result array\n ilg.Emit(OpCodes.Ldloc_0);\n ilg.Emit(OpCodes.Ldc_I4, i);\n ilg.Emit(OpCodes.Ldloc_1);\n ilg.Emit(OpCodes.Stelem_Ref);\n }\n\n ilg.Emit(OpCodes.Ldloc_0);\n ilg.Emit(OpCodes.Ret);\n\n return (Func)dynamicMethod.CreateDelegate(typeof(Func), null);\n }\n\n /*\n private static void SetFieldValues(FieldInfo[] fields, object instance, object[] values, int[] objectIndices, object[] objectGraph) {\n for (int i = 0; i < objectIndices.Length; ++i) {\n var objectIndex = ObjectIndices[i];\n if (objectIndex == 0)\n fields[i].SetValue(instance, values[i]);\n else\n fields[i].SetValue(instance, objectGraph[objectIndex]);\n }\n }\n */\n internal static Action CreateFieldValuesSetter(Type type, FieldInfo[] fields) {\n if (fields.Length == 0) throw new ArgumentException(\"The fields array must be non-empty.\");\n\n // It is important that we use the 8 argument DynamicMethod constructor\n // to associate the method with the type, so that the method is allowed\n // to set readonly (initonly) fields.\n var dynamicMethod = new DynamicMethod(\"FieldValuesSetter\",\n MethodAttributes.Public | MethodAttributes.Static, CallingConventions.Standard,\n null, new Type[]{typeof(object), typeof(object), typeof(object[]), typeof(int[]),\n typeof(object[])},\n type, true);\n var ilg = dynamicMethod.GetILGenerator();\n var isValueType = type.IsValueType;\n\n // arg0: dummy argument (makes delegate invocation faster)\n // arg1: (boxed) object instance\n // arg2: values array\n // arg3: objectIndices array\n // arg4: objectGraph array\n\n // local 0: object index\n ilg.DeclareLocal(typeof(int));\n\n ilg.Emit(OpCodes.Ldarg_1);\n if (!isValueType)\n ilg.Emit(OpCodes.Castclass, type);\n else\n ilg.Emit(OpCodes.Unbox, type); // returns a controlled-mutability pointer\n // which we can't store in a local...\n for (int i = 0; i < fields.Length; ++i) {\n if (i + 1 != fields.Length) ilg.Emit(OpCodes.Dup); // ... so we use OpCodes.Dup to keep it around\n\n var field = fields[i];\n\n // is field value an object in the object graph array?\n ilg.Emit(OpCodes.Ldarg_3);\n ilg.Emit(OpCodes.Ldc_I4, i);\n ilg.Emit(OpCodes.Ldelem, typeof(int));\n ilg.Emit(OpCodes.Stloc_0);\n ilg.Emit(OpCodes.Ldloc_0);\n var label1 = ilg.DefineLabel();\n ilg.Emit(OpCodes.Brtrue, label1);\n\n // load boxed value\n ilg.Emit(OpCodes.Ldarg_2);\n ilg.Emit(OpCodes.Ldc_I4, i);\n ilg.Emit(OpCodes.Ldelem, typeof(object));\n var label2 = ilg.DefineLabel();\n ilg.Emit(OpCodes.Br, label2);\n\n // load object graph array\n ilg.MarkLabel(label1);\n ilg.Emit(OpCodes.Ldarg, 4);\n ilg.Emit(OpCodes.Ldloc_0);\n ilg.Emit(OpCodes.Ldelem, typeof(object));\n\n ilg.MarkLabel(label2);\n // store value into field\n if (field.FieldType != typeof(object))\n ilg.Emit(OpCodes.Unbox_Any, field.FieldType);\n ilg.Emit(OpCodes.Stfld, field);\n }\n\n ilg.Emit(OpCodes.Ret);\n\n return (Action)dynamicMethod.CreateDelegate(typeof(Action), null);\n }\n\n internal static Action CreateISerializableConstructorCaller(ConstructorInfo constructor) {\n var type = constructor.DeclaringType;\n var dynamicMethod = new DynamicMethod(\"SerializableConstructorCaller\",\n MethodAttributes.Public | MethodAttributes.Static, CallingConventions.Standard,\n null, new Type[]{typeof(object), typeof(object), typeof(SerializationInfo), typeof(StreamingContext)},\n type, true);\n var ilg = dynamicMethod.GetILGenerator();\n var isValueType = type.IsValueType;\n ilg.Emit(OpCodes.Ldarg_1);\n if (!isValueType)\n ilg.Emit(OpCodes.Castclass, type);\n else\n ilg.Emit(OpCodes.Unbox, type);\n ilg.Emit(OpCodes.Ldarg_2);\n ilg.Emit(OpCodes.Ldarg_3);\n ilg.Emit(OpCodes.Call, constructor);\n ilg.Emit(OpCodes.Ret);\n return (Action)dynamicMethod.CreateDelegate(typeof(Action), null);\n }\n\n // The following is a non-recursive implementation of David J. Pearce's improved\n // version of Tarjan's algorithm for finding the strongly connected components of\n // a directed graph, see http://homepages.ecs.vuw.ac.nz/~djp/files/P05.pdf\n // The straighforward recursive version is obviously more elegant, but the\n // non-recursive one has the principal advantage of not ending in a stack overflow\n // for large components.\n // (We test this version against the simpler one in CloningTests.fs, of course)\n // Due to the non-recursive implementation we can also exploit that part of\n // what would otherwise be the call stack can be shared with the stack used\n // for holding elements of identified components (see the last paragraph of\n // section 2 in the referenced paper).\n\n // For optimization purposes we use a static stack, which makes\n // FindStronglyConnectedComponents and ComputeTopologicalOrder not thread-safe.\n\n private static int[] TopoIndices = new int[8];\n private static void GrowTopoIndices() {\n var newArray = new int[2*TopoIndices.Length];\n TopoIndices.CopyTo(newArray, 0);\n TopoIndices = newArray;\n }\n private static int GrowTopoIndices(int splitIndex) {\n Debug.Assert(splitIndex >= 0 && splitIndex <= TopoIndices.Length);\n int n = TopoIndices.Length;\n var newArray = new int[2*n];\n Array.Copy(TopoIndices, newArray, splitIndex);\n var newSplitIndex = 2*n;\n int d = n - splitIndex;\n if (d != 0) {\n newSplitIndex -= d;\n Array.Copy(TopoIndices, splitIndex, newArray, newSplitIndex, n - splitIndex);\n }\n TopoIndices = newArray;\n return newSplitIndex;\n }\n\n private static int[] TopoSubIndices = new int[8];\n private static void GrowTopoSubIndices() {\n var newArray = new int[2*TopoSubIndices.Length];\n TopoSubIndices.CopyTo(newArray, 0);\n TopoSubIndices = newArray;\n }\n\n /// Fills the Strongly StronglyConnectedComponent fields of the\n /// states passed in the array. Returns an array mapping each state to an\n /// integer component identifier.\n /// \n /// The object states to traverse. The object with array index\n /// 0 is ignored. All other objects are assumed to be reachable from the object\n /// with array index 1.\n internal static int[] FindStronglyConnectedComponents(State[] states) {\n Debug.Assert(states.Length > 1);\n int[] components = new int[states.Length];\n // The path stack and the component stack are both stored in TopoIndices.\n // The path stack starts at the beginning of TopoIndices, while\n // the component stack starts at the end and progresses in reverse direction.\n int pathStackCount = 0; // number of elements in the path stack\n int componentStackIndex = TopoIndices.Length; // index of element last inserted into component stack\n int counter = 1; // in the paper this variable is called \"index\"\n int reverseCounter = states.Length - 1; // in the paper this variable is called \"C\"\n bool root = true;\n int objectIndex = 1; // states[1] is state for the root object, states[0] is null\n int subIndex = 0;\n var subObjectIndices = states[objectIndex].ObjectIndices;\n components[1] = counter;\n ++counter;\n if (subObjectIndices != null) {\n for (;;) {\n while (subIndex < subObjectIndices.Length) {\n var subObjectIndex = subObjectIndices[subIndex];\n ++subIndex;\n if (subObjectIndex == 0) continue;\n var subObjectComponent = components[subObjectIndex];\n if (subObjectComponent == 0) {\n var subSubObjectIndices = states[subObjectIndex].ObjectIndices;\n if (subSubObjectIndices == null) {\n components[subObjectIndex] = reverseCounter;\n --reverseCounter;\n } else {\n subObjectIndices = subSubObjectIndices;\n components[subObjectIndex] = counter;\n ++counter;\n TopoIndices[pathStackCount] = objectIndex;\n TopoSubIndices[pathStackCount] = root ? subIndex : -subIndex;\n root = true;\n objectIndex = subObjectIndex;\n subIndex = 0;\n ++pathStackCount;\n if (pathStackCount == componentStackIndex)\n componentStackIndex = GrowTopoIndices(componentStackIndex);\n if (pathStackCount == TopoSubIndices.Length)\n GrowTopoSubIndices();\n continue;\n }\n } else if (subObjectComponent < components[objectIndex]) {\n components[objectIndex] = subObjectComponent;\n root = false;\n }\n }\n if (root) {\n if (componentStackIndex < TopoIndices.Length) {\n int component = components[objectIndex];\n if (components[TopoIndices[componentStackIndex]] >= component) {\n int next = componentStackIndex + 1;\n while (next < TopoIndices.Length && components[TopoIndices[next]] >= component) ++next;\n int d = next - componentStackIndex;\n var scc = new int[d + 1];\n for (int i = 0; i < d; ++i) {\n int idx = TopoIndices[componentStackIndex + i];\n scc[1 + i] = idx;\n states[idx].StronglyConnectedComponent = scc;\n components[idx] = reverseCounter;\n --counter;\n }\n scc[0] = objectIndex;\n states[objectIndex].StronglyConnectedComponent = scc;\n componentStackIndex = next;\n }\n }\n components[objectIndex] = reverseCounter;\n --counter;\n --reverseCounter;\n if (pathStackCount == 0) break;\n } else {\n TopoIndices[--componentStackIndex] = objectIndex;\n // we never need to grow the TopoIndices array here\n // because we immediately decrement pathStackCount next\n }\n --pathStackCount;\n int subObjectComponent_ = components[objectIndex];\n objectIndex = TopoIndices[pathStackCount];\n subIndex = TopoSubIndices[pathStackCount];\n if (subIndex > 0) {\n root = true;\n } else {\n subIndex = -subIndex;\n root = false;\n }\n subObjectIndices = states[objectIndex].ObjectIndices;\n if (subObjectComponent_ < components[objectIndex]) {\n components[objectIndex] = subObjectComponent_;\n root = false;\n }\n }\n }\n return components;\n }\n\n private static int[] SccIndexStack = new int[8];\n private static void GrowSccIndexStack() {\n var newStack = new int[2*SccIndexStack.Length];\n SccIndexStack.CopyTo(newStack, 0);\n SccIndexStack = newStack;\n }\n\n /// Returns an array with the topologically sorted indices of the states.\n /// In the returned array the indices of states belonging to the same strongly\n /// connected component are adjacent (but the order within a strongly connected\n /// component is undefined).\n /// \n /// The object states to traverse. The object with array index\n /// 0 is ignored. All other objects are assumed to be reachable from the object\n /// with array index 1.\n internal static int[] ComputeTopologicalOrder(State[] states) {\n Debug.Assert(states.Length > 1);\n\n // Fill the State.StronglyConnectedComponent fields.\n // (We don't need the returned array, so we can recycle it for our purposes.)\n int[] orderedObjectIndices = FindStronglyConnectedComponents(states);\n Array.Clear(orderedObjectIndices, 0, orderedObjectIndices.Length);\n int nextPosition = orderedObjectIndices.Length - 1;\n\n TopoIndices = new int[2];\n TopoSubIndices = new int[2];\n SccIndexStack = new int[2];\n\n // We traverse the graph non-recursively in depth-first order.\n\n int topoStackCount = 0;\n int sccIndexStackCount = 0;\n\n int[] visitedBits = new int[(checked(states.Length + 31))/32];\n\n int objectIndex;\n int[] subObjectIndices;\n {\n var state = states[1];\n if (state.StronglyConnectedComponent == null) {\n objectIndex = 1;\n subObjectIndices = state.ObjectIndices;\n if (subObjectIndices == null) {\n orderedObjectIndices[1] = 1;\n return orderedObjectIndices;\n }\n visitedBits[0] = 1 << 1;\n } else {\n foreach (var sccIndex in state.StronglyConnectedComponent)\n visitedBits[sccIndex/32] |= 1 << (sccIndex%32);\n objectIndex = state.StronglyConnectedComponent[0];\n subObjectIndices = states[objectIndex].ObjectIndices;\n SccIndexStack[0] = 1;\n sccIndexStackCount = 1;\n }\n }\n int subIndex = subObjectIndices.Length - 1;\n\n for (;;) {\n // First we iterate over the sub objects...\n Debug.Assert(subObjectIndices != null);\n\n // (The states array was constructed in breadth-first order, while we construct\n // the topological order using depth-first search. With a bit of luck we can\n // keep the resulting orderedObjectIndices close to a simple increasing sequence\n // by iterating over the sub-objects in the depth-first search in reverse order.)\n while (subIndex >= 0) {\n var subObjectIndex = subObjectIndices[subIndex];\n --subIndex;\n if (subObjectIndex == 0) continue;\n int w = subObjectIndex/32, b = subObjectIndex%32;\n if (((visitedBits[w] >> b) & 1) == 0) {\n var subState = states[subObjectIndex];\n var subSubObjectIndices = subState.ObjectIndices;\n if (subState.StronglyConnectedComponent == null) {\n visitedBits[w] |= 1 << b;\n if (subSubObjectIndices == null) {\n orderedObjectIndices[nextPosition] = subObjectIndex;\n --nextPosition;\n continue;\n }\n subObjectIndices = subSubObjectIndices;\n } else {\n foreach (var sccIndex in subState.StronglyConnectedComponent)\n visitedBits[sccIndex/32] |= 1 << (sccIndex%32);\n subObjectIndex = subState.StronglyConnectedComponent[0];\n subObjectIndices = states[subObjectIndex].ObjectIndices;\n SccIndexStack[sccIndexStackCount] = 1;\n if (++sccIndexStackCount == SccIndexStack.Length) GrowSccIndexStack();\n }\n TopoIndices[topoStackCount] = objectIndex;\n TopoSubIndices[topoStackCount] = subIndex;\n ++topoStackCount;\n if (topoStackCount == TopoIndices.Length) GrowTopoIndices();\n if (topoStackCount == TopoSubIndices.Length) GrowTopoSubIndices();\n objectIndex = subObjectIndex;\n subIndex = subObjectIndices.Length - 1;\n continue;\n }\n }\n\n // ... then we iterate over other object in the same strongly connected component.\n var scc = states[objectIndex].StronglyConnectedComponent;\n if (scc == null) {\n orderedObjectIndices[nextPosition] = objectIndex;\n --nextPosition;\n } else {\n Debug.Assert(sccIndexStackCount > 0);\n int sccIndex = SccIndexStack[sccIndexStackCount - 1];\n if (sccIndex < scc.Length) {\n objectIndex = scc[sccIndex];\n subObjectIndices = states[objectIndex].ObjectIndices;\n subIndex = subObjectIndices.Length - 1;\n SccIndexStack[sccIndexStackCount - 1] = ++sccIndex;\n continue;\n }\n --sccIndexStackCount;\n for (int i = scc.Length - 1; i >= 0; --i) {\n sccIndex = scc[i];\n orderedObjectIndices[nextPosition] = sccIndex;\n --nextPosition;\n }\n }\n if (topoStackCount == 0) break;\n --topoStackCount;\n objectIndex = TopoIndices[topoStackCount];\n subIndex = TopoSubIndices[topoStackCount];\n subObjectIndices = states[objectIndex].ObjectIndices;\n }\n return orderedObjectIndices;\n }\n}\n\n[Flags]\ninternal enum CloneEvents {\n None = 0,\n OnSerializing = 1,\n OnSerialized = 2,\n OnDeserializing = 4,\n OnDeserialized = 8,\n ISerializable = 16,\n IDeserializationCallback = 32,\n IObjectReference = 64\n}\n\ninternal sealed class CloneEventHandlers {\n public readonly CloneEvents Events;\n\n private delegate void Handler(object instance, StreamingContext context);\n\n private readonly Handler OnSerializingHandler;\n private readonly Handler OnSerializedHandler;\n private readonly Handler OnDeserializingHandler;\n private readonly Handler OnDeserializedHandler;\n\n private CloneEventHandlers(CloneEvents events,\n Handler onSerializingHandler,\n Handler onSerializedHandler,\n Handler onDeserializingHandler,\n Handler onDeserializedHandler)\n {\n Events = events;\n OnSerializingHandler = onSerializingHandler;\n OnSerializedHandler = onSerializedHandler;\n OnDeserializingHandler = onDeserializingHandler;\n OnDeserializedHandler = onDeserializedHandler;\n }\n\n public void InvokeOnSerializing(object instance, StreamingContext context) {\n OnSerializingHandler.Invoke(instance, context);\n }\n\n public void InvokeOnSerialized(object instance, StreamingContext context) {\n OnSerializedHandler.Invoke(instance, context);\n }\n\n public void InvokeOnDeserializing(object instance, StreamingContext context) {\n OnDeserializingHandler.Invoke(instance, context);\n }\n\n public void InvokeOnDeserialized(object instance, StreamingContext context) {\n OnDeserializedHandler.Invoke(instance, context);\n }\n\n private static readonly CloneEventHandlers ISerializableOnly = new CloneEventHandlers(CloneEvents.ISerializable, null, null, null, null);\n private static readonly CloneEventHandlers ISerializableAndObjectReferenceOnly = new CloneEventHandlers(CloneEvents.ISerializable | CloneEvents.IObjectReference, null, null, null, null);\n\n private static Handler WithBoxedArgument(Action handler) {\n return (object obj, StreamingContext context) => handler((T)obj, context);\n }\n private static readonly MethodInfo WithBoxedArgumentMethodInfo = typeof(CloneEventHandlers).GetMethod(\"WithBoxedArgument\", BindingFlags.Static | BindingFlags.NonPublic);\n\n private static Handler CreateHandler(Type type, MethodInfo mi) {\n var delegateType = typeof(Action<,>).MakeGenericType(type, typeof(StreamingContext));\n var d = Delegate.CreateDelegate(delegateType, null, mi);\n return (Handler)WithBoxedArgumentMethodInfo.MakeGenericMethod(type).Invoke(null, new object[]{d});\n }\n\n private static readonly Type typeofObject = typeof(object);\n private static readonly Type typeofISerializable = typeof(ISerializable);\n private static readonly Type typeofIObjectReference = typeof(IObjectReference);\n private static readonly Type typeofIDeserializationCallback = typeof(IDeserializationCallback);\n private static readonly Type typeofOnSerializingAttribute = typeof(OnSerializingAttribute);\n private static readonly Type typeofOnSerializedAttribute = typeof(OnSerializedAttribute);\n private static readonly Type typeofOnDeserializingAttribute = typeof(OnDeserializingAttribute);\n private static readonly Type typeofOnDeserializedAttribute = typeof(OnDeserializedAttribute);\n\n public static CloneEventHandlers Create(Type type) {\n Debug.Assert(type != null);\n if (type == typeofObject) return null;\n var events = CloneEvents.None;\n if (typeofISerializable.IsAssignableFrom(type)) events |= CloneEvents.ISerializable;\n if (typeofIObjectReference.IsAssignableFrom(type)) events |= CloneEvents.IObjectReference;\n if (typeofIDeserializationCallback.IsAssignableFrom(type)) events |= CloneEvents.IDeserializationCallback;\n var bt = type;\n for (;;) {\n var methods = bt.GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly);\n for (int i = 0; i < methods.Length; ++i) {\n var mi = methods[i];\n if ( mi.IsDefined(typeofOnSerializingAttribute, false)\n || mi.IsDefined(typeofOnSerializedAttribute, false)\n || mi.IsDefined(typeofOnDeserializingAttribute, false)\n || mi.IsDefined(typeofOnDeserializedAttribute, false)) return CreateContinue(type, events, bt, methods, i);\n }\n bt = bt.BaseType;\n if (bt == null || bt == typeofObject) break;\n if (!bt.IsSerializable) throw new SerializationException(Cloner.BaseTypeNotSerializableMessage(bt, type));\n }\n if (events == 0) return null;\n if (events == CloneEvents.ISerializable) return ISerializableOnly;\n if (events == (CloneEvents.ISerializable | CloneEvents.IObjectReference)) return ISerializableAndObjectReferenceOnly;\n return new CloneEventHandlers(events, null, null, null, null);\n }\n private static CloneEventHandlers CreateContinue(Type type, CloneEvents events, Type baseType, MethodInfo[] methods, int i) {\n Delegate onSerializingHandler = null, onSerializedHandlers = null, onDeserializingHandlers = null, onDeserializedHandlers = null;\n var bt = baseType;\n for (;;) {\n for (; i < methods.Length; ++i) {\n var mi = methods[i];\n if (mi.IsDefined(typeofOnSerializingAttribute, false)) {\n var d = CreateHandler(bt, mi);\n onSerializingHandler = onSerializingHandler == null ? d : Delegate.Combine(d, onSerializingHandler); // call base handler first\n }\n if (mi.IsDefined(typeofOnSerializedAttribute, false)) {\n var d = CreateHandler(bt, mi);\n onSerializedHandlers = onSerializedHandlers == null ? d : Delegate.Combine(d, onSerializedHandlers);\n }\n if (mi.IsDefined(typeofOnDeserializingAttribute, false)) {\n var d = CreateHandler(bt, mi);\n onDeserializingHandlers = onDeserializingHandlers == null ? d : Delegate.Combine(d, onDeserializingHandlers);\n }\n if (mi.IsDefined(typeofOnDeserializedAttribute, false)) {\n var d = CreateHandler(bt, mi);\n onDeserializedHandlers = onDeserializedHandlers == null ? d : Delegate.Combine(d, onDeserializedHandlers);\n }\n }\n bt = bt.BaseType;\n if (bt == null || bt == typeofObject) break;\n if (!bt.IsSerializable) throw new SerializationException(Cloner.BaseTypeNotSerializableMessage(bt, type));\n methods = bt.GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly);\n i = 0;\n }\n Handler onSerializing = null, onSerialized = null, onDeserializing = null, onDeserialized = null;\n if (onSerializingHandler != null) {\n events |= CloneEvents.OnSerializing;\n onSerializing = (Handler)onSerializingHandler;\n }\n if (onSerializedHandlers != null) {\n events |= CloneEvents.OnSerialized;\n onSerialized = (Handler)onSerializedHandlers;\n }\n if (onDeserializingHandlers != null) {\n events |= CloneEvents.OnDeserializing;\n onDeserializing = (Handler)onDeserializingHandlers;\n }\n if (onDeserializedHandlers != null) {\n events |= CloneEvents.OnDeserialized;\n onDeserialized = (Handler)onDeserializedHandlers;\n }\n return new CloneEventHandlers(events, onSerializing, onSerialized, onDeserializing, onDeserialized);\n }\n}\n\n}\n\n#endif" }, { "path": "FParsecCS/ErrorMessage.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Diagnostics;\nusing System.Collections.Generic;\n\nusing Microsoft.FSharp.Core;\n\nnamespace FParsec {\n\npublic enum ErrorMessageType {\n Expected,\n ExpectedString,\n ExpectedCaseInsensitiveString,\n Unexpected,\n UnexpectedString,\n UnexpectedCaseInsensitiveString,\n Message,\n NestedError,\n CompoundError,\n Other\n}\n\n[DebuggerDisplay(\"{GetDebuggerDisplay(),nq}\")]\npublic class ErrorMessage : IEquatable {\n public readonly ErrorMessageType Type;\n\n [DebuggerBrowsable(DebuggerBrowsableState.Never)]\n internal string String;\n\n internal ErrorMessage(ErrorMessageType messageType) {\n Type = messageType;\n }\n\n public class Expected : ErrorMessage {\n public string Label { get { return String; } }\n public Expected(string labelForExpectedInput) : base(ErrorMessageType.Expected) {\n String = labelForExpectedInput;\n }\n }\n\n public class ExpectedString : ErrorMessage {\n public new string String { get { return base.String; } }\n public ExpectedString(string expectedString) : base(ErrorMessageType.ExpectedString) {\n base.String = expectedString;\n }\n }\n\n public class ExpectedCaseInsensitiveString : ErrorMessage {\n public string CaseInsensitiveString { get { return String; } }\n public ExpectedCaseInsensitiveString(string expectedCaseInsensitiveString) : base(ErrorMessageType.ExpectedCaseInsensitiveString) {\n String = expectedCaseInsensitiveString;\n }\n }\n\n public class Unexpected : ErrorMessage {\n public string Label { get { return String; } }\n public Unexpected(string labelForUnexpectedInput) : base(ErrorMessageType.Unexpected) {\n String = labelForUnexpectedInput;\n }\n }\n\n public class UnexpectedString : ErrorMessage {\n public new string String { get { return base.String; } }\n public UnexpectedString(string unexpectedString) : base(ErrorMessageType.UnexpectedString) {\n base.String = unexpectedString;\n }\n }\n\n public class UnexpectedCaseInsensitiveString : ErrorMessage {\n public string CaseInsensitiveString { get { return String; } }\n public UnexpectedCaseInsensitiveString(string unexpectedCaseInsensitiveString) : base(ErrorMessageType.UnexpectedCaseInsensitiveString) {\n String = unexpectedCaseInsensitiveString;\n }\n }\n\n public class Message : ErrorMessage {\n public new string String { get { return base.String; } }\n public Message(string message) : base(ErrorMessageType.Message) {\n base.String = message;\n }\n }\n\n public class NestedError : ErrorMessage {\n public Position Position { get; private set; }\n public object UserState { get; private set; }\n public ErrorMessageList Messages { get; private set; }\n\n public NestedError(Position position, object userState, ErrorMessageList messages) : base(ErrorMessageType.NestedError) {\n Position = position;\n UserState = userState;\n Messages = messages;\n }\n }\n\n public class CompoundError : ErrorMessage {\n public string LabelOfCompound { get { return String; } }\n\n public Position NestedErrorPosition { get; private set; }\n public object NestedErrorUserState { get; private set; }\n public ErrorMessageList NestedErrorMessages { get; private set; }\n\n public CompoundError(string labelOfCompound,\n Position nestedErrorPosition,\n object nestedErrorUserState,\n ErrorMessageList nestedErrorMessages) : base(ErrorMessageType.CompoundError)\n {\n String = labelOfCompound;\n NestedErrorPosition = nestedErrorPosition;\n NestedErrorUserState = nestedErrorUserState;\n NestedErrorMessages = nestedErrorMessages;\n }\n }\n\n public class Other : ErrorMessage {\n public object Data { get; private set; }\n public Other(object data) : base(ErrorMessageType.Other) {\n Data = data;\n }\n }\n\n public override bool Equals(object obj) { return Equals(obj as ErrorMessage); }\n\n public bool Equals(ErrorMessage other) {\n return (object)this == (object)other\n || ( (object)other != null\n && Type == other.Type\n && (Type > ErrorMessageType.Message\n ? EqualsHelper(other)\n : String == other.String));\n }\n\n public static bool operator==(ErrorMessage left, ErrorMessage right) {\n return (object)left == (object)right\n || ( (object)left != null\n && (object)right != null\n && left.Type == right.Type\n && (left.Type > ErrorMessageType.Message\n ? left.EqualsHelper(right)\n : left.String == right.String));\n }\n public static bool operator!=(ErrorMessage left, ErrorMessage right) { return !(left == right); }\n\n private bool EqualsHelper(ErrorMessage other) {\n Debug.Assert(Type == other.Type\n && Type > ErrorMessageType.Message);\n if (Type == ErrorMessageType.NestedError) {\n var ne1 = (NestedError)this;\n var ne2 = (NestedError)other;\n return ne1.Position == ne2.Position\n && ne1.Messages == ne2.Messages\n && LanguagePrimitives.GenericEqualityERComparer.Equals(ne1.UserState, ne2.UserState);\n } else if (Type == ErrorMessageType.CompoundError) {\n if (String != other.String) return false;\n var ce1 = (CompoundError)this;\n var ce2 = (CompoundError)other;\n return ce1.NestedErrorPosition == ce2.NestedErrorPosition\n && ce1.NestedErrorMessages == ce2.NestedErrorMessages\n && LanguagePrimitives.GenericEqualityERComparer.Equals(ce1.NestedErrorUserState, ce2.NestedErrorUserState);\n } else { // ErrorMessageType == ErrorMessageType.Other\n Debug.Assert(Type == ErrorMessageType.Other);\n return ((Other)this).Data == ((Other)other).Data;\n }\n }\n\n public override int GetHashCode() {\n return (int)Type ^ (String == null ? 0 : String.GetHashCode());\n }\n\n private class ErrorMessageComparer : Comparer {\n public override int Compare(ErrorMessage x, ErrorMessage y) {\n if (x == null || y == null) {\n return x == null && y == null ? 0 : (x == null ? -1 : 1);\n }\n int d = (int)x.Type - (int)y.Type;\n if (d != 0) return d;\n var type = x.Type;\n if (type <= ErrorMessageType.Message) {\n Debug.Assert(type >= 0);\n return String.CompareOrdinal(x.String, y.String);\n } else if (type == ErrorMessageType.NestedError) {\n var ne1 = (NestedError)x;\n var ne2 = (NestedError)y;\n var c = Position.Compare(ne1.Position, ne2.Position);\n if (c != 0) return c;\n var msgs1 = ErrorMessageList.ToSortedArray(ne1.Messages);\n var msgs2 = ErrorMessageList.ToSortedArray(ne2.Messages);\n int n = Math.Min(msgs1.Length, msgs2.Length);\n for (int i = 0; i < n; ++i) {\n c = Compare(msgs1[i], msgs2[i]);\n if (c != 0) return c;\n }\n return msgs1.Length - msgs2.Length;\n } else if (type == ErrorMessageType.CompoundError) {\n var c = String.CompareOrdinal(x.String, y.String);\n if (c != 0) return c;\n var ce1 = (CompoundError)x;\n var ce2 = (CompoundError)y;\n c = Position.Compare(ce1.NestedErrorPosition, ce2.NestedErrorPosition);\n if (c != 0) return c;\n var msgs1 = ErrorMessageList.ToSortedArray(ce1.NestedErrorMessages);\n var msgs2 = ErrorMessageList.ToSortedArray(ce2.NestedErrorMessages);\n int n = Math.Min(msgs1.Length, msgs2.Length);\n for (int i = 0; i < n; ++i) {\n c = Compare(msgs1[i], msgs2[i]);\n if (c != 0) return c;\n }\n return msgs1.Length - msgs2.Length;\n } else {\n Debug.Assert(type == ErrorMessageType.Other);\n return 0;\n }\n }\n\t}\n\n internal static Comparer Comparer = new ErrorMessageComparer();\n internal static ErrorMessage[] EmptyArray = new ErrorMessage[0];\n\n internal string GetDebuggerDisplay() {\n switch (Type) {\n case ErrorMessageType.Expected:\n return String == null\n ? \"Expected(null)\"\n : Text.DoubleQuote(\"Expected(\", String, \")\");\n case ErrorMessageType.ExpectedString:\n return String == null\n ? \"ExpectedString(null)\"\n : Text.DoubleQuote(\"ExpectedString(\", String, \")\");\n case ErrorMessageType.ExpectedCaseInsensitiveString:\n return String == null\n ? \"ExpectedCaseInsensitiveString(null)\"\n : Text.DoubleQuote(\"ExpectedCaseInsensitiveString(\", String, \")\");\n case ErrorMessageType.Unexpected:\n return String == null\n ? \"Unexpected(null)\"\n : Text.DoubleQuote(\"Unexpected(\", String, \")\");\n case ErrorMessageType.UnexpectedString:\n return String == null\n ? \"UnexpectedString(null)\"\n : Text.DoubleQuote(\"UnexpectedString(\", String, \")\");\n case ErrorMessageType.UnexpectedCaseInsensitiveString:\n return String == null\n ? \"UnexpectedCaseInsensitiveString(null)\"\n : Text.DoubleQuote(\"UnexpectedCaseInsensitiveString(\", String, \")\");\n case ErrorMessageType.Message:\n return String == null\n ? \"Message(null)\"\n : Text.DoubleQuote(\"Message(\", String, \")\");\n case ErrorMessageType.NestedError: {\n var ne = (NestedError)this;\n var pos = ne.Position == null ? \"null\" : ne.Position.ToString();\n var msgs = ErrorMessageList.GetDebuggerDisplay(ne.Messages);\n return \"NestedError(\" + pos + \", ..., \" + msgs + \")\";\n }\n case ErrorMessageType.CompoundError: {\n var ce = (CompoundError)this;\n var label = ce.String == null ? \"null\" : Text.Escape(ce.String, \"\", \"\\\"\", \"\\\"\", \"\", '\"');\n var pos = ce.NestedErrorPosition == null ? \"\" : ce.NestedErrorPosition.ToString();\n var msgs = ErrorMessageList.GetDebuggerDisplay(ce.NestedErrorMessages);\n return \"CompoundError(\" + label + \", \" + pos + \", ..., \" + msgs + \")\";\n }\n case ErrorMessageType.Other: {\n var oe = (Other)this;\n return oe.Data == null ? \"Other(null)\" : \"Other(\" + oe.ToString() + \")\";\n }\n default:\n throw new InvalidOperationException();\n }\n }\n}\n\n\n}" }, { "path": "FParsecCS/ErrorMessageList.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Diagnostics;\nusing System.Collections.Generic;\n\nnamespace FParsec {\n\n[DebuggerDisplay(\"{ErrorMessageList.GetDebuggerDisplay(this),nq}\"),\n DebuggerTypeProxy(typeof(ErrorMessageList.DebugView))]\npublic sealed class ErrorMessageList : IEquatable {\n public readonly ErrorMessage Head;\n public readonly ErrorMessageList Tail;\n\n public ErrorMessageList(ErrorMessage head, ErrorMessageList tail) {\n var throwNullReferenceExceptionIfHeadIsNull = head.Type;\n Head = head;\n Tail = tail;\n }\n\n public ErrorMessageList(ErrorMessage message) {\n var throwNullReferenceExceptionIfMessageIsNull = message.Type;\n Head = message;\n }\n\n public ErrorMessageList(ErrorMessage message1, ErrorMessage message2) {\n var throwNullReferenceExceptionIfMessage1IsNull = message1.Type;\n Head = message1;\n Tail = new ErrorMessageList(message2);\n }\n\n public static ErrorMessageList Merge(ErrorMessageList list1, ErrorMessageList list2) {\n if ((object)list1 == null) return list2;\n return MergeContinue(list1, list2);\n }\n private static ErrorMessageList MergeContinue(ErrorMessageList list1, ErrorMessageList list2) {\n while ((object)list2 != null) {\n list1 = new ErrorMessageList(list2.Head, list1);\n list2 = list2.Tail;\n }\n return list1;\n }\n\n public static HashSet ToHashSet(ErrorMessageList messages) {\n var msgs = messages;\n var set = new HashSet();\n for (; (object)msgs != null; msgs = msgs.Tail) {\n var msg = msgs.Head;\n Debug.Assert(msg.Type >= 0);\n if (msg.Type <= ErrorMessageType.Message && string.IsNullOrEmpty(msg.String)) continue;\n set.Add(msg);\n }\n return set;\n }\n\n public static ErrorMessage[] ToSortedArray(ErrorMessageList messages) {\n var set = ToHashSet(messages);\n var array = new ErrorMessage[set.Count];\n set.CopyTo(array);\n Array.Sort(array, ErrorMessage.Comparer);\n return array;\n }\n\n public override bool Equals(object obj) { return Equals(obj as ErrorMessageList); }\n\n public bool Equals(ErrorMessageList other) {\n return (object)this == (object)other\n || ( (object)other != null\n && ToHashSet(this).SetEquals(ToHashSet(other)));\n }\n\n public static bool operator==(ErrorMessageList left, ErrorMessageList right) {\n return (object)left == (object)right\n || ( (object)left != null\n && (object)right != null\n && ToHashSet(left).SetEquals(ToHashSet(right)));\n }\n public static bool operator!=(ErrorMessageList left, ErrorMessageList right) { return !(left == right); }\n\n public override int GetHashCode() {\n var set = ToHashSet(this);\n var h = 0;\n foreach (var msg in set)\n h ^= msg.GetHashCode();\n return h;\n }\n\n internal static string GetDebuggerDisplay(ErrorMessageList list) {\n var es = ErrorMessageList.ToSortedArray(list);\n switch (es.Length) {\n case 0: return \"[]\";\n case 1: return \"[\" + es[0].GetDebuggerDisplay() + \"]\";\n case 2: return \"[\" + es[0].GetDebuggerDisplay() + \"; \" + es[1].GetDebuggerDisplay() + \"]\";\n case 3: return \"[\" + es[0].GetDebuggerDisplay() + \"; \" + es[1].GetDebuggerDisplay() + \"; \" + es[2].GetDebuggerDisplay() + \"]\";\n default: return \"[\" + es[0].GetDebuggerDisplay() + \"; \" + es[1].GetDebuggerDisplay() + \"; \" + es[2].GetDebuggerDisplay() + \"; ...]\";\n }\n }\n\n internal class DebugView {\n //[DebuggerBrowsable(DebuggerBrowsableState.Never)]\n private ErrorMessageList List;\n\n public DebugView(ErrorMessageList list) { List = list; }\n\n [DebuggerBrowsable(DebuggerBrowsableState.RootHidden)]\n public ErrorMessage[] Items { get { return ErrorMessageList.ToSortedArray(List); } }\n }\n}\n\n}" }, { "path": "FParsecCS/Errors.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nnamespace FParsec {\n\ninternal static class Errors {\n static private ErrorMessageList Expected(string str) {\n return new ErrorMessageList(new ErrorMessage.Expected(str));\n }\n\n static private ErrorMessageList Unexpected(string str) {\n return new ErrorMessageList(new ErrorMessage.Unexpected(str));\n }\n\n static private ErrorMessageList Message(string str) {\n return new ErrorMessageList(new ErrorMessage.Message(str));\n }\n\n public static readonly ErrorMessageList ExpectedEndOfInput = Expected(Strings.EndOfInput);\n public static readonly ErrorMessageList UnexpectedEndOfInput = Unexpected(Strings.EndOfInput);\n\n public static readonly ErrorMessageList ExpectedAnyChar = Expected(Strings.AnyChar);\n public static readonly ErrorMessageList ExpectedWhitespace = Expected(Strings.Whitespace);\n public static readonly ErrorMessageList ExpectedAsciiUppercaseLetter = Expected(Strings.AsciiUppercaseLetter);\n public static readonly ErrorMessageList ExpectedAsciiLowercaseLetter = Expected(Strings.AsciiLowercaseLetter);\n public static readonly ErrorMessageList ExpectedAsciiLetter = Expected(Strings.AsciiLetter);\n public static readonly ErrorMessageList ExpectedUppercaseLetter = Expected(Strings.UppercaseLetter);\n public static readonly ErrorMessageList ExpectedLowercaseLetter = Expected(Strings.LowercaseLetter);\n public static readonly ErrorMessageList ExpectedLetter = Expected(Strings.Letter);\n public static readonly ErrorMessageList ExpectedBinaryDigit = Expected(Strings.BinaryDigit);\n public static readonly ErrorMessageList ExpectedOctalDigit = Expected(Strings.OctalDigit);\n public static readonly ErrorMessageList ExpectedDecimalDigit = Expected(Strings.DecimalDigit);\n public static readonly ErrorMessageList ExpectedHexadecimalDigit = Expected(Strings.HexadecimalDigit);\n\n public static readonly ErrorMessageList ExpectedNewline = Expected(Strings.Newline);\n public static readonly ErrorMessageList UnexpectedNewline = Unexpected(Strings.Newline);\n\n public static readonly ErrorMessageList ExpectedTab = Expected(Strings.Tab);\n\n public static readonly ErrorMessageList ExpectedFloatingPointNumber = Expected(Strings.FloatingPointNumber);\n\n public static readonly ErrorMessageList ExpectedInt64 = Expected(Strings.Int64);\n public static readonly ErrorMessageList ExpectedInt32 = Expected(Strings.Int32);\n public static readonly ErrorMessageList ExpectedInt16 = Expected(Strings.Int16);\n public static readonly ErrorMessageList ExpectedInt8 = Expected(Strings.Int8);\n public static readonly ErrorMessageList ExpectedUInt64 = Expected(Strings.UInt64);\n public static readonly ErrorMessageList ExpectedUInt32 = Expected(Strings.UInt32);\n public static readonly ErrorMessageList ExpectedUInt16 = Expected(Strings.UInt16);\n public static readonly ErrorMessageList ExpectedUInt8 = Expected(Strings.UInt8);\n\n public static readonly ErrorMessageList ExpectedPrefixOperator = Expected(Strings.PrefixOperator);\n public static readonly ErrorMessageList ExpectedInfixOperator = Expected(Strings.InfixOperator);\n public static readonly ErrorMessageList ExpectedPostfixOperator = Expected(Strings.PostfixOperator);\n public static readonly ErrorMessageList ExpectedInfixOrPostfixOperator = ErrorMessageList.Merge(ExpectedInfixOperator, ExpectedPostfixOperator);\n\n public static readonly ErrorMessageList NumberOutsideOfDoubleRange = Message(Strings.NumberOutsideOfDoubleRange);\n public static readonly ErrorMessageList NumberOutsideOfInt64Range = Message(Strings.NumberOutsideOfInt64Range);\n public static readonly ErrorMessageList NumberOutsideOfInt32Range = Message(Strings.NumberOutsideOfInt32Range);\n public static readonly ErrorMessageList NumberOutsideOfInt16Range = Message(Strings.NumberOutsideOfInt16Range);\n public static readonly ErrorMessageList NumberOutsideOfInt8Range = Message(Strings.NumberOutsideOfInt8Range);\n public static readonly ErrorMessageList NumberOutsideOfUInt64Range = Message(Strings.NumberOutsideOfUInt64Range);\n public static readonly ErrorMessageList NumberOutsideOfUInt32Range = Message(Strings.NumberOutsideOfUInt32Range);\n public static readonly ErrorMessageList NumberOutsideOfUInt16Range = Message(Strings.NumberOutsideOfUInt16Range);\n public static readonly ErrorMessageList NumberOutsideOfUInt8Range = Message(Strings.NumberOutsideOfUInt8Range);\n\n\n public static ErrorMessageList ExpectedAnyCharIn(string chars) {\n return Expected(Strings.AnyCharIn(chars));\n }\n\n public static ErrorMessageList ExpectedAnyCharNotIn(string chars) {\n return Expected(Strings.AnyCharNotIn(chars));\n }\n\n public static ErrorMessageList ExpectedStringMatchingRegex(string regexPattern) {\n return Expected(Strings.StringMatchingRegex(regexPattern));\n }\n\n public static ErrorMessageList ExpectedAnySequenceOfNChars(int n) {\n return Expected(Strings.ExpectedAnySequenceOfNChars(n));\n }\n\n public static ErrorMessageList CouldNotFindString(string str) {\n return Message(Strings.CouldNotFindString(str));\n }\n\n public static ErrorMessageList CouldNotFindCaseInsensitiveString(string str) {\n return Message(Strings.CouldNotFindCaseInsensitiveString(str));\n }\n\n public static ErrorMessageList OperatorsConflict(Position position1, Operator operator1,\n Position position2, Operator operator2)\n {\n return Message(Strings.OperatorsConflict(position1, operator1, position2, operator2));\n }\n\n public static ErrorMessageList UnexpectedNonPrefixOperator(Operator op) {\n return new ErrorMessageList(\n ExpectedPrefixOperator.Head,\n new ErrorMessage.Unexpected(Strings.OperatorToString(op)));\n }\n\n public static ErrorMessageList MissingTernary2ndString(Position position1, Position position2, Operator op) {\n return new ErrorMessageList(\n new ErrorMessage.ExpectedString(op.TernaryRightString),\n new ErrorMessage.Message(Strings.OperatorStringIsRightPartOfTernaryOperator(position1, position2, op)));\n }\n}\n\nnamespace Internal { // the internal namespace contains internal types that must be public for inlining reasons\n public static class ParserCombinatorInInfiniteLoopHelper {\n public static Exception CreateException(string combinatorName, CharStream stream) {\n return new InvalidOperationException(stream.Position.ToString() + \": The combinator '\" + combinatorName + \"' was applied to a parser that succeeds without consuming input and without changing the parser state in any other way. (If no exception had been raised, the combinator likely would have entered an infinite loop.)\");\n }\n }\n}\n\n}\n" }, { "path": "FParsecCS/FParsecCS-LowTrust.csproj", "content": "\n\n \n netstandard2.0;netstandard2.1\n \n\n \n\n\n" }, { "path": "FParsecCS/FParsecCS.csproj", "content": "\n\n \n net6.0\n \n\n \n \n\n" }, { "path": "FParsecCS/FParsecCS.targets", "content": "\n\n \n FParsecCS\n FParsec\n true \n 1591\n false\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "FParsecCS/FastGenericEqualityERComparer.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Collections;\nusing System.Collections.Generic;\n\nusing Microsoft.FSharp.Core;\n\nnamespace FParsec {\n\ninternal static class FastGenericEqualityERComparer {\n // if T is a reference type, accessing the field requires a hash table lookup\n public static readonly EqualityComparer Instance = FastGenericEqualityERComparer.Create();\n\n /// For reference types it's faster to call Instance.Equals directly\n /// (due to limitations of the inliner of the .NET JIT.)\n public static bool Equals(T left, T right) {\n return Instance.Equals(left, right);\n }\n}\n\ninternal static class FastGenericEqualityERComparer {\n public static EqualityComparer Create() {\n if (typeof(T).IsArray) return new ArrayStructuralEqualityERComparer();\n if (typeof(IStructuralEquatable).IsAssignableFrom(typeof(T))) {\n var gct = typeof(T).IsValueType ? typeof(StructStructuralEqualityERComparer<>)\n : typeof(ClassStructuralEqualityERComparer<>);\n var ct = gct.MakeGenericType(typeof(T));\n #if LOW_TRUST || NETSTANDARD1_6\n return (EqualityComparer)Activator.CreateInstance(ct);\n #else\n return (EqualityComparer)System.Runtime.Serialization.FormatterServices.GetUninitializedObject(ct);\n #endif\n }\n return EqualityComparer.Default;\n }\n\n private class ClassStructuralEqualityERComparer : EqualityComparer where T : class, IStructuralEquatable {\n public override bool Equals(T x, T y) {\n return (object)x == (object)y\n || ((object)x != null && x.Equals(y, LanguagePrimitives.GenericEqualityERComparer));\n }\n\n public override int GetHashCode(T obj) {\n if ((object)obj == null) throw new ArgumentNullException(\"obj\");\n return obj.GetHashCode(LanguagePrimitives.GenericEqualityERComparer);\n }\n }\n\n private class StructStructuralEqualityERComparer : EqualityComparer where T : struct, IStructuralEquatable {\n public override bool Equals(T x, T y) {\n return x.Equals(y, LanguagePrimitives.GenericEqualityERComparer);\n }\n\n public override int GetHashCode(T obj) {\n return obj.GetHashCode(LanguagePrimitives.GenericEqualityERComparer);\n }\n }\n\n /// Forwards all work to F#'s GenericEqualityERComparer.\n private class ArrayStructuralEqualityERComparer : EqualityComparer {\n public override bool Equals(T x, T y) {\n return (object)x == (object)y || LanguagePrimitives.GenericEqualityERComparer.Equals(x, y);\n }\n\n public override int GetHashCode(T obj) {\n if ((object)obj == null) throw new ArgumentNullException(\"obj\");\n return LanguagePrimitives.GenericEqualityERComparer.GetHashCode(obj);\n }\n }\n}\n\n}" }, { "path": "FParsecCS/HexFloat.cs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2013\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nnamespace FParsec {\n\npublic static class HexFloat {\n\n// see http://www.quanttec.com/fparsec/reference/charparsers.html#members.floatToHexString\n// for more information on the supported hexadecimal floating-point format\n\n#pragma warning disable 0429 // unreachable expression code\n#pragma warning disable 0162 // unreachable code\n\n// The non-LOW_TRUST code in this class relies on the endianness of floating-point\n// numbers in memory being the same as the normal platform endianness,\n// i.e. on *((uint*)(&s)) and *((ulong*)(&d)) returning the correct IEEE-754 bit\n// representation of the single and double precision numbers s and d.\n// I'm not aware of any .NET/Mono platform where this is not the case.\n// In the unlikely event anyone ever runs this code on a platform where\n// this is not the case the unit tests will detect the problem.\n\n private static ReadOnlySpan AsciiHexValuePlus1s => new byte[] {\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0, 0, 0, 0, 0, 0,\n 0, 11, 12, 13, 14, 15, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 11, 12, 13, 14, 15, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0\n };\n\npublic static string DoubleToHexString(double x) {\n const int expBits = 11; // bits for biased exponent\n const int maxBits = 53; // significant bits (including implicit bit)\n const int maxChars = 24; // \"-0x1.fffffffffffffp-1022\"\n const int maxBiasedExp = (1 << expBits) - 1;\n const int maxExp = 1 << (expBits - 1); // max n for which 0.5*2^n is a double\n const int bias = maxExp - 1;\n\n const int maxFractNibbles = (maxBits - 1 + 3)/4;\n const ulong mask = (1UL << (maxBits - 1)) - 1; // mask for lower (maxBits - 1) bits\n\n ulong xn = unchecked((ulong)BitConverter.DoubleToInt64Bits(x));\n int sign = (int)(xn >> (maxBits - 1 + expBits));\n int e = (int)((xn >> (maxBits - 1)) & maxBiasedExp); // the biased exponent\n ulong s = xn & mask; // the significand (without the implicit bit)\n if (e < maxBiasedExp) {\n if (e == 0 && s == 0) return sign == 0 ? \"0x0.0p0\" : \"-0x0.0p0\";\n Span str = stackalloc char[maxChars];\n int i = 0;\n if (sign != 0) str[i++] = '-';\n str[i++] = '0'; str[i++] = 'x';\n str[i++] = e > 0 ? '1' : '0';\n str[i++] = '.';\n if ((maxBits - 1)%4 > 0) { // normalize fraction to multiple of 4 bits\n s <<= 4 - (maxBits - 1)%4;\n }\n int lastNonNull = i;\n for (int j = 0; j < maxFractNibbles; ++j) {\n int h = unchecked((int) (s >> ((maxFractNibbles - 1 - j) << 2))) & 0xf;\n if (h != 0) lastNonNull = i;\n str[i++] = \"0123456789abcdef\"[h];\n }\n i = lastNonNull + 1;\n str[i++] = 'p';\n if (e >= bias) e -= bias;\n else {\n str[i++] = '-';\n e = e > 0 ? -(e - bias) : bias - 1;\n }\n // e holds absolute unbiased exponent\n int li = e < 10 ? 1 : (e < 100 ? 2 : (e < 1000 ? 3 : 4)); // floor(log(10, e))) + 1\n i += li;\n do {\n int r = e%10; e = e/10;\n str[--i] = (char) (48 + r);\n } while (e > 0);\n i += li;\n return str.Slice(0, i).ToString();\n } else {\n if (s == 0) return sign == 0 ? \"Infinity\" : \"-Infinity\";\n else return \"NaN\";\n }\n}\n\npublic static string SingleToHexString(float x) {\n const int expBits = 8; // bits for biased exponent\n const int maxBits = 24; // significant bits (including implicit bit)\n const int maxChars = 16; // \"-0x1.fffffep-126\"\n const int maxBiasedExp = (1 << expBits) - 1;\n const int maxExp = 1 << (expBits - 1); // max n for which 0.5*2^n is a double\n const int bias = maxExp - 1;\n\n const int maxFractNibbles = (maxBits - 1 + 3)/4;\n const uint mask = (1U << (maxBits - 1)) - 1; // mask for lower (maxBits - 1) bits\n\n #if NETSTANDARD2_0\n uint xn = BitConverter.ToUInt32(BitConverter.GetBytes(x), 0);\n #else\n uint xn = (uint)BitConverter.SingleToInt32Bits(x);\n #endif\n int sign = (int)(xn >> (maxBits - 1 + expBits));\n int e = (int)((xn >> (maxBits - 1)) & maxBiasedExp); // the biased exponent\n uint s = xn & mask; // the significand (without the implicit bit)\n if (e < maxBiasedExp) {\n if (e == 0 && s == 0) return sign == 0 ? \"0x0.0p0\" : \"-0x0.0p0\";\n Span str = stackalloc char[maxChars];\n int i = 0;\n if (sign != 0) str[i++] = '-';\n str[i++] = '0'; str[i++] = 'x';\n str[i++] = e > 0 ? '1' : '0';\n str[i++] = '.';\n int lastNonNull = i;\n if ((maxBits - 1)%4 > 0) { // normalize fraction to multiple of 4 bits\n s <<= 4 - (maxBits - 1)%4;\n }\n for (int j = 0; j < maxFractNibbles; ++j) {\n int h = (int)(s >> ((maxFractNibbles - 1 - j) << 2)) & 0xf;\n if (h != 0) lastNonNull = i;\n str[i++] = \"0123456789abcdef\"[h];\n }\n i = lastNonNull + 1;\n str[i++] = 'p';\n if (e >= bias) e -= bias;\n else {\n str[i++] = '-';\n e = e > 0 ? -(e - bias) : bias - 1;\n }\n // e holds absolute unbiased exponent\n int li = e < 10 ? 1 : (e < 100 ? 2 : 3); // floor(log(10, e))) + 1\n i += li;\n do {\n int r = e%10; e = e/10;\n str[--i] = (char)(48 + r);\n } while (e > 0);\n i += li;\n return str.Slice(0, i).ToString();\n } else {\n if (s == 0) return sign == 0 ? \"Infinity\" : \"-Infinity\";\n else return \"NaN\";\n }\n}\n\n#pragma warning restore 0429\n#pragma warning restore 0162\n\n#if !LOW_TRUST\n unsafe\n#endif\npublic static double DoubleFromHexString(string str) {\n const int expBits = 11; // bits for exponent\n const int maxBits = 53; // significant bits (including implicit bit)\n\n const int maxExp = 1 << (expBits - 1); // max n for which 0.5*2^n is a double\n const int minExp = -maxExp + 3; // min n for which 0.5*2^n is a normal double\n const int minSExp = minExp - (maxBits - 1); // min n for which 0.5*2^n is a subnormal double\n\n const int maxBits2 = maxBits + 2;\n const ulong mask = (1UL << (maxBits - 1)) - 1; // mask for lower (maxBits - 1) bits\n\n if (str == null) throw new ArgumentNullException(\"str\");\n int n = str.Length;\n if (n == 0) goto InvalidFormat;\n\n // n*4 <= Int32.MaxValue protects against an nBits overflow,\n // the additional -minSExp + 10 margin is needed for parsing the exponent\n if (n > (int.MaxValue + minSExp - 10)/4)\n throw new System.FormatException(\"The given hexadecimal string representation of a double precision floating-point number is too long.\");\n\n int sign = 0; // 0 == positive, 1 == negative\n ulong xn = 0; // integer significand with up to maxBits + 2 bits, where the (maxBits + 2)th bit\n // (the least significant bit) is the logical OR of the (maxBits + 2)th and all following input bits\n int nBits = -1; // number of bits in xn, not counting leading zeros\n int exp = 0; // the base-2 exponent\n#if LOW_TRUST\n var s = str;\n#else\n fixed (char* s = str) {\n#endif\n int i = 0;\n // sign\n if (s[0] == '+') i = 1;\n else if (s[0] == '-') {\n i = 1;\n sign = 1;\n }\n // \"0x\" prefix\n if (i + 1 < n && (s[i + 1] == 'x' || s[i + 1] == 'X')) {\n if (s[i] != '0') goto InvalidFormat;\n i += 2;\n }\n bool pastDot = false;\n for (;;) {\n if (i == n) {\n if (!pastDot) exp = nBits;\n if (nBits >= 0) break;\n else goto InvalidFormat;\n }\n char c = s[i++];\n int h;\n if (c < 128 && (h = AsciiHexValuePlus1s[c]) != 0) {\n --h;\n if (nBits <= 0 ) {\n xn |= (uint)h;\n nBits = 0;\n while (h > 0) {\n ++nBits;\n h >>= 1;\n }\n if (pastDot) exp -= 4 - nBits;\n } else if (nBits <= maxBits2 - 4) {\n xn <<= 4;\n xn |= (uint)h;\n nBits += 4;\n } else if (nBits < maxBits2) {\n int nRemBits = maxBits2 - nBits;\n int nSurplusBits = 4 - nRemBits;\n int surplusBits = h & (0xf >> nRemBits);\n // The .NET JIT is not able to emit branch-free code for\n // surplusBits = surplusBits != 0 ? 1 : 0;\n // So we use this version instead:\n surplusBits = (0xfffe >> surplusBits) & 1; // = surplusBits != 0 ? 1 : 0\n xn <<= nRemBits;\n xn |= (uint)((h >> nSurplusBits) | surplusBits);\n nBits += 4;\n } else {\n xn |= (uint)((0xfffe >> h) & 1); // (0xfffe >> h) & 1 == h != 0 ? 1 : 0\n nBits += 4;\n }\n } else if (c == '.') {\n if (pastDot) goto InvalidFormat;\n pastDot = true;\n exp = nBits >= 0 ? nBits : 0; // exponent for integer part of float\n } else if ((c | ' ') == 'p' && nBits >= 0) {\n if (!pastDot) exp = nBits;\n int eSign = 1;\n if (i < n && (s[i] == '-' || s[i] == '+')) {\n if (s[i] == '-') eSign = -1;\n ++i;\n }\n if (i == n) goto InvalidFormat;\n int e = 0;\n do {\n c = s[i++];\n if (((uint)c - (uint)'0') <= 9) {\n if (e <= (int.MaxValue - 9)/10) e = e*10 + (c - '0');\n else e = int.MaxValue - 8;\n } else goto InvalidFormat;\n } while (i < n);\n e*= eSign;\n // either e is exact or |e| >= int.MaxValue - 8\n // |exp| <= n*4 <= int.MaxValue + minSExp - 10\n //\n // Case 1: e and exp have the same sign\n // Case 1.a: e is exact && |exp + e| <= int.MaxValue ==> |exp + e| is exact\n // Case 1.b: |e| >= int.MaxValue - 8 || |exp + e| > int.MaxValue ==> |exp + e| >= int.MaxValue - 8\n // Case 2: e and exp have opposite signs\n // Case 2.a: e is exact ==> |exp + e| is exact\n // Case 2.b: |e| >= int.MaxValue - 8\n // ==> Case e > 0:\n // exp + e >= -(int.MaxValue + minSExp - 10) + (int.MaxValue - 8) = -minSExp + 2 > maxExp\n // Case e < 0:\n // exp + e <= (int.MaxValue + minSExp - 10) - (int.MaxValue - 8) = minSExp - 2\n //\n // hence, |exp + e| is exact || exp + e > maxExp || exp + e < minSExp - 1\n try {\n exp = checked (exp + e);\n } catch (System.OverflowException) {\n exp = e < 0 ? int.MinValue : int.MaxValue;\n }\n break;\n } else {\n --i;\n if (nBits == -1 && i + 3 <= n) {\n if ( ((s[i ] | ' ') == 'i')\n && ((s[i + 1] | ' ') == 'n')\n && ((s[i + 2] | ' ') == 'f')\n && (i + 3 == n\n || (i + 8 == n && ((s[i + 3] | ' ') == 'i')\n && ((s[i + 4] | ' ') == 'n')\n && ((s[i + 5] | ' ') == 'i')\n && ((s[i + 6] | ' ') == 't')\n && ((s[i + 7] | ' ') == 'y'))))\n {\n return sign == 0 ? Double.PositiveInfinity : Double.NegativeInfinity;\n } else if (i + 3 == n && ((s[i] | ' ') == 'n')\n && ((s[i + 1] | ' ') == 'a')\n && ((s[i + 2] | ' ') == 'n'))\n {\n return Double.NaN;\n }\n }\n goto InvalidFormat;\n }\n } // for\n#if !LOW_TRUST\n } // fixed\n#endif\n if (nBits == 0) return sign == 0 ? 0.0 : -0.0;\n if (exp <= maxExp) {\n if (exp >= minExp && nBits <= maxBits) {\n // not subnormal and no rounding is required\n if (nBits < maxBits) xn <<= maxBits - nBits; // normalize significand to maxBits\n xn &= mask; // mask out lower (maxBits - 1) bits, the most significant bit is encoded in exp\n } else {\n if (nBits < maxBits2) xn <<= maxBits2 - nBits; // normalize significand to (maxBits + 2) bits\n int isSubnormal = 0;\n if (exp < minExp) {\n if (exp < minSExp - 1) return sign == 0 ? 0.0 : -0.0; // underflow (minSExp - 1 could still be rounded to minSExp)\n isSubnormal = 1;\n do {\n xn = (xn >> 1) | (xn & 1);\n } while (++exp < minExp);\n if (xn <= 2) return sign == 0 ? 0.0 : -0.0; // underflow\n }\n int r = unchecked((int)xn) & 0x7; // (lsb, bit below lsb, logical OR of all bits below the bit below lsb)\n xn >>= 2; // truncate to maxBits\n if (r >= 6 || r == 3) {\n xn++;\n xn &= mask;\n if (xn == 0) { // rounded to a power of two\n exp += 1;\n if (exp > maxExp) goto Overflow;\n }\n } else {\n xn &= mask;\n }\n exp -= isSubnormal;\n }\n exp -= minExp - 1; // add bias\n xn = (((ulong)sign) << ((maxBits - 1) + expBits)) | (((ulong)exp) << (maxBits - 1)) | xn;\n #if LOW_TRUST\n return BitConverter.Int64BitsToDouble(unchecked((long)xn));\n #else\n return *((double*)(&xn));\n #endif\n }\n\nOverflow:\n string msg = n < 32 ? \"The given string (\\\"\" + str + \"\\\") represents a value either too large or too small for a double precision floating-point number.\"\n : \"The given string represents a value either too large or too small for a double precision floating-point number.\";\n throw new System.OverflowException(msg);\n\nInvalidFormat:\n string errmsg = n < 32 ? \"The given hexadecimal string representation of a double precision floating-point number (\\\"\" + str + \"\\\") is invalid.\"\n : \"The given hexadecimal string representation of a double precision floating-point number is invalid.\";\n throw new System.FormatException(errmsg);\n}\n\n#if !LOW_TRUST\n unsafe\n#endif\npublic static float SingleFromHexString(string str) {\n const int expBits = 8; // bits for exponent\n const int maxBits = 24; // significant bits (including implicit bit)\n\n const int maxExp = 1 << (expBits - 1); // max n for which 0.5*2^n is a double\n const int minExp = -maxExp + 3; // min n for which 0.5*2^n is a normal double\n const int minSExp = minExp - (maxBits - 1); // min n for which 0.5*2^n is a subnormal Single\n\n const int maxBits2 = maxBits + 2;\n const int mask = (1 << (maxBits - 1)) - 1; // mask for lower (maxBits - 1) bits\n\n if (str == null) throw new ArgumentNullException(\"str\");\n int n = str.Length;\n if (n == 0) goto InvalidFormat;\n\n // n*4 <= Int32.MaxValue protects against an nBits overflow,\n // the additional -minSExp + 10 margin is needed for parsing the exponent\n if (n > (int.MaxValue + minSExp - 10)/4)\n throw new System.FormatException(\"The given hexadecimal string representation of a single precision floating-point number is too long.\");\n\n int sign = 0; // 0 == positive, 1 == negative\n int xn = 0; // integer significand with up to maxBits + 2 bits, where the (maxBits + 2)th bit\n // (the least significant bit) is the logical OR of the (maxBits + 2)th and all following input bits\n int nBits = -1; // number of bits in xn, not counting leading zeros\n int exp = 0; // the base-2 exponent\n#if LOW_TRUST\n var s = str;\n#else\n fixed (char* s = str) {\n#endif\n int i = 0;\n // sign\n if (s[0] == '+') i = 1;\n else if (s[0] == '-') {\n i = 1;\n sign = 1;\n }\n // \"0x\" prefix\n if (i + 1 < n && (s[i + 1] == 'x' || s[i + 1] == 'X')) {\n if (s[i] != '0') goto InvalidFormat;\n i += 2;\n }\n bool pastDot = false;\n for (;;) {\n if (i == n) {\n if (!pastDot) exp = nBits;\n if (nBits >= 0) break;\n else goto InvalidFormat;\n }\n char c = s[i++];\n int h;\n if (c < 128 && (h = AsciiHexValuePlus1s[c]) != 0) {\n --h;\n if (nBits <= 0 ) {\n xn |= h;\n nBits = 0;\n while (h > 0) {\n ++nBits;\n h >>= 1;\n }\n if (pastDot) exp -= 4 - nBits;\n } else if (nBits <= maxBits2 - 4) {\n xn <<= 4;\n xn |= h;\n nBits += 4;\n } else if (nBits < maxBits2) {\n int nRemBits = maxBits2 - nBits;\n int nSurplusBits = 4 - nRemBits;\n int surplusBits = h & (0xf >> nRemBits);\n // The .NET JIT is not able to emit branch-free code for\n // surplusBits = surplusBits != 0 ? 1 : 0;\n // So we use this version instead:\n surplusBits = (0xfffe >> surplusBits) & 1; // == surplusBits != 0 ? 1 : 0\n xn <<= nRemBits;\n xn |= (h >> nSurplusBits) | surplusBits;\n nBits += 4;\n } else {\n xn |= (0xfffe >> h) & 1; // (0xfffe >> h) & 1 == h != 0 ? 1 : 0\n nBits += 4;\n }\n } else if (c == '.') {\n if (pastDot) goto InvalidFormat;\n pastDot = true;\n exp = nBits >= 0 ? nBits : 0; // exponent for integer part of float\n } else if ((c | ' ') == 'p' && nBits >= 0) {\n if (!pastDot) exp = nBits;\n int eSign = 1;\n if (i < n && (s[i] == '-' || s[i] == '+')) {\n if (s[i] == '-') eSign = -1;\n ++i;\n }\n if (i == n) goto InvalidFormat;\n int e = 0;\n do {\n c = s[i++];\n if (((uint)c - (uint)'0') <= 9) {\n if (e <= (int.MaxValue - 9)/10) e = e*10 + (c - '0');\n else e = int.MaxValue - 8;\n } else goto InvalidFormat;\n } while (i < n);\n e*= eSign;\n // either e is exact or |e| >= int.MaxValue - 8\n // |exp| <= n*4 <= int.MaxValue + minSExp - 10\n //\n // Case 1: e and exp have the same sign\n // Case 1.a: e is exact && |exp + e| <= int.MaxValue ==> |exp + e| is exact\n // Case 1.b: |e| >= int.MaxValue - 8 || |exp + e| > int.MaxValue ==> |exp + e| >= int.MaxValue - 8\n // Case 2: e and exp have opposite signs\n // Case 2.a: e is exact ==> |exp + e| is exact\n // Case 2.b: |e| >= int.MaxValue - 8\n // ==> Case e > 0:\n // exp + e >= -(int.MaxValue + minSExp - 10) + (int.MaxValue - 8) = -minSExp + 2 > maxExp\n // Case e < 0:\n // exp + e <= (int.MaxValue + minSExp - 10) - (int.MaxValue - 8) = minSExp - 2\n //\n // hence, |exp + e| is exact || exp + e > maxExp || exp + e < minSExp - 1\n try {\n exp = checked (exp + e);\n } catch (System.OverflowException) {\n exp = e < 0 ? int.MinValue : int.MaxValue;\n }\n break;\n } else {\n --i;\n if (nBits == -1 && i + 3 <= n) {\n if ( ((s[i ] | ' ') == 'i')\n && ((s[i + 1] | ' ') == 'n')\n && ((s[i + 2] | ' ') == 'f')\n && (i + 3 == n\n || (i + 8 == n && ((s[i + 3] | ' ') == 'i')\n && ((s[i + 4] | ' ') == 'n')\n && ((s[i + 5] | ' ') == 'i')\n && ((s[i + 6] | ' ') == 't')\n && ((s[i + 7] | ' ') == 'y'))))\n {\n return sign == 0 ? Single.PositiveInfinity : Single.NegativeInfinity;\n } else if (i + 3 == n && ((s[i] | ' ') == 'n')\n && ((s[i + 1] | ' ') == 'a')\n && ((s[i + 2] | ' ') == 'n'))\n {\n return Single.NaN;\n }\n }\n goto InvalidFormat;\n }\n } // for\n#if !LOW_TRUST\n } // fixed\n#endif\n if (nBits == 0) return sign == 0 ? 0.0f : -0.0f;\n if (exp <= maxExp) {\n if (exp >= minExp && nBits <= maxBits) {\n // not subnormal and no rounding is required\n if (nBits < maxBits) xn <<= maxBits - nBits; // normalize significand to maxBits\n xn &= mask; // mask out lower (maxBits - 1) bits, the most significant bit is encoded in exp\n } else {\n if (nBits < maxBits2) xn <<= maxBits2 - nBits; // normalize significand to (maxBits + 2) bits\n int isSubnormal = 0;\n if (exp < minExp) {\n if (exp < minSExp - 1) return sign == 0 ? 0.0f : -0.0f; // underflow (minSExp - 1 could still be rounded to minSExp)\n isSubnormal = 1;\n do {\n xn = (xn >> 1) | (xn & 1);\n } while (++exp < minExp);\n if (xn <= 2) return sign == 0 ? 0.0f : -0.0f; // underflow\n }\n int r = xn & 0x7; // (lsb, bit below lsb, logical OR of all bits below the bit below lsb)\n xn >>= 2; // truncate to maxBits\n if (r >= 6 || r == 3) {\n xn++;\n xn &= mask;\n if (xn == 0) { // rounded to a power of two\n exp += 1;\n if (exp > maxExp) goto Overflow;\n }\n } else {\n xn &= mask;\n }\n exp -= isSubnormal;\n }\n exp -= minExp - 1; // add bias\n xn = (sign << ((maxBits - 1) + expBits)) | (exp << (maxBits - 1)) | xn;\n #if LOW_TRUST\n return BitConverter.ToSingle(BitConverter.GetBytes(xn), 0);\n #else\n return *((float*)(&xn));\n #endif\n }\n\nOverflow:\n string msg = n < 32 ? \"The given string (\\\"\" + str + \"\\\") represents a value either too large or too small for a single precision floating-point number.\"\n : \"The given string represents a value either too large or too small for a single precision floating-point number.\";\n throw new System.OverflowException(msg);\n\nInvalidFormat:\n string errmsg = n < 32 ? \"The given hexadecimal string representation of a single precision floating-point number (\\\"\" + str + \"\\\") is invalid.\"\n : \"The given hexadecimal string representation of a single precision floating-point number is invalid.\";\n throw new System.FormatException(errmsg);\n}\n\n} // class HexFloat\n\n}\n" }, { "path": "FParsecCS/IdentifierValidator.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2012\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Text;\nusing System.Diagnostics;\nusing System.Runtime.CompilerServices;\nusing System.Runtime.InteropServices;\n\nusing Microsoft.FSharp.Core;\n\nnamespace FParsec {\n\n#if !LOW_TRUST\n unsafe\n#endif\npublic sealed class IdentifierValidator {\n\n [Flags]\n internal enum IdentifierCharFlags : byte {\n None = 0,\n\n Continue = 1,\n NonContinue = 2,\n //Start = NonContinue | Continue,\n\n // the following two values are used by the FParsec identifier parser, not this class\n PreCheckContinue = 4,\n PreCheckNonContinue = 8,\n }\n\n public NormalizationForm NormalizationForm { get; set; }\n public bool NormalizeBeforeValidation { get; set; }\n public bool AllowJoinControlCharsAsIdContinueChars { get; set; }\n private readonly IdentifierCharFlags[] AsciiCharOptions;\n\n private void CheckAscii(char asciiChar) {\n if (asciiChar == 0 || asciiChar >= 128)\n throw new ArgumentOutOfRangeException(\"asciiChar\", \"The identifier char settings can only be read or set for non-zero ASCII chars, i.e. chars in the range '\\u0001'-'\\u007f'.\");\n }\n\n public void SetIsAsciiNoIdChar(char asciiChar) { CheckAscii(asciiChar); AsciiCharOptions[asciiChar] = 0; }\n public void SetIsAsciiIdStartChar(char asciiChar) { CheckAscii(asciiChar); AsciiCharOptions[asciiChar] = IdentifierCharFlags.NonContinue | IdentifierCharFlags.Continue; }\n public void SetIsAsciiIdNonStartChar(char asciiChar) { CheckAscii(asciiChar); AsciiCharOptions[asciiChar] = IdentifierCharFlags.Continue; }\n\n public IdentifierValidator() {\n var ascii = new IdentifierCharFlags[128];\n var start = IdentifierCharFlags.NonContinue | IdentifierCharFlags.Continue;\n // defaults as defined by XID_START/XID_CONTINUE\n for (int c = 'A'; c <= 'Z'; ++c) ascii[c] = start;\n for (int c = 'a'; c <= 'z'; ++c) ascii[c] = start;\n for (int c = '0'; c <= '9'; ++c) ascii[c] = IdentifierCharFlags.Continue;\n ascii['_'] = IdentifierCharFlags.Continue;\n AsciiCharOptions = ascii;\n }\n\n internal IdentifierValidator(IdentifierCharFlags[] asciiCharOptions) {\n Debug.Assert(asciiCharOptions.Length == 128);\n AsciiCharOptions = asciiCharOptions;\n }\n\n /// Returns the normalized string, or null in case an invalid identifier\n /// character is found. If an invalid character is found, the string index of the\n /// invalid character is assigned to the out parameter, otherwise -1.\n public string ValidateAndNormalize(string str, out int errorPosition) {\n // Pinning str and asciiOptions to avoid redundant bounds checks would actually\n // slow down the code for small to medium size identifiers because of the\n // (unnecessarily) high overhead associated with C#'s fixed statement. One\n // issue is that the .NET C# compiler emits null and 0-length checks even\n // though the C# standard leaves the respective behaviour undefined and\n // one hence can't rely on them. Another, more severe issue is that the\n // C# compiler puts the whole code inside the scope of the fixed statement\n // into a try-finally block, even if the whole function has no exception\n // handlers. The try-finally block in turn inhibits certain optimizations\n // by the JIT, in particular it seems to prevent the 32-bit .NET JIT from\n // compiling gotos into straighforward jumps.\n var asciiOptions = AsciiCharOptions;\n bool isSecondRound = false;\n bool isOnlyAscii = true;\n int i = 1;\n int length = str.Length; // throws if str is null\n if (length == 0) goto ReturnWithError; // check could be avoided for null-terminated buffer\n\n // Even if NormalizeBeforeValidation is set we first try to validate the\n // identifier without normalization. If we don't get an error, we normalize\n // after validation. If we get an error, we normalize and try\n // to validate the identifier a second time. This doesn't change results\n // because XID identifiers are \"closed under normalization\".\n\n IdStart:\n char c = str[0];\n if (c < 128) {\n if ((asciiOptions[c] & IdentifierCharFlags.NonContinue) == 0) goto Error;\n } else {\n isOnlyAscii = false;\n if (!char.IsSurrogate(c)) {\n if (!IsXIdStartOrSurrogate(c)) goto Error;\n } else {\n if (i == length) goto Error; // check could be avoided for null-terminated buffer\n char c1 = str[1];\n if (c > 0xDBFF || !char.IsLowSurrogate(c1)) goto ReturnWithError;\n int cp = (c - 0xD800)*0x400 + c1 - 0xDC00; // codepoint minus 0x10000\n if (!IsXIdStartSmp(cp)) goto Error;\n ++i;\n }\n }\n if (i < length) {\n if (!AllowJoinControlCharsAsIdContinueChars) {\n for (;;) {\n c = str[i];\n ++i;\n if (c < 128) {\n if ((asciiOptions[c] & IdentifierCharFlags.Continue) == 0) goto Error;\n if (i == length) break;\n } else {\n isOnlyAscii = false;\n if (!char.IsSurrogate(c)) {\n if (!IsXIdContinueOrSurrogate(c)) goto Error;\n if (i == length) break;\n } else {\n if (i == length) goto Error; // check could be avoided for null-terminated buffer\n char c1 = str[i];\n if (c > 0xDBFF || !char.IsLowSurrogate(c1)) goto ReturnWithError;\n int cp = (c - 0xD800)*0x400 + c1 - 0xDC00; // codepoint minus 0x10000\n if (!IsXIdContinueSmp(cp)) goto Error;\n if (++i >= length) break;\n }\n }\n }\n } else { // duplicates the code from the previous case, the only difference being the (*) line\n for (;;) {\n c = str[i];\n ++i;\n if (c < 128) {\n if ((asciiOptions[c] & IdentifierCharFlags.Continue) == 0) goto Error;\n if (i == length) break;\n } else {\n isOnlyAscii = false;\n if (!char.IsSurrogate(c)) {\n if (!IsXIdContinueOrJoinControlOrSurrogate(c)) goto Error; // (*)\n if (i == length) break;\n } else {\n if (i == length) goto Error; // check could be avoided for null-terminated buffer\n char c1 = str[i];\n if (c > 0xDBFF || !char.IsLowSurrogate(c1)) goto ReturnWithError;\n int cp = (c - 0xD800)*0x400 + c1 - 0xDC00; // codepoint minus 0x10000\n if (!IsXIdContinueSmp(cp)) goto Error;\n if (++i >= length) break;\n }\n }\n }\n }\n }\n errorPosition = -1;\n if (NormalizationForm == 0 || (isOnlyAscii | isSecondRound)) return str;\n return str.Normalize(NormalizationForm);\n Error:\n if (NormalizeBeforeValidation && NormalizationForm != 0 && !(isOnlyAscii | isSecondRound)) {\n string nstr;\n try { nstr = str.Normalize(NormalizationForm); } // throws for invalid unicode characters\n catch (ArgumentException) { nstr = str; }\n if ((object)nstr != (object)str) {\n str = nstr;\n length = nstr.Length;\n isSecondRound = true;\n i = 1;\n goto IdStart;\n }\n }\n ReturnWithError:\n errorPosition = i - 1;\n return null;\n }\n\n private class IsIdStartCharOrSurrogateFSharpFunc : FSharpFunc {\n private IdentifierCharFlags[] AsciiCharOptions;\n public IsIdStartCharOrSurrogateFSharpFunc(IdentifierCharFlags[] asciiCharOptions) { AsciiCharOptions = asciiCharOptions; }\n\n public override bool Invoke(char ch) {\n if (ch < 128) return (AsciiCharOptions[ch] & IdentifierCharFlags.NonContinue) != 0;\n return IsXIdStartOrSurrogate(ch);\n }\n }\n\n private class IsIdContinueCharOrSurrogateFSharpFunc : FSharpFunc {\n private IdentifierCharFlags[] AsciiCharOptions;\n public IsIdContinueCharOrSurrogateFSharpFunc(IdentifierCharFlags[] asciiCharOptions) { AsciiCharOptions = asciiCharOptions; }\n\n public override bool Invoke(char ch) {\n if (ch < 128) return (AsciiCharOptions[ch] & IdentifierCharFlags.Continue) != 0;\n return IsXIdContinueOrSurrogate(ch);\n }\n }\n\n private class IsIdContinueCharOrJoinControlOrSurrogateFSharpFunc : FSharpFunc {\n private IdentifierCharFlags[] AsciiCharOptions;\n public IsIdContinueCharOrJoinControlOrSurrogateFSharpFunc(IdentifierCharFlags[] asciiCharOptions) { AsciiCharOptions = asciiCharOptions; }\n\n public override bool Invoke(char ch) {\n if (ch < 128) return (AsciiCharOptions[ch] & IdentifierCharFlags.Continue) != 0;\n return IsXIdContinueOrJoinControlOrSurrogate(ch);\n }\n }\n\n private FSharpFunc isIdStartOrSurrogateFunc;\n public FSharpFunc IsIdStartOrSurrogateFunc { get {\n return isIdStartOrSurrogateFunc\n ?? (isIdStartOrSurrogateFunc = new IsIdStartCharOrSurrogateFSharpFunc(AsciiCharOptions));\n } }\n\n private FSharpFunc isIdContinueOrSurrogateFunc;\n public FSharpFunc IsIdContinueOrSurrogateFunc { get {\n return isIdContinueOrSurrogateFunc\n ?? (isIdContinueOrSurrogateFunc = new IsIdContinueCharOrSurrogateFSharpFunc(AsciiCharOptions));\n } }\n\n private FSharpFunc isIdContinueOrJoinControlOrSurrogateFunc;\n public FSharpFunc IsIdContinueOrJoinControlOrSurrogateFunc { get {\n return isIdContinueOrJoinControlOrSurrogateFunc\n ?? (isIdContinueOrJoinControlOrSurrogateFunc = new IsIdContinueCharOrJoinControlOrSurrogateFSharpFunc(AsciiCharOptions));\n } }\n\n // The XID_START/XID_CONTINUE property data is stored in two multiple-stage lookup tables:\n // the BMP codepoints (0 - 0xFFFF) are stored in a two-stage table and the SMP codepoints (0x10000 - 0x10FFFF)\n // are stored in a three-stage table.\n //\n // Each two-stage table consists of an integer index arrays and one bit array.\n // Each three-stage table consists of two integer index arrays and one bit array.\n //\n // The first stage array is divided into multiple parts: one for XID_START, one for XID_CONTINUE\n // and -- only for the BMP table -- one in which in addition to the XID_CONTINUE chars the two\n // JOIN_CONTROL chars \"zero-width non-joiner\" (ZWNJ, '\\u200C') and \"zero-width joiner\"\n // (ZWJ, '\\u200D') are marked.\n // All codepoints in the BMP reserved for surrogates are marked as XID_START and XID_CONTINUE.\n //\n // The bits in the last stage array are stored in 32-bit words, where each 32-bit word\n // is stored in the platform byte order.\n //\n // To determine whether a codepoint has a property in a three-stage table,\n // three indices are computed:\n // idx1 = the (log_2 table1Length) most significant bits of the codepoint\n // idx2 = table1(START|CONTINUE|CONTINUE_OR_JOIN_CONTROL)[idx]*table2BlockLength\n // + the following (log_2 table2BlockLength) bits of the codepoint\n // idx3 = table2[idx2]*table3BlockLength + the least significant (log_2 table3BlockLength) bits of the codepoint\n // If the bit in table3 at the bit index idx3 is set, the codepoint has the property, otherwise not.\n\n public static bool IsXIdStartOrSurrogate(char bmpCodePoint) { // should get inlined\n return (IsXIdStartOrSurrogate_(bmpCodePoint) & 1u) != 0;\n }\n private static uint IsXIdStartOrSurrogate_(char bmpCodePoint) {\n int cp = bmpCodePoint;\n int idx1 = cp >> XIdBmpTable2Log2BitBlockLength;\n const int f2 = 1 << (XIdBmpTable2Log2BitBlockLength - 5);\n const int m2 = f2 - 1;\n int idx2 = XIdStartBmpTable1[idx1]*f2 + ((cp >> 5) & m2);\n return XIdBmpTable2[idx2] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n\n public static bool IsXIdContinueOrSurrogate(char bmpCodePoint) { // should get inlined\n return (IsXIdContinueOrSurrogate_(bmpCodePoint) & 1u) != 0u;\n }\n private static uint IsXIdContinueOrSurrogate_(char bmpCodePoint) {\n int cp = bmpCodePoint;\n int idx1 = cp >> XIdBmpTable2Log2BitBlockLength;\n const int f2 = 1 << (XIdBmpTable2Log2BitBlockLength - 5);\n const int m2 = f2 - 1;\n int idx2 = XIdContinueBmpTable1[idx1]*f2 + ((cp >> 5) & m2);\n return XIdBmpTable2[idx2] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n\n public static bool IsXIdContinueOrJoinControlOrSurrogate(char bmpCodePoint) { // should get inlined\n return (IsXIdContinueOrJoinControlOrSurrogate_(bmpCodePoint) & 1u) != 0u;\n }\n private static uint IsXIdContinueOrJoinControlOrSurrogate_(char bmpCodePoint) {\n int cp = bmpCodePoint;\n int idx1 = cp >> XIdBmpTable2Log2BitBlockLength;\n const int f2 = 1 << (XIdBmpTable2Log2BitBlockLength - 5);\n const int m2 = f2 - 1;\n int idx2 = XIdContinueOrJoinerBmpTable1[idx1]*f2 + ((cp >> 5) & m2);\n return XIdBmpTable2[idx2] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n\n public static bool IsXIdStartSmp(int smpCodePointMinus0x10000) { // should get inlined\n return (IsXIdStartSmp_(smpCodePointMinus0x10000) & 1u) != 0;\n }\n private static uint IsXIdStartSmp_(int smpCodePointMinus0x10000) {\n int cp = smpCodePointMinus0x10000;\n int idx1 = cp >> (XIdSmpTable2Log2BlockLength + XIdSmpTable3Log2BlockLength);\n const int f2 = 1 << XIdSmpTable2Log2BlockLength,\n f3 = 1 << (XIdSmpTable3Log2BlockLength - 5);\n const int m2 = f2 - 1, m3 = f3 - 1;\n #if !LOW_TRUST\n if ((idx1 & (0xffffffffu << XIdSmpTable1Log2Length)) != 0) throw new IndexOutOfRangeException();\n #endif\n int idx2 = XIdStartSmpTable1[idx1]*f2 + ((cp >> XIdSmpTable3Log2BlockLength) & m2);\n int idx3 = XIdSmpTable2[idx2]*f3 + ((cp >> 5) & m3);\n return XIdSmpTable3[idx3] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n\n public static bool IsXIdContinueSmp(int smpCodePointMinus0x10000) { // should get inlined\n return (IsXIdContinueSmp_(smpCodePointMinus0x10000) & 1u) != 0;\n }\n private static uint IsXIdContinueSmp_(int smpCodePointMinus0x10000) {\n int cp = smpCodePointMinus0x10000;\n int idx1 = cp >> (XIdSmpTable2Log2BlockLength + XIdSmpTable3Log2BlockLength);\n const int f2 = 1 << XIdSmpTable2Log2BlockLength,\n f3 = 1 << (XIdSmpTable3Log2BlockLength - 5);\n const int m2 = f2 - 1, m3 = f3 - 1;\n #if !LOW_TRUST\n if ((idx1 & (0xffffffffu << XIdSmpTable1Log2Length)) != 0) throw new IndexOutOfRangeException();\n #endif\n int idx2 = XIdContinueSmpTable1[idx1]*f2 + ((cp >> XIdSmpTable3Log2BlockLength) & m2);\n int idx3 = XIdSmpTable2[idx2]*f3 + ((cp >> 5) & m3);\n return XIdSmpTable3[idx3] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n\n // tables for Unicode 8.0.0\n\n private const int XIdStartBmpTable1Offset = 0;\n private const int XIdContinueBmpTable1Offset = 256;\n private const int XIdContinueOrJoinerBmpTable1Offset = 512;\n private const int XIdBmpTable1Size = 256;\n private const int XIdBmpTable1Log2Length = 8;\n private const int XIdBmpTable2Offset = 768;\n private const int XIdBmpTable2Size = 2816;\n private const int XIdBmpTable2Log2BitBlockLength = 8;\n\n private const int XIdStartSmpTable1Offset = 3584;\n private const int XIdContinueSmpTable1Offset = 3840;\n private const int XIdSmpTable1Size = 256;\n private const int XIdSmpTable1Log2Length = 8;\n private const int XIdSmpTable2Offset = 4096;\n private const int XIdSmpTable2Size = 704;\n private const int XIdSmpTable2Log2BlockLength = 5;\n private const int XIdSmpTable3Offset = 4800;\n private const int XIdSmpTable3Size = 1504;\n private const int XIdSmpTable3Log2BlockLength = 7;\n\n private static ReadOnlySpan DataArray => new byte[] {\n 0,2,3,4,6,8,10,12,14,16,18,20,22,24,26,28,30,2,32,33,\n 35,2,36,37,39,41,43,45,47,49,2,51,52,55,56,56,56,56,56,56,\n 56,56,56,56,57,59,56,56,61,63,56,56,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,64,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,65,\n 2,2,2,2,66,2,67,69,70,72,74,76,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,78,2,2,2,2,\n 2,2,2,2,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,\n 56,56,56,56,56,56,56,56,56,2,79,80,82,83,84,86,1,2,3,5,\n 7,9,11,13,15,17,19,21,23,25,27,29,31,2,32,34,35,2,36,38,\n 40,42,44,46,48,50,2,51,53,55,56,56,56,56,56,56,56,56,56,56,\n 58,60,56,56,62,63,56,56,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,64,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,65,2,2,2,2,\n 66,2,68,69,71,73,75,77,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,78,2,2,2,2,2,2,2,2,\n 56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,\n 56,56,56,56,56,2,79,81,82,83,85,87,1,2,3,5,7,9,11,13,\n 15,17,19,21,23,25,27,29,31,2,32,34,35,2,36,38,40,42,44,46,\n 48,50,2,51,54,55,56,56,56,56,56,56,56,56,56,56,58,60,56,56,\n 62,63,56,56,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,64,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,65,2,2,2,2,66,2,68,69,\n 71,73,75,77,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,78,2,2,2,2,2,2,2,2,56,56,56,56,\n 56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,56,\n 56,2,79,81,82,83,85,87,0,0,0,0,0,0,0,0,254,255,255,7,\n 254,255,255,7,0,0,0,0,0,4,32,4,255,255,127,255,255,255,127,255,\n 0,0,0,0,0,0,255,3,254,255,255,135,254,255,255,7,0,0,0,0,\n 0,4,160,4,255,255,127,255,255,255,127,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,195,255,3,0,31,80,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,223,184,64,215,255,255,251,255,255,255,\n 255,255,255,255,255,255,191,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,223,184,192,215,255,255,251,255,255,255,255,255,255,255,255,255,191,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,3,252,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,251,252,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,254,255,255,255,127,2,254,255,255,255,\n 255,0,0,0,0,0,0,0,0,0,255,255,255,7,7,0,255,255,255,255,\n 255,255,254,255,255,255,127,2,254,255,255,255,255,0,254,255,255,255,255,191,\n 182,0,255,255,255,7,7,0,0,0,0,0,255,255,255,255,255,7,0,0,\n 0,192,254,255,255,255,255,255,255,255,255,255,255,255,47,0,96,192,0,156,\n 0,0,255,7,255,255,255,255,255,255,255,255,255,195,255,255,255,255,255,255,\n 255,255,255,255,255,255,239,159,255,253,255,159,0,0,253,255,255,255,0,0,\n 0,224,255,255,255,255,255,255,255,255,255,255,63,0,2,0,0,252,255,255,\n 255,7,48,4,0,0,255,255,255,255,255,255,255,231,255,255,255,255,255,255,\n 255,255,255,255,255,255,3,0,255,255,255,255,255,255,63,4,255,255,63,4,\n 16,1,0,0,255,255,255,1,0,0,0,0,0,0,0,0,255,255,31,0,\n 0,0,0,0,0,0,0,0,255,255,255,255,255,63,0,0,255,255,255,15,\n 0,0,0,0,0,0,0,0,255,255,31,0,0,0,0,0,248,255,255,255,\n 240,255,255,255,255,255,255,35,0,0,1,255,3,0,254,255,225,159,249,255,\n 255,253,197,35,0,64,0,176,3,0,3,0,255,255,255,255,255,255,255,255,\n 255,255,255,255,207,255,254,255,239,159,249,255,255,253,197,243,159,121,128,176,\n 207,255,3,0,224,135,249,255,255,253,109,3,0,0,0,94,0,0,28,0,\n 224,191,251,255,255,253,237,35,0,0,1,0,3,0,0,2,238,135,249,255,\n 255,253,109,211,135,57,2,94,192,255,63,0,238,191,251,255,255,253,237,243,\n 191,59,1,0,207,255,0,2,224,159,249,255,255,253,237,35,0,0,0,176,\n 3,0,2,0,232,199,61,214,24,199,255,3,0,0,1,0,0,0,0,0,\n 238,159,249,255,255,253,237,243,159,57,192,176,207,255,2,0,236,199,61,214,\n 24,199,255,195,199,61,129,0,192,255,0,0,224,223,253,255,255,253,255,35,\n 0,0,0,7,3,0,0,0,224,223,253,255,255,253,239,35,0,0,0,64,\n 3,0,6,0,239,223,253,255,255,253,255,227,223,61,96,7,207,255,0,0,\n 238,223,253,255,255,253,239,243,223,61,96,64,207,255,6,0,224,223,253,255,\n 255,255,255,39,0,64,0,128,3,0,0,252,224,255,127,252,255,255,251,47,\n 127,0,0,0,0,0,0,0,238,223,253,255,255,255,255,231,223,125,128,128,\n 207,255,0,252,236,255,127,252,255,255,251,47,127,132,95,255,192,255,12,0,\n 254,255,255,255,255,255,5,0,127,0,0,0,0,0,0,0,150,37,240,254,\n 174,236,5,32,95,0,0,240,0,0,0,0,254,255,255,255,255,255,255,7,\n 255,127,255,3,0,0,0,0,150,37,240,254,174,236,255,59,95,63,255,243,\n 0,0,0,0,1,0,0,0,0,0,0,0,255,254,255,255,255,31,0,0,\n 0,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,3,\n 255,3,160,194,255,254,255,255,255,31,254,255,223,255,255,254,255,255,255,31,\n 64,0,0,0,0,0,0,0,255,255,255,255,255,7,0,128,0,0,63,60,\n 98,192,225,255,3,64,0,0,255,255,255,255,191,32,255,255,255,255,255,247,\n 255,255,255,255,255,255,255,255,255,3,255,255,255,255,255,255,255,255,255,63,\n 255,255,255,255,191,32,255,255,255,255,255,247,255,255,255,255,255,255,255,255,\n 255,61,127,61,255,255,255,255,255,61,255,255,255,255,61,127,61,255,127,255,\n 255,255,255,255,255,255,61,255,255,255,255,255,255,255,255,7,0,0,0,0,\n 255,255,0,0,255,255,255,255,255,255,255,255,255,255,63,63,255,255,61,255,\n 255,255,255,255,255,255,255,231,0,254,3,0,255,255,0,0,255,255,255,255,\n 255,255,255,255,255,255,63,63,254,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,159,255,255,254,255,255,7,\n 255,255,255,255,255,255,255,255,255,199,255,1,255,223,3,0,255,255,3,0,\n 255,255,3,0,255,223,1,0,255,255,255,255,255,255,15,0,0,0,128,16,\n 0,0,0,0,255,223,31,0,255,255,31,0,255,255,15,0,255,223,13,0,\n 255,255,255,255,255,255,255,255,255,255,143,48,255,3,0,0,0,0,0,0,\n 255,255,255,255,255,255,255,255,255,255,255,0,255,255,255,255,255,5,255,255,\n 255,255,255,255,255,255,63,0,0,56,255,3,255,255,255,255,255,255,255,255,\n 255,255,255,0,255,255,255,255,255,7,255,255,255,255,255,255,255,255,63,0,\n 255,255,255,127,0,0,0,0,0,0,255,255,255,63,31,0,255,255,255,255,\n 255,15,255,255,255,3,0,0,0,0,0,0,255,255,255,127,255,15,255,15,\n 192,255,255,255,255,63,31,0,255,255,255,255,255,15,255,255,255,3,255,7,\n 0,0,0,0,255,255,127,0,255,255,255,255,255,255,31,0,0,0,0,0,\n 0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,255,255,255,15,\n 255,255,255,255,255,255,255,127,255,255,255,159,255,3,255,3,128,0,255,63,\n 0,0,0,0,0,0,0,0,224,255,255,255,255,255,15,0,224,15,0,0,\n 0,0,0,0,248,255,255,255,1,192,0,252,255,255,255,255,63,0,0,0,\n 255,255,255,255,255,255,255,255,255,15,255,3,0,248,15,0,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,15,0,255,255,255,255,15,0,0,0,\n 0,224,0,252,255,255,255,63,0,0,0,0,0,0,0,0,0,0,0,0,\n 0,222,99,0,255,255,255,255,255,255,255,0,255,227,255,255,255,255,255,63,\n 0,0,0,0,0,0,0,0,0,0,247,255,255,255,127,3,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,63,240,\n 255,255,63,63,255,255,255,255,63,63,255,170,255,255,255,63,255,255,255,255,\n 255,255,223,95,220,31,207,15,255,31,220,31,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,2,128,0,0,255,31,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,0,128,1,0,16,0,0,0,2,128,\n 0,0,255,31,0,0,0,0,0,0,255,31,226,255,1,0,0,48,0,0,\n 0,0,0,128,1,0,16,0,0,0,2,128,0,0,255,31,0,0,0,0,\n 0,0,255,31,226,255,1,0,132,252,47,63,80,253,255,243,224,67,0,0,\n 255,255,255,255,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,127,255,255,\n 255,255,255,127,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 31,120,12,0,255,255,255,255,255,127,255,255,255,255,255,127,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,31,248,15,0,255,255,255,255,\n 191,32,255,255,255,255,255,255,255,128,0,0,255,255,127,0,127,127,127,127,\n 127,127,127,127,0,0,0,0,255,255,255,255,191,32,255,255,255,255,255,255,\n 255,128,0,128,255,255,127,0,127,127,127,127,127,127,127,127,255,255,255,255,\n 224,0,0,0,254,3,62,31,254,255,255,255,255,255,255,255,255,255,127,224,\n 254,255,255,255,255,255,255,255,255,255,255,247,224,0,0,0,254,255,62,31,\n 254,255,255,255,255,255,255,255,255,255,127,230,254,255,255,255,255,255,255,255,\n 255,255,255,247,224,255,255,255,255,63,254,255,255,255,255,255,255,255,255,255,\n 255,127,0,0,255,255,255,7,0,0,0,0,0,0,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,63,0,\n 0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,63,0,0,0,0,0,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,0,0,\n 0,0,0,0,0,0,255,255,255,255,255,63,255,31,255,255,0,12,0,0,\n 255,255,255,255,255,127,0,128,255,255,255,63,255,255,255,255,255,255,255,255,\n 255,255,0,0,255,31,255,255,255,15,0,0,255,255,255,255,255,255,240,191,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,3,0,0,0,128,255,\n 252,255,255,255,255,255,255,255,255,255,255,255,255,249,255,255,255,63,255,0,\n 0,0,0,0,0,0,128,255,187,247,255,255,7,0,0,0,255,255,255,255,\n 255,255,15,0,252,255,255,255,255,255,15,0,0,0,0,0,0,0,252,40,\n 255,255,255,255,255,0,0,0,255,255,255,255,255,255,15,0,255,255,255,255,\n 255,255,255,255,31,0,255,3,255,255,255,40,0,252,255,255,63,0,255,255,\n 127,0,0,0,255,255,255,31,240,255,255,255,255,255,7,0,0,128,0,0,\n 223,255,0,124,255,255,255,255,255,63,255,255,255,255,15,0,255,255,255,31,\n 255,255,255,255,255,255,255,255,1,128,255,3,255,255,255,127,255,255,255,255,\n 255,1,0,0,247,15,0,0,255,255,127,196,255,255,255,255,255,255,98,62,\n 5,0,0,56,255,7,28,0,255,255,255,255,255,255,127,0,255,63,255,3,\n 255,255,127,252,255,255,255,255,255,255,255,255,7,0,0,56,255,255,124,0,\n 126,126,126,0,127,127,255,255,255,255,255,247,63,0,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,7,0,0,0,126,126,126,0,127,127,255,255,\n 255,255,255,247,63,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,55,255,3,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,15,0,255,255,127,248,255,255,255,255,255,15,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,63,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,3,0,0,0,0,127,0,248,160,255,253,127,95,219,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,3,0,0,0,248,255,255,255,255,255,\n 127,0,248,224,255,253,127,95,219,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,3,0,0,0,248,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,63,240,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,63,0,0,255,255,255,255,255,255,\n 255,255,252,255,255,255,255,255,255,0,0,0,0,0,255,3,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,138,170,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,31,255,255,0,0,255,255,24,0,0,224,0,0,\n 0,0,138,170,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,\n 0,0,0,0,254,255,255,7,254,255,255,7,192,255,255,255,255,255,255,63,\n 255,255,255,127,252,252,252,28,0,0,0,0,0,0,255,3,254,255,255,135,\n 254,255,255,7,192,255,255,255,255,255,255,255,255,255,255,127,252,252,252,28,\n 0,0,0,0,\n 0,2,4,5,6,7,8,7,7,7,7,10,7,12,14,7,16,16,16,16,\n 16,16,16,16,16,16,17,18,19,7,7,20,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,1,3,4,5,\n 6,7,9,7,7,7,7,11,7,13,15,7,16,16,16,16,16,16,16,16,\n 16,16,17,18,19,7,7,20,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,21,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,\n 7,7,7,7,7,7,7,7,7,7,7,7,0,1,2,3,3,5,7,9,\n 10,11,13,3,10,10,14,3,15,16,17,18,19,21,23,24,25,26,3,3,\n 3,3,3,3,0,1,2,4,3,6,8,9,10,12,13,3,10,10,14,3,\n 15,16,17,18,20,22,23,24,25,26,3,3,3,3,3,3,27,29,31,33,\n 35,37,39,3,3,41,3,43,45,47,49,3,3,51,3,3,3,53,3,3,\n 3,3,3,3,3,3,3,3,28,30,32,34,36,38,40,3,3,42,3,44,\n 46,48,50,3,3,52,3,3,3,53,3,3,3,3,3,3,3,3,3,3,\n 10,10,10,10,10,10,10,49,54,10,55,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,10,10,10,10,10,10,10,10,\n 56,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,10,10,10,10,57,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,10,10,10,10,58,60,62,64,3,3,3,3,3,3,65,67,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,10,10,10,10,\n 59,61,63,64,3,3,3,3,3,3,66,68,69,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,70,71,3,3,\n 3,3,3,3,69,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,70,72,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,76,77,78,10,10,79,80,81,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,73,74,75,3,3,3,76,77,78,10,\n 10,79,80,82,3,3,3,3,83,84,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,10,85,3,3,\n 3,3,3,3,3,3,3,3,87,88,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,10,86,3,3,3,3,3,3,3,3,3,3,\n 87,88,3,3,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,\n 10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,\n 10,10,10,10,10,10,10,10,10,89,10,10,10,10,10,10,10,10,10,10,\n 10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,\n 10,10,90,10,91,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,\n 10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,\n 10,10,10,10,10,10,10,10,10,92,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,10,10,10,10,11,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,10,93,3,3,3,3,3,3,3,3,3,3,3,3,\n 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,255,239,255,255,\n 127,255,255,183,255,63,255,63,0,0,0,0,255,255,255,255,255,255,255,255,\n 255,255,255,255,255,255,255,7,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,255,255,255,31,\n 255,255,255,255,255,255,1,0,0,0,0,0,255,255,255,31,255,255,255,255,\n 255,255,1,0,1,0,0,0,255,255,255,255,0,0,255,255,255,7,255,255,\n 255,255,63,0,255,255,255,255,0,0,255,255,255,7,255,255,255,255,255,7,\n 255,255,255,63,255,255,255,255,15,255,62,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,63,0,0,0,0,\n 0,0,0,0,0,0,0,0,255,255,255,63,255,3,0,0,0,0,0,0,\n 0,0,0,0,255,255,255,255,255,0,255,255,255,255,255,255,15,0,0,0,\n 255,255,255,255,255,255,127,0,255,255,63,0,255,0,0,0,63,253,255,255,\n 255,255,191,145,255,255,63,0,255,255,127,0,255,255,255,127,0,0,0,0,\n 0,0,0,0,255,255,55,0,255,255,63,0,255,255,255,3,0,0,0,0,\n 0,0,0,0,255,255,255,255,255,255,255,192,0,0,0,0,0,0,0,0,\n 1,0,239,254,255,255,15,0,0,0,0,0,255,255,255,31,111,240,239,254,\n 255,255,15,135,0,0,0,0,255,255,255,31,255,255,255,31,0,0,0,0,\n 255,254,255,255,31,0,0,0,255,255,255,31,0,0,0,0,255,254,255,255,\n 127,0,0,0,255,255,255,255,255,255,63,0,255,255,63,0,255,255,7,0,\n 255,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,255,1,0,0,0,0,0,0,255,255,255,255,255,255,7,0,\n 255,255,255,255,255,255,7,0,248,255,255,255,255,255,255,0,0,0,0,0,\n 0,0,0,0,255,255,255,255,255,255,255,255,127,0,0,0,192,255,0,128,\n 248,255,255,255,255,255,0,0,0,0,255,255,255,1,0,0,255,255,255,255,\n 255,255,255,7,0,0,255,255,255,1,255,3,248,255,255,255,127,0,0,0,\n 0,0,255,255,255,255,71,0,255,255,255,255,255,255,223,255,0,0,255,255,\n 255,255,79,0,248,255,255,255,255,255,7,0,30,0,0,20,0,0,0,0,\n 255,255,255,255,255,255,255,255,31,28,255,23,0,0,0,0,255,255,251,255,\n 255,15,0,0,0,0,0,0,0,0,0,0,255,255,251,255,255,255,255,0,\n 0,0,0,0,0,0,0,0,127,189,255,191,255,1,255,255,255,255,255,127,\n 0,0,0,0,127,189,255,191,255,1,255,255,255,255,255,255,255,7,255,3,\n 224,159,249,255,255,253,237,35,0,0,1,224,3,0,0,0,239,159,249,255,\n 255,253,237,243,159,57,129,224,207,31,31,0,255,255,255,255,255,255,0,0,\n 176,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,191,0,255,3,\n 0,0,0,0,255,255,255,255,255,127,0,0,0,0,0,15,0,0,0,0,\n 255,255,255,255,255,255,63,255,1,0,0,63,0,0,0,0,255,255,255,255,\n 255,255,0,0,16,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,\n 17,0,255,3,0,0,0,0,255,255,255,255,255,7,0,0,0,0,0,0,\n 0,0,0,0,255,255,255,255,255,255,255,0,255,3,0,0,0,0,0,0,\n 255,255,255,3,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,227,\n 255,15,255,3,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,0,0,0,128,0,0,0,0,255,255,255,255,255,255,255,255,\n 255,3,0,128,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,1,\n 255,255,255,255,255,255,255,255,255,255,255,255,255,127,0,0,255,255,255,255,\n 255,255,255,255,15,0,0,0,0,0,0,0,255,255,255,255,255,127,0,0,\n 0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,127,0,0,0,\n 0,0,0,0,255,255,255,255,255,255,255,1,255,255,255,127,0,0,0,0,\n 255,255,255,255,255,255,255,1,255,255,255,127,255,3,0,0,0,0,0,0,\n 0,0,0,0,0,0,255,255,255,63,0,0,0,0,0,0,0,0,0,0,\n 0,0,255,255,255,63,31,0,255,255,255,255,255,255,0,0,15,0,0,0,\n 248,255,255,224,255,255,255,255,255,255,127,0,15,0,255,3,248,255,255,224,\n 255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,31,0,1,0,0,0,0,0,255,255,255,255,255,255,255,255,\n 31,0,255,255,255,255,255,127,0,0,248,255,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,128,255,255,0,0,0,0,0,0,0,0,0,0,0,0,\n 3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,7,255,31,255,1,255,3,0,0,0,0,\n 0,0,0,0,0,0,0,0,255,1,255,99,0,0,0,0,0,0,0,0,\n 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,224,227,7,248,\n 231,15,0,0,0,60,0,0,0,0,0,0,0,0,0,0,0,0,0,0,\n 0,0,0,0,28,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,\n 255,255,223,255,255,255,255,255,255,255,255,223,100,222,255,235,239,255,255,255,\n 255,255,255,255,191,231,223,223,255,255,255,123,95,252,253,255,255,255,255,255,\n 255,255,255,255,63,255,255,255,253,255,255,247,255,255,255,247,255,255,223,255,\n 255,255,223,255,255,127,255,255,255,127,255,255,255,253,255,255,255,253,255,255,\n 247,15,0,0,0,0,0,0,255,253,255,255,255,253,255,255,247,207,255,255,\n 255,255,255,255,255,255,255,255,255,255,127,248,255,255,255,255,255,31,32,0,\n 16,0,0,248,254,255,0,0,0,0,0,0,0,0,0,0,255,255,255,255,\n 255,255,255,255,31,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,\n 31,0,127,0,0,0,0,0,239,255,255,255,150,254,247,10,132,234,150,170,\n 150,247,247,94,255,251,255,15,238,251,255,15,0,0,0,0,0,0,0,0,\n 255,255,255,255,255,255,255,255,255,255,127,0,0,0,0,0,255,255,255,255,\n 255,255,31,0,255,255,255,255,255,255,255,255,255,255,255,63,255,255,255,255,\n 255,255,255,255,255,255,255,255,255,255,255,255,3,0,0,0,0,0,0,0,\n 0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,0,\n\n };\n\n#if LOW_TRUST\n private static ReadOnlySpan XIdStartBmpTable1 => DataArray.Slice(XIdStartBmpTable1Offset, XIdBmpTable1Size);\n private static ReadOnlySpan XIdContinueBmpTable1 => DataArray.Slice(XIdContinueBmpTable1Offset, XIdBmpTable1Size);\n private static ReadOnlySpan XIdContinueOrJoinerBmpTable1 => DataArray.Slice(XIdContinueOrJoinerBmpTable1Offset, XIdBmpTable1Size);\n private static readonly uint[] XIdBmpTable2 = Buffer.CopyUIntsStoredInLittleEndianByteArray(DataArray, XIdBmpTable2Offset, XIdBmpTable2Size);\n\n private static ReadOnlySpan XIdStartSmpTable1 => DataArray.Slice(XIdStartSmpTable1Offset, XIdSmpTable1Size);\n private static ReadOnlySpan XIdContinueSmpTable1 => DataArray.Slice(XIdContinueSmpTable1Offset, XIdSmpTable1Size);\n private static ReadOnlySpan XIdSmpTable2 => DataArray.Slice(XIdSmpTable2Offset, XIdSmpTable2Size);\n private static readonly uint[] XIdSmpTable3 = Buffer.CopyUIntsStoredInLittleEndianByteArray(DataArray, XIdSmpTable3Offset, XIdSmpTable3Size);\n#else\n private static byte* Data => (byte*)Unsafe.AsPointer(ref MemoryMarshal.GetReference(DataArray));\n\n private static byte* XIdStartBmpTable1 => Data + XIdStartBmpTable1Offset;\n private static byte* XIdContinueBmpTable1 => Data + XIdContinueBmpTable1Offset;\n private static byte* XIdContinueOrJoinerBmpTable1 => Data + XIdContinueOrJoinerBmpTable1Offset;\n private static readonly uint* XIdBmpTable2 = Buffer.LoadLittleEndianUInt32Data(Data, XIdBmpTable2Offset, XIdBmpTable2Size);\n\n private static byte* XIdStartSmpTable1 => Data + XIdStartSmpTable1Offset;\n private static byte* XIdContinueSmpTable1 => Data + XIdContinueSmpTable1Offset;\n private static byte* XIdSmpTable2 => Data + XIdSmpTable2Offset;\n private static readonly uint* XIdSmpTable3 = Buffer.LoadLittleEndianUInt32Data(Data, XIdSmpTable3Offset, XIdSmpTable3Size);\n\n#endif\n\n}\n}" }, { "path": "FParsecCS/ManyChars.cs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Text;\nusing Microsoft.FSharp.Core;\n\nnamespace FParsec {\n\n#if !LOW_TRUST\ninternal unsafe struct _16CharBuffer {\n public UInt64 UInt64_0;\n public UInt64 UInt64_1;\n public UInt64 UInt64_2;\n public UInt64 UInt64_3;\n}\n#endif\n\ninternal class Many1Chars : FSharpFunc, Reply> {\n protected FSharpFunc, Reply> CharParser1;\n protected FSharpFunc, Reply> CharParser;\n\n public Many1Chars(FSharpFunc, Reply> charParser1,\n FSharpFunc, Reply> charParser)\n {\n CharParser1 = charParser1;\n CharParser = charParser;\n }\n\n public override Reply Invoke(CharStream stream) {\n var reply = CharParser1.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok)\n return ParseRestOfString(stream, reply.Result, reply.Error);\n else\n return new Reply{Status = reply.Status, Error = reply.Error};\n }\n\n#if !LOW_TRUST\n unsafe\n#endif\n protected Reply ParseRestOfString(CharStream stream, char firstChar, ErrorMessageList error) {\n #if LOW_TRUST\n var sb = new StringBuilder(16);\n sb.Append(firstChar);\n #else\n _16CharBuffer buffer_; // produces more efficient code on .NET than stackalloc char[16]\n char* buffer = (char*)(&buffer_);\n buffer[0] = firstChar;\n char[] chars = null;\n uint n = 1;\n #endif\n for (;;) {\n var tag = stream.StateTag;\n var reply = CharParser.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok) {\n if (tag == stream.StateTag)\n throw Internal.ParserCombinatorInInfiniteLoopHelper.CreateException(\"manyChars\", stream);\n error = reply.Error;\n #if LOW_TRUST\n sb.Append(reply.Result);\n #else\n var i = n%16;\n if (i != 0) {\n buffer[i] = reply.Result;\n ++n;\n } else {\n if (chars == null) chars = new char[32];\n else if (n == chars.Length) {\n var newChars = new char[2*chars.Length];\n Array.Copy(chars, newChars, chars.Length);\n chars = newChars;\n }\n for (i = 0; i < 16; ++i)\n chars[n - 16 + i] = buffer[i];\n buffer[0] = reply.Result;\n ++n;\n }\n #endif\n } else if (reply.Status == ReplyStatus.Error && tag == stream.StateTag) {\n string str;\n #if LOW_TRUST\n str = sb.ToString();\n #else\n if (n <= 16) str = new String(buffer, 0, (int)n);\n else {\n for (uint i = (n - 1) & 0x7ffffff0u; i < n; ++i)\n chars[i] = buffer[i%16];\n str = new string(chars, 0, (int)n);\n }\n #endif\n error = ErrorMessageList.Merge(error, reply.Error);\n return new Reply{Status = ReplyStatus.Ok, Result = str, Error = error};\n } else {\n error = tag == stream.StateTag ? ErrorMessageList.Merge(error, reply.Error) : reply.Error;\n return new Reply{Status = reply.Status, Error = error};\n }\n }\n }\n\n public FSharpFunc, Reply> AsFSharpFunc { get { return this; } }\n}\n\n\ninternal class ManyChars : Many1Chars {\n public ManyChars(FSharpFunc, Reply> charParser1,\n FSharpFunc, Reply> charParser)\n : base(charParser1, charParser) { }\n\n public override Reply Invoke(CharStream stream) {\n var tag = stream.StateTag;\n var reply = CharParser1.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok)\n return ParseRestOfString(stream, reply.Result, reply.Error);\n else if (reply.Status == ReplyStatus.Error && tag == stream.StateTag)\n return new Reply{Status = ReplyStatus.Ok, Result = \"\", Error = reply.Error};\n else\n return new Reply{Status = reply.Status, Error = reply.Error};\n }\n}\n\ninternal class Many1CharsTill : FSharpFunc, Reply> {\n protected FSharpFunc, Reply> CharParser1;\n protected FSharpFunc, Reply> CharParser;\n protected FSharpFunc, Reply> EndParser;\n protected OptimizedClosures.FSharpFunc Mapping;\n\n public Many1CharsTill(FSharpFunc, Reply> charParser1,\n FSharpFunc, Reply> charParser,\n FSharpFunc, Reply> endParser,\n FSharpFunc> mapping)\n {\n CharParser1 = charParser1;\n CharParser = charParser;\n EndParser = endParser;\n Mapping = (OptimizedClosures.FSharpFunc)(object)OptimizedClosures.FSharpFunc.Adapt(mapping);\n }\n\n public override Reply Invoke(CharStream stream) {\n var reply = CharParser1.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok)\n return ParseRestOfString(stream, reply.Result, reply.Error);\n else\n return new Reply{Status = reply.Status, Error = reply.Error};\n }\n\n#if !LOW_TRUST\n unsafe\n#endif\n protected Reply ParseRestOfString(CharStream stream, char firstChar, ErrorMessageList error) {\n #if LOW_TRUST\n var sb = new StringBuilder(16);\n sb.Append(firstChar);\n #else\n _16CharBuffer buffer_; // produces more efficient code than stackalloc char[16]\n char* buffer = (char*)(&buffer_);\n buffer[0] = firstChar;\n char[] chars = null;\n uint n = 1;\n #endif\n for (;;) {\n var tag = stream.StateTag;\n var eReply = EndParser.Invoke(stream);\n if (eReply.Status == ReplyStatus.Error && tag == stream.StateTag) {\n var reply = CharParser.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok) {\n if (tag == stream.StateTag)\n throw Internal.ParserCombinatorInInfiniteLoopHelper.CreateException(\"manyCharsTill\", stream);\n error = reply.Error;\n #if LOW_TRUST\n sb.Append(reply.Result);\n #else\n var i = n%16;\n if (i != 0) {\n buffer[i] = reply.Result;\n ++n;\n } else {\n if (chars == null) chars = new char[32];\n else if (n == chars.Length) {\n var newChars = new char[2*chars.Length];\n Array.Copy(chars, newChars, chars.Length);\n chars = newChars;\n }\n for (i = 0; i < 16; ++i)\n chars[n - 16 + i] = buffer[i];\n buffer[0] = reply.Result;\n ++n;\n }\n #endif\n } else {\n error = tag == stream.StateTag\n ? ErrorMessageList.Merge(ErrorMessageList.Merge(error, eReply.Error), reply.Error)\n : reply.Error;\n return new Reply{Status = reply.Status, Error = error};\n }\n } else if (eReply.Status == ReplyStatus.Ok) {\n string str;\n #if LOW_TRUST\n str = sb.ToString();\n #else\n if (n <= 16) str = new String(buffer, 0, (int)n);\n else {\n for (uint i = (n - 1) & 0x7ffffff0; i < n; ++i)\n chars[i] = buffer[i%16];\n str = new string(chars, 0, (int)n);\n }\n #endif\n var result = Mapping.Invoke(str, eReply.Result);\n error = tag == stream.StateTag\n ? ErrorMessageList.Merge(error, eReply.Error)\n : eReply.Error;\n return new Reply{Status = ReplyStatus.Ok, Result = result, Error = error};\n } else {\n error = tag == stream.StateTag\n ? ErrorMessageList.Merge(error, eReply.Error)\n : eReply.Error;\n return new Reply{Status = eReply.Status, Error = error};\n }\n }\n }\n\n public FSharpFunc, Reply> AsFSharpFunc { get { return this; } }\n}\n\ninternal class ManyCharsTill : Many1CharsTill {\n public ManyCharsTill(FSharpFunc, Reply> charParser1,\n FSharpFunc, Reply> charParser,\n FSharpFunc, Reply> endParser,\n FSharpFunc> mapping)\n : base(charParser1, charParser, endParser, mapping) { }\n\n public override Reply Invoke(CharStream stream) {\n var tag = stream.StateTag;\n var eReply = EndParser.Invoke(stream);\n if (eReply.Status == ReplyStatus.Error && tag == stream.StateTag) {\n var reply = CharParser1.Invoke(stream);\n if (reply.Status == ReplyStatus.Ok) {\n return ParseRestOfString(stream, reply.Result, reply.Error);\n } else {\n var error = tag == stream.StateTag\n ? ErrorMessageList.Merge(eReply.Error, reply.Error)\n : reply.Error;\n return new Reply{Status = reply.Status, Error = error};\n }\n } else if (eReply.Status == ReplyStatus.Ok) {\n var result = Mapping.Invoke(\"\", eReply.Result);\n return new Reply{Status = ReplyStatus.Ok, Result = result, Error = eReply.Error};\n } else {\n return new Reply{Status = eReply.Status, Error = eReply.Error};\n }\n }\n}\n\n\n\n}" }, { "path": "FParsecCS/OperatorPrecedenceParser.cs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nusing Microsoft.FSharp.Core;\nusing System.Diagnostics;\nusing System.Collections.Generic;\n\nnamespace FParsec {\n\npublic enum Associativity {\n None = 0,\n Left = 1,\n Right = 2\n}\n\n\npublic enum OperatorType {\n Infix = 0,\n Prefix = 1,\n Postfix = 2\n}\n\n\npublic class Operator {\n public OperatorType Type { get; private set; }\n\n public string String { get; protected set; }\n internal FSharpFunc, Reply> AfterStringParser { get; private set; }\n\n public string TernaryRightString { get; protected set; }\n internal FSharpFunc, Reply> AfterTernaryRightStringParser { get; private set; }\n public bool IsTernary { get { return TernaryRightString != null; } }\n\n public int Precedence { get; protected set; }\n public Associativity Associativity { get; protected set; }\n public bool IsAssociative { get { return Associativity != Associativity.None; } }\n\n internal OptimizedClosures.FSharpFunc Mapping1 { get; private set; }\n internal OptimizedClosures.FSharpFunc Mapping2 { get; private set; }\n internal OptimizedClosures.FSharpFunc Mapping3 { get; private set; }\n\n private Operator() {}\n static readonly internal Operator ZeroPrecedenceOperator = new Operator{Type = OperatorType.Prefix};\n\n private Operator(OperatorType type,\n string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence)\n {\n Debug.Assert(type >= OperatorType.Infix && type <= OperatorType.Postfix);\n Type = type;\n if (string.IsNullOrEmpty(operatorString)) throw new ArgumentException(\"operatorString\", \"The operator string must not be empty.\");\n String = operatorString;\n if (afterStringParser == null) throw new ArgumentNullException(\"afterStringParser\");\n AfterStringParser = afterStringParser;\n if (precedence < 1) throw new ArgumentOutOfRangeException(\"precedence\", \"The operator precedence must be greater than 0.\");\n Precedence = precedence;\n }\n\n internal Operator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n Associativity associativity,\n FSharpFunc>> mapping)\n : this(OperatorType.Infix, operatorString, afterStringParser, precedence)\n {\n if (associativity < Associativity.None || associativity > Associativity.Right)\n throw new ArgumentOutOfRangeException(\"associativity\", \"The associativity argument is invalid.\");\n Associativity = associativity;\n if (mapping == null) throw new ArgumentNullException(\"mapping\");\n Mapping2 = OptimizedClosures.FSharpFunc.Adapt(mapping);\n }\n\n internal Operator(OperatorType type,\n string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n bool isAssociative,\n FSharpFunc> mapping)\n : this(type, operatorString, afterStringParser, precedence)\n {\n Debug.Assert(type == OperatorType.Prefix || type == OperatorType.Postfix);\n Associativity = !isAssociative ? Associativity.None :\n type == OperatorType.Prefix ? Associativity.Right : Associativity.Left;\n if (mapping == null) throw new ArgumentNullException(\"mapping\");\n Mapping1 = OptimizedClosures.FSharpFunc.Adapt(mapping);\n }\n\n\n internal Operator(string leftString,\n FSharpFunc, Reply> afterLeftStringParser,\n string rightString,\n FSharpFunc, Reply> afterRightStringParser,\n int precedence,\n Associativity associativity,\n FSharpFunc>>>> mapping)\n {\n Type = OperatorType.Infix;\n if (string.IsNullOrEmpty(leftString)) throw new ArgumentException(\"leftString\", \"The operator strings must not be empty.\");\n String = leftString;\n if (afterLeftStringParser == null) throw new ArgumentNullException(\"afterLeftStringParser\");\n AfterStringParser = afterLeftStringParser;\n if (string.IsNullOrEmpty(rightString)) throw new ArgumentException(\"rightString\", \"The operator strings must not be empty.\");\n TernaryRightString = rightString;\n if (afterRightStringParser == null) throw new ArgumentNullException(\"afterRightStringParser\");\n AfterTernaryRightStringParser = afterRightStringParser;\n if (precedence < 1) throw new ArgumentOutOfRangeException(\"precedence\", \"The operator precedence must be greater than 0.\");\n Precedence = precedence;\n if (associativity < Associativity.None || associativity > Associativity.Right)\n throw new ArgumentOutOfRangeException(\"associativity\", \"The associativity argument is invalid.\");\n Associativity = associativity;\n if (mapping == null) throw new ArgumentNullException(\"mapping\");\n Mapping3 = OptimizedClosures.FSharpFunc.Adapt(mapping);\n }\n\n protected class NoAfterStringUnaryMappingAdapter\n : OptimizedClosures.FSharpFunc\n {\n private FSharpFunc Mapping;\n public NoAfterStringUnaryMappingAdapter(FSharpFunc mapping) { Mapping = mapping; }\n public override TTerm Invoke(TAfterString afterString, TTerm term) { return Mapping.Invoke(term); }\n }\n\n protected class NoAfterStringBinaryMappingAdapter\n : OptimizedClosures.FSharpFunc\n {\n private OptimizedClosures.FSharpFunc Mapping;\n public NoAfterStringBinaryMappingAdapter(OptimizedClosures.FSharpFunc mapping) { Mapping = mapping; }\n public override TTerm Invoke(TAfterString afterString, TTerm leftTerm, TTerm rightTerm) {\n return Mapping.Invoke(leftTerm, rightTerm);\n }\n }\n\n protected class NoAfterStringTernaryMappingAdapter\n : OptimizedClosures.FSharpFunc\n {\n private OptimizedClosures.FSharpFunc Mapping;\n public NoAfterStringTernaryMappingAdapter(OptimizedClosures.FSharpFunc mapping) { Mapping = mapping; }\n public override TTerm Invoke(TAfterString afterLeftString, TAfterString afterRightString,\n TTerm leftTerm, TTerm middleTerm, TTerm rightTerm)\n {\n return Mapping.Invoke(leftTerm, middleTerm, rightTerm);\n }\n }\n\n}\n\npublic sealed class InfixOperator : Operator {\n public InfixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n Associativity associativity,\n FSharpFunc> mapping)\n : base(operatorString, afterStringParser, precedence, associativity,\n mapping == null ? null : new NoAfterStringBinaryMappingAdapter(OptimizedClosures.FSharpFunc.Adapt(mapping))) {}\n\n public InfixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n Associativity associativity,\n Unit dummy, // disambiguates overloads in F#\n FSharpFunc>> mapping)\n : base(operatorString, afterStringParser, precedence, associativity, mapping) {}\n}\n\npublic sealed class PrefixOperator : Operator {\n public PrefixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n bool isAssociative,\n FSharpFunc mapping)\n : base(OperatorType.Prefix, operatorString, afterStringParser, precedence, isAssociative,\n mapping == null ? null : new NoAfterStringUnaryMappingAdapter(mapping)) {}\n\n public PrefixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n bool isAssociative,\n Unit dummy, // disambiguates overloads in F#\n FSharpFunc> mapping)\n : base(OperatorType.Prefix, operatorString, afterStringParser, precedence, isAssociative, mapping) {}\n}\n\n public sealed class PostfixOperator : Operator {\n public PostfixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n bool isAssociative,\n FSharpFunc mapping)\n : base(OperatorType.Postfix, operatorString, afterStringParser, precedence, isAssociative,\n mapping == null ? null : new NoAfterStringUnaryMappingAdapter(mapping)) {}\n\n public PostfixOperator(string operatorString,\n FSharpFunc, Reply> afterStringParser,\n int precedence,\n bool isAssociative,\n Unit dummy, // disambiguates overloads in F#\n FSharpFunc> mapping)\n : base(OperatorType.Postfix, operatorString, afterStringParser, precedence, isAssociative, mapping) {}\n}\n\npublic sealed class TernaryOperator : Operator {\n public TernaryOperator(string leftString,\n FSharpFunc, Reply> afterLeftStringParser,\n string rightString,\n FSharpFunc, Reply> afterRightStringParser,\n int precedence,\n Associativity associativity,\n FSharpFunc>> mapping)\n : base(leftString, afterLeftStringParser, rightString, afterRightStringParser, precedence, associativity,\n mapping == null ? null : new NoAfterStringTernaryMappingAdapter(OptimizedClosures.FSharpFunc.Adapt(mapping))) {}\n\n public TernaryOperator(string leftString,\n FSharpFunc, Reply> afterLeftStringParser,\n string rightString,\n FSharpFunc, Reply> afterRightStringParser,\n int precedence,\n Associativity associativity,\n Unit dummy, // disambiguates overloads in F#\n FSharpFunc>>>> mapping)\n : base(leftString, afterLeftStringParser, rightString, afterRightStringParser, precedence, associativity, mapping) {}\n}\n\n\npublic class OperatorPrecedenceParser : FSharpFunc, Reply> {\n\n internal struct OperatorData { // declared as struct, so we can allocate it on the stack\n internal Operator Operator;\n internal TAfterString AfterStringValue;\n internal CharStreamIndexToken IndexToken;\n internal long Line;\n internal long LineBegin;\n }\n\n /// The length of LhsOps and RhsOps. Must be a power of 2.\n internal const int OpsArrayLength = 128;\n\n // LhsOps and RhsOps are arrays of operator arrays. LhsOps contains the prefix\n // operator definitions, RhsOps contains all other operator definitions.\n // Both have a fixed size of OpsArrayLength (which must be a power of 2).\n // All operators beginning with the same char modulo OpsArrayLength are\n // grouped together in the same inner array. The inner arrays are sorted\n // by the Operator.String property in descending lexical order.\n // The index of an inner array in the outer array is given by the\n // inner array's operator strings' first char modulo oppArrayLength.\n // An empty inner array is represended by null.\n\n private readonly Operator[][] LhsOps = new Operator[OpsArrayLength][];\n private readonly Operator[][] RhsOps = new Operator[OpsArrayLength][];\n\n // initialized to 0\n private int PrefixOpCount;\n private int InfixOpCount;\n private int PostfixOpCount;\n\n private ErrorMessageList ExpectedInfixOrPostfixOperator; // initialized to null\n\n private readonly Dictionary> Reserved = new Dictionary>();\n\n // The following two members aren't static because accessing static members of generic types is rather expensive.\n\n /// ParsePrefixOp returns this value to signal that it backtracked and we should try to parse a term.\n private readonly Operator ErrorOp = Operator.ZeroPrecedenceOperator;\n\n /// Can not be readonly because it is passed as as a ref (for performance reasons), but it is never mutated.\n private OperatorData ZeroPrecedenceOperatorData = new OperatorData{Operator = Operator.ZeroPrecedenceOperator};\n\n public FSharpFunc, Reply> TermParser { get; set; }\n\n public FSharpFunc<\n Tuple, TAfterString>,\n ErrorMessageList>\n MissingTernary2ndStringErrorFormatter { get; set; }\n\n\n // C# really needs type abbreviations (or better type inference)\n private OptimizedClosures.FSharpFunc<\n Tuple, TAfterString>,\n Tuple, TAfterString>,\n ErrorMessageList>\n _OperatorConflictErrorFormatter;\n public FSharpFunc<\n Tuple, TAfterString>,\n FSharpFunc, TAfterString>,\n ErrorMessageList>>\n OperatorConflictErrorFormatter {\n get { return _OperatorConflictErrorFormatter; }\n set { _OperatorConflictErrorFormatter = OptimizedClosures.FSharpFunc, TAfterString>,Tuple, TAfterString>, ErrorMessageList>\n .Adapt(value); }\n }\n\n public OperatorPrecedenceParser() {\n MissingTernary2ndStringErrorFormatter = new DefaultMissingTernary2ndStringErrorFormatter();\n OperatorConflictErrorFormatter = new DefaultOperatorConflictErrorFormatter();\n }\n\n public FSharpFunc, Reply> ExpressionParser { get { return this; } }\n\n private bool FindPosition(Operator[][] ops, string str, out int arrayIndex, out int indexInArray) {\n var c0 = str[0];\n int i = c0 & (OpsArrayLength - 1);\n arrayIndex = i;\n var array = ops[i];\n int c = -1;\n int j = 0;\n if (array != null) {\n for (j = 0; j < array.Length; ++j) {\n c = String.CompareOrdinal(str, array[j].String);\n if (c >= 0) break;\n }\n }\n indexInArray = j;\n return c == 0;\n }\n\n private void ThrowDefinitionConflictException(Operator op,\n Operator oldOp)\n {\n throw new ArgumentException(\"The definition of the \" + op.ToString() + \" conflicts with (or duplicates) the previous definition of the \" + oldOp.ToString() + \".\");\n }\n\n public void AddOperator(Operator op) {\n Operator oldOp;\n if ( Reserved.TryGetValue(op.String, out oldOp)\n || (op.IsTernary && Reserved.TryGetValue(op.TernaryRightString, out oldOp)))\n {\n ThrowDefinitionConflictException(op, oldOp);\n }\n var ops = op.Type == OperatorType.Prefix ? LhsOps : RhsOps;\n int i, j;\n if (FindPosition(ops, op.String, out i, out j))\n ThrowDefinitionConflictException(op, ops[i][j]);\n if (op.IsTernary) {\n int i2, j2;\n // make sure the Ternary2ndString isn't registered as an operator\n if (FindPosition(LhsOps, op.TernaryRightString, out i2, out j2))\n ThrowDefinitionConflictException(op, LhsOps[i2][j2]);\n if (FindPosition(RhsOps, op.TernaryRightString, out i2, out j2))\n ThrowDefinitionConflictException(op, RhsOps[i2][j2]);\n Reserved.Add(op.TernaryRightString, op);\n }\n var array = ops[i];\n if (array == null) {\n ops[i] = new Operator[1]{op};\n } else {\n int n = array.Length;\n var newArray = new Operator[n + 1];\n if (j != 0) Array.Copy(array, 0, newArray, 0, j);\n newArray[j] = op;\n if (j != n) Array.Copy(array, j, newArray, j + 1, n - j);\n ops[i] = newArray;\n }\n if (op.Type == OperatorType.Infix) {\n ++InfixOpCount;\n if (InfixOpCount == 1) {\n ExpectedInfixOrPostfixOperator = PostfixOpCount == 0\n ? Errors.ExpectedInfixOperator\n : Errors.ExpectedInfixOrPostfixOperator;\n }\n } else if (op.Type == OperatorType.Postfix) {\n ++PostfixOpCount;\n if (PostfixOpCount == 1) {\n ExpectedInfixOrPostfixOperator = InfixOpCount == 0\n ? Errors.ExpectedPostfixOperator\n : Errors.ExpectedInfixOrPostfixOperator;\n }\n } else ++PrefixOpCount;\n }\n\n public bool RemoveInfixOperator(string opString) { return Remove(OperatorType.Infix, opString); }\n public bool RemovePrefixOperator(string opString) { return Remove(OperatorType.Prefix, opString); }\n public bool RemovePostfixOperator(string opString) { return Remove(OperatorType.Postfix, opString); }\n public bool RemoveTernaryOperator(string opStringLeft, string opStringRight) {\n Operator reservedOp;\n if (!Reserved.TryGetValue(opStringRight, out reservedOp) || opStringLeft != reservedOp.String) return false;\n Reserved.Remove(opStringRight);\n return Remove(OperatorType.Infix, opStringLeft);\n }\n\n public bool RemoveOperator(Operator op) {\n var ops = op.Type == OperatorType.Prefix ? LhsOps : RhsOps;\n int i, j;\n if (!FindPosition(ops, op.String, out i, out j)) return false;\n if (op != ops[i][j]) return false;\n return op.IsTernary ? RemoveTernaryOperator(op.String, op.TernaryRightString)\n : Remove(op.Type, op.String);\n }\n\n private bool Remove(OperatorType operatorType, string opString) {\n var ops = operatorType == OperatorType.Prefix ? LhsOps : RhsOps ;\n int i, j;\n if (!FindPosition(ops, opString, out i, out j)) return false;\n var array = ops[i];\n var n = array.Length;\n if (n == 1) ops[i] = null;\n else {\n var newArray = new Operator[n - 1];\n if (j != 0) Array.Copy(array, 0, newArray, 0, j);\n if (j + 1 != n) Array.Copy(array, j + 1, newArray, j, n - j - 1);\n ops[i] = newArray;\n }\n if (operatorType == OperatorType.Infix) {\n --InfixOpCount;\n if (InfixOpCount == 0) {\n ExpectedInfixOrPostfixOperator = PostfixOpCount == 0 ? null : Errors.ExpectedPostfixOperator;\n }\n } else if (operatorType == OperatorType.Postfix) {\n --PostfixOpCount;\n if (PostfixOpCount == 0) {\n ExpectedInfixOrPostfixOperator = InfixOpCount == 0 ? null : Errors.ExpectedInfixOperator;\n }\n } else --PrefixOpCount;\n return true;\n }\n\n public IEnumerable> Operators { get {\n var result = new Operator[PrefixOpCount + InfixOpCount + PostfixOpCount];\n var n = 0;\n if (PrefixOpCount != 0) {\n foreach (var array in LhsOps)\n if (array != null)\n foreach (var op in array)\n result[n++] = op;\n }\n if ((InfixOpCount | PostfixOpCount) != 0) {\n foreach (var array in RhsOps)\n if (array != null)\n foreach (var op in array)\n result[n++] = op;\n }\n Debug.Assert(n == result.Length);\n return result;\n } }\n\n private\n Operator\n PeekOp(CharStream stream, Operator[][] ops)\n {\n var cs = stream.Peek2();\n var c1 = cs.Char1;\n var c0 = cs.Char0;\n var array = ops[c0 & (OpsArrayLength - 1)];\n if (array != null) {\n foreach (var op in array) {\n var s = op.String;\n if (s[0] == c0) {\n if ( s.Length <= 1\n || (s[1] == c1 && (s.Length == 2 || stream.Match(s)))) return op;\n } else if (s[0] < c0) break;\n }\n }\n return null;\n }\n\n public override Reply Invoke(CharStream stream) {\n Reply reply = new Reply();\n reply.Status = ReplyStatus.Ok;\n var nextOp = ParseExpression(ref ZeroPrecedenceOperatorData, ref reply, stream);\n Debug.Assert(nextOp == null);\n return reply;\n }\n\n // =============================================================================\n // NOTE: The main complication in the below code arises from the handling of the\n // backtracking related to the after-string-parser. Please see the reference\n // documentation for an explanation of the after-string-parser behaviour.\n // =============================================================================\n\n internal\n Operator\n ParseExpression(ref OperatorData prevOpData, // prevOpData is passed as ref for performance reasons, but is not mutated\n ref Reply reply,\n CharStream stream)\n {\n Operator op;\n if (PrefixOpCount != 0 && ((op = PeekOp(stream, LhsOps)) != null)) {\n op = ParsePrefixOp(ref prevOpData, op, ref reply, stream);\n // ParsePrefixOp returns ErrorOp when it backtracks and we should try to parse a term\n if (op == null) goto Break;\n if (op != ErrorOp) goto CheckNextOp;\n }\n var error = reply.Error;\n var stateTag = stream.StateTag;\n reply = TermParser.Invoke(stream); // <-- this is where we parse the terms\n if (stateTag == stream.StateTag) {\n error = ErrorMessageList.Merge(error, reply.Error);\n if (PrefixOpCount != 0) error = ErrorMessageList.Merge(error, Errors.ExpectedPrefixOperator);\n reply.Error = error;\n }\n if (reply.Status != ReplyStatus.Ok) goto ReturnNull;\n op = PeekOp(stream, RhsOps);\n CheckNextOp:\n if (op != null) {\n var prevOp = prevOpData.Operator;\n if (prevOp.Precedence > op.Precedence) goto Break;\n if (prevOp.Precedence < op.Precedence) goto Continue;\n // prevOp.Precedence == op.Precedence\n if (op.Type == OperatorType.Infix) {\n var assoc = prevOp.Associativity & op.Associativity;\n if (assoc == Associativity.Left || prevOp.Type == OperatorType.Prefix) goto Break;\n if (assoc == Associativity.Right) goto Continue;\n } else {\n if (prevOp.Type == OperatorType.Infix) goto Continue;\n Debug.Assert(prevOp.Type == OperatorType.Prefix && op.Type == OperatorType.Postfix);\n if ((prevOp.Associativity | op.Associativity) != Associativity.None) goto Break;\n }\n HandlePossibleConflict(ref prevOpData, op, ref reply, stream);\n } else {\n error = ErrorMessageList.Merge(reply.Error, ExpectedInfixOrPostfixOperator);\n reply.Error = error;\n }\n ReturnNull:\n op = null;\n Break:\n return op;\n Continue:\n return ParseExpressionContinue(ref prevOpData, op, ref reply, stream);\n }\n\n /// Parses the following prefix operators, plus the expression the operators apply to.\n private\n Operator\n ParsePrefixOp(ref OperatorData prevOpData,\n Operator op,\n ref Reply reply,\n CharStream stream)\n {\n var opData = new OperatorData();\n opData.Line = stream.Line;\n opData.LineBegin = stream.LineBegin;\n opData.IndexToken = stream.IndexToken;\n opData.Operator = op;\n var userState = stream.UserState;\n #if DEBUG\n var ok = stream.Skip(op.String);\n Debug.Assert(ok);\n #else\n stream.Skip((uint)op.String.Length);\n #endif\n var stateTag = stream.StateTag;\n var asReply = op.AfterStringParser.Invoke(stream);\n if (asReply.Status == ReplyStatus.Ok) {\n opData.AfterStringValue = asReply.Result;\n var prevOp = prevOpData.Operator;\n if ( prevOp.Precedence != op.Precedence\n || prevOp.Type != OperatorType.Prefix\n || (prevOp.Associativity | op.Associativity) != Associativity.None)\n {\n reply.Error = asReply.Error;\n var nextOp = ParseExpression(ref opData, ref reply, stream);\n if (reply.Status == ReplyStatus.Ok)\n reply.Result = op.Mapping1.Invoke(opData.AfterStringValue, reply.Result);\n return nextOp;\n }\n // backtrack to the beginning of the operator\n stream.Seek(opData.IndexToken);\n stream.SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(opData.Line);\n stream.SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(opData.LineBegin);\n stream.UserState = userState;\n stream.StateTag = stateTag - 1;\n ReportConflict(ref prevOpData, op, asReply.Result, ref reply, stream);\n return null;\n } else if (asReply.Status == ReplyStatus.Error && stateTag == stream.StateTag) {\n // backtrack to the beginning of the operator\n stream.Seek(opData.IndexToken);\n stream.StateTag = stateTag - 1;\n return ErrorOp;\n } else {\n reply.Error = asReply.Error;\n reply.Status = asReply.Status;\n return null;\n }\n }\n\n /// Parses (higher-precedence) infix and postfix operators after the first term, together with the argument expressions.\n private\n Operator\n ParseExpressionContinue(ref OperatorData prevOpData,\n Operator op,\n ref Reply reply,\n CharStream stream)\n {\n var opData = new OperatorData();\n for (;;) {\n opData.Line = stream.Line;\n opData.LineBegin = stream.LineBegin;\n opData.IndexToken = stream.IndexToken;\n opData.Operator = op;\n #if DEBUG\n var ok = stream.Skip(op.String);\n Debug.Assert(ok);\n #else\n stream.Skip((uint)op.String.Length);\n #endif\n var stateTag = stream.StateTag;\n var asReply = op.AfterStringParser.Invoke(stream);\n if (asReply.Status == ReplyStatus.Ok) {\n opData.AfterStringValue = asReply.Result;\n reply.Error = asReply.Error;\n if (op.Type == OperatorType.Infix) {\n var result1 = reply.Result;\n if (!op.IsTernary) {\n var nextOp = ParseExpression(ref opData, ref reply, stream);\n if (reply.Status == ReplyStatus.Ok)\n reply.Result = op.Mapping2.Invoke(opData.AfterStringValue, result1, reply.Result);\n op = nextOp;\n if (op == null) break;\n goto CheckNextOp;\n } else {\n ParseExpression(ref ZeroPrecedenceOperatorData, ref reply, stream);\n if (reply.Status != ReplyStatus.Ok) goto ReturnNull;\n var result2 = reply.Result;\n if (stream.Skip(op.TernaryRightString)) {\n stateTag = stream.StateTag;\n asReply = op.AfterTernaryRightStringParser.Invoke(stream);\n if (asReply.Status == ReplyStatus.Ok) {\n reply.Error = asReply.Error;\n var nextOp = ParseExpression(ref opData, ref reply, stream);\n if (reply.Status == ReplyStatus.Ok)\n reply.Result = op.Mapping3.Invoke(opData.AfterStringValue, asReply.Result, result1, result2, reply.Result);\n op = nextOp;\n if (op == null) break;\n goto CheckNextOp;\n } else if (asReply.Status != ReplyStatus.Error || stateTag != stream.StateTag) {\n reply.Error = asReply.Error;\n reply.Status = asReply.Status;\n goto ReturnNull;\n } else {\n // backtrack\n stream.Skip(-op.TernaryRightString.Length);\n stream.StateTag -= 2;\n }\n }\n HandleMissingTernary2ndStringError(ref opData, ref reply, stream);\n goto ReturnNull;\n }\n } else {\n Debug.Assert(op.Type == OperatorType.Postfix);\n reply.Result = op.Mapping1.Invoke(opData.AfterStringValue, reply.Result);\n var lastOp = op;\n op = PeekOp(stream, RhsOps);\n // we check for adjacent postfix operators here ...\n if (op != null) {\n if (op.Type == OperatorType.Postfix && lastOp.Precedence <= op.Precedence) {\n if ( lastOp.Precedence < op.Precedence\n || (lastOp.Associativity | op.Associativity) != Associativity.None) continue;\n // ... so we can report conflicting postfix operators\n HandlePossibleConflict(ref opData, op, ref reply, stream);\n goto ReturnNull;\n }\n } else {\n reply.Error = ErrorMessageList.Merge(reply.Error, ExpectedInfixOrPostfixOperator);\n break;\n }\n }\n CheckNextOp:\n var prevOp = prevOpData.Operator;\n if (prevOp.Precedence < op.Precedence) continue;\n if (prevOp.Precedence > op.Precedence) break;\n // prevOp.Precedence == op.Precedence\n if (op.Type == OperatorType.Infix) {\n var assoc = prevOp.Associativity & op.Associativity;\n if (assoc == Associativity.Left || prevOp.Type == OperatorType.Prefix) break;\n if (assoc == Associativity.Right) continue;\n } else { // op.OperatorType == OperatorType.Postfix\n if (prevOp.Type == OperatorType.Infix) continue;\n Debug.Assert(prevOp.Type == OperatorType.Prefix);\n if ((prevOp.Associativity | op.Associativity) != Associativity.None) break;\n }\n HandlePossibleConflict(ref prevOpData, op, ref reply, stream);\n } else { // asReply.Status != ReplyStatus.Ok\n if (asReply.Status == ReplyStatus.Error && stateTag == stream.StateTag) {\n // backtrack\n stream.Seek(opData.IndexToken);\n stream.StateTag -= 2;\n reply.Error = ErrorMessageList.Merge(reply.Error, ExpectedInfixOrPostfixOperator);\n } else {\n reply.Error = asReply.Error;\n reply.Status = asReply.Status;\n }\n }\n ReturnNull:\n op = null;\n break;\n }\n return op;\n }\n\n private void HandleMissingTernary2ndStringError(ref OperatorData opData,\n ref Reply reply,\n CharStream stream)\n {\n var firstStringIndex = opData.IndexToken.GetIndex(stream);\n var firstStringColumn = firstStringIndex - opData.LineBegin + 1;\n var firstStringPos = new Position(stream.Name, firstStringIndex, opData.Line, firstStringColumn);\n var secondStringPos = stream.Position;\n var error1 = ExpectedInfixOrPostfixOperator;\n var error2 = MissingTernary2ndStringErrorFormatter.Invoke(Tuple.Create(firstStringPos, secondStringPos, (TernaryOperator)opData.Operator, opData.AfterStringValue));\n reply.Error = ErrorMessageList.Merge(reply.Error, ErrorMessageList.Merge(error1, error2));\n reply.Status = ReplyStatus.Error;\n }\n\n private void HandlePossibleConflict(ref OperatorData prevOpData,\n Operator op,\n ref Reply reply,\n CharStream stream)\n {\n // \"possible\" conflict, because it's not a conflict when the\n // after-string-parser fails without changing the parser state.\n var state = stream.State;\n var ok = stream.Skip(op.String);\n Debug.Assert(ok);\n var stateTag = stream.StateTag;\n var asReply = op.AfterStringParser.Invoke(stream);\n if (asReply.Status == ReplyStatus.Ok) {\n stream.BacktrackTo(ref state);\n ReportConflict(ref prevOpData, op, asReply.Result, ref reply, stream);\n } else if (asReply.Status == ReplyStatus.Error && stateTag == stream.StateTag) {\n // backtrack and ignore the operator\n stream.BacktrackTo(ref state);\n reply.Error = ErrorMessageList.Merge(reply.Error, ExpectedInfixOrPostfixOperator);\n } else {\n // report AfterStringParser error instead of conflict\n reply.Error = asReply.Error;\n reply.Status = asReply.Status;\n }\n }\n\n private void ReportConflict(ref OperatorData prevOpData,\n Operator op,\n TAfterString afterStringValue,\n ref Reply reply,\n CharStream stream)\n {\n var prevOpIndex = prevOpData.IndexToken.GetIndex(stream);\n var prevOpColumn = prevOpIndex - prevOpData.LineBegin + 1;\n var prevOpPos = new Position(stream.Name, prevOpIndex, prevOpData.Line, prevOpColumn);\n var error = _OperatorConflictErrorFormatter.Invoke(\n Tuple.Create(prevOpPos, prevOpData.Operator, prevOpData.AfterStringValue),\n Tuple.Create(stream.Position, op, afterStringValue));\n reply.Error = ErrorMessageList.Merge(reply.Error, error);\n reply.Status = ReplyStatus.Error;\n }\n\n private sealed class DefaultMissingTernary2ndStringErrorFormatter\n : FSharpFunc, TAfterString>, ErrorMessageList>\n {\n public override ErrorMessageList Invoke(Tuple, TAfterString> value) {\n var position1 = value.Item1;\n var position2 = value.Item2;\n var op = value.Item3;\n return Errors.MissingTernary2ndString(position1, position2, op);\n }\n }\n\n private sealed class DefaultOperatorConflictErrorFormatter\n : OptimizedClosures.FSharpFunc, TAfterString>,\n Tuple, TAfterString>,\n ErrorMessageList>\n {\n public override ErrorMessageList Invoke(Tuple, TAfterString> arg1, Tuple, TAfterString> arg2) {\n return Errors.OperatorsConflict(arg1.Item1, arg1.Item2, arg2.Item1, arg2.Item2);\n }\n }\n\n}\n\n\n}" }, { "path": "FParsecCS/Position.cs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2009\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nnamespace FParsec {\n\npublic sealed class Position : IEquatable, IComparable, IComparable {\n public long Index { get; private set; }\n public long Line { get; private set; }\n public long Column { get; private set; }\n public string StreamName { get; private set; }\n\n public Position(string streamName, long index, long line, long column) {\n StreamName = streamName; Index = index; Line = line; Column = column;\n }\n\n public override string ToString() {\n var ln = String.IsNullOrEmpty(StreamName) ? \"(Ln: \" : Text.Escape(StreamName, \"\", \"(\\\"\", \"\\\", Ln: \", \"\", '\"');\n return ln + Line.ToString() + \", Col: \" + Column.ToString() + \")\";\n }\n\n public override bool Equals(object obj) {\n return Equals(obj as Position);\n }\n public bool Equals(Position other) {\n return (object)this == (object)other\n || ( (object)other != null\n && Index == other.Index\n && Line == other.Line\n && Column == other.Column\n && StreamName == other.StreamName);\n }\n public static bool operator==(Position left, Position right) {\n return (object)left == null ? (object)right == null : left.Equals(right);\n }\n public static bool operator!=(Position left, Position right) { return !(left == right); }\n\n public override int GetHashCode() {\n return Index.GetHashCode();\n }\n\n public static int Compare(Position left, Position right) {\n if ((object)left != null) return left.CompareTo(right);\n return (object)right == null ? 0 : -1;\n }\n\n public int CompareTo(Position other) {\n if ((object)this == (object)other) return 0;\n if ((object)other == null) return 1;\n int r = String.CompareOrdinal(StreamName, other.StreamName);\n if (r != 0) return r;\n r = Line.CompareTo(other.Line);\n if (r != 0) return r;\n r = Column.CompareTo(other.Column);\n if (r != 0) return r;\n return Index.CompareTo(other.Index);\n }\n int IComparable.CompareTo(object value) {\n Position position = value as Position;\n if ((object)position != null) return CompareTo(position);\n if (value == null) return 1;\n throw new ArgumentException(\"Object must be of type Position.\");\n }\n}\n\n}\n" }, { "path": "FParsecCS/Properties/AssemblyInfo.cs", "content": "using System.Reflection;\nusing System.Runtime.CompilerServices;\nusing System.Runtime.InteropServices;\n\n[assembly: ComVisible(false)]\n\n#if LOW_TRUST\n [assembly: System.Security.AllowPartiallyTrustedCallers]\n [assembly: System.Security.SecurityTransparent]\n#endif\n\n[assembly: InternalsVisibleTo (\"FParsec\" + FParsec.CommonAssemblyInfo.StrongNamePublicKey)]\n[assembly: InternalsVisibleTo (FParsec.CommonAssemblyInfo.TestAssemblyName + FParsec.CommonAssemblyInfo.StrongNamePublicKey)]\n\nnamespace FParsec {\n\ninternal static partial class CommonAssemblyInfo {\n public const string TestAssemblyName = \"Test\";\n\n#if STRONG_NAME\n public const string StrongNamePublicKey =\n \", PublicKey=002400000480000094000000060200000024000052534131000400000100010077c6be48a40f5b\" +\n \"194ec9f992e5b512bbbba33e211354d9ee50c3214decddad8356470a9a19a9ee84637cbd6ff690\" +\n \"9527d3973741dbe0a69b1461eeae774af9a78de45618ffd6fe7c7d52e0441b92f3bc7e8fb5757f\" +\n \"b8b1611a0b6b8c9f9ef64edcf51d44218ae040f3015373fd261d30f8e1f5a1f914fd9ebcde7d7e\" +\n \"f42dbaa5\";\n#else\n public const string StrongNamePublicKey = \"\";\n#endif\n};\n\n}\n" }, { "path": "FParsecCS/Reply.cs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nnamespace FParsec {\n\npublic enum ReplyStatus {\n Ok = 1,\n Error = 0,\n FatalError = -1\n}\n\n[System.Diagnostics.DebuggerDisplay(\"{GetDebuggerDisplay(),nq}\")]\npublic struct Reply : IEquatable> {\n public ErrorMessageList Error;\n public TResult Result;\n public ReplyStatus Status;\n\n public Reply(TResult result) {\n Result = result;\n Error = null;\n Status = ReplyStatus.Ok;\n }\n\n public Reply(ReplyStatus status, ErrorMessageList error) {\n Status = status;\n Error = error;\n Result = default(TResult);\n }\n\n public Reply(ReplyStatus status, TResult result, ErrorMessageList error) {\n Status = status;\n Error = error;\n Result = result;\n }\n\n public override bool Equals(object other) {\n if (!(other is Reply)) return false;\n return Equals((Reply) other);\n }\n public bool Equals(Reply other) {\n return Status == other.Status\n && (Status != ReplyStatus.Ok || FastGenericEqualityERComparer.Instance.Equals(Result, other.Result))\n && Error == other.Error;\n }\n public override int GetHashCode() {\n return (int)Status\n ^ (Status != ReplyStatus.Ok ? 0 : FastGenericEqualityERComparer.Instance.GetHashCode(Result));\n }\n public static bool operator==(Reply r1, Reply r2) { return r1.Equals(r2); }\n public static bool operator!=(Reply r1, Reply r2) { return !r1.Equals(r2); }\n\n private string GetDebuggerDisplay() {\n if (Status == ReplyStatus.Ok) {\n string result;\n if (Result == null)\n result = typeof(TResult) == typeof(Microsoft.FSharp.Core.Unit) ? \"()\" : \"null\";\n else if (typeof(TResult) == typeof(string))\n result = Text.DoubleQuote(Result.ToString());\n else\n result = Result.ToString();\n\n return Error == null\n ? \"Reply(\" + result + \")\"\n : \"Reply(Ok, \" + result + \", \" + ErrorMessageList.GetDebuggerDisplay(Error) + \")\";\n } else {\n var status = Status == ReplyStatus.Error ? \"Error\" :\n Status == ReplyStatus.FatalError ? \"FatalError\" :\n \"(ReplyStatus)\" + ((int)Status).ToString();\n\n return Error == null\n ? \"Reply(\" + status + \", NoErrorMessages)\"\n : \"Reply(\" + status + \", \" + ErrorMessageList.GetDebuggerDisplay(Error) + \")\";\n }\n }\n}\n\n}" }, { "path": "FParsecCS/StringBuffer.cs", "content": "// Copyright (c) Stephan Tolksdorf 2009\n// License: Simplified BSD License. See accompanying documentation.\n\n#if !LOW_TRUST\n\n#if DEBUG\n #define DEBUG_STRINGBUFFER\n#endif\n\nusing System;\nusing System.Runtime.InteropServices;\nusing System.Diagnostics;\n\nnamespace FParsec {\n\n/// A substring of a pinned string on the large object heap.\n/// StringBuffers are cached in a pool and hence need to be properly disposed.\ninternal unsafe sealed class StringBuffer : IDisposable {\n private PoolSegment Segment;\n public string String { get { return Segment == null ? \"\" : Segment.String; } }\n public char* StringPointer { get { return Segment == null ? null : Segment.StringPointer; } }\n public int Index { get; private set; }\n public int Length { get; private set; }\n\n private StringBuffer(PoolSegment segment, int index, int length) {\n Segment = segment;\n Index = index;\n Length = length;\n }\n\n private sealed class FreeChunk {\n public PoolSegment Segment;\n\n // free chunks in each segment form a doubly-linked list ordered by index\n public FreeChunk PrevInSegment;\n public FreeChunk NextInSegment;\n\n public static FreeChunk Smallest;\n public static FreeChunk Largest;\n\n // all free chunks together form a doubly-linked list ordered by size\n public FreeChunk PrevInSize;\n public FreeChunk NextInSize;\n\n public int Index;\n public int Size;\n\n public FreeChunk(PoolSegment segment, int index, int size) {\n Debug.Assert(segment.FirstFreeChunk == null && index >= 0 && size > 0 && index + size <= segment.Size);\n Segment = segment;\n Index = index;\n Size = size;\n segment.FirstFreeChunk = this;\n InsertIntoSizeList();\n }\n\n public FreeChunk(PoolSegment segment, FreeChunk prevInSegment, FreeChunk nextInSegment, int index, int size) {\n Debug.Assert(index >= 0 && size > 0 && index + size <= segment.Size);\n Segment = segment;\n Index = index;\n Size = size;\n PrevInSegment = prevInSegment;\n NextInSegment = nextInSegment;\n if (prevInSegment != null) {\n Debug.Assert(prevInSegment.Index + prevInSegment.Size < index);\n prevInSegment.NextInSegment = this;\n } else {\n Debug.Assert(segment.FirstFreeChunk == nextInSegment);\n segment.FirstFreeChunk = this;\n }\n if (nextInSegment != null) {\n Debug.Assert(index + size < nextInSegment.Index);\n nextInSegment.PrevInSegment = this;\n }\n InsertIntoSizeList();\n }\n\n private void InsertIntoSizeList() {\n var largest = FreeChunk.Largest;\n if (largest != null) {\n if (largest.Size <= Size) {\n largest.NextInSize = this;\n PrevInSize = largest;\n FreeChunk.Largest = this;\n } else {\n NextInSize = largest;\n var prev = largest.PrevInSize;\n largest.PrevInSize = this;\n if (prev != null) {\n PrevInSize = prev;\n prev.NextInSize = this;\n if (Size < prev.Size) MoveAfterSizeHasDecreased();\n } else FreeChunk.Smallest = this;\n }\n } else {\n FreeChunk.Smallest = this;\n FreeChunk.Largest = this;\n }\n }\n\n public void Remove() {\n var prev = PrevInSegment;\n var next = NextInSegment;\n if (prev != null) prev.NextInSegment = next;\n else Segment.FirstFreeChunk = next;\n if (next != null) next.PrevInSegment = prev;\n\n prev = PrevInSize;\n next = NextInSize;\n if (prev != null) prev.NextInSize = next;\n else Smallest = next;\n if (next != null) next.PrevInSize = prev;\n else Largest = prev;\n }\n\n // the following two methods are dual to each other,\n // i.e. one can be transformed into the other by way of simple search & replace\n public void MoveAfterSizeHasDecreased() {\n Debug.Assert(Size < PrevInSize.Size);\n var prev = PrevInSize;\n var next = NextInSize;\n if (next != null) next.PrevInSize = prev;\n else Largest = prev;\n prev.NextInSize = next;\n next = prev;\n prev = prev.PrevInSize;\n while (prev != null && prev.Size > Size) {\n next = prev;\n prev = prev.PrevInSize;\n }\n NextInSize = next;\n next.PrevInSize = this;\n PrevInSize = prev;\n if (prev != null) prev.NextInSize = this;\n else Smallest = this;\n }\n\n public void MoveAfterSizeHasIncreased() {\n Debug.Assert(Size > NextInSize.Size);\n var next = NextInSize;\n var prev = PrevInSize;\n if (prev != null) prev.NextInSize = next;\n else Smallest = next;\n next.PrevInSize = prev;\n prev = next;\n next = next.NextInSize;\n while (next != null && next.Size < Size) {\n prev = next;\n next = next.NextInSize;\n }\n PrevInSize = prev;\n prev.NextInSize = this;\n NextInSize = next;\n if (next != null) next.PrevInSize = this;\n else Largest = this;\n }\n }\n\n private const int MinChunkSize = 1536; // 3 * 2^9\n // segment sizes must be multiple of MinChunkSize and large enough to allocated on the LargeObjectHeap\n private const int FirstSegmentSmallSize = 42 * MinChunkSize; // 64 512\n private const int FirstSegmentLargeSize = 128 * MinChunkSize; // 3 * 2^16 = 196 608 (default CharStream block size)\n private const int MaxSegmentSize = 640 * MinChunkSize; // 983 040\n\n private static int MaxNumberOfUnusedSegments = 3;\n\n private static int NumberOfUnusedSegments;\n\n private sealed class PoolSegment : IDisposable {\n // segments form a doubly-linked list in the order they were constructed\n\n /// the last allocated segment\n private static PoolSegment Last;\n\n private PoolSegment Next;\n private PoolSegment Prev;\n\n public string String { get; private set; }\n /// String.Length - x, where x > 0\n public int Size { get; private set; }\n public char* StringPointer { get; private set; }\n private GCHandle StringHandle;\n\n public FreeChunk FirstFreeChunk;\n\n public PoolSegment(int size, int firstBufferSize) {\n Debug.Assert(firstBufferSize > 0 && firstBufferSize <= size && (size <= MaxSegmentSize || firstBufferSize == size));\n // + 1, so that no chunk can span the full string, which helps avoiding accidentally passing a reference to the internal buffer string to the \"outside world\"\n String = new String('\\u0000', size + 1);\n Size = size;\n StringHandle = GCHandle.Alloc(String, GCHandleType.Pinned);\n StringPointer = (char*)StringHandle.AddrOfPinnedObject();\n if (Last != null) {\n Last.Next = this;\n Prev = Last;\n }\n Last = this;\n if (firstBufferSize < size)\n new FreeChunk(this, firstBufferSize, size - firstBufferSize); // inserts itself into the lists\n }\n\n public void Dispose() {\n if (StringPointer != null) {\n Debug.Assert(FirstFreeChunk == null);\n if (FirstFreeChunk != null) throw new InvalidOperationException();\n if (Prev != null) Prev.Next = Next;\n if (Next != null) Next.Prev = Prev;\n else Last = Prev;\n StringPointer = null;\n StringHandle.Free();\n }\n }\n\n public static StringBuffer AllocateStringBufferInNewSegment(int length) {\n int segmentSize = length > MaxSegmentSize\n ? length\n : (Last == null && length <= FirstSegmentLargeSize)\n ? (length <= FirstSegmentSmallSize ? FirstSegmentSmallSize : FirstSegmentLargeSize)\n : MaxSegmentSize;\n return new StringBuffer(new PoolSegment(segmentSize, length), 0, length);\n }\n\n [Conditional(\"DEBUG_STRINGBUFFER\")]\n public void AssertIntegrity() {\n Debug.Assert(StringPointer != null);\n int sumOfSegmentSizes = 0;\n { // check list of segments\n var segment = Last;\n Debug.Assert(segment.Next == null);\n var prev = segment.Prev;\n sumOfSegmentSizes += segment.Size;\n bool visitedThis = segment == this;\n while (prev != null) {\n Debug.Assert(segment == prev.Next);\n segment = prev;\n prev = prev.Prev;\n sumOfSegmentSizes += segment.Size;\n visitedThis = visitedThis || segment == this;\n }\n Debug.Assert(visitedThis);\n }\n { // check segment list of free chunks ordered by index\n var chunk = FirstFreeChunk;\n if (chunk != null) {\n Debug.Assert( chunk.Index >= 0 && chunk.Size > 0\n && (chunk.PrevInSize != null ? chunk.Size >= chunk.PrevInSize.Size : chunk == FreeChunk.Smallest)\n && (chunk.NextInSize != null ? chunk.Size <= chunk.NextInSize.Size : chunk == FreeChunk.Largest));\n int chunkEnd = chunk.Index + chunk.Size;\n var next = chunk.NextInSegment;\n while (next != null) {\n Debug.Assert( (chunk == next.PrevInSegment && chunkEnd < next.Index && next.Size > 0)\n && (next.PrevInSize != null ? next.Size >= next.PrevInSize.Size : next == FreeChunk.Smallest)\n && (next.NextInSize != null ? next.Size <= next.NextInSize.Size : next == FreeChunk.Largest));\n chunk = next;\n chunkEnd = chunk.Index + chunk.Size;\n next = chunk.NextInSegment;\n }\n Debug.Assert(chunkEnd <= Size);\n }\n }\n { // check global list of free chunks ordered by size\n int free = 0;\n var chunk = FreeChunk.Smallest;\n if (chunk == null) Debug.Assert(FreeChunk.Largest == null);\n else {\n Debug.Assert(chunk.Size > 0 && chunk.PrevInSize == null);\n free += chunk.Size;\n var next = chunk.NextInSize;\n while (next != null) {\n Debug.Assert(chunk == next.PrevInSize && chunk.Size <= next.Size);\n chunk = next;\n free += chunk.Size;\n next = chunk.NextInSize;\n }\n Debug.Assert(chunk == FreeChunk.Largest);\n }\n Debug.Assert(Allocated == sumOfSegmentSizes - free);\n }\n }\n }\n\n\n /// Sum of the lengths of all currently allocated StringBuffers\n private static int Allocated = 0;\n private static object SyncRoot = new Object();\n\n public static StringBuffer Create(int minLength) {\n int size = unchecked(minLength + (MinChunkSize - 1));\n if (size > (MinChunkSize - 1)) { // minLength > 0 && minLength <= System.Int32.MaxValue - (MinChunkSize - 1)\n size -= (int)((uint)size%(uint)MinChunkSize); // round down to multiple of MinChunkSize\n lock (SyncRoot) {\n Allocated += size;\n FreeChunk chunk = FreeChunk.Largest;\n if (chunk != null) { // find smallest free chunk that is large enough to hold the buffer\n if (size > 10*MinChunkSize) {\n var prev = chunk.PrevInSize;\n while (prev != null && prev.Size >= size) {\n chunk = prev;\n prev = prev.PrevInSize;\n }\n } else {\n chunk = FreeChunk.Smallest;\n var next = chunk.NextInSize;\n while (chunk.Size < size && next != null) {\n chunk = next;\n next = next.NextInSize;\n }\n }\n if (size <= chunk.Size) {\n int index = chunk.Index;\n if (index == 0 && chunk.Size == chunk.Segment.Size) --NumberOfUnusedSegments;\n if (size != chunk.Size) {\n chunk.Index += size;\n chunk.Size -= size;\n var prev = chunk.PrevInSize;\n if (prev != null && chunk.Size < prev.Size) chunk.MoveAfterSizeHasDecreased();\n } else chunk.Remove();\n chunk.Segment.AssertIntegrity();\n return new StringBuffer(chunk.Segment, index, size);\n }\n }\n return PoolSegment.AllocateStringBufferInNewSegment(size);\n }\n } else {\n if (minLength < 0) throw new ArgumentOutOfRangeException(\"minLength\", \"minLength is negative.\");\n else if (minLength > 0) throw new ArgumentOutOfRangeException(\"minLength\", \"minLength is too large. The maximum string buffer length is approximately 2^30.\");\n return new StringBuffer(null, 0, 0);\n }\n }\n\n public void Dispose() {\n int size = Length;\n Length = -1;\n if (size > 0) {\n lock (SyncRoot) {\n Allocated -= size;\n if (size <= MaxSegmentSize) {\n FreeChunk prev = null;\n FreeChunk next = Segment.FirstFreeChunk;\n while (next != null && Index > next.Index) {\n prev = next;\n next = next.NextInSegment;\n }\n if (prev == null || prev.Index + prev.Size != Index) {\n if (next != null && Index + size == next.Index) {\n next.Index = Index;\n next.Size += size;\n var nextNext = next.NextInSize;\n if (nextNext != null && next.Size > nextNext.Size) next.MoveAfterSizeHasIncreased();\n } else {\n new FreeChunk(Segment, prev, next, Index, size); // inserts itself into the lists\n }\n } else {\n if (next != null && Index + size == next.Index) {\n prev.Size += size + next.Size;\n next.Remove();\n } else {\n prev.Size += size;\n }\n if (prev.NextInSize != null && prev.Size > prev.NextInSize.Size) prev.MoveAfterSizeHasIncreased();\n }\n Segment.AssertIntegrity();\n var first = Segment.FirstFreeChunk;\n if (first.Size == Segment.Size && ++NumberOfUnusedSegments > MaxNumberOfUnusedSegments) {\n --NumberOfUnusedSegments;\n first.Remove();\n Segment.Dispose();\n }\n } else { // size > MaxSegmentSize\n Debug.Assert(size == Segment.Size);\n Segment.Dispose();\n }\n }\n }\n }\n}\n\n}\n\n#endif" }, { "path": "FParsecCS/Strings.cs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\n\nnamespace FParsec {\n\ninternal static class Strings {\n\n static internal string Quote(string stringToQuote) {\n return Text.SingleQuote(stringToQuote);\n }\n static internal string Quote(string prefix, string stringToQuote, string postfix) {\n return Text.SingleQuote(prefix, stringToQuote, postfix);\n }\n\n static internal string AsciiQuote(string prefix, string stringToQuote, string postfix) {\n return Text.AsciiEscape(stringToQuote, prefix, \"'\", \"'\", postfix, '\\'');\n }\n\n static internal string QuoteCaseInsensitive(string caseInsensitiveStringToQuote) {\n return Quote(\"\", caseInsensitiveStringToQuote, \" (case-insensitive)\");\n }\n\n static private string OrdinalEnding(int value) {\n if (value < 1) throw new ArgumentOutOfRangeException(\"value\", \"The value must be greater than 0.\");\n var n100 = value%100;\n var n10 = value%10;\n if (n100 < 11 || n100 > 13) {\n if (n10 == 1) return \"st\";\n if (n10 == 2) return \"nd\";\n if (n10 == 3) return \"rd\";\n }\n return \"th\";\n }\n\n public static readonly string EndOfInput = \"end of input\";\n public static readonly string AnyChar = \"any char\";\n public static readonly string Whitespace = \"whitespace\";\n public static readonly string AsciiUppercaseLetter = \"Ascii uppercase letter\";\n public static readonly string AsciiLowercaseLetter = \"Ascii lowercase letter\";\n public static readonly string AsciiLetter = \"Ascii letter\";\n public static readonly string UppercaseLetter = \"uppercase letter\";\n public static readonly string LowercaseLetter = \"lowercase letter\";\n public static readonly string Letter = \"letter\";\n public static readonly string BinaryDigit = \"binary digit\";\n public static readonly string OctalDigit = \"octal digit\";\n public static readonly string DecimalDigit = \"decimal digit\";\n public static readonly string HexadecimalDigit = \"hexadecimal digit\";\n public static readonly string Newline = \"newline\";\n public static readonly string Tab = \"tab\";\n public static readonly string FloatingPointNumber = \"floating-point number\";\n public static readonly string Int64 = \"integer number (64-bit, signed)\";\n public static readonly string Int32 = \"integer number (32-bit, signed)\";\n public static readonly string Int16 = \"integer number (16-bit, signed)\";\n public static readonly string Int8 = \"integer number (8-bit, signed)\";\n public static readonly string UInt64 = \"integer number (64-bit, unsigned)\";\n public static readonly string UInt32 = \"integer number (32-bit, unsigned)\";\n public static readonly string UInt16 = \"integer number (16-bit, unsigned)\";\n public static readonly string UInt8 = \"integer number (8-bit, unsigned)\";\n\n public static readonly string Identifier = \"identifier\";\n public static readonly string IdentifierContainsInvalidCharacterAtIndicatedPosition = \"The identifier contains an invalid character at the indicated position.\";\n\n\n public static readonly string NumberOutsideOfDoubleRange = \"This number is outside the allowable range for double precision floating-pointer numbers.\";\n\n public static readonly string NumberOutsideOfInt64Range = \"This number is outside the allowable range for signed 64-bit integers.\";\n public static readonly string NumberOutsideOfInt32Range = \"This number is outside the allowable range for signed 32-bit integers.\";\n public static readonly string NumberOutsideOfInt16Range = \"This number is outside the allowable range for signed 16-bit integers.\";\n public static readonly string NumberOutsideOfInt8Range = \"This number is outside the allowable range for signed 8-bit integers.\";\n\n public static readonly string NumberOutsideOfUInt64Range = \"This number is outside the allowable range for unsigned 64-bit integers.\";\n public static readonly string NumberOutsideOfUInt32Range = \"This number is outside the allowable range for unsigned 32-bit integers.\";\n public static readonly string NumberOutsideOfUInt16Range = \"This number is outside the allowable range for unsigned 16-bit integers.\";\n public static readonly string NumberOutsideOfUInt8Range = \"This number is outside the allowable range for unsigned 8-bit integers.\";\n\n public static readonly string InfixOperator = \"infix operator\";\n public static readonly string TernaryOperator = \"ternary operator\";\n public static readonly string PrefixOperator = \"prefix operator\";\n public static readonly string PostfixOperator = \"postfix operator\";\n\n private static readonly string AnyCharIn1 = \"any char in \";\n private static readonly string AnyCharIn2 = \"\";\n\n private static readonly string AnyCharNotIn1 = \"any char not in \";\n private static readonly string AnyCharNotIn2 = \"\";\n\n private static readonly string AnySequenceOfNChars1 = \"any sequence of \";\n private static readonly string AnySequenceOfNChars2 = \" chars\";\n\n private static readonly string CouldNotFindString1 = \"Could not find the string \";\n private static readonly string CouldNotFindString2 = \".\";\n\n private static readonly string CouldNotFindCaseInsensitiveString1 = \"Could not find the case-insensitive string \";\n private static readonly string CouldNotFindCaseInsensitiveString2 = \".\";\n\n private static readonly string StringMatchingRegex1 = \"string matching the regex \";\n private static readonly string StringMatchingRegex2 = \"\";\n\n private static readonly string ErrorPositionStreamNameFormat = \" {0}:\";\n private static readonly string ErrorPositionUnaccountedNewlinesFormat = \" (+{0})\";\n private static readonly string ErrorPositionUtf16ColumnFormat = \" (UTF16-Col: {0})\";\n private static readonly string ErrorPositionFormat = \"Error in{0} Ln: {1}{2} Col: {3}{4}\";\n // 0: ErrorPositionStreamName or \"\"\n // 1: line\n // 2: ErrorPositionUnaccountedNewlines or \"\"\n // 3: column\n // 4: ErrorPositionUtf16Col\n\n public static string ErrorPosition(Position position) {\n var name = string.IsNullOrEmpty(position.StreamName) ? \"\" : string.Format(ErrorPositionStreamNameFormat, position.StreamName);\n return string.Format(ErrorPositionFormat, name, position.Line, \"\", position.Column, \"\");\n }\n\n public static string ErrorPosition(Position position, int unaccountedNewlines, long column, long utf16Column) {\n var name = string.IsNullOrEmpty(position.StreamName) ? \"\" : string.Format(ErrorPositionStreamNameFormat, position.StreamName);\n var nlCorrection = unaccountedNewlines == 0 ? \"\" : string.Format(ErrorPositionUnaccountedNewlinesFormat, unaccountedNewlines);\n var utf16Col = column == utf16Column ? \"\" : string.Format(ErrorPositionUtf16ColumnFormat, utf16Column);\n return string.Format(ErrorPositionFormat, name, position.Line, nlCorrection, column, utf16Col);\n }\n\n public static readonly string Note = \"Note: \";\n public static readonly string Expecting = \"Expecting: \";\n public static readonly string Unexpected = \"Unexpected: \";\n public static readonly string Comma = \", \";\n public static readonly string Or = \" or \";\n public static readonly string And = \" and \";\n private static readonly string CompoundCouldNotBeParsedBecauseFormat = \"{0} could not be parsed because: \";\n\n public static string CompoundCouldNotBeParsedBecause(string compoundLabel) {\n return string.Format(CompoundCouldNotBeParsedBecauseFormat, compoundLabel);\n }\n\n public static readonly string ParserBacktrackedAfter = \"The parser backtracked after: \";\n public static readonly string OtherErrors = \"Other error messages: \";\n public static readonly string UnknownErrors = \"Unknown Error(s)\";\n public static readonly string Utf16ColumnCountOnlyCountsEachTabAs1Char = \" The UTF-16 column count only counts each tab as 1 char.\";\n public static readonly string ExactPositionBetweenCaretsDependsOnDisplayUnicodeCapabilities = \"The exact error position between the two ^ depends on the unicode capabilities of the display.\";\n public static readonly string ErrorOccurredAtEndOfInputStream = \"The error occurred at the end of the input stream.\";\n public static readonly string ErrorOccurredOnAnEmptyLine = \"The error occurred on an empty line.\";\n public static readonly string ErrorOccurredAtEndOfLine = \"The error occurred at the end of the line.\";\n public static readonly string ErrorOccurredAtSecondCharInNewline = \"The error occured at the 2nd char in the newline char sequence '\\r\\n'.\";\n\n private static readonly string NonAssociative = \"non-associative\";\n private static readonly string LeftAssociative = \"left-associative\";\n private static readonly string RightAssociative = \"right-associative\";\n\n private static readonly string OperatorToStringFormat = \"{0} {1} (precedence: {2}{3}{4})\";\n // 0: InfixOperator/TernaryOperator/...\n // 1: operator strings\n // 2: precedence\n // 3: Comma if 4 is not empty, otherwise empty\n // 4: LeftAssociative/RightAssociative/... or empty if operator is an associative prefix or postfix operator\n\n // It would be more precise to write \"UTF-16 colum\" here,\n // but that would probably only confuse users in most situations.\n private static readonly string RelativePositionOnTheSameLine = \"on the same line at column {0}\";\n private static readonly string RelativePositionOnPreviousLine = \"on the previous line column {0}\";\n private static readonly string RelativePositionOnLineAbove = \"{0} lines above column {1}\";\n private static readonly string RelativePositionOnDifferentLine = \"at (Ln: {0}, Col: {1} )\";\n private static readonly string RelativePositionInDifferentFile = \"at ({0}, Ln: {1}, Col: {2})\";\n\n private static readonly string OperatorsConflictsFormat = \"The {1} conflicts with the {0} {2}.\";\n // 0: previous operator\n // 1: current operator\n // 2: relative position of previous operator\n\n private static readonly string OperatorStringIsRightPartOfTernaryOperatorFormat = \"{0} is the right part of the ternary operator {1}. The left part is {2}.\";\n\n\n private static readonly string ColumnCountAssumesTabStopDistanceOfNChars1 = \"The column count assumes a tab stop distance of \";\n private static readonly string ColumnCountAssumesTabStopDistanceOfNChars2 = \" chars.\";\n\n private static readonly string ErrorOccurredAtNthCharInCombiningCharacterSequence1 = \"The error occurred at the \";\n private static readonly string ErrorOccurredAtNthCharInCombiningCharacterSequence2 = \" char in the combining character sequence \";\n private static readonly string ErrorOccurredAtNthCharInCombiningCharacterSequence3 = \".\";\n\n private static readonly string InputContainsAtLeastNUnaccountedNewlines1 = \"The input contains at least \";\n private static readonly string InputContainsAtLeastNUnaccountedNewlines2Singular = \" newline in the input that wasn't properly registered in the parser stream state.\";\n private static readonly string InputContainsAtLeastNUnaccountedNewlines2Plural = \" newlines in the input that weren't properly registered in the parser stream state.\";\n\n private static readonly string ErrorOccurredAtBeginningOfSurrogatePair1 = \"The error occurred at the beginning of the surrogate pair \";\n private static readonly string ErrorOccurredAtBeginningOfSurrogatePair2 = \".\";\n\n private static readonly string ErrorOccurredAtSecondCharInSurrogatePair1 = \"The error occurred at the second char in the surrogate pair \";\n private static readonly string ErrorOccurredAtSecondCharInSurrogatePair2 = \".\";\n\n private static readonly string CharAtErrorPositionIsIsolatedHighSurrogate1 = \"The char at the error position ('\";\n private static readonly string CharAtErrorPositionIsIsolatedHighSurrogate2 = \"') is an isolated high surrogate.\";\n\n private static readonly string CharAtErrorPositionIsIsolatedLowSurrogate1 = \"The char at the error position ('\";\n private static readonly string CharAtErrorPositionIsIsolatedLowSurrogate2 = \"') is an isolated low surrogate.\";\n\n private static readonly string CharBeforeErrorPositionIsIsolatedHighSurrogate1 = \"The char before the error position ('\";\n private static readonly string CharBeforeErrorPositionIsIsolatedHighSurrogate2 = \"') is an isolated high surrogate.\";\n\n private static readonly string CharBeforeErrorPositionIsIsolatedLowSurrogate1 = \"The char before the error position ('\";\n private static readonly string CharBeforeErrorPositionIsIsolatedLowSurrogate2 = \"') is an isolated low surrogate.\";\n\n\n public static string AnyCharIn(string chars) {\n //return Quote(Strings.AnyCharIn1, chars, Strings.AnyCharIn2);\n return Strings.AnyCharIn1 + \"‘\" + chars + \"’\" + Strings.AnyCharIn2; // Review: Should we use different quotes if the string contains ‘ or ’ chars?\n }\n\n public static string AnyCharNotIn(string chars) {\n //return Quote(Strings.AnyCharNotIn1, chars, Strings.AnyCharNotIn2);\n return Strings.AnyCharNotIn1 + \"‘\" + chars + \"’\" + Strings.AnyCharNotIn2;\n }\n\n public static string StringMatchingRegex(string regexPattern) {\n return Quote(Strings.StringMatchingRegex1, regexPattern, Strings.StringMatchingRegex2);\n }\n\n public static string ExpectedAnySequenceOfNChars(int n) {\n return Strings.AnySequenceOfNChars1 + n.ToString() + Strings.AnySequenceOfNChars2;\n }\n\n public static string CouldNotFindString(string str) {\n return Quote(Strings.CouldNotFindString1, str, Strings.CouldNotFindString2);\n }\n\n public static string CouldNotFindCaseInsensitiveString(string str) {\n return Quote(Strings.CouldNotFindCaseInsensitiveString1, str, Strings.CouldNotFindCaseInsensitiveString2);\n }\n\n internal static string OperatorToString(Operator op) {\n var type = op.Type == OperatorType.Infix ? (op.IsTernary ? TernaryOperator : InfixOperator) :\n op.Type == OperatorType.Prefix ? PrefixOperator : PostfixOperator;\n var opString = op.IsTernary ? Quote(Quote(\"\", op.String, \" \"), op.TernaryRightString, \"\") : Quote(op.String);\n var comma = op.Type != OperatorType.Infix && op.IsAssociative ? \"\" : Comma;\n var assoc = op.Type != OperatorType.Infix\n ? (op.IsAssociative ? \"\" : NonAssociative)\n : (op.Associativity == Associativity.Left ? LeftAssociative :\n op.Associativity == Associativity.Right ? RightAssociative : NonAssociative);\n return String.Format(OperatorToStringFormat, type, opString, op.Precedence, comma, assoc);\n }\n\n private static string RelativePosition(Position previousPosition, Position currentPosition) {\n if (previousPosition.StreamName == currentPosition.StreamName) {\n if (previousPosition.Line == currentPosition.Line)\n return String.Format(RelativePositionOnTheSameLine, previousPosition.Column);\n long diff = currentPosition.Line - previousPosition.Line;\n if (diff == 1)\n return String.Format(RelativePositionOnPreviousLine, previousPosition.Column);\n if (diff <= 3)\n return String.Format(RelativePositionOnLineAbove, diff, previousPosition.Column);\n return String.Format(RelativePositionOnDifferentLine, previousPosition.Line, previousPosition.Column);\n }\n return String.Format(RelativePositionInDifferentFile, Quote(previousPosition.StreamName), previousPosition.Line, previousPosition.Column);\n }\n\n public static string OperatorsConflict(Position previousPosition, Operator previousOperator,\n Position currentPosition, Operator currentOperator)\n {\n var prevOpString = OperatorToString(previousOperator);\n var currentOpString = OperatorToString(currentOperator);\n var relativePosition = RelativePosition(previousPosition, currentPosition);\n return String.Format(OperatorsConflictsFormat, prevOpString, currentOpString, relativePosition);\n }\n\n public static string OperatorStringIsRightPartOfTernaryOperator(Position position1, Position position2, Operator op) {\n return String.Format(OperatorStringIsRightPartOfTernaryOperatorFormat,\n Quote(op.TernaryRightString),\n Quote(Quote(\"\", op.String, \" \"), op.TernaryRightString, \"\"),\n RelativePosition(position1, position2));\n }\n\n public static string ColumnCountAssumesTabStopDistanceOfNChars(int n) {\n return ColumnCountAssumesTabStopDistanceOfNChars1 + n.ToString() + ColumnCountAssumesTabStopDistanceOfNChars2;\n }\n\n public static string ErrorOccurredAtNthCharInCombiningCharacterSequence(int n, string textElement) {\n return AsciiQuote(ErrorOccurredAtNthCharInCombiningCharacterSequence1 + n.ToString() + OrdinalEnding(n) + ErrorOccurredAtNthCharInCombiningCharacterSequence2,\n textElement,\n ErrorOccurredAtNthCharInCombiningCharacterSequence3);\n }\n\n public static string InputContainsAtLeastNUnaccountedNewlines(int n) {\n return InputContainsAtLeastNUnaccountedNewlines1 + n.ToString() + (n == 1 ? InputContainsAtLeastNUnaccountedNewlines2Singular\n : InputContainsAtLeastNUnaccountedNewlines2Plural);\n }\n\n public static string ErrorOccurredAtBeginningOfSurrogatePair(string surrogatePair) {\n return AsciiQuote(ErrorOccurredAtBeginningOfSurrogatePair1, surrogatePair, ErrorOccurredAtBeginningOfSurrogatePair2);\n }\n\n\n public static string ErrorOccurredAtSecondCharInSurrogatePair(string surrogatePair) {\n return AsciiQuote(ErrorOccurredAtSecondCharInSurrogatePair1, surrogatePair, ErrorOccurredAtSecondCharInSurrogatePair2);\n }\n\n\n public static string CharAtErrorPositionIsIsolatedHighSurrogate(char ch) {\n return CharAtErrorPositionIsIsolatedHighSurrogate1 + Text.HexEscape(ch) + CharAtErrorPositionIsIsolatedHighSurrogate2;\n }\n\n public static string CharAtErrorPositionIsIsolatedLowSurrogate(char ch) {\n return CharAtErrorPositionIsIsolatedLowSurrogate1 + Text.HexEscape(ch) + CharAtErrorPositionIsIsolatedLowSurrogate2;\n }\n\n public static string CharBeforeErrorPositionIsIsolatedHighSurrogate(char ch) {\n return CharBeforeErrorPositionIsIsolatedHighSurrogate1 + Text.HexEscape(ch) + CharBeforeErrorPositionIsIsolatedHighSurrogate2;\n }\n\n public static string CharBeforeErrorPositionIsIsolatedLowSurrogate(char ch) {\n return CharBeforeErrorPositionIsIsolatedLowSurrogate1 + Text.HexEscape(ch) + CharBeforeErrorPositionIsIsolatedLowSurrogate2;\n }\n\n\n}\n\n}\n\n" }, { "path": "FParsecCS/Text.cs", "content": "// Copyright (c) Stephan Tolksdorf 2009-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nusing System;\nusing System.Text;\nusing System.Diagnostics;\nusing System.Runtime.CompilerServices;\nusing System.Runtime.InteropServices;\n\nnamespace FParsec {\n\npublic static class Text {\n\n/// Detects the presence of an encoding preamble in the first count bytes of the byte buffer.\n/// If detectEncoding is false, this function only searches for the preamble of the given default encoding,\n/// otherwise also for any of the standard unicode byte order marks (UTF-8, UTF-16 LE/BE, UTF-32 LE/BE).\n/// If an encoding different from the given default encoding is detected, the new encoding\n/// is assigned to the encoding reference.\n/// Returns the number of bytes in the detected preamble, or 0 if no preamble is detected.\n/// \ninternal static int DetectPreamble(byte[] buffer, int count, ref Encoding encoding, bool detectEncoding) {\n Debug.Assert(count >= 0);\n if (detectEncoding && count >= 2) {\n switch (buffer[0]) {\n case 0xEF:\n if (buffer[1] == 0xBB && count > 2 && buffer[2] == 0xBF) {\n if (encoding.CodePage != 65001)\n encoding = Encoding.UTF8;\n return 3;\n }\n break;\n case 0xFE:\n if (buffer[1] == 0xFF) {\n if (encoding.CodePage != 1201)\n encoding = Encoding.BigEndianUnicode;\n return 2;\n }\n break;\n case 0xFF:\n if (buffer[1] == 0xFE) {\n if (count >= 4 && buffer[2] == 0x00 && buffer[3] == 0x00) {\n if (encoding.CodePage != 12000)\n encoding = Encoding.UTF32; // UTF-32 little endian\n return 4;\n } else {\n if (encoding.CodePage != 1200)\n encoding = Encoding.Unicode; // UTF-16 little endian\n return 2;\n }\n }\n break;\n case 0x00:\n if (buffer[1] == 0x00 && count >= 4 && buffer[2] == 0xFE && buffer[3] == 0xFF) {\n if (encoding.CodePage != 12001)\n encoding = new UTF32Encoding(true, true); // UTF-32 big endian\n return 4;\n }\n break;\n }\n }\n#if NETSTANDARD2_0\n byte[] preamble = encoding.GetPreamble();\n#else\n ReadOnlySpan preamble = encoding.Preamble;\n#endif\n if (preamble.Length > 0 && count >= preamble.Length) {\n int i = 0;\n while (buffer[i] == preamble[i]) {\n if (++i == preamble.Length) return preamble.Length;\n }\n }\n return 0;\n}\n\n#if !LOW_TRUST\n/// Reads all remaining chars into the given buffer. If the remaining stream\n/// content holds more than the given maximum number of chars, an exception will be thrown.\ninternal unsafe static int ReadAllRemainingCharsFromStream(char* buffer, int maxCount, byte[] byteBuffer, int byteBufferIndex, int byteBufferCount, System.IO.Stream stream, long streamPosition, Decoder decoder, bool flush) {\n Debug.Assert(maxCount > 0 && byteBufferIndex >= 0 && byteBufferIndex < byteBufferCount);\n fixed (byte* pByteBuffer = byteBuffer) {\n int bufferCount = 0;\n for (;;) {\n try {\n bufferCount += decoder.GetChars(pByteBuffer + byteBufferIndex, byteBufferCount - byteBufferIndex,\n buffer + bufferCount, maxCount - bufferCount, flush);\n } catch (DecoderFallbackException e) {\n e.Data.Add(\"Stream.Position\", streamPosition - (byteBufferCount - byteBufferIndex) + e.Index);\n throw;\n }\n if (flush) break;\n byteBufferIndex = 0; // GetChars consumed all bytes in the byte buffer\n byteBufferCount = stream.Read(byteBuffer, 0, byteBuffer.Length);\n streamPosition += byteBufferCount;\n flush = byteBufferCount == 0;\n }\n return bufferCount;\n }\n}\n#endif\n\n\n/// Returns a case-folded copy of the string argument. All chars are mapped\n/// using the (non-Turkic) 1-to-1 case folding mappings (v. 6.0) for Unicode code\n/// points in the Basic Multilingual Plane, i.e. code points below 0x10000.\n/// If the argument is null, null is returned.\n#if LOW_TRUST\nstatic public string FoldCase(string str) {\n char[] cftable = CaseFoldTable.FoldedChars;\n if (str != null) {\n for (int i = 0; i < str.Length; ++i) {\n char c = str[i];\n char cfc = cftable[c];\n if (c != cfc) {\n StringBuilder sb = new StringBuilder(str);\n sb[i++] = cfc;\n for (; i < str.Length; ++i) {\n c = str[i];\n cfc = cftable[c];\n if (c != cfc) sb[i] = cfc;\n }\n return sb.ToString();\n }\n }\n }\n return str;\n}\n#else\nstatic unsafe public string FoldCase(string str) {\n if (str != null) {\n fixed (char* src0 = str) {\n char* end = src0 + str.Length;\n char* cftable = CaseFoldTable.FoldedChars;\n char* src = src0;\n for (;;) { // src is null-terminated, so we can always read one char\n char c = *src;\n if (c == cftable[c]) {\n if (++src >= end) break;\n } else {\n string newString = new String('\\u0000', str.Length);\n fixed (char* dst_ = newString) {\n src = src0;\n char* dst = dst_;\n do {\n *dst = cftable[*src];\n ++src; ++dst;\n } while (src != end);\n }\n return newString;\n }\n }\n }\n }\n return str;\n}\n#endif\n\n#if !LOW_TRUST\n unsafe\n#endif\nstatic public char FoldCase(char ch) {\n return CaseFoldTable.FoldedChars[ch];\n}\n\ninternal static int FindNewlineOrEOSChar(string str) {\n int i;\n for (i = 0; i < str.Length; ++i) {\n char c = str[i];\n // '\\n' = '\\u000A', '\\r' = '\\u000D'\n if (unchecked(c - 0xEu) < 0xFFFFu - 0xEu) continue;\n if (c == '\\n' || c == '\\r' || c == '\\uffff') goto Return;\n }\n i = -1;\nReturn:\n return i;\n}\n\n/// Returns the given string with all occurrences of \"\\r\\n\" and \"\\r\" replaced\n/// by \"\\n\". If the argument is null, null is returned.\n#if LOW_TRUST\nstatic public string NormalizeNewlines(string str) {\n if (str == null || str.Length == 0) return str;\n int nCR = 0;\n int nCRLF = 0;\n for (int i = 0; i < str.Length; ++i) {\n if (str[i] == '\\r') {\n if (i + 1 < str.Length && str[i + 1] == '\\n') ++nCRLF;\n else ++nCR;\n }\n }\n if (nCRLF == 0) {\n return nCR == 0 ? str : str.Replace('\\r', '\\n');\n } else {\n return CopyWithNormalizedNewlines(str, 0, str.Length, nCRLF, nCR);\n }\n}\nstatic internal string CopyWithNormalizedNewlines(string src, int index, int length, int nCRLF, int nCR) {\n Debug.Assert(length > 0 && nCRLF >= 0 && nCR >= 0 && (nCRLF | nCR) != 0);\n if (nCRLF != 0) {\n StringBuilder sb = new StringBuilder(length - nCRLF);\n int end = index + length;\n int i0 = index;\n if (nCR == 0) {\n int nn = nCRLF;\n int i = index;\n for (;;) {\n char c = src[i++];\n if (c == '\\r') {\n sb.Append(src, i0, i - i0 - 1).Append('\\n');\n ++i; // skip over the '\\n' in \"\\r\\n\"\n i0 = i;\n if (--nn == 0) break;\n }\n }\n } else {\n int nn = nCRLF + nCR;\n int i = index;\n for (;;) {\n char c = src[i++];\n if (c == '\\r') {\n sb.Append(src, i0, i - i0 - 1).Append('\\n');\n if (i < end && src[i] == '\\n') ++i; // skip over the '\\n' in \"\\r\\n\"\n i0 = i;\n if (--nn == 0) break;\n }\n }\n }\n if (i0 < end) sb.Append(src, i0, end - i0);\n return sb.ToString();\n } else {\n return new StringBuilder(src, index, length, length).Replace('\\r', '\\n').ToString();\n }\n}\n#else\nstatic unsafe public string NormalizeNewlines(string str) {\n int length;\n if (str == null || (length = str.Length) == 0) return str;\n fixed (char* src = str) { // the char buffer is guaranteed to be null-terminated (C# language specification on fixed statement)\n int nCR = 0;\n int nCRLF = 0;\n for (int i = 0; i < length; ++i) {\n if (src[i] == '\\r') {\n if (src[i + 1] == '\\n') ++nCRLF; // relies on null-termination\n else ++nCR;\n }\n }\n if (nCRLF == 0) {\n return nCR == 0 ? str : str.Replace('\\r', '\\n');\n } else {\n return CopyWithNormalizedNewlines(src, length, nCRLF, nCR);\n }\n }\n}\nstatic unsafe internal string CopyWithNormalizedNewlines(char* src, int length, int nCRLF, int nCR) {\n Debug.Assert(length > 0 && nCRLF >= 0 && nCR >= 0 && (nCRLF | nCR) != 0);\n string newString = new String('\\n', length - nCRLF);\n fixed (char* dst_ = newString) {\n char* dst = dst_;\n char* end = src + length;\n if (nCRLF != 0) {\n if (nCR == 0) {\n int nn = nCRLF;\n for (;;) {\n char c = *src;\n ++src;\n if (c != '\\r') {\n *dst = c;\n ++dst;\n } else {\n ++src; // skip over the '\\n' in \"\\r\\n\"\n *dst = '\\n';\n ++dst;\n if (--nn == 0) break;\n }\n }\n } else {\n int nn = nCRLF + nCR;\n for (;;) {\n char c = *src;\n ++src;\n if (c != '\\r') {\n *dst = c;\n ++dst;\n } else {\n if (*src == '\\n') ++src; // skip over the '\\n' in \"\\r\\n\" (relies on null-termination)\n *dst = '\\n';\n ++dst;\n if (--nn == 0) break;\n }\n }\n }\n } else {\n int nn = nCR;\n for (;;) {\n char c = *src;\n ++src;\n if (c != '\\r') {\n *dst = c;\n ++dst;\n } else {\n *dst = '\\n';\n ++dst;\n if (--nn == 0) break;\n }\n }\n }\n // copy remaining chars\n #if UNALIGNED_READS\n if (src != end) {\n uint len = Buffer.PositiveDistance(src, end);\n if ((unchecked((int)dst) & 2) != 0) { // align dest\n *dst = *src;\n ++src; ++dst; --len;\n }\n while (len >= 8) {\n ((int*)dst)[0] = ((int*)src)[0];\n ((int*)dst)[1] = ((int*)src)[1];\n ((int*)dst)[2] = ((int*)src)[2];\n ((int*)dst)[3] = ((int*)src)[3];\n src += 8; dst += 8; len -= 8;\n }\n if ((len & 4) != 0) {\n ((int*)dst)[0] = ((int*)src)[0];\n ((int*)dst)[1] = ((int*)src)[1];\n src += 4; dst += 4;\n }\n if ((len & 2) != 0) {\n ((int*)dst)[0] = ((int*)src)[0];\n src += 2; dst += 2;\n }\n if ((len & 1) != 0) {\n *dst = *src;\n }\n }\n #else\n while (src < end) {\n *dst = *src;\n ++src; ++dst;\n }\n #endif\n }\n return newString;\n}\n#endif\n\n\n/// Returns System.Globalization.StringInfo(str).LengthInTextElements\npublic static int CountTextElements(string str)\n{\n#if NET\n int count = 0, index = 0;\n while (index < str.Length)\n {\n index += System.Globalization.StringInfo.GetNextTextElementLength(str, index);\n count++;\n }\n return count;\n#else\n return new System.Globalization.StringInfo(str).LengthInTextElements;\n#endif\n}\n\n[Obsolete(\"Use System.Char.IsSurrogate instead.\")]\npublic static bool IsSurrogate(char ch) => char.IsSurrogate(ch);\n[Obsolete(\"Use System.Char.IsHighSurrogate instead.\")]\npublic static bool IsHighSurrogate(char ch) => char.IsHighSurrogate(ch);\n[Obsolete(\"Use System.Char.IsLowSurrogate instead.\")]\npublic static bool IsLowSurrogate(char ch) => char.IsLowSurrogate(ch);\n\n#if LOW_TRUST\n\npublic static bool IsWhitespace(char ch) {\n return System.Char.IsWhiteSpace(ch);\n}\n\n#else\n\ninternal unsafe static class IsWhitespaceHelper {\n\n // we use the same data structure and algorithm as for IdentifierValidator\n\n private static ReadOnlySpan DataArray => new byte[] {\n 0,1,1,1,1,1,1,1,1,1,1,2,1,1,1,1,3,1,1,1,\n 1,1,1,1,4,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,\n 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,\n 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,\n 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,\n 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,\n 1,1,1,1,1,1,1,1,0,1,2,2,3,1,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,\n 2,2,2,2,1,2,2,2,2,2,2,2,2,2,2,2,4,5,6,2,\n 2,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,2,2,\n 2,2,2,2,2,2,2,2,0,62,0,0,1,0,0,0,0,0,0,0,\n 32,0,0,0,255,7,0,0,0,131,0,0,0,0,0,128,\n };\n\n private const int Table1Offset = 0;\n private const int Table1Size = 128;\n private const int Table1Log2Length = 7;\n private const int Table2Offset = 128;\n private const int Table2Size = 80;\n private const int Table2Log2BlockLength = 4;\n private const int Table3Offset = Table2Offset + Table2Size;\n private const int Table3Size = 28;\n private const int Table3Log2BlockLength = 5;\n\n#if LOW_TRUST\n private static ReadOnlySpan Table1 => DataArray.Slice(0, Table2Offset);\n private static ReadOnlySpan Table2 => DataArray.Slice(Table2Offset, Table2Size);\n private static readonly uint[] Table3 = Buffer.CopyUIntsStoredInLittleEndianByteArray(DataArray, Table3Offset, Table3Size);\n#else\n private static byte* Data => (byte*)Unsafe.AsPointer(ref MemoryMarshal.GetReference(DataArray));\n private static byte* Table1 => Data + Table1Offset;\n private static byte* Table2 => Data + Table2Offset;\n private static readonly uint* Table3 = Buffer.LoadLittleEndianUInt32Data(Data, Table3Offset, Table3Size);\n#endif\n\n public static uint IsWhitespace_(char ch) {\n int cp = ch;\n int idx1 = cp >> (Table2Log2BlockLength + Table3Log2BlockLength);\n const int f2 = 1 << Table2Log2BlockLength;\n const int m2 = f2 - 1;\n int idx2 = Table1[idx1]*f2 + ((cp >> Table3Log2BlockLength) & m2);\n int idx3 = Table2[idx2];\n return Table3[idx3] >> (cp /* & 0x1fu */); // C#'s operator>> masks with 0x1fu, no matter whether we do too\n }\n}\n\n/// A faster implementation of System.Char.IsWhiteSpace.\npublic static bool IsWhitespace(char ch) { // should get inlined\n return (IsWhitespaceHelper.IsWhitespace_(ch) & 1u) != 0;\n}\n#endif\n\ninternal static string HexEscape(char c)\n{\n Span cs = stackalloc char[6];\n cs[0] = '\\\\';\n cs[1] = 'u';\n int n = c;\n for (int j = 5; j > 1; --j) {\n cs[j] = \"0123456789abcdef\"[n & 0xf];\n n >>= 4;\n }\n return cs.ToString();\n}\n\ninternal static string EscapeChar(char c) {\n switch (c) {\n case '\\\\': return \"\\\\\\\\\";\n case '\\'': return \"\\\\\\'\";\n case '\\\"': return \"\\\\\\\"\";\n case '\\r': return \"\\\\r\";\n case '\\n': return \"\\\\n\";\n case '\\t': return \"\\\\t\";\n case '\\f': return \"\\\\f\";\n case '\\v': return \"\\\\v\";\n case '\\a': return \"\\\\a\";\n case '\\b': return \"\\\\b\";\n default: return HexEscape(c);\n }\n}\n\ninternal static string Concat(string str0, string str1, string str2, string str3, string str4) {\n#if !NET\n return str0 + str1 + str2 + str3 + str4;\n#else\n int length = str0.Length + str1.Length + str2.Length + str3.Length + str4.Length;\n return string.Create(length, (str0, str1, str2, str3, str4), static (span, x) =>\n {\n int i = 0;\n x.str0.CopyTo(span[i..]);\n i += x.str0.Length;\n x.str1.CopyTo(span[i..]);\n i += x.str1.Length;\n x.str2.CopyTo(span[i..]);\n i += x.str2.Length;\n x.str3.CopyTo(span[i..]);\n i += x.str3.Length;\n x.str4.CopyTo(span[i..]);\n });\n#endif\n}\n\ninternal static string Escape(string str, string prefix1, string prefix2, string postfix1, string postfix2, char escapedQuoteChar) {\n Debug.Assert(str != null && prefix1 != null && prefix2 != null && postfix1 != null && postfix2 != null);\n StringBuilder sb = null;\n int i0 = 0;\n int i = 0;\n for (;;) {\n if (i >= str.Length) break;\n char c = str[i];\n ++i;\n if (c > '\\'' && c < '\\u007f') {\n if (c != '\\\\') continue;\n } else if (c == ' ' || ( !Char.IsControl(c) && c != escapedQuoteChar\n && (c < '\\u2028' || c > '\\u2029'))) continue;\n if (sb is null) {\n sb = new StringBuilder(str.Length + prefix1.Length + prefix2.Length + postfix1.Length + postfix2.Length + 8);\n sb.Append(prefix1).Append(prefix2);\n }\n int n = i - i0 - 1;\n if (n != 0) sb.Append(str, i0, n);\n i0 = i;\n sb.Append(EscapeChar(c));\n }\n if (sb is null) return Concat(prefix1, prefix2, str, postfix1, postfix2);\n if (i0 != i) sb.Append(str, i0, i - i0);\n return sb.Append(postfix1).Append(postfix2).ToString();\n}\n\ninternal static string AsciiEscape(string str, string prefix1, string prefix2, string postfix1, string postfix2, char escapedQuoteChar) {\n Debug.Assert(str != null && prefix1 != null && prefix2 != null && postfix1 != null && postfix2 != null);\n StringBuilder sb = null;\n int i0 = 0;\n int i = 0;\n for (;;) {\n if (i >= str.Length) break;\n char c = str[i];\n ++i;\n if (c > '\\'' && c < '\\u007f') {\n if (c != '\\\\') continue;\n } else if (c == ' ' || (c >= ' ' && c <= '\\'' && c != escapedQuoteChar)) continue;\n if (sb is null) {\n sb = new StringBuilder(str.Length + prefix1.Length + prefix2.Length + postfix1.Length + postfix2.Length + 8);\n sb.Append(prefix1).Append(prefix2);\n }\n int n = i - i0 - 1;\n if (n != 0) sb.Append(str, i0, n);\n i0 = i;\n sb.Append(EscapeChar(c));\n }\n if (sb is null) return Concat(prefix1, prefix2, str, postfix1, postfix2);\n if (i0 != i) sb.Append(str, i0, i - i0);\n return sb.Append(postfix1).Append(postfix2).ToString();\n}\n\ninternal static string SingleQuote(string str) {\n return Escape(str, \"\", \"'\", \"'\", \"\", '\\'');\n}\n\ninternal static string SingleQuote(string prefix, string str, string postfix) {\n return Escape(str, prefix, \"'\", \"'\", postfix, '\\'');\n}\n\ninternal static string DoubleQuote(string str) {\n return Escape(str, \"\", \"\\\"\", \"\\\"\", \"\", '\"');\n}\n\ninternal static string DoubleQuote(string prefix, string str, string postfix) {\n return Escape(str, prefix, \"\\\"\", \"\\\"\", postfix, '\"');\n}\n\n\n} // class Text\n\n}" }, { "path": "NuGet.config", "content": "\n\n \n \n \n \n\n" }, { "path": "Samples/Calculator/Calculator-LowTrust.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Calculator/Calculator.fsproj", "content": "\n\n \n Calculator\n Calculator\n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Calculator/Calculator.targets", "content": "\n \n Calculator\n Calculator\n Exe\n false\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Calculator/InterpLexYacc-LowTrust.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Calculator/calculator.fs", "content": "\n// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n// the parser definition\n////////////////////////\n\nopen FParsec\n\nlet ws = spaces // skips any whitespace\n\nlet str_ws s = pstring s >>. ws\n\n// we calculate with double precision floats\nlet number = pfloat .>> ws\n\n// we set up an operator precedence parser for parsing the arithmetic expressions\nlet opp = new OperatorPrecedenceParser()\nlet expr = opp.ExpressionParser\nopp.TermParser <- number <|> between (str_ws \"(\") (str_ws \")\") expr\n\n// operator definitions follow the schema\n// operator type, string, trailing whitespace parser, precedence, associativity, function to apply\n\nopp.AddOperator(InfixOperator(\"+\", ws, 1, Associativity.Left, (+)))\nopp.AddOperator(InfixOperator(\"-\", ws, 1, Associativity.Left, (-)))\nopp.AddOperator(InfixOperator(\"*\", ws, 2, Associativity.Left, (*)))\nopp.AddOperator(InfixOperator(\"/\", ws, 2, Associativity.Left, (/)))\nopp.AddOperator(InfixOperator(\"^\", ws, 3, Associativity.Right, fun x y -> System.Math.Pow(x, y)))\nopp.AddOperator(PrefixOperator(\"-\", ws, 4, true, fun x -> -x))\n\n// we also want to accept the operators \"exp\" and \"log\", but we don't want to accept\n// expressions like \"logexp\" 2, so we require that non-symbolic operators are not\n// followed by letters\n\nlet ws1 = nextCharSatisfiesNot isLetter >>. ws\nopp.AddOperator(PrefixOperator(\"log\", ws1, 4, true, System.Math.Log))\nopp.AddOperator(PrefixOperator(\"exp\", ws1, 4, true, System.Math.Exp))\n\nlet completeExpression = ws >>. expr .>> eof // we append the eof parser to make\n // sure all input is consumed\n\n// running and testing the parser\n/////////////////////////////////\n\nlet calculate s = run completeExpression s\n\nlet equals expectedValue r =\n match r with\n | Success (v, _, _) when v = expectedValue -> ()\n | Success (v, _, _) -> failwith \"Math is hard, let's go shopping!\"\n | Failure (msg, err, _) -> printf \"%s\" msg; failwith msg\n\nlet test() =\n calculate \"10.5 + 123.25 + 877\" |> equals 1010.75\n calculate \"10/2 + 123.125 + 877\" |> equals 1005.125\n calculate \"(123 + log 1 + 877) * 9/3\" |> equals 3000.\n calculate \" ( ( exp 0 + (6 / ( 1 +2 ) )- 123456 )/ 2+123 + 877) * 3^2 / 3\" |> equals (-182179.5)\n printfn \"No errors\"\n\n// currently the program only executes some tests\ndo test()\n\n" }, { "path": "Samples/FSharpParsingSample/FParsecVersion/InterpFParsec-LowTrust.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/FSharpParsingSample/FParsecVersion/InterpFParsec.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/FSharpParsingSample/FParsecVersion/InterpFParsec.targets", "content": "\n\n \n InterpFParsec\n InterpFParsec\n Exe\n false\n $(MSBuildProjectDirectory)/../LexYaccVersion/test.lang\n \n\n \n \n \n \n \n \n \n \n\n\n" }, { "path": "Samples/FSharpParsingSample/FParsecVersion/main.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008\n// License: Simplified BSD License. See accompanying documentation.\n\n// This is a port of the parsing sample that came with the F# 1.9.4.19\n// distribution.\n\nopen FParsec\n\nopen Ast\n\n[]\nlet main(argv: string[]) =\n if argv.Length <> 1 then\n printf \"usage: interp.exe \\n\"\n exit 1\n\n // Run the parser prog on the file path in argv[0]\n // If the file has no byte order marks, System.Text.Encoding.Default\n // is assumed to be the encoding.\n let fileName = argv[0]\n let result =\n runParserOnFile Parser.prog () fileName System.Text.Encoding.UTF8\n\n let myProg =\n match result with\n | Success (v, _, _) -> v\n | Failure (msg, _, _) ->\n System.Console.WriteLine(msg)\n exit 1\n\n // count statements\n printf \"#stmts = %d\\n\" (List.length (match myProg with Prog l -> l));\n printf \"running program...\\n\";\n Interp.prog myProg\n 0\n\n" }, { "path": "Samples/FSharpParsingSample/FParsecVersion/parser.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008.\n// License: Simplified BSD License. See accompanying documentation.\n\n// Compare this parser implementation with the implementation in ../LexYaccVersion.\n\nmodule Parser\n\nopen System\nopen System.Collections.Generic\n\nopen FParsec\n\nopen Ast\n\n// some lexical definitions\n///////////////////////////\n\nlet ws = spaces // skips any whitespace\n\nlet str s = pstring s >>. ws\n\n// identifiers are strings of lower ascii chars that are not keywords\nlet identifierString = many1Satisfy isLower .>> ws // [a-z]+\nlet keywords = [\"while\"; \"begin\"; \"end\"; \"do\"; \"if\"; \"then\"; \"else\"; \"print\"; \"decr\"]\nlet keywordsSet = new HashSet(keywords)\nlet isKeyword str = keywordsSet.Contains(str)\n\n//open FParsec.StaticMapping\n//let isKeyword = createStaticStringMapping false [for kw in keywords -> (kw, true)]\n\nlet identifier : Parser =\n let expectedIdentifier = expected \"identifier\"\n fun stream ->\n let state = stream.State\n let reply = identifierString stream\n if reply.Status <> Ok || not (isKeyword reply.Result) then reply\n else // result is keyword, so backtrack to before the string\n stream.BacktrackTo(state)\n Reply(Error, expectedIdentifier)\n\n\nlet numberFormat = NumberLiteralOptions.AllowMinusSign\n ||| NumberLiteralOptions.AllowFraction\n ||| NumberLiteralOptions.AllowExponent\nlet numberLit = numberLiteral numberFormat \"number\" .>> ws\n\n\n// parsers for the original grammar productions\n///////////////////////////////////////////////\n\nlet pval = identifier |>> Val\n\nlet number =\n numberLit\n |>> fun nl -> // an overflow will throw an exception, as in the original sample\n if nl.IsInteger then Int (int32 nl.String)\n else Float (float nl.String)\n\n// expr and decr are mutually recursive grammar grammar productions.\n// In order to break the cyclic dependency, we make expr a parser that\n// forwards all calls to a parser in a reference cell.\nlet expr, exprRef = createParserForwardedToRef() // initially exprRef holds a reference to a dummy parser\n\nlet pdecr = str \"decr\" >>. str \"(\" >>. expr .>> str \")\" |>> Decr\n\n// replace dummy parser reference in exprRef\ndo exprRef:= choice [pval; pdecr; number] // we need to try pval first, so we don't\n // accidentally try to parse an identifier\n // starting with \"decr...\" as a Decr statement\n // (this is a disadvantage of not having a tokenizer)\n\n\nlet stmt, stmtRef = createParserForwardedToRef()\n\nlet stmtList = sepBy1 stmt (str \";\")\n\nlet assign =\n pipe2 identifier (str \":=\" >>. expr) (fun id e -> Assign(id, e))\n\nlet print = str \"print\" >>. expr |>> Print\n\nlet pwhile =\n pipe2 (str \"while\" >>. expr) (str \"do\" >>. stmt) (fun e s -> While(e, s))\n\nlet seq =\n str \"begin\" >>. stmtList .>> str \"end\" |>> Seq\n\nlet ifthen =\n pipe3 (str \"if\" >>. expr) (str \"then\" >>. stmt) (opt (str \"else\" >>. stmt))\n (fun e s1 optS2 ->\n match optS2 with\n | None -> IfThen(e, s1)\n | Some s2 -> IfThenElse(e, s1, s2))\n\ndo stmtRef:= choice [assign; ifthen; pwhile; seq; print] // try assign first, so that an\n // identifier starting with a\n // keyword doesn't trigger an error\nlet prog =\n ws >>. stmtList .>> eof |>> Prog\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/Doc.html", "content": "\n\n\nThe F# Parsing Sample\n\n \n \n\n \n\n\n\n\n

The F# Parsing Sample

\n\n

Links: Up

\n\n

\n\nThis sample shows how to write a simple parser and lexer using\nF#, and the tools fslex and fsyacc in particular.\n

\nThe sample is made up of the F# type declarations \nfor the Abstract Syntax Tree types in ast.fs,\nthe definition of the lexer in lex.fsl,\nthe definition of the token types and \nparser in pars.fsy , an interpeter for the language in interp.fs and an\nF# driver program that plugs the parser and the interpreter together, in main.fs. \n

\n

Suggested Exercises

\n
  • \nAdd an extra construct to the abstract syntax in ast.fs, e.g. an addition node\nfor expressions. Define the semantics for the expression in interp.fs. \nThen add appropriate tokens and rules in the parser and lexer.\n
\n\n\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/InterpLexYacc.fsproj", "content": "\n \n net6\n Exe\n false\n $(MSBuildProjectDirectory)/test.lang\n \n\n \n \n \n \n --unicode\n \n \n --module Parser\n \n \n \n \n \n \n \n\n \n \n \n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/ast.fs", "content": "// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n\nmodule Ast\n\ntype Expr =\n | Val of string\n | Int of int\n | Float of float\n | Decr of Expr\n\ntype Stmt =\n | Assign of string * Expr\n | While of Expr * Stmt\n | Seq of Stmt list\n | IfThen of Expr * Stmt\n | IfThenElse of Expr * Stmt * Stmt\n | Print of Expr\n\ntype Prog = Prog of Stmt list\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/interp.fs", "content": "\n// Original code:\n// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n\n// Modifications:\n// Copyright (c) Stephan Tolksdorf 2015.\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule Interp\n\nopen Ast\n\nopen System.Collections.Generic\n\ntype Value = INT of int | FLOAT of float\n\ntype State = Dictionary\n\nlet printVal os v =\n match v with\n | INT n -> Printf.fprintf os \"%d\" n\n | FLOAT f -> Printf.fprintf os \"%g\" f\n\nlet rec prog (Prog l ) =\n stmts (new Dictionary<_,_>()) l\n\nand stmts s l =\n List.iter (stmt s) l\n\nand stmt (s: State) st =\n match st with\n | Assign (a,b) ->\n s[a] <- expr s b\n | While (a,b) ->\n while expr s a <> INT 0 do\n stmt s b\n | Seq l ->\n stmts s l\n | IfThen (g,t) ->\n if (expr s g <> INT 0) then stmt s t\n | IfThenElse (g,t,e) ->\n if (expr s g <> INT 0) then stmt s t\n else stmt s e\n | Print (e) ->\n Printf.printf \"--> %a\\n\" printVal (expr s e)\n stdout.Flush()\n\nand expr (s: State) e =\n match e with\n | Val n ->\n match s.TryGetValue(n) with\n | true, v -> v\n | false, _ -> Printf.eprintf \"warning: location %s not defined\\n\" n;\n INT 0\n | Expr.Int n -> INT n\n | Expr.Float f -> FLOAT f\n | Decr e2 ->\n match expr s e2 with\n | INT n -> INT (n-1)\n | FLOAT f -> failwith \"cannot decrement a float\"\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/lex.fs", "content": "# 13 \"lex.fsl\"\n \n\nmodule Lex\n\nopen FSharp.Text.Lexing\nopen Parser\n\nlet lexeme = LexBuffer.LexemeString\n\n// Fslex generated parsers follow the same pattern as OCamllex\n// and Mossmllex generated parsers, and do not update line number\n// information automatically, partly because the knowledge of when\n// a newline has occured is best placed in the lexer rules.\n// Thus the following boiler-plate code is very useful:\n\nlet newline (lexbuf: LexBuffer<_>) =\n lexbuf.StartPos <- lexbuf.StartPos.NextLine\n\n# 20 \"lex.fs\"\nlet trans : uint16[] array = \n [| \n (* State 0 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 1us; 2us; 65535us; 65535us; 3us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 1us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 11us; 12us; 65535us; 65535us; 65535us; 17us; 65535us; 65535us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 14us; 13us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 15us; 5us; 15us; 7us; 6us; 15us; 15us; 15us; 8us; 15us; 15us; 15us; 15us; 15us; 15us; 10us; 15us; 15us; 15us; 9us; 15us; 15us; 4us; 15us; 15us; 15us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 18us; |];\n (* State 1 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 2 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 3 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 2us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 4 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 49us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 5 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 45us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 6 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 41us; 26us; 40us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 7 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 37us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 36us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 8 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 35us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 9 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 32us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 10 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 28us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 11 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 12 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 13 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 14 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 27us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 15 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 16 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 21us; 65535us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 17 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 16us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 18 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 19 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 21us; 65535us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 19us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 20 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 24us; 24us; 24us; 24us; 24us; 24us; 24us; 24us; 24us; 24us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 21 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 22us; 22us; 22us; 22us; 22us; 22us; 22us; 22us; 22us; 22us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 22 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 23 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 23us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 20us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 24 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 25 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 25us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 26 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 27 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 28 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 29us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 29 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 30us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 30 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 31us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 31 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 32 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 33us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 33 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 34us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 34 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 35 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 36 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 37 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 38us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 38 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 39us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 39 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 40 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 44us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 41 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 42us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 42 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 43us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 43 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 44 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 45 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 46us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 46 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 47us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 47 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 48us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 48 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 49 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 50us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 50 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 51us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 51 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 52us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n (* State 52 *)\n [| 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 26us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; 65535us; |];\n |] \nlet actions : uint16[] = [|65535us; 0us; 1us; 65535us; 15us; 15us; 15us; 15us; 15us; 15us; 15us; 11us; 12us; 13us; 65535us; 15us; 16us; 65535us; 18us; 16us; 65535us; 65535us; 17us; 17us; 17us; 17us; 15us; 14us; 15us; 15us; 15us; 9us; 15us; 15us; 7us; 6us; 5us; 15us; 15us; 10us; 15us; 15us; 15us; 8us; 4us; 15us; 15us; 15us; 3us; 15us; 15us; 15us; 2us; |]\nlet _fslex_tables = FSharp.Text.Lexing.UnicodeTables.Create(trans,actions)\nlet rec _fslex_dummy () = _fslex_dummy() \n// Rule token\nand token lexbuf =\n match _fslex_tables.Interpret(0,lexbuf) with\n | 0 -> ( \n# 54 \"lex.fsl\"\n token lexbuf \n# 139 \"lex.fs\"\n )\n | 1 -> ( \n# 55 \"lex.fsl\"\n newline lexbuf; token lexbuf \n# 144 \"lex.fs\"\n )\n | 2 -> ( \n# 56 \"lex.fsl\"\n WHILE \n# 149 \"lex.fs\"\n )\n | 3 -> ( \n# 57 \"lex.fsl\"\n BEGIN \n# 154 \"lex.fs\"\n )\n | 4 -> ( \n# 58 \"lex.fsl\"\n END \n# 159 \"lex.fs\"\n )\n | 5 -> ( \n# 59 \"lex.fsl\"\n DO \n# 164 \"lex.fs\"\n )\n | 6 -> ( \n# 60 \"lex.fsl\"\n IF \n# 169 \"lex.fs\"\n )\n | 7 -> ( \n# 61 \"lex.fsl\"\n THEN \n# 174 \"lex.fs\"\n )\n | 8 -> ( \n# 62 \"lex.fsl\"\n ELSE \n# 179 \"lex.fs\"\n )\n | 9 -> ( \n# 63 \"lex.fsl\"\n PRINT \n# 184 \"lex.fs\"\n )\n | 10 -> ( \n# 64 \"lex.fsl\"\n DECR \n# 189 \"lex.fs\"\n )\n | 11 -> ( \n# 65 \"lex.fsl\"\n LPAREN \n# 194 \"lex.fs\"\n )\n | 12 -> ( \n# 66 \"lex.fsl\"\n RPAREN \n# 199 \"lex.fs\"\n )\n | 13 -> ( \n# 67 \"lex.fsl\"\n SEMI \n# 204 \"lex.fs\"\n )\n | 14 -> ( \n# 68 \"lex.fsl\"\n ASSIGN \n# 209 \"lex.fs\"\n )\n | 15 -> ( \n# 70 \"lex.fsl\"\n ID(lexeme lexbuf) \n# 214 \"lex.fs\"\n )\n | 16 -> ( \n# 72 \"lex.fsl\"\n INT (int32 (lexeme lexbuf)) \n# 219 \"lex.fs\"\n )\n | 17 -> ( \n# 74 \"lex.fsl\"\n FLOAT (float (lexeme lexbuf)) \n# 224 \"lex.fs\"\n )\n | 18 -> ( \n# 75 \"lex.fsl\"\n EOF \n# 229 \"lex.fs\"\n )\n | _ -> failwith \"token\"\n\n# 3000000 \"lex.fs\"\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/lex.fsl", "content": "// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n//\n\n// This file is a sample lexer specification for use with F# Lex (fslex.exe).\n\n//--------------------------------------------------------------\n// Part I. Supporting F# definitions. Everything between braces\n// is F# code added to the generated file.\n\n{\n\nmodule Lex\n\nopen FSharp.Text.Lexing\nopen Parser\n\nlet lexeme = LexBuffer.LexemeString\n\n// Fslex generated parsers follow the same pattern as OCamllex\n// and Mossmllex generated parsers, and do not update line number\n// information automatically, partly because the knowledge of when\n// a newline has occured is best placed in the lexer rules.\n// Thus the following boiler-plate code is very useful:\n\nlet newline (lexbuf: LexBuffer<_>) =\n lexbuf.StartPos <- lexbuf.StartPos.NextLine\n}\n\n//--------------------------------------------------------------\n// Part II. Define some regular expressions\n//\n// These are some regular expression definitions\n\nlet digit = ['0'-'9']\nlet whitespace = [' ' '\\t' ]\nlet newline = ('\\n' | '\\r' '\\n')\n\n//--------------------------------------------------------------\n// Part III. Token generators and rules.\n//\n// These are the rules specifying the tokens matched by the lexer\n//\n// This lexer has only one generator ('token')\n//\n// Generator definitions can take arguments, e.g.\n//\n// rule token arg1 arg2 = parse ...\n\n\nrule token = parse\n| whitespace { token lexbuf } // keep lexing!\n| newline { newline lexbuf; token lexbuf } // record line break and keep lexing!\n| \"while\" { WHILE } // return a token!\n| \"begin\" { BEGIN } // ...\n| \"end\" { END }\n| \"do\" { DO }\n| \"if\" { IF }\n| \"then\" { THEN }\n| \"else\" { ELSE }\n| \"print\" { PRINT }\n| \"decr\" { DECR }\n| \"(\" { LPAREN }\n| \")\" { RPAREN }\n| \";\" { SEMI }\n| \":=\" { ASSIGN } // ...\n| ['a'-'z']+\n { ID(lexeme lexbuf) } // return a token carrying data!\n| ['-']?digit+\n { INT (int32 (lexeme lexbuf)) } // return a token carrying data!\n| ['-']?digit+('.'digit+)?(['e''E']digit+)?\n { FLOAT (float (lexeme lexbuf)) }\n| eof { EOF } // return a token for the EOF\n\n//\n// Additional generator definitions would go here, e.g.\n//\n// and comment arg1 arg2 = parse ...\n//\n// and string arg1 arg2 = parse ...\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/main.fs", "content": "\n// Original code:\n// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n\n// Modifications:\n// Copyright (c) Stephan Tolksdorf 2015.\n// License: Simplified BSD License. See accompanying documentation.\n\n// This program uses FsLex and FsYacc:\n// http://fsprojects.github.io/FsLexYacc/\n\nopen Ast\nopen Printf\n\nopen FSharp.Text.Lexing\n\n[]\nlet main(argv: string[]) =\n\n if argv.Length <> 1 then\n printf \"usage: interp.exe \\n\"\n exit 1\n\n let stream = new System.IO.StreamReader(argv[0], System.Text.Encoding.UTF8)\n let myProg =\n\n // Create the lexer, presenting the bytes to the lexer as ASCII regardless of the original\n // encoding of the stream (the lexer specification\n // is designed to consume ASCII)\n let lexbuf = LexBuffer.FromTextReader(stream)\n\n // Call the parser\n try\n Parser.start Lex.token lexbuf\n with e ->\n let pos = lexbuf.EndPos\n printf \"error near line %d, character %d\\n%s\\n\" pos.Line pos.Column (e.ToString());\n exit 1\n\n // Now look at the resulting AST, e.g. count the number of top-level\n // statements, and then the overall number of nodes.\n printf \"#stmts = %d\\n\" (List.length (match myProg with Prog l -> l));\n printf \"running program...\\n\";\n Interp.prog myProg\n 0\n\n\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/pars.fs", "content": "// Implementation file for parser generated by fsyacc\nmodule Parser\n#nowarn \"64\";; // turn off warnings that type variables used in production annotations are instantiated to concrete type\nopen FSharp.Text.Lexing\nopen FSharp.Text.Parsing.ParseHelpers\n# 10 \"pars.fsy\"\n\n\n// This prelude is F# code that is available throughout this file. In this\n// case we just open a module to reveal some datatype definitions.\nopen Ast\n\n\n# 14 \"pars.fs\"\n// This type is the type of tokens accepted by the parser\ntype token = \n | DECR\n | LPAREN\n | RPAREN\n | WHILE\n | DO\n | END\n | BEGIN\n | IF\n | THEN\n | ELSE\n | PRINT\n | SEMI\n | ASSIGN\n | EOF\n | FLOAT of (System.Double)\n | INT of (System.Int32)\n | ID of (string)\n// This type is used to give symbolic names to token indexes, useful for error messages\ntype tokenId = \n | TOKEN_DECR\n | TOKEN_LPAREN\n | TOKEN_RPAREN\n | TOKEN_WHILE\n | TOKEN_DO\n | TOKEN_END\n | TOKEN_BEGIN\n | TOKEN_IF\n | TOKEN_THEN\n | TOKEN_ELSE\n | TOKEN_PRINT\n | TOKEN_SEMI\n | TOKEN_ASSIGN\n | TOKEN_EOF\n | TOKEN_FLOAT\n | TOKEN_INT\n | TOKEN_ID\n | TOKEN_end_of_input\n | TOKEN_error\n// This type is used to give symbolic names to token indexes, useful for error messages\ntype nonTerminalId = \n | NONTERM__startstart\n | NONTERM_start\n | NONTERM_Prog\n | NONTERM_Expr\n | NONTERM_Stmt\n | NONTERM_StmtList\n\n// This function maps tokens to integer indexes\nlet tagOfToken (t:token) = \n match t with\n | DECR -> 0 \n | LPAREN -> 1 \n | RPAREN -> 2 \n | WHILE -> 3 \n | DO -> 4 \n | END -> 5 \n | BEGIN -> 6 \n | IF -> 7 \n | THEN -> 8 \n | ELSE -> 9 \n | PRINT -> 10 \n | SEMI -> 11 \n | ASSIGN -> 12 \n | EOF -> 13 \n | FLOAT _ -> 14 \n | INT _ -> 15 \n | ID _ -> 16 \n\n// This function maps integer indexes to symbolic token ids\nlet tokenTagToTokenId (tokenIdx:int) = \n match tokenIdx with\n | 0 -> TOKEN_DECR \n | 1 -> TOKEN_LPAREN \n | 2 -> TOKEN_RPAREN \n | 3 -> TOKEN_WHILE \n | 4 -> TOKEN_DO \n | 5 -> TOKEN_END \n | 6 -> TOKEN_BEGIN \n | 7 -> TOKEN_IF \n | 8 -> TOKEN_THEN \n | 9 -> TOKEN_ELSE \n | 10 -> TOKEN_PRINT \n | 11 -> TOKEN_SEMI \n | 12 -> TOKEN_ASSIGN \n | 13 -> TOKEN_EOF \n | 14 -> TOKEN_FLOAT \n | 15 -> TOKEN_INT \n | 16 -> TOKEN_ID \n | 19 -> TOKEN_end_of_input\n | 17 -> TOKEN_error\n | _ -> failwith \"tokenTagToTokenId: bad token\"\n\n/// This function maps production indexes returned in syntax errors to strings representing the non terminal that would be produced by that production\nlet prodIdxToNonTerminal (prodIdx:int) = \n match prodIdx with\n | 0 -> NONTERM__startstart \n | 1 -> NONTERM_start \n | 2 -> NONTERM_Prog \n | 3 -> NONTERM_Expr \n | 4 -> NONTERM_Expr \n | 5 -> NONTERM_Expr \n | 6 -> NONTERM_Expr \n | 7 -> NONTERM_Stmt \n | 8 -> NONTERM_Stmt \n | 9 -> NONTERM_Stmt \n | 10 -> NONTERM_Stmt \n | 11 -> NONTERM_Stmt \n | 12 -> NONTERM_Stmt \n | 13 -> NONTERM_StmtList \n | 14 -> NONTERM_StmtList \n | _ -> failwith \"prodIdxToNonTerminal: bad production index\"\n\nlet _fsyacc_endOfInputTag = 19 \nlet _fsyacc_tagOfErrorTerminal = 17\n\n// This function gets the name of a token as a string\nlet token_to_string (t:token) = \n match t with \n | DECR -> \"DECR\" \n | LPAREN -> \"LPAREN\" \n | RPAREN -> \"RPAREN\" \n | WHILE -> \"WHILE\" \n | DO -> \"DO\" \n | END -> \"END\" \n | BEGIN -> \"BEGIN\" \n | IF -> \"IF\" \n | THEN -> \"THEN\" \n | ELSE -> \"ELSE\" \n | PRINT -> \"PRINT\" \n | SEMI -> \"SEMI\" \n | ASSIGN -> \"ASSIGN\" \n | EOF -> \"EOF\" \n | FLOAT _ -> \"FLOAT\" \n | INT _ -> \"INT\" \n | ID _ -> \"ID\" \n\n// This function gets the data carried by a token as an object\nlet _fsyacc_dataOfToken (t:token) = \n match t with \n | DECR -> (null : System.Object) \n | LPAREN -> (null : System.Object) \n | RPAREN -> (null : System.Object) \n | WHILE -> (null : System.Object) \n | DO -> (null : System.Object) \n | END -> (null : System.Object) \n | BEGIN -> (null : System.Object) \n | IF -> (null : System.Object) \n | THEN -> (null : System.Object) \n | ELSE -> (null : System.Object) \n | PRINT -> (null : System.Object) \n | SEMI -> (null : System.Object) \n | ASSIGN -> (null : System.Object) \n | EOF -> (null : System.Object) \n | FLOAT _fsyacc_x -> Microsoft.FSharp.Core.Operators.box _fsyacc_x \n | INT _fsyacc_x -> Microsoft.FSharp.Core.Operators.box _fsyacc_x \n | ID _fsyacc_x -> Microsoft.FSharp.Core.Operators.box _fsyacc_x \nlet _fsyacc_gotos = [| 0us; 65535us; 1us; 65535us; 0us; 1us; 1us; 65535us; 0us; 2us; 5us; 65535us; 8us; 9us; 12us; 13us; 14us; 15us; 21us; 22us; 27us; 28us; 6us; 65535us; 0us; 29us; 16us; 17us; 18us; 29us; 23us; 24us; 25us; 26us; 30us; 31us; 2us; 65535us; 0us; 3us; 18us; 19us; |]\nlet _fsyacc_sparseGotoTableRowOffsets = [|0us; 1us; 3us; 5us; 11us; 18us; |]\nlet _fsyacc_stateToProdIdxsTableElements = [| 1us; 0us; 1us; 0us; 1us; 1us; 2us; 2us; 14us; 1us; 3us; 1us; 4us; 1us; 5us; 1us; 6us; 1us; 6us; 1us; 6us; 1us; 6us; 1us; 7us; 1us; 7us; 1us; 7us; 1us; 8us; 1us; 8us; 1us; 8us; 1us; 8us; 1us; 9us; 2us; 9us; 14us; 1us; 9us; 2us; 10us; 11us; 2us; 10us; 11us; 2us; 10us; 11us; 2us; 10us; 11us; 1us; 11us; 1us; 11us; 1us; 12us; 1us; 12us; 1us; 13us; 1us; 14us; 1us; 14us; |]\nlet _fsyacc_stateToProdIdxsTableRowOffsets = [|0us; 2us; 4us; 6us; 9us; 11us; 13us; 15us; 17us; 19us; 21us; 23us; 25us; 27us; 29us; 31us; 33us; 35us; 37us; 39us; 42us; 44us; 47us; 50us; 53us; 56us; 58us; 60us; 62us; 64us; 66us; 68us; |]\nlet _fsyacc_action_rows = 32\nlet _fsyacc_actionTableElements = [|5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 0us; 49152us; 0us; 16385us; 1us; 16386us; 11us; 30us; 0us; 16387us; 0us; 16388us; 0us; 16389us; 1us; 32768us; 1us; 8us; 4us; 32768us; 0us; 7us; 14us; 6us; 15us; 5us; 16us; 4us; 1us; 32768us; 2us; 10us; 0us; 16390us; 1us; 32768us; 12us; 12us; 4us; 32768us; 0us; 7us; 14us; 6us; 15us; 5us; 16us; 4us; 0us; 16391us; 4us; 32768us; 0us; 7us; 14us; 6us; 15us; 5us; 16us; 4us; 1us; 32768us; 4us; 16us; 5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 0us; 16392us; 5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 2us; 32768us; 5us; 20us; 11us; 30us; 0us; 16393us; 4us; 32768us; 0us; 7us; 14us; 6us; 15us; 5us; 16us; 4us; 1us; 32768us; 8us; 23us; 5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 1us; 16394us; 9us; 25us; 5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 0us; 16395us; 4us; 32768us; 0us; 7us; 14us; 6us; 15us; 5us; 16us; 4us; 0us; 16396us; 0us; 16397us; 5us; 32768us; 3us; 14us; 6us; 18us; 7us; 21us; 10us; 27us; 16us; 11us; 0us; 16398us; |]\nlet _fsyacc_actionTableRowOffsets = [|0us; 6us; 7us; 8us; 10us; 11us; 12us; 13us; 15us; 20us; 22us; 23us; 25us; 30us; 31us; 36us; 38us; 44us; 45us; 51us; 54us; 55us; 60us; 62us; 68us; 70us; 76us; 77us; 82us; 83us; 84us; 90us; |]\nlet _fsyacc_reductionSymbolCounts = [|1us; 1us; 1us; 1us; 1us; 1us; 4us; 3us; 4us; 3us; 4us; 6us; 2us; 1us; 3us; |]\nlet _fsyacc_productionToNonTerminalTable = [|0us; 1us; 2us; 3us; 3us; 3us; 3us; 4us; 4us; 4us; 4us; 4us; 4us; 5us; 5us; |]\nlet _fsyacc_immediateActions = [|65535us; 49152us; 16385us; 65535us; 16387us; 16388us; 16389us; 65535us; 65535us; 65535us; 16390us; 65535us; 65535us; 16391us; 65535us; 65535us; 65535us; 16392us; 65535us; 65535us; 16393us; 65535us; 65535us; 65535us; 65535us; 65535us; 16395us; 65535us; 16396us; 16397us; 65535us; 16398us; |]\nlet _fsyacc_reductions () = [| \n# 184 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> Ast.Prog in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n raise (FSharp.Text.Parsing.Accept(Microsoft.FSharp.Core.Operators.box _1))\n )\n : 'gentype__startstart));\n# 193 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> 'gentype_Prog in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 37 \"pars.fsy\"\n _1 \n )\n# 37 \"pars.fsy\"\n : Ast.Prog ));\n# 204 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> 'gentype_StmtList in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 40 \"pars.fsy\"\n Prog(List.rev(_1)) \n )\n# 40 \"pars.fsy\"\n : 'gentype_Prog));\n# 215 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> string in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 42 \"pars.fsy\"\n Val(_1); \n )\n# 42 \"pars.fsy\"\n : 'gentype_Expr));\n# 226 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> System.Int32 in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 43 \"pars.fsy\"\n Int(_1) \n )\n# 43 \"pars.fsy\"\n : 'gentype_Expr));\n# 237 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> System.Double in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 44 \"pars.fsy\"\n Float(_1) \n )\n# 44 \"pars.fsy\"\n : 'gentype_Expr));\n# 248 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _3 = parseState.GetInput(3) :?> 'gentype_Expr in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 45 \"pars.fsy\"\n Decr(_3) \n )\n# 45 \"pars.fsy\"\n : 'gentype_Expr));\n# 259 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> string in\n let _3 = parseState.GetInput(3) :?> 'gentype_Expr in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 47 \"pars.fsy\"\n Assign(_1,_3) \n )\n# 47 \"pars.fsy\"\n : 'gentype_Stmt));\n# 271 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _2 = parseState.GetInput(2) :?> 'gentype_Expr in\n let _4 = parseState.GetInput(4) :?> 'gentype_Stmt in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 48 \"pars.fsy\"\n While(_2,_4) \n )\n# 48 \"pars.fsy\"\n : 'gentype_Stmt));\n# 283 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _2 = parseState.GetInput(2) :?> 'gentype_StmtList in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 49 \"pars.fsy\"\n Seq(List.rev(_2)) \n )\n# 49 \"pars.fsy\"\n : 'gentype_Stmt));\n# 294 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _2 = parseState.GetInput(2) :?> 'gentype_Expr in\n let _4 = parseState.GetInput(4) :?> 'gentype_Stmt in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 50 \"pars.fsy\"\n IfThen(_2,_4) \n )\n# 50 \"pars.fsy\"\n : 'gentype_Stmt));\n# 306 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _2 = parseState.GetInput(2) :?> 'gentype_Expr in\n let _4 = parseState.GetInput(4) :?> 'gentype_Stmt in\n let _6 = parseState.GetInput(6) :?> 'gentype_Stmt in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 51 \"pars.fsy\"\n IfThenElse(_2,_4,_6) \n )\n# 51 \"pars.fsy\"\n : 'gentype_Stmt));\n# 319 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _2 = parseState.GetInput(2) :?> 'gentype_Expr in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 52 \"pars.fsy\"\n Print(_2) \n )\n# 52 \"pars.fsy\"\n : 'gentype_Stmt));\n# 330 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> 'gentype_Stmt in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 55 \"pars.fsy\"\n [_1] \n )\n# 55 \"pars.fsy\"\n : 'gentype_StmtList));\n# 341 \"pars.fs\"\n (fun (parseState : FSharp.Text.Parsing.IParseState) ->\n let _1 = parseState.GetInput(1) :?> 'gentype_StmtList in\n let _3 = parseState.GetInput(3) :?> 'gentype_Stmt in\n Microsoft.FSharp.Core.Operators.box\n (\n (\n# 56 \"pars.fsy\"\n _3 :: _1 \n )\n# 56 \"pars.fsy\"\n : 'gentype_StmtList));\n|]\n# 354 \"pars.fs\"\nlet tables : FSharp.Text.Parsing.Tables<_> = \n { reductions= _fsyacc_reductions ();\n endOfInputTag = _fsyacc_endOfInputTag;\n tagOfToken = tagOfToken;\n dataOfToken = _fsyacc_dataOfToken; \n actionTableElements = _fsyacc_actionTableElements;\n actionTableRowOffsets = _fsyacc_actionTableRowOffsets;\n stateToProdIdxsTableElements = _fsyacc_stateToProdIdxsTableElements;\n stateToProdIdxsTableRowOffsets = _fsyacc_stateToProdIdxsTableRowOffsets;\n reductionSymbolCounts = _fsyacc_reductionSymbolCounts;\n immediateActions = _fsyacc_immediateActions;\n gotos = _fsyacc_gotos;\n sparseGotoTableRowOffsets = _fsyacc_sparseGotoTableRowOffsets;\n tagOfErrorTerminal = _fsyacc_tagOfErrorTerminal;\n parseError = (fun (ctxt:FSharp.Text.Parsing.ParseErrorContext<_>) -> \n match parse_error_rich with \n | Some f -> f ctxt\n | None -> parse_error ctxt.Message);\n numTerminals = 20;\n productionToNonTerminalTable = _fsyacc_productionToNonTerminalTable }\nlet engine lexer lexbuf startState = tables.Interpret(lexer, lexbuf, startState)\nlet start lexer lexbuf : Ast.Prog =\n engine lexer lexbuf 0 :?> _\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/pars.fsi", "content": "// Signature file for parser generated by fsyacc\nmodule Parser\ntype token = \n | DECR\n | LPAREN\n | RPAREN\n | WHILE\n | DO\n | END\n | BEGIN\n | IF\n | THEN\n | ELSE\n | PRINT\n | SEMI\n | ASSIGN\n | EOF\n | FLOAT of (System.Double)\n | INT of (System.Int32)\n | ID of (string)\ntype tokenId = \n | TOKEN_DECR\n | TOKEN_LPAREN\n | TOKEN_RPAREN\n | TOKEN_WHILE\n | TOKEN_DO\n | TOKEN_END\n | TOKEN_BEGIN\n | TOKEN_IF\n | TOKEN_THEN\n | TOKEN_ELSE\n | TOKEN_PRINT\n | TOKEN_SEMI\n | TOKEN_ASSIGN\n | TOKEN_EOF\n | TOKEN_FLOAT\n | TOKEN_INT\n | TOKEN_ID\n | TOKEN_end_of_input\n | TOKEN_error\ntype nonTerminalId = \n | NONTERM__startstart\n | NONTERM_start\n | NONTERM_Prog\n | NONTERM_Expr\n | NONTERM_Stmt\n | NONTERM_StmtList\n/// This function maps tokens to integer indexes\nval tagOfToken: token -> int\n\n/// This function maps integer indexes to symbolic token ids\nval tokenTagToTokenId: int -> tokenId\n\n/// This function maps production indexes returned in syntax errors to strings representing the non terminal that would be produced by that production\nval prodIdxToNonTerminal: int -> nonTerminalId\n\n/// This function gets the name of a token as a string\nval token_to_string: token -> string\nval start : (FSharp.Text.Lexing.LexBuffer<'cty> -> token) -> FSharp.Text.Lexing.LexBuffer<'cty> -> ( Ast.Prog ) \n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/pars.fsy", "content": "// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n//\n\n// This example shows how to write an F# Yacc parser file which creates\n// nodes that carry F# values.\n\n%{\n\n// This prelude is F# code that is available throughout this file. In this\n// case we just open a module to reveal some datatype definitions.\nopen Ast\n\n%}\n\n// The start token becomes a parser function in the compiled code: */\n%start start\n\n// These are the terminal tokens of the grammar along with the types of\n// the data carried by each token:\n%token ID\n%token INT\n%token FLOAT\n%token DECR LPAREN RPAREN WHILE DO END BEGIN IF THEN ELSE PRINT SEMI ASSIGN EOF\n\n// This is the type of the data produced by a successful reduction of the 'start'\n// symbol:\n%type < Ast.Prog > start\n\n%%\n\n// These are the rules of the grammar along with the F# code of the\n// actions executed as rules are reduced. In this case the actions\n// produce data using F# data construction terms.\nstart: Prog { $1 }\n\n\nProg: StmtList { Prog(List.rev($1)) }\n\nExpr: ID { Val($1); }\n | INT { Int($1) }\n | FLOAT { Float($1) }\n | DECR LPAREN Expr RPAREN { Decr($3) }\n\nStmt: ID ASSIGN Expr { Assign($1,$3) }\n | WHILE Expr DO Stmt { While($2,$4) }\n | BEGIN StmtList END { Seq(List.rev($2)) }\n | IF Expr THEN Stmt { IfThen($2,$4) }\n | IF Expr THEN Stmt ELSE Stmt { IfThenElse($2,$4,$6) }\n | PRINT Expr { Print($2) }\n\nStmtList:\n | Stmt { [$1] }\n | StmtList SEMI Stmt { $3 :: $1 }\n\n\n" }, { "path": "Samples/FSharpParsingSample/LexYaccVersion/test.lang", "content": "\na := 1;\nb := 0;\nif a then d := 20;\nif b then d := 40;\nprint d;\nwhile d do \n begin\n d := decr(d);\n print d\n end;\nprint d\n\n\n\n" }, { "path": "Samples/FSharpParsingSample/readme.txt", "content": "This sample is derived from the parsing sample that shipped with the F# 1.9.4.19\ndistribution originally published at\nhttp://research.microsoft.com/fsharp/release.aspx\n\nThe original sample code was provided by Microsoft under the following licence:\n\n// Copyright (c) Microsoft Corporation 2005-2006.\n// This sample code is provided \"as is\" without warranty of any kind.\n// We disclaim all warranties, either express or implied, including the\n// warranties of merchantability and fitness for a particular purpose.\n" }, { "path": "Samples/JSON/JsonParser-LowTrust.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/JSON/JsonParser.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/JSON/JsonParser.targets", "content": "\n\n \n JsonParser\n JsonParser\n Exe\n false\n $(MSBuildProjectDirectory)/test_json.txt\n \n\n \n \n \n \n \n \n \n\n\n" }, { "path": "Samples/JSON/PegParser-LowTrust.fsproj", "content": "\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/JSON/ast.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule Ast\n\n[]\ntype Json = JString of string\n | JNumber of float\n | JBool of bool\n | JNull\n | JList of Json list\n | JObject of Map\n with\n member private t.StructuredFormatDisplay =\n match t with\n | JString s -> box (\"\\\"\" + s + \"\\\"\")\n | JNumber f -> box f\n | JBool b -> box b\n | JNull -> box \"null\"\n | JList l -> box l\n | JObject m -> Map.toList m :> obj\n" }, { "path": "Samples/JSON/main.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008\n// License: Simplified BSD License. See accompanying documentation.\n\n// See parser.fs for more information.\n\nopen FParsec.CharParsers\n\nopen Ast\nopen Parser\n\n\n[]\nlet main(args: string[]) =\n if args.Length <> 1 then\n printf \"usage: json.exe \\n\"\n exit 1\n\n // The parser is run on the file path in args[0].\n // If the file has no byte order marks, System.Text.Encoding.Default\n // is assumed to be the encoding.\n // The parser result will be the abstract syntax tree of the input file.\n let result = parseJsonFile args[0] System.Text.Encoding.UTF8\n // for the moment we just print out the AST\n match result with\n | Success (v, _, _) ->\n printf \"The AST of the input file is:\\n%A\\n\" v\n 0\n | Failure (msg, err, _) ->\n printfn \"%s\" msg\n 1\n\n\n" }, { "path": "Samples/JSON/parser.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule Parser\n\nopen FParsec\n\nopen Ast\n\n// This is a general JSON parser that will parse any JSON file into an AST.\n// See e.g. http://www.json.org/, for a specification of JSON.\n\n// The FParsec tutorial discusses this parser in detail.\n\n// Note that in typical applications you often don't need to parse any general\n// JSON file, but only files describing objects of a certain type. In those cases\n// it might be more convenient to parse the input with specialized parsers\n// instead of using the indirect approach via an intermediate AST. The parser\n// definitions below should be useful in any case.\n\nlet jnull = stringReturn \"null\" JNull\nlet jtrue = stringReturn \"true\" (JBool true)\nlet jfalse = stringReturn \"false\" (JBool false)\n\nlet jnumber = pfloat |>> JNumber // pfloat will accept a little more than specified by JSON\n // as valid numbers (such as NaN or Infinity), but that makes\n // it only more robust\n\nlet str s = pstring s\n\nlet stringLiteral =\n let escape = anyOf \"\\\"\\\\/bfnrt\"\n |>> function\n | 'b' -> \"\\b\"\n | 'f' -> \"\\u000C\"\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c // every other char is mapped to itself\n\n let unicodeEscape =\n /// converts a hex char ([0-9a-fA-F]) to its integer number (0-15)\n let hex2int c = (int c &&& 15) + (int c >>> 6)*9\n\n str \"u\" >>. pipe4 hex hex hex hex (fun h3 h2 h1 h0 ->\n (hex2int h3)*4096 + (hex2int h2)*256 + (hex2int h1)*16 + hex2int h0\n |> char |> string\n )\n\n let escapedCharSnippet = str \"\\\\\" >>. (escape <|> unicodeEscape)\n let normalCharSnippet = manySatisfy (fun c -> c <> '\"' && c <> '\\\\')\n\n between (str \"\\\"\") (str \"\\\"\")\n (stringsSepBy normalCharSnippet escapedCharSnippet)\n\nlet jstring = stringLiteral |>> JString\n\n// jvalue, jlist and jobject are three mutually recursive grammar productions.\n// In order to break the cyclic dependency, we make jvalue a parser that\n// forwards all calls to a parser in a reference cell.\nlet jvalue, jvalueRef = createParserForwardedToRef() // initially jvalueRef holds a reference to a dummy parser\n\nlet ws = spaces // skips any whitespace\n\nlet listBetweenStrings sOpen sClose pElement f =\n between (str sOpen) (str sClose)\n (ws >>. sepBy (pElement .>> ws) (str \",\" .>> ws) |>> f)\n\nlet keyValue = tuple2 stringLiteral (ws >>. str \":\" >>. ws >>. jvalue)\n\nlet jlist = listBetweenStrings \"[\" \"]\" jvalue JList\nlet jobject = listBetweenStrings \"{\" \"}\" keyValue (Map.ofList >> JObject)\n\ndo jvalueRef := choice [jobject\n jlist\n jstring\n jnumber\n jtrue\n jfalse\n jnull]\n\nlet json = ws >>. jvalue .>> ws .>> eof\n\nlet parseJsonString str = run json str\n\n// UTF8 is the default, but it will detect UTF16 or UTF32 byte-order marks automatically\nlet parseJsonFile fileName encoding =\n runParserOnFile json () fileName encoding\n\nlet parseJsonStream stream encoding =\n runParserOnStream json () \"\" stream System.Text.Encoding.UTF8" }, { "path": "Samples/JSON/test_json.txt", "content": "{\n \"glossary\": {\n \"title\": \"example glossary\",\n \"GlossDiv\": {\n \"title\": \"S\",\n \"GlossList\": {\n \"GlossEntry\": {\n \"ID\": \"SGML\",\n \"SortAs\": \"SGML\",\n \"GlossTerm\": \"Standard Generalized Markup Language\",\n \"Acronym\": \"SGML\",\n \"Abbrev\": \"ISO 8879:1986\",\n \"GlossDef\": {\n \"para\": \"A meta-markup language, used to create markup languages such as DocBook.\",\n \"GlossSeeAlso\": [\"GML\", \"XML\"]\n },\n \"GlossSee\": \"markup\"\n }\n }\n }\n }\n}" }, { "path": "Samples/PEG/PegParser-LowTrust.fsproj", "content": "\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/PEG/PegParser.fsproj", "content": "\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/PEG/PegParser.targets", "content": "\n\n \n PegParser\n PegParser\n Exe\n false\n $(MSBuildProjectDirectory)/test_peg.txt\n \n\n \n \n \n \n \n \n \n\n\n" }, { "path": "Samples/PEG/ast.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2008\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule Ast\n\ntype Grammar = Definition list\n\nand Definition = Def of string * Expression\n\nand Range =\n | Char of char\n | Range of char * char\n\nand Expression =\n /// expression1 / expression2 / ...\n | Alt of Expression list\n /// expression1 expression2 ...\n | Seq of Expression list\n /// expression?\n | Opt of Expression\n /// expression*\n | Star of Expression\n /// expression+\n | Plus of Expression\n /// &expression\n | And of Expression\n /// !expression\n | Not of Expression\n | Class of Range list\n | Literal of string\n | Identifier of string\n | Dot" }, { "path": "Samples/PEG/main.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2008\n// License: Simplified BSD License. See accompanying documentation.\n\n// This is a simple parser for PEG grammars.\n// See parser.fs for more information.\n\nopen FParsec\n\nopen Ast\n\n[]\nlet main(args: string[]) =\n if args.Length <> 1 then\n printf \"usage: peg.exe \\n\"\n exit 1\n\n // The parser is run on the file path in args[0].\n // If the file has no byte order marks, System.Text.Encoding.Default\n // is assumed to be the encoding.\n // The parser result will be the abstract syntax tree of the input file.\n let fileName = args[0]\n let result = runParserOnFile Parser.pGrammar () fileName System.Text.Encoding.UTF8\n // for the moment we just print out the AST\n match result with\n | Success (v, _, _) -> printf \"The ast for the input file is:\\n%A\\n\" v\n | Failure (msg, err, _) -> printf \"%s\\n\" msg\n 0\n" }, { "path": "Samples/PEG/parser.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule Parser\n\nopen System\nopen FParsec\n\nopen Ast\n\n// The following is a close translation of the grammar on page 2 of\n// Parsing Expression Grammars: A Recognition-Based Syntactic Foundation, Bryan Ford.\n// 31st ACM Symposium on Principles of Programming Languages, January 14-16, 2004, Venice, Italy.\n// http://www.bford.info/pub/lang/peg.pdf\n\n// If you're new to FParsec, take a look at\n// http://www.quanttec.com/fparsec/reference/parser-overview.html\n\n// some abbreviations\nlet str s = pstring s\n\n// Lexical syntax\n\nlet pEndOfFile = eof\n//let pEndOfLine = skipNewline\n//let pSpace = skipAnyOf \" \\t\\n\"\n\nlet pComment = str \"#\" >>. skipRestOfLine true\n\nlet pSpacing = // literal translation:\n // skipManyChars (pSpace <|> pComment)\n // more efficient:\n skipSepBy spaces pComment\n\nlet LEFTARROW = str \"<-\" >>. pSpacing\nlet SLASH = str \"/\" >>. pSpacing\nlet AND = str \"&\" >>. pSpacing\nlet NOT = str \"!\" >>. pSpacing\n//let QUESTION = str \"?\" .>> pSpacing\n//let STAR = str \"*\" .>> pSpacing\n//let PLUS = str \"+\" .>> pSpacing\nlet OPEN = str \"(\" >>. pSpacing\nlet CLOSE = str \")\" >>. pSpacing\nlet DOT = str \".\" >>. pSpacing\n\n// Instead of the odd octal escapes in the original grammar,\n// we accept the usual UTF16 character escapes '\\uxxxx'\nlet pChar =\n let escape = anyOf \"nrt'\\\"[]\\\\\" |>> function\n | 'n' -> '\\n'\n | 'r' -> '\\r'\n | 't' -> '\\t'\n | c -> c\n\n let unicodeEscape =\n str \"u\" >>. pipe4 hex hex hex hex (fun h3 h2 h1 h0 ->\n let hex2int c = (int c &&& 15) + (int c >>> 6)*9 // hex char to int\n (hex2int h3)*4096 + (hex2int h2)*256 + (hex2int h1)*16 + hex2int h0\n |> char\n )\n\n satisfy ((<>) '\\\\') <|> (str \"\\\\\" >>. (escape <|> unicodeEscape))\n\n\nlet pRange =\n pipe2 pChar (opt (str \"-\" >>. pChar))\n (fun c1 c2Opt ->\n match c2Opt with\n | None -> Char c1\n | Some(c2) -> Range(c1, c2))\n\n\nlet pClass = str \"[\" >>. (manyTill pRange (str \"]\") .>> pSpacing |>> Class)\n\nlet pLiteralString = ( (str \"\\'\" >>. (manyCharsTill pChar (str \"\\'\")))\n <|> (str \"\\\"\" >>. (manyCharsTill pChar (str \"\\\"\" )))) .>> pSpacing\nlet pLiteral = pLiteralString |>> Literal\n\n\nlet isIdentifierStart = fun c -> isAsciiLetter c || c = '_' // \"A-Za-z_\"\nlet isIdentifierCont = fun c -> isAsciiLetter c || isDigit c || c = '_' // A-Za-z_0-9\nlet pIdentifierString = many1Satisfy2 isIdentifierStart isIdentifierCont .>> pSpacing\nlet pIdentifier = pIdentifierString |>> Identifier\n\nlet pDot = DOT >>% Dot\n\n\n// Hierarchical syntax\n\n// expression, sequence, prefix, suffix and primary are mutually recursive\n// grammar productions. In order to break the cyclic dependency, we make\n// pPrimary a parser that forwards all calls to a parser in a reference cell.\nlet pPrimary, pPrimaryRef = createParserForwardedToRef() // initially pPrimary holds a reference to a dummy parser\n\nlet pSuffix =\n // returns 'x' if there is no '?', '*' or '+'\n pipe2 pPrimary (anyOf \"?*+\" <|>% 'x')\n (fun p c ->\n match c with\n | '?' -> Opt p\n | '*' -> Star p\n | '+' -> Plus p\n | _ -> p)\n\n\nlet pPrefix = choice [AND >>. (pSuffix |>> And)\n NOT >>. (pSuffix |>> Not)\n pSuffix]\n .>> pSpacing\n\nlet pSequence = many pPrefix |>> function\n | [exp] -> exp\n | exps -> Seq exps\n\nlet pExpression = sepBy1 pSequence SLASH |>> function\n | [exp] -> exp\n | exps -> Alt exps // only use an Alt for more than one alternative\n\npPrimaryRef:= choice [pIdentifier .>>? notFollowedByString \"<-\" // backtracks to the beginning if the id is followed by \"<-\"\n between OPEN CLOSE pExpression\n pLiteral\n pClass\n pDot]\n\nlet pDefinition = pipe2 pIdentifierString (LEFTARROW >>. pExpression)\n (fun s e -> Def (s, e))\n\nlet pGrammar: Parser<_, unit> = // one type annotation is enough for the whole parser\n pSpacing >>. many1 pDefinition .>> pEndOfFile\n" }, { "path": "Samples/PEG/test_peg.txt", "content": "# Source: \n# Parsing Expression Grammars: A Recognition-Based Syntactic Foundation, Bryan Ford.\n# 31st ACM Symposium on Principles of Programming Languages, January 14-16, 2004, Venice, Italy.\n# http://www.bford.info/pub/lang/peg.pdf\n\n# PEG formally describing its own ASCII syntax\n\n# Hierarchical syntax\nGrammar <- Spacing Definition+ EndOfFile\nDefinition <- Identifier LEFTARROW Expression\nExpression <- Sequence (SLASH Sequence)*\nSequence <- Prefix*\nPrefix <- (AND / NOT)? Suffix\nSuffix <- Primary (QUESTION / STAR / PLUS)?\nPrimary <- Identifier !LEFTARROW\n / OPEN Expression CLOSE\n / Literal / Class / DOT\n \n# Lexical syntax\nIdentifier <- IdentStart IdentCont* Spacing\nIdentStart <- [a-zA-Z_]\nIdentCont <- IdentStart / [0-9]\nLiteral <- ['] (!['] Char)* ['] Spacing\n / [\"] (![\"] Char)* [\"] Spacing\nClass <- '[' (!']' Range)* ']' Spacing\nRange <- Char '-' Char / Char\nChar <- '\\\\' [nrt'\"\\[\\]\\\\]\n / '\\\\' [0-2][0-7][0-7]\n / '\\\\' [0-7][0-7]?\n / !'\\\\' .\n\nLEFTARROW <- '<-' Spacing\nSLASH <- '/' Spacing\nAND <- '&' Spacing\nNOT <- '!' Spacing\nQUESTION <- '?' Spacing\nSTAR <- '*' Spacing\nPLUS <- '+' Spacing\nOPEN <- '(' Spacing\nCLOSE <- ')' Spacing\nDOT <- '.' Spacing\n\nSpacing <- (Space / Comment)*\nComment <- '#' (!EndOfLine .)* EndOfLine\nSpace <- ' ' / '\\t' / EndOfLine\nEndOfLine <- '\\r\\n' / '\\n' / '\\r'\nEndOfFile <- !.\n\n\n\n" }, { "path": "Samples/Tutorial/Tutorial-LowTrust.fsproj", "content": "\n\n \n net6;\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Tutorial/Tutorial.fsproj", "content": "\n\n \n net6\n \n\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Tutorial/Tutorial.targets", "content": "\n\n \n Tutorial\n Tutorial\n Exe\n false\n \n\n \n \n \n \n\n\n" }, { "path": "Samples/Tutorial/tutorial.fs", "content": "// Copyright (c) Stephan Tolksdorf 2011\n// License: Simplified BSD License. See accompanying documentation.\n\n// Source code for the tutorial in the documentation\n\n// 2 Parsing a single float\n\nopen FParsec\n\nlet test p str =\n match run p str with\n | Success(result, _, _) -> printfn \"Success: %A\" result\n | Failure(errorMsg, _, _) -> printfn \"Failure: %s\" errorMsg\n\ntest pfloat \"1.25\"\ntest pfloat \"1.25E 2\"\n\n// 3 Parsing a float between brackets\n\nlet str s = pstring s\nlet floatBetweenBrackets = str \"[\" >>. pfloat .>> str \"]\"\n\ntest floatBetweenBrackets \"[1.0]\"\ntest floatBetweenBrackets \"[]\"\ntest floatBetweenBrackets \"[1.0]\"\n\n// 4 Abstracting parsers\n\nlet betweenStrings s1 s2 p = str s1 >>. p .>> str s2\nlet floatBetweenBrackets_ = pfloat |> betweenStrings \"[\" \"]\"\nlet floatBetweenDoubleBrackets_ = pfloat |> betweenStrings \"[[\" \"]]\"\n\ntest floatBetweenBrackets_ \"[1.0]\"\ntest floatBetweenDoubleBrackets_ \"[[1.0]]\"\n\nlet between_ pBegin pEnd p = pBegin >>. p .>> pEnd\nlet betweenStrings_ s1 s2 p = p |> between_ (str s1) (str s2)\n\n// 5 Parsing a list of floats\n\ntest (many floatBetweenBrackets) \"\"\ntest (many floatBetweenBrackets) \"[1.0]\"\ntest (many floatBetweenBrackets) \"[2][3][4]\"\ntest (many floatBetweenBrackets) \"[1][2.0E]\"\n\ntest (many1 floatBetweenBrackets) \"(1)\"\n\ntest (many1 (floatBetweenBrackets \"float between brackets\")) \"(1)\"\n\nlet floatList = str \"[\" >>. sepBy pfloat (str \",\") .>> str \"]\"\n\ntest floatList \"[]\"\ntest floatList \"[1.0]\"\ntest floatList \"[4,5,6]\"\n\ntest floatList \"[1.0,\"\n\n// 6 Handling whitespace\n\ntest floatBetweenBrackets \"[1.0, 2.0]\"\n\nlet ws = spaces\nlet str_ws s = pstring s .>> ws\nlet float_ws = pfloat .>> ws\nlet numberList = str_ws \"[\" >>. sepBy float_ws (str_ws \",\") .>> str_ws \"]\"\n\ntest numberList @\"[ 1 ,\n 2 ] \"\n\ntest numberList @\"[ 1,\n 2; 3]\"\n\nlet numberListFile = ws >>. numberList .>> eof\ntest numberListFile \" [1, 2, 3] [4]\"\n\n// 7 Parsing string data\n\ntest (many (str \"a\" <|> str \"b\")) \"abba\"\n\ntest (skipStringCI \"\" >>. pfloat) \"1.0\"\n\nlet identifier =\n let isIdentifierFirstChar c = isLetter c || c = '_'\n let isIdentifierChar c = isLetter c || isDigit c || c = '_'\n\n many1Satisfy2L isIdentifierFirstChar isIdentifierChar \"identifier\"\n .>> ws // skips trailing whitepace\n\ntest identifier \"_\"\ntest identifier \"_test1=\"\ntest identifier \"1\"\n\nlet stringLiteral =\n let normalChar = satisfy (fun c -> c <> '\\\\' && c <> '\"')\n let unescape c = match c with\n | 'n' -> '\\n'\n | 'r' -> '\\r'\n | 't' -> '\\t'\n | c -> c\n let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> unescape)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (manyChars (normalChar <|> escapedChar))\n\ntest stringLiteral \"\\\"abc\\\"\"\ntest stringLiteral \"\\\"abc\\\\\\\"def\\\\\\\\ghi\\\"\"\ntest stringLiteral \"\\\"abc\\\\def\\\"\"\n\nlet stringLiteral2 =\n let normalCharSnippet = many1Satisfy (fun c -> c <> '\\\\' && c <> '\"')\n let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> function\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (manyStrings (normalCharSnippet <|> escapedChar))\n\ntest stringLiteral2 \"\\\"abc\\\"\"\ntest stringLiteral2 \"\\\"abc\\\\\\\"def\\\\\\\\ghi\\\"\"\ntest stringLiteral2 \"\\\"abc\\\\def\\\"\"\n\nlet stringLiteral3 =\n let normalCharSnippet = manySatisfy (fun c -> c <> '\\\\' && c <> '\"')\n let escapedChar = pstring \"\\\\\" >>. (anyOf \"\\\\nrt\\\"\" |>> function\n | 'n' -> \"\\n\"\n | 'r' -> \"\\r\"\n | 't' -> \"\\t\"\n | c -> string c)\n between (pstring \"\\\"\") (pstring \"\\\"\")\n (stringsSepBy normalCharSnippet escapedChar)\n\ntest stringLiteral3 \"\\\"abc\\\"\"\ntest stringLiteral3 \"\\\"abc\\\\\\\"def\\\\\\\\ghi\\\"\"\ntest stringLiteral3 \"\\\"abc\\\\def\\\"\"\n\n// 8 Sequentially applying parsers\n\nlet product = pipe2 float_ws (str_ws \"*\" >>. float_ws)\n (fun x y -> x * y)\n\ntest product \"3 * 5\";;\n\ntype StringConstant = StringConstant of string * string\n\nlet stringConstant = pipe3 identifier (str_ws \"=\") stringLiteral\n (fun id _ str -> StringConstant(id, str))\n\ntest stringConstant \"myString = \\\"stringValue\\\"\"\n\ntest (float_ws .>>. (str_ws \",\" >>. float_ws)) \"123, 456\"\n\nlet pipe7 p1 p2 p3 p4 p5 p6 p7 f =\n pipe4 p1 p2 p3 (tuple4 p4 p5 p6 p7)\n (fun x1 x2 x3 (x4, x5, x6, x7) -> f x1 x2 x3 x4 x5 x6 x7)\n\n// 9 Parsing alternatives\n\nlet boolean = (stringReturn \"true\" true)\n <|> (stringReturn \"false\" false)\n\ntest boolean \"false\"\ntest boolean \"true\"\ntest boolean \"tru\"\n\ntest ((ws >>. str \"a\") <|> (ws >>. str \"b\")) \" b\"\n\ntest (ws >>. (str \"a\" <|> str \"b\")) \" b\"\n" }, { "path": "Test/AllTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\n\nlet run() =\n printfn \"Testing FParsec.Buffer ...\"\n FParsec.Test.BufferTests.run()\n printfn \"Testing FParsec.CharSet ...\"\n FParsec.Test.CharSetTests.run()\n printfn \"Testing FParsec.HexFloat ...\"\n FParsec.Test.HexFloatTests.run()\n printfn \"Testing FParsec.Text ...\"\n FParsec.Test.TextTests.run()\n#if !LOW_TRUST\n #if !DISABLE_STREAM_BACKTRACKING_TESTS\n // In .NET Core System.Text.Decoder no longer support serialization, see https://github.com/stephan-tolksdorf/fparsec/issues/95\n printfn \"Testing FParsec.Cloning ...\"\n FParsec.Test.CloningTests.run()\n #endif\n printfn \"Testing FParsec.StringBuffer ...\"\n FParsec.Test.StringBufferTests.run()\n#endif\n printfn \"Testing FParsec.CharStream ...\"\n FParsec.Test.CharStreamTests.run()\n printfn \"Testing FParsec.Primitives ...\"\n FParsec.Test.PrimitivesTests.run()\n printfn \"Testing FParsec.CharParsers ...\"\n FParsec.Test.CharParsersTests.run()\n printfn \"Testing FParsec.OperatorPrecedenceParserTests ...\"\n FParsec.Test.OperatorPrecedenceParserTests.run()\n printfn \"Testing FParsec.IdentifierValidator ...\"\n FParsec.Test.IdentifierValidatorTests.run()\n#if !LOW_TRUST\n printfn \"Testing FParsec.StaticMapping ... \"\n printfn \"(this can take a while)\"\n if System.Diagnostics.Debugger.IsAttached then\n printfn \"Note: When the Visual Studio debugger is attached, this test requires lots of memory.\"\n FParsec.Test.RangeTests.run()\n FParsec.Test.StaticMappingTests.run()\n#endif\n printfn \"No error was found.\"\n\n[]\nlet main _argv =\n\n#if NETCOREAPP\n System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);\n#endif\n\n try\n run()\n 0\n with\n | ex ->\n printfn $\"error: {ex}\"\n 1\n" }, { "path": "Test/BufferTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.BufferTests\n\nopen System\nopen System.Runtime.InteropServices\nopen Microsoft.FSharp.NativeInterop\nopen FParsec.Test.Test\n\n#nowarn \"9\" // \"Uses of this construct may result in the generation of unverifiable .NET IL code.\"\n\ntype Buffer = FParsec.Buffer\n\nlet testSwapByteOrder() =\n Buffer.SwapByteOrder(0xffffffffu) |> Equal 0xffffffffu\n Buffer.SwapByteOrder(0x00000000u) |> Equal 0x00000000u\n Buffer.SwapByteOrder(0x12345678u) |> Equal 0x78563412u\n Buffer.SwapByteOrder(0xffffffffffffffffUL) |> Equal 0xffffffffffffffffUL\n Buffer.SwapByteOrder(0x0000000000000000UL) |> Equal 0x0000000000000000UL\n Buffer.SwapByteOrder(0x123456789abcdef0UL) |> Equal 0xf0debc9a78563412UL\n#if LOW_TRUST\n let array = [|0x12345678u; 0x9abcdef0u; 0x12345678u|]\n Buffer.SwapByteOrder(array)\n array |> Equal [|0x78563412u; 0xf0debc9au; 0x78563412u;|]\n#else\n let p = NativePtr.stackalloc 3\n NativePtr.set p 0 0x12345678u\n NativePtr.set p 1 0x9abcdef0u\n NativePtr.set p 2 0x12345678u\n Buffer.SwapByteOrder(Span<_>(NativePtr.toVoidPtr p, 3))\n NativePtr.get p 0 |> Equal 0x78563412u\n NativePtr.get p 1 |> Equal 0xf0debc9au\n NativePtr.get p 2 |> Equal 0x78563412u\n#endif\n Buffer.SwapByteOrder(0x01020304u) |> Equal 0x04030201u\n\n#if !LOW_TRUST\n\nlet testCopy() =\n let n = 64\n let bytes = Array.init n (fun i -> byte i)\n let buffer1 = Array.zeroCreate n : byte[]\n let buffer2 = Array.zeroCreate n : byte[]\n let handle = GCHandle.Alloc(buffer2, GCHandleType.Pinned)\n let buffer2Ptr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject()) : nativeptr\n\n for iSrc = 0 to n do\n for iDst = 0 to n do\n for size = 0 to min (n - iSrc) (n - iDst) do\n Array.blit bytes 0 buffer1 0 n\n Array.blit bytes 0 buffer2 0 n\n System.Buffer.BlockCopy(buffer1, iSrc, buffer1, iDst, size)\n Buffer.Copy(NativePtr.add buffer2Ptr iDst,\n NativePtr.add buffer2Ptr iSrc, size)\n if buffer1 <> buffer2 then\n Fail()\n\n try Buffer.Copy(NativePtr.ofNativeInt 0n, NativePtr.ofNativeInt 0n, -1)\n Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\nlet testEqual() =\n let n = 16\n let buffer1 = NativePtr.stackalloc n\n let buffer2 = NativePtr.stackalloc n\n for i = 0 to n - 1 do\n NativePtr.set buffer1 i (uint32 i)\n NativePtr.set buffer2 i (uint32 i)\n\n for length = 0 to n do\n for i = 0 to length - 1 do\n Buffer.Equals(buffer1, buffer2, length) |> True\n NativePtr.set buffer2 i 0xffffffffu\n Buffer.Equals(buffer1, buffer2, length) |> False\n NativePtr.set buffer2 i (uint32 i)\n\n\n#endif\n\nlet run() =\n testSwapByteOrder()\n#if !LOW_TRUST\n testCopy()\n testEqual()\n#endif\n" }, { "path": "Test/CharParsersTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.CharParsersTests\n\nopen System.Text.RegularExpressions\n\nopen FParsec\nopen FParsec.Error\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nopen FParsec.Test.Test\n\ntype NLO = NumberLiteralOptions\ntype NLF = NumberLiteralResultFlags\n\nlet testCharParsers() =\n pchar ' ' |> ROk \" \" 1 ' '\n pchar '\\t' |> ROk \"\\t\\t\" 1 '\\t'\n pchar ' ' |> RError \"\" 0 (expectedString \" \")\n pchar ' ' |> RError \"x\" 0 (expectedString \" \")\n\n pchar '\\r' |> RError \"_\\r\" 0 Errors.ExpectedNewline\n newline |> RError \"_\\n\" 0 Errors.ExpectedNewline\n newline |> RError \"\" 0 Errors.ExpectedNewline\n\n pchar '\\n' |> ROkNL \"\\r\" 1 '\\n'\n newline |> ROkNL \"\\r\" 1 '\\n'\n pchar '\\r' |> ROkNL \"\\r\" 1 '\\r'\n pchar '\\n' |> ROkNL \"\\r\\n\" 2 '\\n'\n pchar '\\r' |> ROkNL \"\\r\\n\" 2 '\\r'\n pchar '\\n' |> ROkNL \"\\n\" 1 '\\n'\n pchar '\\r' |> ROkNL \"\\n\" 1 '\\r'\n\n skipChar '\\t' |> ROk \"\\t\" 1 ()\n charReturn '\\t' 0 |> ROk \"\\t\" 1 0\n skipNewline |> ROkNL \"\\n\" 1 ()\n newlineReturn 0 |> ROkNL \"\\r\\n\" 2 0\n\n try pchar EOS |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n anyChar |> RError \"\" 0 Errors.ExpectedAnyChar\n skipAnyChar |> RError \"\" 0 Errors.ExpectedAnyChar\n\n anyChar |> ROk \" \" 1 ' '\n anyChar |> ROk \"\\ufffe\" 1 '\\ufffe'\n skipAnyChar |> ROk \" \" 1 ()\n anyChar |> ROk \"\\t\\t\" 1 '\\t'\n skipAnyChar |> ROk \"\\t\\t\" 1 ()\n\n anyChar |> ROkNL \"\\r\\n\" 2 '\\n'\n skipAnyChar |> ROkNL \"\\r\\n\" 2 ()\n anyChar |> ROkNL \"\\n\\n\" 1 '\\n'\n skipAnyChar |> ROkNL \"\\n\\n\" 1 ()\n\n satisfy (fun c -> true) |> RError \"\" 0 NoErrorMessages\n skipSatisfy (fun c -> true) |> RError \"\" 0 NoErrorMessages\n satisfyL (fun c -> true) \"test\" |> RError \"\" 0 (expected \"test\")\n skipSatisfyL (fun c -> true) \"test\" |> RError \"\" 0 (expected \"test\")\n\n satisfy ((=) '1') |> ROk \"1\" 1 '1'\n satisfy ((=) '\\t') |> ROk \"\\t\" 1 '\\t'\n satisfy ((=) '1') |> ROk \"11\" 1 '1'\n satisfy ((=) '1') |> RError \"0\" 0 NoErrorMessages\n satisfyL ((=) '1') \"test\" |> RError \"2\" 0 (expected \"test\")\n satisfyL ((=) '\\r') \"test\" |> RError \"\\r\" 0 (expected \"test\")\n satisfy ((=) '\\n') |> ROkNL \"\\r\" 1 '\\n'\n satisfy ((=) '\\n') |> ROkNL \"\\r\\n\" 2 '\\n'\n satisfy ((=) '\\n') |> ROkNL \"\\n\" 1 '\\n'\n\n skipSatisfy ((=) '1') |> ROk \"1\" 1 ()\n skipSatisfy ((=) '\\t') |> ROk \"\\t\" 1 ()\n skipSatisfy ((=) '1') |> ROk \"11\" 1 ()\n skipSatisfy ((=) '1') |> RError \"0\" 0 NoErrorMessages\n skipSatisfyL ((=) '1') \"test\" |> RError \"2\" 0 (expected \"test\")\n skipSatisfyL ((=) '\\r') \"test\" |> RError \"\\r\" 0 (expected \"test\")\n skipSatisfy ((=) '\\n') |> ROkNL \"\\r\" 1 ()\n skipSatisfy ((=) '\\n') |> ROkNL \"\\r\\n\" 2 ()\n skipSatisfy ((=) '\\n') |> ROkNL \"\\n\" 1 ()\n\nlet testAnyNoneOf() =\n anyOf \"1\" |> ROk \"1\" 1 '1'\n anyOf \"1\" |> RError \"2\" 0 (Errors.ExpectedAnyCharIn(\"1\"))\n noneOf \"1\" |> RError \"1\" 0 (Errors.ExpectedAnyCharNotIn(\"1\"))\n noneOf \"1\" |> ROk \"2\" 1 '2'\n skipAnyOf \"1\" |> ROk \"1\" 1 ()\n skipAnyOf \"1\" |> RError \"2\" 0 (Errors.ExpectedAnyCharIn(\"1\"))\n skipNoneOf \"1\" |> RError \"1\" 0 (Errors.ExpectedAnyCharNotIn(\"1\"))\n skipNoneOf \"1\" |> ROk \"2\" 1 ()\n\n//#nowarn \"44\" // \"This construct is deprecated.\"\n\nlet testSpecialCharParsers() =\n for i = 0 to 1023 do\n let c = char i\n isUpper c |> Equal (System.Char.IsUpper(c))\n isLower c |> Equal (System.Char.IsLower(c))\n isLetter c |> Equal (System.Char.IsLetter(c))\n isAsciiUpper c |> Equal (c <= '\\u007f' && System.Char.IsUpper(c))\n isAsciiLower c |> Equal (c <= '\\u007f' && System.Char.IsLower(c))\n isAsciiLetter c |> Equal (c <= '\\u007f' && System.Char.IsLetter(c))\n isDigit c |> Equal (c >= '0' && c <= '9')\n isHex c |> Equal ((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F'))\n isOctal c |> Equal (c >= '0' && c <= '7')\n\n asciiUpper |> ROk \"A\" 1 'A'\n asciiUpper |> RError \"a\" 0 Errors.ExpectedAsciiUppercaseLetter\n asciiLower |> ROk \"a\" 1 'a'\n asciiLower |> RError \"A\" 0 Errors.ExpectedAsciiLowercaseLetter\n asciiLetter |> ROk \"A\" 1 'A'\n asciiLetter |> RError \"1\" 0 Errors.ExpectedAsciiLetter\n\n upper |> ROk \"Ä\" 1 'Ä'\n upper |> RError \"ä\" 0 Errors.ExpectedUppercaseLetter\n lower |> ROk \"ä\" 1 'ä'\n lower |> RError \"Ä\" 0 Errors.ExpectedLowercaseLetter\n letter |> ROk \"Ä\" 1 'Ä'\n letter |> RError \"1\" 0 Errors.ExpectedLetter\n\n digit |> ROk \"1\" 1 '1'\n digit |> RError \"a\" 0 Errors.ExpectedDecimalDigit\n hex |> ROk \"a\" 1 'a'\n hex |> RError \"g\" 0 Errors.ExpectedHexadecimalDigit\n octal |> ROk \"7\" 1 '7'\n octal |> RError \"8\" 0 Errors.ExpectedOctalDigit\n\n tab |> ROk \"\\t\" 1 '\\t'\n tab |> RError \"\\r\" 0 Errors.ExpectedTab\n\n unicodeNewline |> ROkNL \"\\r\" 1 '\\n'\n unicodeNewline |> ROkNL \"\\r\\n\" 2 '\\n'\n unicodeNewline |> ROkNL \"\\n\" 1 '\\n'\n unicodeNewline |> ROkNL \"\\u0085\" 1 '\\n'\n unicodeNewline |> ROkNL \"\\u2028\" 1 '\\n'\n unicodeNewline |> ROkNL \"\\u2029\" 1 '\\n'\n unicodeNewline |> RError \"\\f\" 0 Errors.ExpectedNewline\n unicodeNewline |> RError \"\\t\" 0 Errors.ExpectedNewline\n unicodeNewline |> RError \"\" 0 Errors.ExpectedNewline\n\n skipUnicodeNewline |> ROkNL \"\\u2028\" 1 ()\n\n let count p = many p |>> List.fold (fun c x -> c + 1) 0\n\n match run (count unicodeNewline) \"\\n\\r\\r\\n\\u0085\\u2028\\u2029\\r\\n\" with\n | Success(c,_,pos) -> c |> Equal 7; pos.Index |> Equal 9L; pos.Line |> Equal 8L; pos.Column |> Equal 1L\n | Failure _ -> Fail()\n\n spaces |> ROk \"\" 0 ()\n spaces |> ROk \" \" 1 ()\n spaces |> ROk \" \" 2 ()\n spaces1 |> RError \"\" 0 Errors.ExpectedWhitespace\n spaces1 |> ROk \" \" 1 ()\n spaces1 |> ROk \" \" 2 ()\n\n unicodeSpaces |> ROk \"\" 0 ()\n unicodeSpaces |> ROk \" \" 1 ()\n unicodeSpaces |> ROk \" \\u200A\" 2 () // '\\u200A' is a \"hair space\" (interestingly, the '\\u200B' \"zero width space\" character is not recognized as white space)\n unicodeSpaces1 |> RError \"\" 0 Errors.ExpectedWhitespace\n unicodeSpaces1 |> ROk \" \" 1 ()\n unicodeSpaces1 |> ROk \" \\u200A\" 2 ()\n\n match run spaces \"\\n \\r\\t\\t\\r\\n\\n \" with\n | Success(_, _, pos) -> pos.Index |> Equal 9L; pos.Line |> Equal 5L; pos.Column |> Equal 2L\n | _ -> Fail()\n match run spaces1 \"\\n \\r\\t\\t\\r\\n\\n \" with\n | Success(_, _, pos) -> pos.Index |> Equal 9L; pos.Line |> Equal 5L; pos.Column |> Equal 2L\n | _ -> Fail()\n\n match run unicodeSpaces \"\\n \\r\\t\\t\\r\\n\\n \\u0085\\u000C\\u2028\\u2029 \\r\\n\\t\\u200A\" with\n | Success(_, _, pos) -> pos.Index |> Equal 18L; pos.Line |> Equal 9L; pos.Column |> Equal 3L\n | _ -> Fail()\n match run unicodeSpaces1 \"\\n \\r\\t\\t\\r\\n\\n \\u0085\\u000C\\u2028\\u2029 \\r\\n\\t\\u200A\" with\n | Success(_, _, pos) -> pos.Index |> Equal 18L; pos.Line |> Equal 9L; pos.Column |> Equal 3L\n | _ -> Fail()\n\n eof |> ROk \"\" 0 ()\n (pchar '1' >>. eof) |> ROk \"1\" 1 ()\n eof |> RError \"1\" 0 Errors.ExpectedEndOfInput\n\n\nlet testStringParsers() =\n pstring \"\" |> ROk \"1\" 0 \"\"\n\n pstring \"1\" |> RError \"\" 0 (expectedString \"1\")\n pstring \"1\" |> RError \"2\" 0 (expectedString \"1\")\n pstring \"1\" |> ROk \"1\" 1 \"1\"\n\n pstring \"12\" |> RError \"\" 0 (expectedString \"12\")\n pstring \"12\" |> RError \"1\" 0 (expectedString \"12\")\n pstring \"12\" |> RError \"22\" 0 (expectedString \"12\")\n pstring \"12\" |> RError \"13\" 0 (expectedString \"12\")\n pstring \"12\" |> ROk \"12\" 2 \"12\"\n\n pstring \"test\" |> RError \"pest\" 0 (expectedString \"test\")\n pstring \"test\" |> ROk \"test\" 4 \"test\"\n skipString \"test\" |> ROk \"test\" 4 ()\n stringReturn \"test\" -1 |> ROk \"test\" 4 -1\n\n try pstring \"\\r\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try pstring \"\\n\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try pstring \"\\uffff\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try pstring \"\\r1\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try pstring \"1\\n\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try pstring \"12\\n\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n pstringCI \"t\" |> RError \"p\" 0 (expectedStringCI \"t\")\n pstringCI \"t\" |> ROk \"t\" 1 \"t\"\n pstringCI \"t\" |> ROk \"T\" 1 \"T\"\n pstringCI \"T\" |> ROk \"t\" 1 \"t\"\n pstringCI \"T\" |> ROk \"T\" 1 \"T\"\n skipStringCI \"t\" |> RError \"p\" 0 (expectedStringCI \"t\")\n skipStringCI \"t\" |> ROk \"t\" 1 ()\n skipStringCI \"t\" |> ROk \"T\" 1 ()\n skipStringCI \"T\" |> ROk \"t\" 1 ()\n skipStringCI \"T\" |> ROk \"T\" 1 ()\n\n pstringCI \"tEsT\" |> RError \"pest\" 0 (expectedStringCI \"tEsT\")\n pstringCI \"tEsT\" |> ROk \"TeSt\" 4 \"TeSt\"\n skipStringCI \"tEsT\" |> RError \"pest\" 0 (expectedStringCI \"tEsT\")\n skipStringCI \"tEsT\" |> ROk \"TeSt\" 4 ()\n stringCIReturn \"tEsT\" -1 |> ROk \"TeSt\" 4 -1\n\n try skipStringCI \"\\n\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try skipStringCI \"12\\n\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n anyString 3 |> RError \"12\" 0 (Errors.ExpectedAnySequenceOfNChars(3))\n skipAnyString 3 |> RError \"12\" 0 (Errors.ExpectedAnySequenceOfNChars(3))\n anyString 3 |> ROkNL \"12\\r\\n4\" 4 \"12\\n\"\n skipAnyString 3 |> ROkNL \"12\\r\\n4\" 4 ()\n\n skipped (skipAnyString 3) |> RError \"12\" 0 (Errors.ExpectedAnySequenceOfNChars(3))\n skipAnyString 3 |> withSkippedString (fun str () -> str) |> RError \"12\" 0 (Errors.ExpectedAnySequenceOfNChars(3))\n skipped (skipAnyString 3) |> ROk \"123\" 3 \"123\"\n skipAnyString 3 |> withSkippedString (fun str () -> str) |> ROk \"123\" 3 \"123\"\n skipped (skipAnyString 3) |> ROkNL \"12\\r\\n4\" 4 \"12\\n\"\n skipAnyString 3 |> withSkippedString (fun str () -> str) |> ROkNL \"12\\r\\n4\" 4 \"12\\n\"\n\n restOfLine true |> ROk \"\" 0 \"\"\n skipRestOfLine true |> ROk \"\" 0 ()\n restOfLine true |> ROkNL \"\\r\\n1\" 2 \"\"\n skipRestOfLine true |> ROkNL \"\\r\\n1\" 2 ()\n restOfLine true |> ROkNL \" \\r\\n1\" 4 \" \"\n skipRestOfLine true |> ROkNL \" \\r\\n1\" 4 ()\n\n restOfLine false |> ROk \"\" 0 \"\"\n skipRestOfLine false |> ROk \"\" 0 ()\n restOfLine false |> ROk \"\\r\\n1\" 0 \"\"\n skipRestOfLine false |> ROk \"\\r\\n1\" 0 ()\n restOfLine false |> ROk \" \\r\\n1\" 2 \" \"\n skipRestOfLine false |> ROk \" \\r\\n1\" 2 ()\n\n regex \"abc\" |> ROk \"abc\" 3 \"abc\"\n (anyChar >>. regex \"abc\") |> ROk \"_abc\" 4 \"abc\"\n regex \".*\\r\\r\\n.*\" |> ROkNL \"abc\\r\\r\\nabc\" 9 \"abc\\n\\nabc\"\n regex \"abc\" |> RError \"ab\" 0 (Errors.ExpectedStringMatchingRegex(\"abc\"))\n regexL \"abc\" \"test\" |> RError \"ab\" 0 (expected \"test\")\n\nlet testIdentifier() =\n // We do most of the testing in IdentifierValidatorTests.fs.\n // Here we only test the identifier parser wrapper.\n\n let U = System.Char.ConvertFromUtf32\n let ud800 = string (char 0xd800)\n let a_ud800 = \"a\" + ud800\n let mc2 = \"MC\" + (string '²')\n let s1 = U 0x00010280\n\n let expectedIdentifierError = expected Strings.Identifier\n let invalidCharacterError = messageError Strings.IdentifierContainsInvalidCharacterAtIndicatedPosition\n\n let defaultOpts = IdentifierOptions()\n identifier defaultOpts |> RError \"\" 0 expectedIdentifierError\n identifier defaultOpts |> RError \"1\" 0 expectedIdentifierError\n identifier defaultOpts |> RFatalError ud800 0 invalidCharacterError\n identifier defaultOpts |> RFatalError a_ud800 1 invalidCharacterError\n\n identifier defaultOpts |> ROk \"a\" 1 \"a\"\n identifier defaultOpts |> ROk \"abc1\" 4 \"abc1\"\n\n identifier defaultOpts |> ROk s1 2 s1\n\n identifier defaultOpts |> RFatalError \"क्‍\" 2 invalidCharacterError\n identifier (IdentifierOptions(allowJoinControlChars=true)) |> ROk \"क्‍\" 3 \"क्‍\"\n\n identifier (IdentifierOptions(label=\"test\")) |> RError \"1\" 0 (expected \"test\")\n identifier (IdentifierOptions(invalidCharMessage=\"test\")) |> RFatalError \"क्‍\" 2 (messageError \"test\")\n\n identifier defaultOpts |> ROk \"ϒ\\u0308\" 2 \"ϒ\\u0308\"\n identifier defaultOpts |> ROk mc2 2 \"MC\"\n let normOpts = IdentifierOptions(normalization=System.Text.NormalizationForm.FormKC)\n let preNormOpts = IdentifierOptions(normalization=System.Text.NormalizationForm.FormKC,\n normalizeBeforeValidation=true,\n preCheckContinue= fun c -> FParsec.IdentifierValidator.IsXIdContinueOrSurrogate(c) || c > '\\u007f')\n\n identifier normOpts |> ROk \"ϒ\\u0308\" 2 \"\\u03AB\"\n\n identifier normOpts |> ROk mc2 2 \"MC\"\n identifier preNormOpts |> ROk mc2 3 \"MC2\"\n\n let abOpts = IdentifierOptions(isAsciiIdStart=((=) 'a'), isAsciiIdContinue=((=) 'b'))\n\n identifier abOpts |> RError \"b\" 0 (expected Strings.Identifier)\n identifier abOpts |> ROk \"aa\" 1 \"a\"\n identifier abOpts |> ROk \"abc\" 2 \"ab\"\n\n let abNonAsciiOpts = IdentifierOptions(isAsciiIdStart=((=) 'a'), isAsciiIdContinue=((=) 'b'),\n allowAllNonAsciiCharsInPreCheck = true)\n\n identifier abNonAsciiOpts |> RError \"b\" 0 (expected Strings.Identifier)\n identifier abNonAsciiOpts |> ROk \"aa\" 1 \"a\"\n identifier abNonAsciiOpts |> ROk \"abc\" 2 \"ab\"\n identifier abNonAsciiOpts |> ROk \"abä\" 3 \"abä\"\n identifier abNonAsciiOpts |> RFatalError \"ab\\uFB1C\" 2 invalidCharacterError\n\n let abPreOpts = IdentifierOptions(isAsciiIdStart=((=) 'a'), isAsciiIdContinue=((=) 'b'),\n preCheckStart = (fun c -> c >= 'a' && c <= 'b'),\n preCheckContinue = (fun c -> c >= 'b' && c <= 'c'))\n identifier abPreOpts |> RFatalError \"b\" 0 invalidCharacterError\n identifier abPreOpts |> RFatalError \"abc\" 2 invalidCharacterError\n\n let abPreNonAsciiOpts = IdentifierOptions(isAsciiIdStart=((=) 'a'), isAsciiIdContinue=((=) 'b'),\n preCheckStart = (fun c -> c >= 'a' && c <= 'b'),\n preCheckContinue = (fun c -> c >= 'b' && c <= 'c'),\n allowAllNonAsciiCharsInPreCheck = true)\n identifier abPreNonAsciiOpts |> RFatalError \"b\" 0 invalidCharacterError\n identifier abPreNonAsciiOpts |> RFatalError \"abc\" 2 invalidCharacterError\n identifier abPreNonAsciiOpts |> ROk \"abä\" 3 \"abä\"\n\n\nlet testManySatisfy() =\n manySatisfy isDigit |> ROk \"\" 0 \"\"\n manySatisfy2 isHex isDigit |> ROk \"\" 0 \"\"\n manySatisfy isDigit |> ROk \"123\" 3 \"123\"\n manySatisfy2 isHex isDigit |> ROk \"a23a\" 3 \"a23\"\n\n skipManySatisfy isDigit |> ROk \"\" 0 ()\n skipManySatisfy2 isHex isDigit |> ROk \"\" 0 ()\n skipManySatisfy isDigit |> ROk \"123\" 3 ()\n skipManySatisfy2 isHex isDigit |> ROk \"a23a\" 3 ()\n\n many1Satisfy isDigit |> RError \"a\" 0 NoErrorMessages\n many1Satisfy2 isHex isDigit |> RError \"g\" 0 NoErrorMessages\n many1SatisfyL isDigit \"test\" |> RError \"a\" 0 (expected \"test\")\n many1Satisfy2L isHex isDigit \"test\" |> RError \"g\" 0 (expected \"test\")\n many1Satisfy isDigit |> ROk \"123\" 3 \"123\"\n many1Satisfy2 isHex isDigit |> ROk \"a23a\" 3 \"a23\"\n\n skipMany1SatisfyL isDigit \"test\" |> RError \"a\" 0 (expected \"test\")\n skipMany1Satisfy2L isHex isDigit \"test\" |> RError \"g\" 0 (expected \"test\")\n skipMany1Satisfy isDigit |> ROk \"123\" 3 ()\n skipMany1Satisfy2 isHex isDigit |> ROk \"a23a\" 3 ()\n\n manyMinMaxSatisfy 0 3 isDigit |> ROk \"1234\" 3 \"123\"\n manyMinMaxSatisfy 3 3 isDigit |> ROk \"1234\" 3 \"123\"\n manyMinMaxSatisfyL 4 4 isDigit \"test\" |> RError \"123a\" 0 (expected \"test\")\n manyMinMaxSatisfy2 0 3 isHex isDigit |> ROk \"a234\" 3 \"a23\"\n manyMinMaxSatisfy2 3 3 isHex isDigit |> ROk \"a234\" 3 \"a23\"\n manyMinMaxSatisfy2L 4 4 isHex isDigit \"test\" |> RError \"a23a\" 0 (expected \"test\")\n\n skipManyMinMaxSatisfy 0 3 isDigit |> ROk \"1234\" 3 ()\n skipManyMinMaxSatisfy 3 3 isDigit |> ROk \"1234\" 3 ()\n skipManyMinMaxSatisfyL 4 4 isDigit \"test\" |> RError \"123a\" 0 (expected \"test\")\n skipManyMinMaxSatisfy2 0 3 isHex isDigit |> ROk \"a234\" 3 ()\n skipManyMinMaxSatisfy2 3 3 isHex isDigit |> ROk \"a234\" 3 ()\n skipManyMinMaxSatisfy2L 4 4 isHex isDigit \"test\" |> RError \"a23a\" 0 (expected \"test\")\n\n try manyMinMaxSatisfy 0 -1 isDigit |> ROk \"1234\" 3 \"123\"; Fail()\n with :? System.ArgumentException -> ()\n\n try skipManyMinMaxSatisfy 0 -1 isDigit |> ROk \"1234\" 3 (); Fail()\n with :? System.ArgumentException -> ()\n\nlet testMany() =\n let ps1 = (constantTestParsers '1' (expected \"1\"))[1..] // no parser that returns OK without changing the state\n let ps2 = (constantTestParsers '2' (expected \"2\"))[1..]\n let ps3 = (constantTestParsers '3' (expected \"3\"))[1..]\n\n let content = \"the content doesn't matter\"\n use stream = new FParsec.CharStream(content, 0, content.Length)\n\n let many1Chars2Ref p1 p = Inline.Many((fun c -> (new System.Text.StringBuilder()).Append(c: char)),\n (fun sb c -> sb.Append(c)),\n (fun sb -> sb.ToString()),\n p, p1)\n let manyChars2Ref p1 p = many1Chars2Ref p1 p <|>% \"\"\n\n let manySeq2 = seq {for p2 in ps2 do\n for p3 in ps3 do\n yield [p2; p3]}\n\n for p1 in ps1 do\n for ps in manySeq2 do\n let p_1, p_2, pr = seqParserAndReset2 ps\n\n checkParser (manyChars2 p1 p_1) (manyChars2Ref p1 p_2) stream; pr()\n checkParser (many1Chars2 p1 p_1) (many1Chars2Ref p1 p_2) stream; pr()\n\n manyChars digit |> ROkE \"123\" 3 \"123\" Errors.ExpectedDecimalDigit\n many1Chars digit |> ROkE \"123\" 3 \"123\" Errors.ExpectedDecimalDigit\n\n try manyChars (preturn ' ') stream |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n let anyCharWithIndexMessage : Parser =\n fun stream ->\n let c = stream.ReadCharOrNewline()\n if c <> EOS then\n Reply(Ok, c, messageError (string stream.Index))\n else\n Reply(Error, Errors.ExpectedAnyChar)\n\n let sb = new System.Text.StringBuilder()\n for i = 1 to 200 do\n let s = sb.Append(char (i%10)).ToString()\n manyChars (anyCharWithIndexMessage)\n |> ROkE s s.Length s (mergeErrors (messageError (string i)) Errors.ExpectedAnyChar)\n\n sb.Length <- 0 // Clear() is only supported in >= .NET 4\n for i = 1 to 200 do\n let s = sb.Append(char (i%10)).ToString()\n manyCharsTill (anyCharWithIndexMessage) eof\n |> ROkE s s.Length s (messageError (string i))\n\n let eps1 = constantTestParsers 1 (expected \"11\")\n let eps2 = constantTestParsers 2 (expected \"22\")\n let eps3 = constantTestParsers 3 (expected \"33\")\n\n let manyCharsTillRef p endp = Inline.ManyTill((fun c -> (new System.Text.StringBuilder()).Append(c: char)),\n (fun sb c -> sb.Append(c)),\n (fun sb _ -> sb.ToString()),\n p, endp,\n resultForEmptySequence = (fun _ -> \"\"))\n\n let many1CharsTillRef p endp = pipe2 p (manyCharsTillRef p endp) (fun c0 s -> string c0 + s)\n\n let manyTillSeq =\n seq {for endp1 in eps1 do\n for p1 in ps1 do\n for endp2 in eps2 do\n for p2 in ps2 do\n for endp3 in eps3 do\n for p3 in ps3[1..] do\n yield [p1; p2; p3;], [endp1; endp2; endp3; eps3[1]]}\n\n for ps, es in manyTillSeq do\n let p_1, p_2, pr = seqParserAndReset2 ps\n let e_1, e_2, er = seqParserAndReset2 es\n checkParser (manyCharsTill p_1 e_1) (manyCharsTillRef p_2 e_2) stream; pr(); er()\n checkParser (many1CharsTill p_1 e_1) (many1CharsTillRef p_2 e_2) stream; pr(); er()\n\n manyCharsTill2 letter digit (pchar '.') |> ROk \"a23.\" 4 \"a23\"\n many1CharsTill2 letter digit (pchar '.') |> ROk \"a23.\" 4 \"a23\"\n\n manyCharsTillApply digit (pchar '.') (fun str c -> str + string c) |> ROk \"23.\" 3 \"23.\"\n many1CharsTillApply digit (pchar '.') (fun str c -> str + string c) |> ROk \"23.\" 3 \"23.\"\n\n try manyCharsTill (preturn ' ') (fail \"t\") stream |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n try many1CharsTill (preturn ' ') (fail \"t\") stream |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n let sps1 = constantTestParsers \"1\" (expected \"1\")\n let sps2 = constantTestParsers \"2\" (expected \"2\")\n let sps3 = constantTestParsers \"3\" (expected \"3\")\n let sps4 = constantTestParsers \"4\" (expected \"4\")\n let sps5 = constantTestParsers \"5\" (expected \"5\")\n let sps6 = constantTestParsers \"6\" (expected \"6\")\n let sps7 = constantTestParsers \"7\" (expected \"7\")\n\n let manyStringsRef p = many p |>> List.fold (fun acc s -> acc + s) \"\"\n let many1StringsRef p = many1 p |>> List.reduce (+)\n\n let manySeq7 = seq {for p1 in sps1[1..] do\n for p2 in sps2[1..] do\n for p3 in sps3[1..] do\n for p4 in sps4[1..] do\n for p5 in sps5[1..] do\n for p6 in sps6[1..] do\n for p7 in sps7[1..] do\n yield [p1;p2;p3;p4;p5;p6;p7]}\n\n let sw = new System.Diagnostics.Stopwatch()\n for ps in manySeq7 do\n let p_1, p_2, pr = seqParserAndReset2 ps\n\n checkParser (manyStrings p_1) (manyStringsRef p_2) stream; pr()\n checkParser (many1Strings p_1) (many1StringsRef p_2) stream\n\n manyStrings2 (pstring \"1\") (pstring \"2\") |> ROkE \"12223\" 4 \"1222\" (expectedString \"2\")\n\n try manyStrings (preturn \"1\") stream |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n let sepByTestParsers r1 e1 r2 e2 =\n let p1s = constantTestParsers r1 e1\n let p2s = constantTestParsers r2 e2\n seq {for p1 in p1s[1..] do\n for p2 in p2s do\n yield p1, p2}\n\n let sepBySeq3 =\n seq {for p1 in (constantTestParsers \"1\" (expected \"p1\"))[1..] do\n for sep1, p2 in sepByTestParsers \"a\" (expected \"sep1\") \"2\" (expected \"p2\") do\n for sep2, p3 in sepByTestParsers \"b\" (expected \"sep2\") \"3\" (expected \"p3\") do\n for sep3, p4 in sepByTestParsers \"c\" (expected \"sep3\") \"4\" (expected \"p4\") do\n yield [p1; p2; p3; p4], [sep1; sep2; sep3]\n\n // We exclude the following parameter combinations from regular test runs\n // because executing all of them just takes too much time.\n (*\n for sep4, p5 in sepByTestParsers \"d\" (expected \"sep4\") \"5\" (expected \"p5\") do\n yield [p1; p2; p3; p4; p5], [sep1; sep2; sep3; sep4]\n\n for p1, sep1 in sepByTestParsers \"1\" (expected \"p1\") \"a\" (expected \"sep1\") do\n for p2, sep2 in sepByTestParsers \"2\" (expected \"p2\") \"b\" (expected \"sep2\") do\n for p3, sep3 in sepByTestParsers \"3\" (expected \"p3\") \"c\" (expected \"sep3\") do\n for p4, sep4 in sepByTestParsers \"4\" (expected \"p4\") \"d\" (expected \"sep4\") do\n for p5 in (constantTestParsers \"5\" (expected \"p5\"))[1..] do\n yield [p1; p2; p3; p4; p5], [sep1; sep2; sep3; sep4] *)\n }\n\n let expectedStringsSepByResultForSepByReply (reply: Reply) =\n if reply.Status <> Ok then null\n else match reply.Result with\n | [] -> \"\"\n | [_] -> \"1\"\n | [_;_] -> \"1a2\"\n | [_;_;_] -> \"1a2b3\"\n | [_;_;_;_] -> \"1a2b3c4\"\n | [_;_;_;_;_] -> \"1a2b3c4d5\"\n | _ -> failwith \"stringsSepByTest\"\n\n let mutable i = 0\n let userState0 = stream.UserState\n let tag0 = stream.StateTag\n for ps, ss in sepBySeq3 do\n i <- i + 1\n let p, pr = seqParserAndReset ps\n let s, sr = seqParserAndReset ss\n checkParser (stringsSepBy p s)\n (fun stream ->\n pr(); sr()\n let r = sepBy p s stream\n let result = expectedStringsSepByResultForSepByReply r\n Reply(r.Status, result, r.Error)) stream\n pr(); sr();\n checkParser (stringsSepBy1 p s)\n (fun stream ->\n pr(); sr()\n let r = sepBy1 p s stream\n let result = expectedStringsSepByResultForSepByReply r\n Reply(r.Status, result, r.Error)) stream\n\n try stringsSepBy (preturn \"1\") (preturn \";\") stream |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n\nlet testSkipToString() =\n charsTillString \"abc\" false System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindString(\"abc\"))\n charsTillString \"abc\" false System.Int32.MaxValue |> ROk \"abc\" 0 \"\"\n charsTillString \"abc\" false 0 |> ROk \"abc\" 0 \"\"\n charsTillString \"abc\" false System.Int32.MaxValue |> ROk \"abdabc\" 3 \"abd\"\n charsTillString \"abc\" false 3 |> ROk \"abdabc\" 3 \"abd\"\n charsTillString \"abc\" false 2 |> RError \"abdabc\" 2 (Errors.CouldNotFindString(\"abc\"))\n\n charsTillStringCI \"AbC\" false System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n charsTillStringCI \"AbC\" false System.Int32.MaxValue |> ROk \"aBc\" 0 \"\"\n charsTillStringCI \"AbC\" false 0 |> ROk \"abc\" 0 \"\"\n charsTillStringCI \"AbC\" false System.Int32.MaxValue |> ROk \"aBdaBc\" 3 \"aBd\"\n charsTillStringCI \"AbC\" false 3 |> ROk \"aBdaBc\" 3 \"aBd\"\n charsTillStringCI \"AbC\" false 2 |> RError \"aBdaBc\" 2 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n\n skipCharsTillString \"abc\" false System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindString(\"abc\"))\n skipCharsTillString \"abc\" false System.Int32.MaxValue |> ROk \"abc\" 0 ()\n skipCharsTillString \"abc\" false 0 |> ROk \"abc\" 0 ()\n skipCharsTillString \"abc\" false System.Int32.MaxValue |> ROk \"abdabc\" 3 ()\n skipCharsTillString \"abc\" false 3 |> ROk \"abdabc\" 3 ()\n skipCharsTillString \"abc\" false 2 |> RError \"abdabc\" 2 (Errors.CouldNotFindString(\"abc\"))\n\n skipCharsTillStringCI \"AbC\" false System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n skipCharsTillStringCI \"AbC\" false System.Int32.MaxValue |> ROk \"aBc\" 0 ()\n skipCharsTillStringCI \"AbC\" false 0 |> ROk \"abc\" 0 ()\n skipCharsTillStringCI \"AbC\" false System.Int32.MaxValue |> ROk \"aBdaBc\" 3 ()\n skipCharsTillStringCI \"AbC\" false 3 |> ROk \"aBdaBc\" 3 ()\n skipCharsTillStringCI \"AbC\" false 2 |> RError \"aBdaBc\" 2 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n\n charsTillString \"abc\" true System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindString(\"abc\"))\n charsTillString \"abc\" true System.Int32.MaxValue |> ROk \"abc\" 3 \"\"\n charsTillString \"abc\" true 0 |> ROk \"abc\" 3 \"\"\n charsTillString \"abc\" true System.Int32.MaxValue |> ROk \"abdabc\" 6 \"abd\"\n charsTillString \"abc\" true 3 |> ROk \"abdabc\" 6 \"abd\"\n charsTillString \"abc\" true 2 |> RError \"abdabc\" 2 (Errors.CouldNotFindString(\"abc\"))\n\n charsTillStringCI \"AbC\" true System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n charsTillStringCI \"AbC\" true System.Int32.MaxValue |> ROk \"aBc\" 3 \"\"\n charsTillStringCI \"AbC\" true 0 |> ROk \"abc\" 3 \"\"\n charsTillStringCI \"AbC\" true System.Int32.MaxValue |> ROk \"aBdaBc\" 6 \"aBd\"\n charsTillStringCI \"AbC\" true 3 |> ROk \"aBdaBc\" 6 \"aBd\"\n charsTillStringCI \"AbC\" true 2 |> RError \"aBdaBc\" 2 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n\n skipCharsTillString \"abc\" true System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindString(\"abc\"))\n skipCharsTillString \"abc\" true System.Int32.MaxValue |> ROk \"abc\" 3 ()\n skipCharsTillString \"abc\" true 0 |> ROk \"abc\" 3 ()\n skipCharsTillString \"abc\" true System.Int32.MaxValue |> ROk \"abdabc\" 6 ()\n skipCharsTillString \"abc\" true 3 |> ROk \"abdabc\" 6 ()\n skipCharsTillString \"abc\" true 2 |> RError \"abdabc\" 2 (Errors.CouldNotFindString(\"abc\"))\n\n skipCharsTillStringCI \"AbC\" true System.Int32.MaxValue |> RError \"abbab\" 5 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n skipCharsTillStringCI \"AbC\" true System.Int32.MaxValue |> ROk \"aBc\" 3 ()\n skipCharsTillStringCI \"AbC\" true 0 |> ROk \"abc\" 3 ()\n skipCharsTillStringCI \"AbC\" true System.Int32.MaxValue |> ROk \"aBdaBc\" 6 ()\n skipCharsTillStringCI \"AbC\" true 3 |> ROk \"aBdaBc\" 6 ()\n skipCharsTillStringCI \"AbC\" true 2 |> RError \"aBdaBc\" 2 (Errors.CouldNotFindCaseInsensitiveString(\"AbC\"))\n\n try charsTillString \"1\\r\" false 1 |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try charsTillStringCI \"1\\r\" false 1 |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try skipCharsTillString \"1\\r\" false 1 |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try skipCharsTillStringCI \"1\\r\" false 1 |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n try charsTillString \"1\" false -1 |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try charsTillStringCI \"1\" false -1 |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try skipCharsTillString \"1\" false -1 |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try skipCharsTillStringCI \"1\" false -1 |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n\nlet testNumberParsers() =\n let ROkI content i result parser = ROk content i result parser\n let ROk content result parser = ROk content (content.Length - 1) result parser\n\n let testNumberLiteral() =\n let all = NLO.AllowSuffix\n ||| NLO.AllowMinusSign\n ||| NLO.AllowPlusSign\n ||| NLO.AllowFraction\n ||| NLO.AllowFractionWOIntegerPart\n ||| NLO.AllowExponent\n ||| NLO.AllowHexadecimal\n ||| NLO.AllowBinary\n ||| NLO.AllowOctal\n ||| NLO.AllowInfinity\n ||| NLO.AllowNaN\n\n\n numberLiteral all \"nl\" |> RError \"|\" 0 (expected \"nl\")\n numberLiteral all \"nl\" |> RError \"+|\" 0 (expected \"nl\")\n numberLiteral all \"nl\" |> RError \"-|\" 0 (expected \"nl\")\n numberLiteral all \"nl\" |> RError \"+n\" 0 (expected \"nl\")\n numberLiteral all \"nl\" |> RError \"-n\" 0 (expected \"nl\")\n numberLiteral all \"nl\" |> ROk \"0|\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0|\" (NumberLiteral(\"+0\", NLF.HasPlusSign ||| NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"-0|\" (NumberLiteral(\"-0\", NLF.HasMinusSign ||| NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"0u|\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 1), 'u', EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0az|\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 2), 'a', 'z', EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0uAZ|\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 3), 'u', 'A', 'Z', EOS))\n numberLiteral all \"nl\" |> ROk \"0ulLF|\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 4), 'u', 'l', 'L', 'F'))\n\n let all2 = all ||| NLO.IncludeSuffixCharsInString\n\n numberLiteral all2 \"nl\" |> ROk \"0u|\" (NumberLiteral(\"0u\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 1), 'u', EOS, EOS, EOS))\n numberLiteral all2 \"nl\" |> ROk \"0az|\" (NumberLiteral(\"0az\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 2), 'a', 'z', EOS, EOS))\n numberLiteral all2 \"nl\" |> ROk \"0uAZ|\" (NumberLiteral(\"0uAZ\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 3), 'u', 'A', 'Z', EOS))\n numberLiteral all2 \"nl\" |> ROk \"0ulLF|\" (NumberLiteral(\"0ulLF\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| ((enum) 4), 'u', 'l', 'L', 'F'))\n\n numberLiteral all \"nl\" |> ROk \".0|\" (NumberLiteral(\".0\", NLF.IsDecimal ||| NLF.HasFraction, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"1.|\" (NumberLiteral(\"1.\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \".0E0|\" (NumberLiteral(\".0E0\", NLF.IsDecimal ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0e-123f|\" (NumberLiteral(\"+0e-123\", NLF.IsDecimal ||| NLF.HasPlusSign ||| NLF.HasIntegerPart ||| NLF.HasExponent ||| ((enum) 1), 'f', EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0.1E+123|\" (NumberLiteral(\"0.1E+123\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"0.0E0|\" (NumberLiteral(\"0.0E0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"9.9E9|\" (NumberLiteral(\"9.9E9\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"00.00E00|\" (NumberLiteral(\"00.00E00\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"99.99E99|\" (NumberLiteral(\"99.99E99\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"-909.090e-09909z|\" (NumberLiteral(\"-909.090e-09909\", NLF.HasMinusSign ||| NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent ||| (enum) 1, 'z', EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"0x.0|\" (NumberLiteral(\"0x.0\", NLF.IsHexadecimal ||| NLF.HasFraction, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0x0.|\" (NumberLiteral(\"0x0.\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0X.fP0|\" (NumberLiteral(\"0X.fP0\", NLF.IsHexadecimal ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0xFp-0f|\" (NumberLiteral(\"+0xFp-0\", NLF.HasPlusSign ||| NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasExponent ||| ((enum) 1), 'f', EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xf.0AP+123|\" (NumberLiteral(\"0xf.0AP+123\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"0x0.0P0|\" (NumberLiteral(\"0x0.0P0\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xff.fp9|\" (NumberLiteral(\"0xff.fp9\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xa.aP9|\" (NumberLiteral(\"0xa.aP9\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xF.Fp0|\" (NumberLiteral(\"0xF.Fp0\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xA.AP9|\" (NumberLiteral(\"0xA.AP9\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0x00.00P00|\" (NumberLiteral(\"0x00.00P00\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xff.ffp99|\" (NumberLiteral(\"0xff.ffp99\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xaa.aaP99|\" (NumberLiteral(\"0xaa.aaP99\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xFF.FFp00|\" (NumberLiteral(\"0xFF.FFp00\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0xAA.AAP99|\" (NumberLiteral(\"0xAA.AAP99\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0x0afFA0.afFA0P+9099A|\" (NumberLiteral(\"+0x0afFA0.afFA0P+9099\", NLF.HasPlusSign ||| NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| NLF.HasExponent ||| (enum) 1, 'A', EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"0b02\" (NumberLiteral(\"0b0\", NLF.IsBinary ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"-0B0102\" (NumberLiteral(\"-0B010\", NLF.HasMinusSign ||| NLF.IsBinary ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0B010ul|\" (NumberLiteral(\"+0B010\", NLF.HasPlusSign ||| NLF.IsBinary ||| NLF.HasIntegerPart ||| (enum) 2, 'u', 'l', EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"-0o08\" (NumberLiteral(\"-0o0\", NLF.HasMinusSign ||| NLF.IsOctal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"0O0778\" (NumberLiteral(\"0O077\", NLF.IsOctal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+0o1770ul|\" (NumberLiteral(\"+0o1770\", NLF.HasPlusSign ||| NLF.IsOctal ||| NLF.HasIntegerPart ||| (enum) 2, 'u', 'l', EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"Infinityy\" (NumberLiteral(\"Infinity\", NLF.IsInfinity, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"-InFINitYy\" (NumberLiteral(\"-InFINitY\", NLF.HasMinusSign ||| NLF.IsInfinity, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"+iNfi\" (NumberLiteral(\"+iNf\", NLF.HasPlusSign ||| NLF.IsInfinity, EOS, EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> ROk \"NaNn\" (NumberLiteral(\"NaN\", NLF.IsNaN, EOS, EOS, EOS, EOS))\n numberLiteral all \"nl\" |> ROk \"-nAna\" (NumberLiteral(\"-nAn\", NLF.HasMinusSign ||| NLF.IsNaN, EOS, EOS, EOS, EOS))\n\n numberLiteral all \"nl\" |> RError \".a\" 1 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \".ea\" 1 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \".E-1\" 1 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \".1ea\" 3 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"-1ea\" 3 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"1.e-a\" 4 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"1e+a\" 3 Errors.ExpectedDecimalDigit\n\n numberLiteral all \"nl\" |> RError \"0x.g\" 3 Errors.ExpectedHexadecimalDigit\n numberLiteral all \"nl\" |> RError \"0x.pa\" 3 Errors.ExpectedHexadecimalDigit\n numberLiteral all \"nl\" |> RError \"0x.p-1\" 3 Errors.ExpectedHexadecimalDigit\n numberLiteral all \"nl\" |> RError \"+0x.1pa\" 6 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"0x1pa\" 4 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"0x1.p-a\" 6 Errors.ExpectedDecimalDigit\n numberLiteral all \"nl\" |> RError \"0x1p+a\" 5 Errors.ExpectedDecimalDigit\n\n numberLiteral all \"nl\" |> RError \"0b3\" 2 Errors.ExpectedBinaryDigit\n numberLiteral all \"nl\" |> RError \"-0b.0\" 3 Errors.ExpectedBinaryDigit\n numberLiteral all \"nl\" |> RError \"+0ou\" 3 Errors.ExpectedOctalDigit\n numberLiteral all \"nl\" |> RError \"0o.0\" 2 Errors.ExpectedOctalDigit\n\n numberLiteral (all ^^^ NLO.AllowPlusSign) \"nl\" |> RError \"+1|\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowPlusSign) \"nl\" |> ROk \"-1|\" (NumberLiteral(\"-1\", NLF.HasMinusSign ||| NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowMinusSign) \"nl\" |> RError \"-1|\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowMinusSign) \"nl\" |> ROk \"+1|\" (NumberLiteral(\"+1\", NLF.HasPlusSign ||| NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ (NLO.AllowPlusSign ||| NLO.AllowMinusSign)) \"nl\" |> ROk \"1|\" (NumberLiteral(\"1\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n\n numberLiteral (all ^^^ NLO.AllowFractionWOIntegerPart) \"nl\" |> RError \".0|\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowFractionWOIntegerPart) \"nl\" |> RError \"0x.0|\" 2 Errors.ExpectedHexadecimalDigit\n\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> ROk \"1.\" (NumberLiteral(\"1\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> ROkI \"10.10E2\" 2 (NumberLiteral(\"10\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> RError \".1\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> RError \".1\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> ROkI \"0x0.1p2\" 3 (NumberLiteral(\"0x0\", NLF.IsHexadecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowFraction) \"nl\" |> ROkI \"10.10E2\" 2 (NumberLiteral(\"10\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n\n numberLiteral (all ^^^ NLO.AllowExponent) \"nl\" |> ROkI \"1e1\" 2 (NumberLiteral(\"1\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| (enum) 1, 'e', EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowExponent) \"nl\" |> ROkI \"1.0e1\" 4 (NumberLiteral(\"1.0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| (enum) 1, 'e', EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowExponent) \"nl\" |> ROkI \"0x1p1\" 4 (NumberLiteral(\"0x1\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| (enum) 1, 'p', EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowExponent) \"nl\" |> ROkI \"0x1.0p1\" 6 (NumberLiteral(\"0x1.0\", NLF.IsHexadecimal ||| NLF.HasIntegerPart ||| NLF.HasFraction ||| (enum) 1, 'p', EOS, EOS, EOS))\n\n numberLiteral (all ^^^ NLO.AllowSuffix) \"nl\" |> ROk \"0u\" (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowSuffix) \"nl\" |> ROk \"0x1u\" (NumberLiteral(\"0x1\", NLF.IsHexadecimal ||| NLF.HasIntegerPart, EOS, EOS, EOS, EOS))\n\n numberLiteral (all ^^^ NLO.AllowBinary) \"nl\" |> ROkI \"0b1|\" 2 (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| (enum) 1, 'b', EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowOctal) \"nl\" |> ROkI \"0o1|\" 2 (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| (enum) 1, 'o', EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowHexadecimal) \"nl\" |> ROkI \"0x1|\" 2 (NumberLiteral(\"0\", NLF.IsDecimal ||| NLF.HasIntegerPart ||| (enum) 1, 'x', EOS, EOS, EOS))\n\n numberLiteral (all ^^^ NLO.AllowInfinity) \"nl\" |> RError \"Infinity|\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowInfinity) \"nl\" |> ROk \"NaN|\" (NumberLiteral(\"NaN\", NLF.IsNaN, EOS, EOS, EOS, EOS))\n numberLiteral (all ^^^ NLO.AllowNaN) \"nl\" |> RError \"NaN|\" 0 (expected \"nl\")\n numberLiteral (all ^^^ NLO.AllowNaN) \"nl\" |> ROk \"Infinity|\" (NumberLiteral(\"Infinity\", NLF.IsInfinity, EOS, EOS, EOS, EOS))\n\n testNumberLiteral()\n\n let testPfloat() =\n pfloat |> RError \"\" 0 Errors.ExpectedFloatingPointNumber\n pfloat |> RError \"-0x\" 3 Errors.ExpectedHexadecimalDigit\n pfloat |> ROk \"0|\" 0.\n pfloat |> ROk \"+0|\" 0.\n pfloat |> ROk \"-0|\" -0.\n pfloat |> ROk \"0x0|\" 0.\n pfloat |> ROk \"+0x0|\" 0.\n pfloat |> ROk \"-0X0|\" -0.\n pfloat |> ROk \"+123|\" 123.\n pfloat |> ROk \"+0x123|\" (floatOfHexString \"0x123\")\n pfloat |> ROk \"+123e2|\" 123e2\n pfloat |> ROk \"+0x123p2|\" (floatOfHexString \"0x123p2\")\n pfloat |> ROk \"-123.456e123|\" -123.456e123\n\n pfloat |> ROk \"1e99999|\" System.Double.PositiveInfinity\n pfloat |> ROk \"0x1p99999|\" System.Double.PositiveInfinity\n pfloat |> ROk \"-1e99999|\" System.Double.NegativeInfinity\n pfloat |> ROk \"-0x1p99999|\" System.Double.NegativeInfinity\n\n pfloat |> ROk \"-0x123cde.123afAcEp123|\" (floatOfHexString \"-0x123cde.123afAcEp123\")\n pfloat |> ROk \"-0x1.fffffffffffffp1023|\" -System.Double.MaxValue\n pfloat |> ROk \"-0x1.fffffffffffffp1024|\" System.Double.NegativeInfinity\n pfloat |> ROk \"Inf|\" System.Double.PositiveInfinity\n pfloat |> ROk \"-Infinity|\" System.Double.NegativeInfinity\n pfloat >>% 1 |> ROk \"NaN|\" 1\n\n testPfloat()\n\n let testPuint64() =\n let expectedE = Errors.ExpectedUInt64\n let overflowE = Errors.NumberOutsideOfUInt64Range\n\n puint64 |> RError \"\" 0 expectedE\n puint64 |> RError \"+1\" 0 expectedE\n puint64 |> RError \"-1\" 0 expectedE\n puint64 |> RFatalError \"18446744073709551620\" 0 overflowE\n puint64 |> RFatalError \"18446744073709551619\" 0 overflowE\n puint64 |> RFatalError \"18446744073709551618\" 0 overflowE\n puint64 |> RFatalError \"18446744073709551617\" 0 overflowE\n puint64 |> RFatalError \"18446744073709551616\" 0 overflowE\n puint64 |> RFatalError \"0000018446744073709551616\" 0 overflowE\n puint64 |> RFatalError \"111111111111111111111\" 0 overflowE\n\n puint64 |> ROk \"0|\" 0UL\n puint64 |> ROk \"000|\" 0UL\n puint64 |> ROk \"12345678901234567890|\" 12345678901234567890UL\n puint64 |> ROk \"18446744073709551615|\" System.UInt64.MaxValue\n puint64 |> ROk \"018446744073709551614|\" (System.UInt64.MaxValue - 1UL)\n puint64 |> ROk \"018446744073709551613|\" (System.UInt64.MaxValue - 2UL)\n puint64 |> ROk \"018446744073709551612|\" (System.UInt64.MaxValue - 3UL)\n puint64 |> ROk \"018446744073709551611|\" (System.UInt64.MaxValue - 4UL)\n puint64 |> ROk \"018446744073709551610|\" (System.UInt64.MaxValue - 5UL)\n puint64 |> ROk \"018446744073709551609|\" (System.UInt64.MaxValue - 6UL)\n puint64 |> ROk \"0000018446744073709551615|\" System.UInt64.MaxValue\n\n puint64 |> RError \"0x\" 2 Errors.ExpectedHexadecimalDigit\n puint64 |> RError \"+0x1\" 0 expectedE\n puint64 |> RFatalError \"0x10000000000000000\" 0 overflowE\n puint64 |> RFatalError \"0x11111111111111111\" 0 overflowE\n puint64 |> RFatalError \"0Xfffffffffffffffff\" 0 overflowE\n\n puint64 |> ROk \"0x0|\" 0UL\n puint64 |> ROk \"0x000|\" 0UL\n puint64 |> ROk \"0x1234567890abcdef|\" 0x1234567890abcdefUL\n puint64 |> ROk \"0X1234567890ABCDEF|\" 0x1234567890abcdefUL\n puint64 |> ROk \"0xffffffffffffffff|\" System.UInt64.MaxValue\n puint64 |> ROk \"0xfffffffffffffffe|\" (System.UInt64.MaxValue - 1UL)\n puint64 |> ROk \"0xfffffffffffffff1|\" (System.UInt64.MaxValue - 14UL)\n puint64 |> ROk \"0xfffffffffffffff0|\" (System.UInt64.MaxValue - 15UL)\n puint64 |> ROk \"0xffffffffffffffef|\" (System.UInt64.MaxValue - 16UL)\n puint64 |> ROk \"0x00000ffffffffffffffff|\" System.UInt64.MaxValue\n\n puint64 |> RError \"0o\" 2 Errors.ExpectedOctalDigit\n puint64 |> RError \"+0o1\" 0 expectedE\n puint64 |> RFatalError \"0o2000000000000000000001\" 0 overflowE\n puint64 |> RFatalError \"0o2000000000000000000000\" 0 overflowE\n puint64 |> RFatalError \"0o7777777777777777777777\" 0 overflowE\n puint64 |> RFatalError \"0O77777777777777777777777\" 0 overflowE\n\n puint64 |> ROk \"0o0|\" 0UL\n puint64 |> ROk \"0o000|\" 0UL\n puint64 |> ROk \"0o1234567123456701234567|\" 0o1234567123456701234567UL\n puint64 |> ROk \"0o1777777777777777777777|\" System.UInt64.MaxValue\n puint64 |> ROk \"0o1777777777777777777776|\" (System.UInt64.MaxValue - 1UL)\n puint64 |> ROk \"0o1777777777777777777771|\" (System.UInt64.MaxValue - 6UL)\n puint64 |> ROk \"0o1777777777777777777770|\" (System.UInt64.MaxValue - 7UL)\n puint64 |> ROk \"0o1777777777777777777767|\" (System.UInt64.MaxValue - 8UL)\n puint64 |> ROk \"0O000001777777777777777777777|\" System.UInt64.MaxValue\n\n puint64 |> RError \"0b\" 2 Errors.ExpectedBinaryDigit\n puint64 |> RError \"+0b1\" 0 expectedE\n puint64 |> RFatalError \"0b10000000000000000000000000000000000000000000000000000000000000001\" 0 overflowE\n puint64 |> RFatalError \"0b10000000000000000000000000000000000000000000000000000000000000000\" 0 overflowE\n puint64 |> RFatalError \"0b11111111111111111111111111111111111111111111111111111111111111111\" 0 overflowE\n\n puint64 |> ROk \"0b0|\" 0UL\n puint64 |> ROk \"0b000|\" 0UL\n puint64 |> ROk \"0b1111111111111111111111111111111111111111111111111111111111111111|\" System.UInt64.MaxValue\n puint64 |> ROk \"0b1111111111111111111111111111111111111111111111111111111111111110|\" (System.UInt64.MaxValue - 1UL)\n puint64 |> ROk \"0b1111111111111111111111111111111111111111111111111111111111111101|\" (System.UInt64.MaxValue - 2UL)\n puint64 |> ROk \"0b1111111111111111111111111111111111111111111111111111111111111100|\" (System.UInt64.MaxValue - 3UL)\n puint64 |> ROk \"0B000001111111111111111111111111111111111111111111111111111111111111111|\" System.UInt64.MaxValue\n\n testPuint64()\n\n let testPint64() =\n let expectedE = Errors.ExpectedInt64\n let overflowE = Errors.NumberOutsideOfInt64Range\n\n pint64 |> RFatalError \"18446744073709551615\" 0 overflowE\n pint64 |> RFatalError \"+00018446744073709551615\" 0 overflowE\n pint64 |> RFatalError \"-18446744073709551615\" 0 overflowE\n pint64 |> RFatalError \"-0018446744073709551615\" 0 overflowE\n\n pint64 |> RFatalError \"9223372036854775808\" 0 overflowE\n pint64 |> RFatalError \"+0009223372036854775808\" 0 overflowE\n pint64 |> RFatalError \"-9223372036854775809\" 0 overflowE\n pint64 |> RFatalError \"-09223372036854775809\" 0 overflowE\n\n pint64 |> ROk \"9223372036854775807|\" System.Int64.MaxValue\n pint64 |> ROk \"+000009223372036854775807|\" System.Int64.MaxValue\n pint64 |> ROk \"-9223372036854775808|\" System.Int64.MinValue\n pint64 |> ROk \"-009223372036854775808|\" System.Int64.MinValue\n\n pint64 |> RFatalError \"0xffffffffffffffff\" 0 overflowE\n pint64 |> RFatalError \"+0x000ffffffffffffffff\" 0 overflowE\n pint64 |> RFatalError \"-0xffffffffffffffff\" 0 overflowE\n pint64 |> RFatalError \"-0X0ffffffffffffffff\" 0 overflowE\n\n pint64 |> RFatalError \"0x8000000000000000\" 0 overflowE\n pint64 |> RFatalError \"+0x0008000000000000000\" 0 overflowE\n pint64 |> RFatalError \"-0x8000000000000001\" 0 overflowE\n pint64 |> RFatalError \"-0x0008000000000000001\" 0 overflowE\n\n pint64 |> ROk \"0x7fffffffffffffff|\" System.Int64.MaxValue\n pint64 |> ROk \"+0x000007fffffffffffffff|\" System.Int64.MaxValue\n pint64 |> ROk \"-0x8000000000000000|\" System.Int64.MinValue\n pint64 |> ROk \"-0x008000000000000000|\" System.Int64.MinValue\n\n pint64 |> RFatalError \"0o2000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"+0o002000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"-0o0002000000000000000000000\" 0 overflowE\n\n pint64 |> RFatalError \"0o1000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"0o1000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"+0o001000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"-0o1000000000000000000001\" 0 overflowE\n pint64 |> RFatalError \"-0O001000000000000000000001\" 0 overflowE\n\n pint64 |> ROk \"0o777777777777777777777|\" System.Int64.MaxValue\n pint64 |> ROk \"+0o00777777777777777777777|\" System.Int64.MaxValue\n pint64 |> ROk \"-0o1000000000000000000000|\" System.Int64.MinValue\n pint64 |> ROk \"-0O001000000000000000000000|\" System.Int64.MinValue\n\n pint64 |> RFatalError \"+0b00011111111111111111111111111111111111111111111111111111111111111111\" 0 overflowE\n pint64 |> RFatalError \"-0b011111111111111111111111111111111111111111111111111111111111111111\" 0 overflowE\n\n pint64 |> RFatalError \"+0B1000000000000000000000000000000000000000000000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"0b0001000000000000000000000000000000000000000000000000000000000000000\" 0 overflowE\n pint64 |> RFatalError \"-0b0001000000000000000000000000000000000000000000000000000000000000001\" 0 overflowE\n pint64 |> RFatalError \"-0b1000000000000000000000000000000000000000000000000000000000000001\" 0 overflowE\n\n pint64 |> ROk \"0b111111111111111111111111111111111111111111111111111111111111111|\" System.Int64.MaxValue\n pint64 |> ROk \"+0b00111111111111111111111111111111111111111111111111111111111111111|\" System.Int64.MaxValue\n pint64 |> ROk \"-0b1000000000000000000000000000000000000000000000000000000000000000|\"System.Int64.MinValue\n pint64 |> ROk \"-0B0001000000000000000000000000000000000000000000000000000000000000000|\" System.Int64.MinValue\n\n testPint64()\n\n let testPuint32() =\n let expectedE = Errors.ExpectedUInt32\n let overflowE = Errors.NumberOutsideOfUInt32Range\n\n puint32 |> RError \"\" 0 expectedE\n puint32 |> RError \"+1\" 0 expectedE\n puint32 |> RError \"-1\" 0 expectedE\n\n puint32 |> RFatalError \"4294967300\" 0 overflowE\n puint32 |> RFatalError \"4294967299\" 0 overflowE\n puint32 |> RFatalError \"4294967298\" 0 overflowE\n puint32 |> RFatalError \"4294967297\" 0 overflowE\n puint32 |> RFatalError \"4294967296\" 0 overflowE\n puint32 |> RFatalError \"000004294967296\" 0 overflowE\n puint32 |> RFatalError \"11111111111\" 0 overflowE\n\n puint32 |> ROk \"0|\" 0u\n puint32 |> ROk \"000|\" 0u\n puint32 |> ROk \"1234567890|\" 1234567890u\n puint32 |> ROk \"4294967295|\" System.UInt32.MaxValue\n puint32 |> ROk \"4294967294|\" (System.UInt32.MaxValue - 1u)\n puint32 |> ROk \"4294967293|\" (System.UInt32.MaxValue - 2u)\n puint32 |> ROk \"4294967292|\" (System.UInt32.MaxValue - 3u)\n puint32 |> ROk \"4294967291|\" (System.UInt32.MaxValue - 4u)\n puint32 |> ROk \"4294967290|\" (System.UInt32.MaxValue - 5u)\n puint32 |> ROk \"4294967289|\" (System.UInt32.MaxValue - 6u)\n puint32 |> ROk \"000004294967295|\" System.UInt32.MaxValue\n\n puint32 |> RError \"0x\" 2 Errors.ExpectedHexadecimalDigit\n puint32 |> RError \"+0x1\" 0 expectedE\n puint32 |> RFatalError \"0x100000001\" 0 overflowE\n puint32 |> RFatalError \"0x100000000\" 0 overflowE\n puint32 |> RFatalError \"0x111111111\" 0 overflowE\n puint32 |> RFatalError \"0Xfffffffff\" 0 overflowE\n\n puint32 |> ROk \"0x0|\" 0u\n puint32 |> ROk \"0x000|\" 0u\n puint32 |> ROk \"0x1234abcd|\" 0x1234abcdu\n puint32 |> ROk \"0X1234ABCD|\" 0x1234abcdu\n puint32 |> ROk \"0xffffffff|\" System.UInt32.MaxValue\n puint32 |> ROk \"0xfffffffe|\" (System.UInt32.MaxValue - 1u)\n puint32 |> ROk \"0xfffffff1|\" (System.UInt32.MaxValue - 14u)\n puint32 |> ROk \"0xfffffff0|\" (System.UInt32.MaxValue - 15u)\n puint32 |> ROk \"0xffffffef|\" (System.UInt32.MaxValue - 16u)\n puint32 |> ROk \"0x00000ffffffff|\" System.UInt32.MaxValue\n\n puint32 |> RError \"0o\" 2 Errors.ExpectedOctalDigit\n puint32 |> RError \"+0o1\" 0 expectedE\n puint32 |> RFatalError \"0o40000000001\" 0 overflowE\n puint32 |> RFatalError \"0o40000000000\" 0 overflowE\n puint32 |> RFatalError \"0o777777777777\" 0 overflowE\n puint32 |> RFatalError \"0O7777777777777\" 0 overflowE\n\n puint32 |> ROk \"0o0|\" 0u\n puint32 |> ROk \"0o000|\" 0u\n puint32 |> ROk \"0o12345670123|\" 0o12345670123u\n puint32 |> ROk \"0o37777777777|\" System.UInt32.MaxValue\n puint32 |> ROk \"0o37777777776|\" (System.UInt32.MaxValue - 1u)\n puint32 |> ROk \"0o37777777771|\" (System.UInt32.MaxValue - 6u)\n puint32 |> ROk \"0o37777777770|\" (System.UInt32.MaxValue - 7u)\n puint32 |> ROk \"0o37777777767|\" (System.UInt32.MaxValue - 8u)\n puint32 |> ROk \"0O0000037777777777|\" System.UInt32.MaxValue\n\n puint32 |> RError \"0b\" 2 Errors.ExpectedBinaryDigit\n puint32 |> RError \"+0b1\" 0 expectedE\n puint32 |> RFatalError \"0b100000000000000000000000000000001\" 0 overflowE\n puint32 |> RFatalError \"0b100000000000000000000000000000000\" 0 overflowE\n puint32 |> RFatalError \"0B111111111111111111111111111111111\" 0 overflowE\n\n puint32 |> ROk \"0b0|\" 0u\n puint32 |> ROk \"0b000|\" 0u\n puint32 |> ROk \"0b11111111111111111111111111111111|\" System.UInt32.MaxValue\n puint32 |> ROk \"0b11111111111111111111111111111110|\" (System.UInt32.MaxValue - 1u)\n puint32 |> ROk \"0b11111111111111111111111111111101|\" (System.UInt32.MaxValue - 2u)\n puint32 |> ROk \"0b11111111111111111111111111111100|\" (System.UInt32.MaxValue - 3u)\n puint32 |> ROk \"0B0000011111111111111111111111111111111|\" System.UInt32.MaxValue\n\n testPuint32()\n\n let testPint32() =\n let expectedE = Errors.ExpectedInt32\n let overflowE = Errors.NumberOutsideOfInt32Range\n\n pint32 |> RFatalError \"4294967295\" 0 overflowE\n pint32 |> RFatalError \"+4294967295\" 0 overflowE\n pint32 |> RFatalError \"-4294967295\" 0 overflowE\n pint32 |> RFatalError \"-004294967295\" 0 overflowE\n\n pint32 |> RFatalError \"2147483648\" 0 overflowE\n pint32 |> RFatalError \"+0002147483648\" 0 overflowE\n pint32 |> RFatalError \"-2147483649\" 0 overflowE\n pint32 |> RFatalError \"-02147483649\" 0 overflowE\n\n pint32 |> ROk \"2147483647|\" System.Int32.MaxValue\n pint32 |> ROk \"+000002147483647|\" System.Int32.MaxValue\n pint32 |> ROk \"-2147483648|\" System.Int32.MinValue\n pint32 |> ROk \"-002147483648|\" System.Int32.MinValue\n\n pint32 |> RFatalError \"0xffffffffffffffff\" 0 overflowE\n pint32 |> RFatalError \"+0x000ffffffffffffffff\" 0 overflowE\n pint32 |> RFatalError \"-0xffffffffffffffff\" 0 overflowE\n pint32 |> RFatalError \"-0X0ffffffffffffffff\" 0 overflowE\n\n pint32 |> RFatalError \"0x80000000\" 0 overflowE\n pint32 |> RFatalError \"+0x00080000000\" 0 overflowE\n pint32 |> RFatalError \"-0x80000001\" 0 overflowE\n pint32 |> RFatalError \"-0x00080000001\" 0 overflowE\n\n pint32 |> ROk \"0x7fffffff|\" System.Int32.MaxValue\n pint32 |> ROk \"+0x000007fffffff|\" System.Int32.MaxValue\n pint32 |> ROk \"-0x80000000|\" System.Int32.MinValue\n pint32 |> ROk \"-0x0080000000|\" System.Int32.MinValue\n\n pint32 |> RFatalError \"0o40000000000\" 0 overflowE\n pint32 |> RFatalError \"+0o0040000000000\" 0 overflowE\n pint32 |> RFatalError \"-0o00040000000000\" 0 overflowE\n\n pint32 |> RFatalError \"0o20000000000\" 0 overflowE\n pint32 |> RFatalError \"+0o0020000000000\" 0 overflowE\n pint32 |> RFatalError \"-0o20000000001\" 0 overflowE\n pint32 |> RFatalError \"-0O0020000000001\" 0 overflowE\n\n pint32 |> ROk \"0o17777777777|\" System.Int32.MaxValue\n pint32 |> ROk \"+0o0017777777777|\" System.Int32.MaxValue\n pint32 |> ROk \"-0o20000000000|\" System.Int32.MinValue\n pint32 |> ROk \"-0O0020000000000|\" System.Int32.MinValue\n\n pint32 |> RFatalError \"+0b000111111111111111111111111111111111\" 0 overflowE\n pint32 |> RFatalError \"-0b0111111111111111111111111111111111\" 0 overflowE\n\n pint32 |> RFatalError \"+0B1000000000000000000000000000000000000000000000000000000000000000\" 0 overflowE\n pint32 |> RFatalError \"0b0001000000000000000000000000000000000000000000000000000000000000000\" 0 overflowE\n pint32 |> RFatalError \"-0b0001000000000000000000000000000000000000000000000000000000000000001\" 0 overflowE\n pint32 |> RFatalError \"-0b1000000000000000000000000000000000000000000000000000000000000001\" 0 overflowE\n\n pint32 |> ROk \"0b1111111111111111111111111111111|\" System.Int32.MaxValue\n pint32 |> ROk \"+0b001111111111111111111111111111111|\" System.Int32.MaxValue\n pint32 |> ROk \"-0b10000000000000000000000000000000|\"System.Int32.MinValue\n pint32 |> ROk \"-0B00010000000000000000000000000000000|\" System.Int32.MinValue\n\n testPint32()\n\n let testPintOther() =\n let overflowInt32 = Errors.NumberOutsideOfInt32Range\n let overflowInt16 = Errors.NumberOutsideOfInt16Range\n let overflowInt8 = Errors.NumberOutsideOfInt8Range\n let overflowUInt32 = Errors.NumberOutsideOfUInt32Range\n let overflowUInt16 = Errors.NumberOutsideOfUInt16Range\n let overflowUInt8 = Errors.NumberOutsideOfUInt8Range\n\n puint32 |> RError \"+0|\" 0 Errors.ExpectedUInt32\n puint32 |> RFatalError \"4294967296|\" 0 overflowUInt32\n puint32 |> RFatalError \"00004294967296|\" 0 overflowUInt32\n puint32 |> RFatalError \"11111111111|\" 0 overflowUInt32\n\n puint32 |> ROk \"0|\" 0u\n puint32 |> ROk \"000|\" 0u\n puint32 |> ROk \"1234567890|\" 1234567890u\n puint32 |> ROk \"0001234567890|\" 1234567890u\n puint32 |> ROk \"4294967295|\" System.UInt32.MaxValue\n puint32 |> ROk \"0004294967295|\" System.UInt32.MaxValue\n\n pint32 |> RError \"+|\" 0 Errors.ExpectedInt32\n pint32 |> RFatalError \"2147483648|\" 0 overflowInt32\n pint32 |> RFatalError \"-00002147483649|\" 0 overflowInt32\n pint32 |> RFatalError \"11111111111|\" 0 overflowInt32\n\n pint32 |> ROk \"0|\" 0\n pint32 |> ROk \"+000|\" 0\n pint32 |> ROk \"1234567890|\" 1234567890\n pint32 |> ROk \"0001234567890|\" 1234567890\n pint32 |> ROk \"2147483647|\" System.Int32.MaxValue\n pint32 |> ROk \"+0002147483647|\" System.Int32.MaxValue\n pint32 |> ROk \"-2147483648|\" System.Int32.MinValue\n pint32 |> ROk \"-0002147483648|\" System.Int32.MinValue\n\n pint32 |> RFatalError \"0x80000000|\" 0 overflowInt32\n pint32 |> ROk \"0x7fffffff|\" 0x7fffffff\n pint32 |> ROk \"-0x80000000|\" -0x80000000\n pint32 |> RFatalError \"-0x80000001|\" 0 overflowInt32\n\n puint32 |> RFatalError \"0x100000000|\" 0 overflowUInt32\n puint32 |> ROk \"0xffffffff|\" 0xffffffffu\n\n pint16 |> RFatalError \"0x8000|\" 0 overflowInt16\n pint16 |> ROk \"0x7fff|\" 0x7fffs\n pint16 |> ROk \"-0x8000|\" -0x8000s\n pint16 |> RFatalError \"-0x8001|\" 0 overflowInt16\n\n puint16 |> RFatalError \"0x10000|\" 0 overflowUInt16\n puint16 |> ROk \"0xffff|\" 0xffffus\n\n pint8 |> RFatalError \"0x80|\" 0 overflowInt8\n pint8 |> ROk \"0x7f|\" 0x7fy\n pint8 |> ROk \"-0x80|\" -0x80y\n pint8 |> RFatalError \"-0x81|\" 0 overflowInt8\n\n puint8 |> RFatalError \"0x100|\" 0 overflowUInt8\n puint8 |> ROk \"0xff|\" 0xffuy\n\n testPintOther()\n\nlet testFollowedBy() =\n notFollowedByEof |> ROk \" \" 0 ()\n notFollowedByEof |> RError \"\" 0 Errors.UnexpectedEndOfInput\n\n followedByNewline |> RError \"1\" 0 Errors.ExpectedNewline\n followedByNewline |> RError \" \" 0 Errors.ExpectedNewline\n followedByNewline |> ROk \"\\r\" 0 ()\n followedByNewline |> ROk \"\\n\" 0 ()\n\n notFollowedByNewline |> ROk \"1\" 0 ()\n notFollowedByNewline |> ROk \" \" 0 ()\n notFollowedByNewline |> RError \"\\r\" 0 Errors.UnexpectedNewline\n notFollowedByNewline |> RError \"\\n\" 0 Errors.UnexpectedNewline\n\n followedByString \"a\" |> ROk \"a\" 0 ()\n followedByString \"a\" |> RError \"A\" 0 (expectedString \"a\")\n followedByString \"123\" |> ROk \"123\" 0 ()\n followedByString \"123\" |> RError \"124\" 0 (expectedString \"123\")\n notFollowedByString \"a\" |> ROk \"A\" 0 ()\n notFollowedByString \"a\" |> RError \"a\" 0 (unexpectedString \"a\")\n notFollowedByString \"123\" |> ROk \"124\" 0 ()\n notFollowedByString \"123\" |> RError \"123\" 0 (unexpectedString \"123\")\n\n try followedByString \"13\\r\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try notFollowedByString \"13\\r\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n followedByStringCI \"A\" |> ROk \"a\" 0 ()\n followedByStringCI \"A\" |> ROk \"A\" 0 ()\n followedByStringCI \"A\" |> RError \"B\" 0 (expectedStringCI \"A\")\n followedByStringCI \"aBc\" |> ROk \"AbC\" 0 ()\n followedByStringCI \"aBc\" |> RError \"Abd\" 0 (expectedStringCI \"aBc\")\n notFollowedByStringCI \"A\" |> ROk \"B\" 0 ()\n notFollowedByStringCI \"A\" |> RError \"a\" 0 (unexpectedStringCI \"A\")\n notFollowedByStringCI \"A\" |> RError \"A\" 0 (unexpectedStringCI \"A\")\n notFollowedByStringCI \"aBc\" |> ROk \"Abd\" 0 ()\n notFollowedByStringCI \"aBc\" |> RError \"AbC\" 0 (unexpectedStringCI \"aBc\")\n\n try followedByStringCI \"13\\r\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try notFollowedByStringCI \"13\\r\" |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n let one chr = fun c -> if c = chr then true\n else Fail()\n\n let oneN chr = fun c -> if c = chr then false\n else Fail()\n\n let eos1 = fun c -> Fail()\n\n nextCharSatisfies (one '2') |> ROk \"2\" 0 ()\n nextCharSatisfies (one '\\n') |> ROk \"\\n\" 0 ()\n nextCharSatisfies (one '\\n') |> ROk \"\\r\\n\" 0 ()\n nextCharSatisfies (one '\\n') |> ROk \"\\r\" 0 ()\n nextCharSatisfies eos1 |> RError \"\" 0 NoErrorMessages\n nextCharSatisfies (oneN '1') |> RError \"1\" 0 NoErrorMessages\n nextCharSatisfies (oneN '\\n') |> RError \"\\r\" 0 NoErrorMessages\n\n nextCharSatisfiesNot (oneN '2') |> ROk \"2\" 0 ()\n nextCharSatisfiesNot (oneN '\\n') |> ROk \"\\n\" 0 ()\n nextCharSatisfiesNot (oneN '\\n') |> ROk \"\\r\\n\" 0 ()\n nextCharSatisfiesNot (oneN '\\n') |> ROk \"\\r\" 0 ()\n nextCharSatisfiesNot eos1 |> ROk \"\" 0 ()\n nextCharSatisfiesNot (one '1') |> RError \"1\" 0 NoErrorMessages\n nextCharSatisfiesNot (one '\\n') |> RError \"\\r\" 0 NoErrorMessages\n\n let two (str: string) =\n fun c0 c1 -> if c0 = str[0] && c1 = str[1] then true\n else Fail()\n let twoN (str: string) =\n fun c0 c1 -> if c0 = str[0] && c1 = str[1] then false\n else Fail()\n\n let eos2 = fun c0 c1 -> Fail()\n\n next2CharsSatisfy (two \"12\") |> ROk \"12\" 0 ()\n next2CharsSatisfy (two \"\\n2\") |> ROk \"\\r2\" 0 ()\n next2CharsSatisfy (two \"\\n2\") |> ROk \"\\r\\n2\" 0 ()\n next2CharsSatisfy (two \"\\n2\") |> ROk \"\\n2\" 0 ()\n next2CharsSatisfy (two \"\\n\\n\") |> ROk \"\\n\\r\" 0 ()\n next2CharsSatisfy (two \"\\n\\n\") |> ROk \"\\r\\r\" 0 ()\n next2CharsSatisfy (two \"\\n\\n\") |> ROk \"\\r\\n\\r\" 0 ()\n next2CharsSatisfy eos2 |> RError \"\" 0 NoErrorMessages\n next2CharsSatisfy eos2 |> RError \"1\" 0 NoErrorMessages\n next2CharsSatisfy eos2 |> RError \"\\r\" 0 NoErrorMessages\n next2CharsSatisfy eos2 |> RError \"\\r\\n\" 0 NoErrorMessages\n next2CharsSatisfy eos2 |> RError \"\\n\" 0 NoErrorMessages\n next2CharsSatisfy (twoN \"13\") |> RError \"13\" 0 NoErrorMessages\n next2CharsSatisfy (twoN \"\\n\\t\") |> RError \"\\n\\t\" 0 NoErrorMessages\n next2CharsSatisfy (twoN \"\\n\\t\") |> RError \"\\r\\n\\t\" 0 NoErrorMessages\n next2CharsSatisfy (twoN \"\\n\\t\") |> RError \"\\r\\t\" 0 NoErrorMessages\n\n next2CharsSatisfyNot (twoN \"12\") |> ROk \"12\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n2\") |> ROk \"\\r2\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n2\") |> ROk \"\\r\\n2\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n2\") |> ROk \"\\n2\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n\\n\") |> ROk \"\\n\\r\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n\\n\") |> ROk \"\\r\\r\" 0 ()\n next2CharsSatisfyNot (twoN \"\\n\\n\") |> ROk \"\\r\\n\\r\" 0 ()\n next2CharsSatisfyNot eos2 |> ROk \"\" 0 ()\n next2CharsSatisfyNot eos2 |> ROk \"1\" 0 ()\n next2CharsSatisfyNot eos2 |> ROk \"\\r\" 0 ()\n next2CharsSatisfyNot eos2 |> ROk \"\\r\\n\" 0 ()\n next2CharsSatisfyNot eos2 |> ROk \"\\n\" 0 ()\n next2CharsSatisfyNot (two \"13\") |> RError \"13\" 0 NoErrorMessages\n next2CharsSatisfyNot (two \"\\n\\t\") |> RError \"\\n\\t\" 0 NoErrorMessages\n next2CharsSatisfyNot (two \"\\n\\t\") |> RError \"\\r\\n\\t\" 0 NoErrorMessages\n next2CharsSatisfyNot (two \"\\n\\t\") |> RError \"\\r\\t\" 0 NoErrorMessages\n\n anyChar >>. previousCharSatisfies (one '1') |> ROk \"12\" 1 ()\n anyChar >>. previousCharSatisfies (one '\\n') |> ROkNL \"\\n1\" 1 ()\n anyChar >>. previousCharSatisfies (one '\\n') |> ROkNL \"\\r\\n1\" 2 ()\n anyChar >>. previousCharSatisfies (one '\\n') |> ROkNL \"\\r1\" 1 ()\n anyChar >>. previousCharSatisfies (oneN '0') |> RError \"01\" 1 NoErrorMessages\n previousCharSatisfies eos1 |> RError \"1\" 0 NoErrorMessages\n previousCharSatisfies eos1 |> RError \"\" 0 NoErrorMessages\n\n anyChar >>. previousCharSatisfiesNot (oneN '1') |> ROk \"12\" 1 ()\n anyChar >>. previousCharSatisfiesNot (oneN '\\n') |> ROkNL \"\\n1\" 1 ()\n anyChar >>. previousCharSatisfiesNot (oneN '\\n') |> ROkNL \"\\r\\n1\" 2 ()\n anyChar >>. previousCharSatisfiesNot (oneN '\\n') |> ROkNL \"\\r1\" 1 ()\n anyChar >>. previousCharSatisfiesNot (one '0') |> RError \"01\" 1 NoErrorMessages\n previousCharSatisfiesNot eos1 |> ROk \"1\" 0 ()\n previousCharSatisfiesNot eos1 |> ROk \"\" 0 ()\n\nlet testUserStateParsers() =\n use stream = new CharStream<_>(\"test\")\n stream.UserState <- 1\n\n let reply = getUserState stream\n reply.Status |> Equal Ok\n reply.Error |> Equal NoErrorMessages\n reply.Result |> Equal 1\n\n let reply = setUserState 2 stream\n reply.Status |> Equal Ok\n reply.Error |> Equal NoErrorMessages\n stream.UserState |> Equal 2\n\n let reply = updateUserState (fun i -> i + 1) stream\n reply.Status |> Equal Ok\n reply.Error |> Equal NoErrorMessages\n stream.UserState |> Equal 3\n\n let reply = userStateSatisfies ((=) 3) stream\n reply.Status |> Equal Ok\n reply.Error |> Equal NoErrorMessages\n\n let reply = userStateSatisfies ((<>) 3) stream\n reply.Status |> Equal Error\n reply.Error |> Equal NoErrorMessages\n\nlet run() =\n testCharParsers()\n testAnyNoneOf()\n testSpecialCharParsers()\n testStringParsers()\n testIdentifier()\n testManySatisfy()\n testMany()\n testSkipToString()\n testNumberParsers()\n testFollowedBy()\n testUserStateParsers()\n\n" }, { "path": "Test/CharSetTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.CharSetTests\n\nopen FParsec.Test.Test\n\nlet basicTests() =\n let test s (sin: string) (sout: string) =\n let cs = FParsec.CharSet(s)\n for c in sin do\n cs.Contains(c) |> True\n for c in sout do\n cs.Contains(c) |> False\n\n test \"\" \"\" \"a\\u0000\\uffff\"\n test \"a\" \"a\" \"\\u0000\\uffff\"\n test \"\\u0000\\uffffa\" \"a\\u0000\\uffffa\" \"b\\u0001\\ufffe\"\n test \"\\u0002\\u0001\\u0399\\u0400\\u0401\\u0399\\u0400\\u0401\\uffffabc123\" \"\\u0002\\u0001\\u0399\\u0400\\u0401\\uffffabc123\" \"\\u0000\\u0398\\u0402\\ufffed0\"\n\n\nlet moreTests() =\n let rand = new System.Random(12345)\n\n for j = 0 to 20000 do\n let n = rand.Next(1, 100)\n let cs = Array.zeroCreate n\n for i = 1 to n/2 do\n let r = rand.Next()\n cs[i*2 - 2] <- char r\n cs[i*2 - 1] <- char (r >>> 16)\n if n%2 = 1 then cs[cs.Length - 1] <- char (rand.Next())\n\n let set = FParsec.CharSet(new string(cs))\n\n Array.sortInPlace cs\n\n let mutable c_1 = '\\uffff'\n let mutable c = cs[0]\n\n for i = 0 to n - 1 do\n set.Contains(c) |> True\n if c <> c_1 && int c - 1 <> int c_1 then\n set.Contains(char (int c - 1)) |> False\n if i + 1 < n then\n let c1 = cs[i + 1]\n if c < '\\uffff' && c <> c1 && int c + 1 <> int c1 then\n set.Contains(char (int c + 1)) |> False\n c_1 <- c\n c <- c1\n\nlet run() =\n basicTests()\n moreTests()\n" }, { "path": "Test/CharStreamTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.CharStreamTests\n\n#nowarn \"9\" // \"Uses of this construct may result in the generation of unverifiable .NET IL code.\"\n#nowarn \"51\" // \"The address-of operator may result in non-verifiable code.\"\n\nopen System.Text\nopen System.Text.RegularExpressions\n\nopen Microsoft.FSharp.NativeInterop\n\nopen FParsec\nopen FParsec.Test.Test\n\n\nlet EOS = CharStream.EndOfStreamChar\n\nlet testNonStreamConstructors() =\n let s = \"1234567890\"\n let cs = s.ToCharArray()\n\n let regex = new System.Text.RegularExpressions.Regex(\".*\")\n\n let testStream (stream: CharStream) (index: int) (length: int) (indexOffset: int64) (supportsRegex: bool) =\n stream.IndexOfFirstChar |> Equal indexOffset\n stream.Index |> Equal indexOffset\n stream.LineBegin |> Equal indexOffset\n stream.Line |> Equal 1L\n stream.Encoding |> Equal System.Text.Encoding.Unicode\n\n if length > 0 then\n stream.Peek() |> Equal s[index]\n stream.Skip(s.Substring(index, length)) |> True\n stream.Index |> Equal (indexOffset + int64 length)\n stream.IsEndOfStream |> Equal true\n stream.Seek(indexOffset)\n stream.Index |> Equal indexOffset\n if supportsRegex then\n stream.Match(regex).Value |> Equal (s.Substring(index, length))\n else\n try stream.Match(regex).Value |> ignore; Fail()\n with :? System.NotSupportedException -> ()\n else\n stream.Peek() |> Equal EOS\n stream.IsEndOfStream |> True\n\n let testStringStream() =\n use stream = new CharStream(s)\n testStream stream 0 s.Length 0L true\n\n use stream = new CharStream(s, 0, s.Length)\n testStream stream 0 s.Length 0L true\n use stream = new CharStream(s, 0, s.Length, 1000L)\n testStream stream 0 s.Length 1000L true\n use stream = new CharStream(s, 1, s.Length - 1)\n testStream stream 1 (s.Length - 1) 0L true\n use stream = new CharStream(s, 1, 1, 1000L)\n testStream stream 1 1 1000L true\n use stream = new CharStream(s, 1, 0, 1000L)\n testStream stream 1 0 1000L true\n\n try new CharStream((null: string), 1, 10) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try new CharStream(s, -1, 1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(s, 11, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(s, 1, 10) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(s, 0, 10, -1L) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(s, 0, 10, (1L <<< 60)) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n testStringStream()\n\n #if LOW_TRUST\n #else\n let testCharArrayStream() =\n use stream = new CharStream(cs, 0, s.Length)\n testStream stream 0 s.Length 0L false\n use stream = new CharStream(cs, 0, s.Length, 1000L)\n testStream stream 0 s.Length 1000L false\n use stream = new CharStream(cs, 1, s.Length - 1)\n testStream stream 1 (s.Length - 1) 0L false\n use stream = new CharStream(cs, 1, 1, 1000L)\n testStream stream 1 1 1000L false\n use stream = new CharStream(cs, 1, 0, 1000L)\n testStream stream 1 0 1000L false\n\n try new CharStream((null: char[]), 1, 10) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try new CharStream(cs, -1, 1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cs, 11, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cs, 1, 10) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cs, 0, 10, -1L) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cs, 0, 10, (1L <<< 60)) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n testCharArrayStream()\n\n let testCharPointerStream() =\n let handle = System.Runtime.InteropServices.GCHandle.Alloc(cs, System.Runtime.InteropServices.GCHandleType.Pinned)\n let cp = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n\n use stream = new CharStream(NativePtr.add cp 0, s.Length)\n testStream stream 0 s.Length 0L false\n use stream = new CharStream(NativePtr.add cp 0, s.Length, 1000L)\n testStream stream 0 s.Length 1000L false\n use stream = new CharStream(NativePtr.add cp 1, s.Length - 1)\n testStream stream 1 (s.Length - 1) 0L false\n use stream = new CharStream(NativePtr.add cp 1, 1, 1000L)\n testStream stream 1 1 1000L false\n use stream = new CharStream(NativePtr.add cp 1, 0, 1000L)\n testStream stream 1 0 1000L false\n\n try new CharStream(NativePtr.ofNativeInt 0n, 10) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try new CharStream(cp, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n if sizeof = 4 then\n try new CharStream(cp, System.Int32.MaxValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cp, 10, -1L) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try new CharStream(cp, 10, (1L <<< 60)) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n handle.Free()\n ()\n testCharPointerStream()\n #endif\n\nlet testStreamConstructorArgumentChecking() =\n let encoding = System.Text.Encoding.UTF8\n let str = \"1234567890\"\n let streamBytes = Array.append (encoding.GetPreamble()) (encoding.GetBytes(str))\n use stream = new System.IO.MemoryStream(streamBytes)\n try new CharStream((null: System.IO.Stream), false, encoding) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try new CharStream(stream, null) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n\n let tempFilePath = System.IO.Path.GetTempFileName()\n use nonReadableStream = new System.IO.FileStream(tempFilePath, System.IO.FileMode.Open, System.IO.FileAccess.Write)\n try new CharStream(nonReadableStream, encoding) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n nonReadableStream.Write(streamBytes, 0, streamBytes.Length)\n nonReadableStream.Dispose()\n\n try new CharStream((null: string), encoding) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try new CharStream(\"\", (null: System.Text.Encoding)) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n\n use charStream = new CharStream(tempFilePath, System.Text.Encoding.ASCII, true)\n charStream.Name |> Equal tempFilePath\n charStream.Read(str.Length + 1) |> Equal str\n charStream.Dispose()\n\n System.IO.File.Delete(tempFilePath)\n\n\ntype CustomPreambleUTF8Encoding(preamble: byte[]) =\n inherit System.Text.UTF8Encoding()\n override t.GetPreamble() = preamble\n\nlet testEncodingDetection() =\n let s = \"1234567890\"\n let iso8859_1 = System.Text.Encoding.GetEncoding(28591) // an encoding we can't detect\n\n let test (e: System.Text.Encoding) =\n let bs0 = e.GetPreamble()\n use cs0 = new CharStream(new System.IO.MemoryStream(bs0, false), iso8859_1);\n cs0.Encoding.CodePage |> Equal (e.CodePage)\n\n bs0[1] <- 33uy\n use cs0 = new CharStream(new System.IO.MemoryStream(bs0, false), iso8859_1);\n cs0.Encoding|> ReferenceEqual iso8859_1\n\n let bs = Array.append (e.GetPreamble()) (e.GetBytes(s))\n use cs = new CharStream(new System.IO.MemoryStream(bs, false), iso8859_1);\n cs.Encoding.CodePage |> Equal (e.CodePage)\n cs.Read(s.Length) |> Equal s\n use cs2 = new CharStream(new System.IO.MemoryStream(bs, false), e);\n cs2.Encoding |> ReferenceEqual e\n cs2.Read(s.Length) |> Equal s\n use cs3 = new CharStream(new System.IO.MemoryStream(bs, false), false, iso8859_1, false);\n cs3.Encoding |> ReferenceEqual iso8859_1\n\n test (System.Text.UTF32Encoding(false, true))\n test (System.Text.UTF32Encoding(true, true))\n\n test (System.Text.UnicodeEncoding(false, true))\n test (System.Text.UnicodeEncoding(true, true))\n test (System.Text.UTF8Encoding(true))\n\n let e = CustomPreambleUTF8Encoding([|0uy;1uy;2uy;3uy;4uy|])\n let bs = Array.append (e.GetPreamble()) (e.GetBytes(s))\n use cs = new CharStream(new System.IO.MemoryStream(bs, false), e);\n cs.Encoding.CodePage |> Equal (e.CodePage)\n cs.Read(s.Length) |> Equal s\n\n\n\n/// creates a CharStream with block size 8 and block overlap 3\nlet createMultiBlockTestStream byteStream encoding =\n new CharStream(byteStream, false,\n encoding, true,\n #if LOW_TRUST\n #else\n #if DISABLE_STREAM_BACKTRACKING_TESTS\n 128, 64,\n #else\n 8, 3,\n #endif\n #endif\n 16);\n\n\nlet createMultiBlockUtf8TestStream (chars: char[]) =\n let e = System.Text.Encoding.UTF8\n let bs = e.GetBytes(chars)\n createMultiBlockTestStream (new System.IO.MemoryStream(bs, false)) e\n\ntype NonSeekableMemoryStream(bytes: byte[]) =\n inherit System.IO.MemoryStream(bytes)\n override t.Seek(offset, origin) = raise (System.NotSupportedException())\n override t.CanSeek = false\n\n#if !LOW_TRUST\n\n[]\ntype NonSerializableUTF8Decoder() =\n inherit System.Text.Decoder()\n let decoder = System.Text.Encoding.UTF8.GetDecoder()\n override t.GetCharCount(bytes: byte[], index: int, count: int) : int =\n raise (System.NotImplementedException())\n override t.GetChars(bytes: byte[], byteIndex: int, byteCount: int, chars: char[], charIndex: int): int =\n raise (System.NotImplementedException())\n\n override t.Reset() = decoder.Reset()\n override t.Convert(bytes, byteCount, chars, charCount, flush, bytesUsed: byref, charsUsed: byref, completed: byref) =\n decoder.Convert(bytes, byteCount, chars, charCount, flush, &bytesUsed, &charsUsed, &completed)\n\n interface System.Runtime.Serialization.ISerializable with\n member t.GetObjectData(info, context) = raise (System.NotSupportedException())\n\ntype UTF8EncodingWithNonSerializableDecoder() =\n inherit System.Text.UTF8Encoding()\n override t.GetDecoder() = new NonSerializableUTF8Decoder() :> System.Text.Decoder\n\nlet testNonSeekableCharStreamHandling() =\n let str = \"1234567890ABCDEFGHIJKLMNOPQ\"\n let streamBytes = Array.append (System.Text.Encoding.UTF8.GetPreamble()) (System.Text.Encoding.UTF8.GetBytes(str))\n\n let testNonSeekableByteStream() =\n let encoding = System.Text.Encoding.UTF8\n use byteStream = new NonSeekableMemoryStream(streamBytes)\n use stream = createMultiBlockTestStream byteStream System.Text.Encoding.Unicode\n stream.Skip(9)\n try stream.Skip(-9) |> ignore\n Fail()\n with :? System.NotSupportedException as e -> ()\n\n use byteStream2 = new NonSeekableMemoryStream(streamBytes[..(6 + 3)])\n use stream2 = createMultiBlockTestStream byteStream2 System.Text.Encoding.Unicode\n stream2.Read(7) |> Equal str[..6]\n stream2.IsEndOfStream |> True\n\n testNonSeekableByteStream()\n\n let testNonSerializableEncoding() =\n let nsEncoding = UTF8EncodingWithNonSerializableDecoder()\n use byteStream = new System.IO.MemoryStream(streamBytes)\n use stream = createMultiBlockTestStream byteStream nsEncoding\n // seeking forward should work\n stream.Read(str.Length) |> Equal str\n stream.IsEndOfStream |> True\n // ... and backtracking to the first block should work too\n stream.SkipAndPeek(-str.Length) |> Equal str[0]\n stream.Seek(int64 str.Length - 1L)\n stream.Read() |> Equal str[str.Length - 1]\n stream.IsEndOfStream |> True\n // ... but backtracking to a block other than the first should fail\n try\n stream.Seek(8L)\n Fail()\n with :? System.NotSupportedException as e -> ()\n testNonSerializableEncoding()\n\n#endif\n\n\nlet testDecoderFallbackExceptionHandling() =\n let encoding = System.Text.Encoding.GetEncoding(\"utf-32\", System.Text.EncoderFallback.ExceptionFallback, System.Text.DecoderFallback.ExceptionFallback)\n\n let getStreamBytes bytes =\n Array.concat [|[|0x00uy|]; encoding.GetPreamble(); bytes|]\n\n let test (byteStream: System.IO.Stream) multiBlock (position: int64) =\n try\n use stream = if not multiBlock then new CharStream<_>(byteStream, encoding)\n else createMultiBlockTestStream byteStream encoding\n stream.Read(int position + 4) |> ignore\n Fail()\n with :? System.Text.DecoderFallbackException as e ->\n unbox (e.Data[\"Stream.Position\"]) |> Equal position\n\n let shortStreamBytes = getStreamBytes (encoding.GetBytes(\"123\\u00005\"))\n shortStreamBytes[1 + 4 + 3*4 + 1] <- 0xd8uy\n\n use shortByteStream = new System.IO.MemoryStream(shortStreamBytes)\n shortByteStream.ReadByte() |> ignore\n test shortByteStream false (int64 (1 + 4 + 3*4))\n use nsShortByteStream = new NonSeekableMemoryStream(shortStreamBytes)\n nsShortByteStream.ReadByte() |> ignore\n test nsShortByteStream false (int64 ( 4 + 3*4))\n\n let longStreamBytes = getStreamBytes (encoding.GetBytes(\"12345678901\\u00003\"))\n longStreamBytes[1 + 4 + 11*4 + 1] <- 0xd8uy\n\n use longByteStream = new System.IO.MemoryStream(longStreamBytes)\n longByteStream.ReadByte() |> ignore\n test longByteStream true (int64 (1 + 4 + 11*4))\n\n use nsLongByteStream = new NonSeekableMemoryStream(longStreamBytes)\n nsLongByteStream.ReadByte() |> ignore\n test nsLongByteStream true (int64 ( 4 + 11*4))\n\n\n\nlet testEmptyStream (stream: CharStream<_>) =\n\n let index0 = stream.Index\n stream.IsBeginOfStream |> True\n stream.IsEndOfStream |> True\n stream.IndexOfFirstChar |> Equal index0\n stream.IndexOfLastCharPlus1 |> Equal index0\n\n stream.Seek(index0 + 1L); stream.Index |> Equal index0\n\n stream.Peek() |> Equal EOS\n stream.Peek2() |> Equal (TwoChars(EOS, EOS))\n stream.Peek(0) |> Equal EOS\n stream.Peek(1) |> Equal EOS\n stream.Peek(-1) |> Equal EOS\n stream.Peek(System.Int32.MaxValue) |> Equal EOS\n stream.Peek(System.Int32.MinValue) |> Equal EOS\n stream.Peek(0u) |> Equal EOS\n stream.Peek(1u) |> Equal EOS\n stream.Peek(System.UInt32.MaxValue) |> Equal EOS\n stream.PeekString(0) |> Equal \"\"\n stream.PeekString(1) |> Equal \"\"\n stream.PeekString(System.Int32.MaxValue) |> Equal \"\"\n let array = [|'x'|]\n stream.PeekString(array, 0, 1) |> Equal 0\n array[0] |> Equal 'x'\n #if LOW_TRUST\n #else\n let handle = System.Runtime.InteropServices.GCHandle.Alloc(array, System.Runtime.InteropServices.GCHandleType.Pinned)\n let arrayPtr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n\n stream.PeekString(arrayPtr, 1) |> Equal 0\n array[0] |> Equal 'x'\n #endif\n\n stream.Read() |> Equal EOS; stream.Index |> Equal index0\n stream.Read(0) |> Equal \"\"; stream.Index |> Equal index0\n stream.Read(1) |> Equal \"\"; stream.Index |> Equal index0\n stream.Read(System.Int32.MaxValue) |> Equal \"\"; stream.Index |> Equal index0\n stream.Read(array, 0, 1) |> Equal 0; stream.Index |> Equal index0\n #if LOW_TRUST\n #else\n stream.Read(arrayPtr, 1) |> Equal 0\n array[0] |> Equal 'x'; stream.Index |> Equal index0\n #endif\n\n stream.Match(EOS) |> False\n stream.Match(\"\") |> True\n stream.Match(\"x\") |> False\n stream.MatchCaseFolded(EOS) |> False\n stream.MatchCaseFolded(\"\") |> True\n stream.MatchCaseFolded(\"x\") |> False\n stream.Match([||],0,0) |> True\n stream.Match([|'x'|],0,1) |> False\n #if LOW_TRUST\n #else\n stream.Match(arrayPtr, 0) |> True\n stream.Match(arrayPtr, 1) |> False\n stream.MatchCaseFolded(arrayPtr, 0) |> True\n stream.MatchCaseFolded(arrayPtr, 1) |> False\n #endif\n stream.Match(Regex(\"x\")).Success |> False\n\n stream.Skip(EOS) |> False; stream.Index |> Equal index0\n stream.Skip(\"\") |> True; stream.Index |> Equal index0\n stream.Skip(\"x\") |> False; stream.Index |> Equal index0\n stream.Skip(EOS) |> False; stream.Index |> Equal index0\n stream.Skip(\"\") |> True; stream.Index |> Equal index0\n stream.Skip(\"x\") |> False; stream.Index |> Equal index0\n stream.Skip([||], 0, 0) |> True; stream.Index |> Equal index0\n stream.Skip([|'x'|], 0, 1) |> False; stream.Index |> Equal index0\n #if LOW_TRUST\n #else\n stream.Skip(arrayPtr, 0) |> True; stream.Index |> Equal index0\n stream.Skip(arrayPtr, 1) |> False; stream.Index |> Equal index0\n stream.Skip(arrayPtr, 0) |> True; stream.Index |> Equal index0\n stream.Skip(arrayPtr, 1) |> False; stream.Index |> Equal index0\n #endif\n\n let tag = stream.StateTag\n stream.Skip(); stream.Index |> Equal index0\n stream.StateTag |> Equal tag\n\n stream.Skip(0); stream.Index |> Equal index0\n stream.Skip(1); stream.Index |> Equal index0\n stream.Skip(System.Int32.MaxValue); stream.Index |> Equal index0\n\n try stream.Skip(-1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Skip(System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n stream.Skip(0L); stream.Index |> Equal index0\n stream.Skip(1L); stream.Index |> Equal index0\n stream.Skip(System.Int64.MaxValue); stream.Index |> Equal index0\n\n try stream.Skip(-1L) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Skip(System.Int64.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n stream.Skip(0u); stream.Index |> Equal index0\n stream.Skip(1u); stream.Index |> Equal index0\n stream.Skip(System.UInt32.MaxValue); stream.Index |> Equal index0\n\n stream.SkipAndPeek(0) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(1) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(System.Int32.MaxValue) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(-1) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(System.Int32.MinValue) |> Equal EOS; stream.Index |> Equal index0\n\n stream.SkipAndPeek(0u) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(1u) |> Equal EOS; stream.Index |> Equal index0\n stream.SkipAndPeek(System.UInt32.MaxValue) |> Equal EOS; stream.Index |> Equal index0\n\n let state = stream.State\n stream.ReadFrom(state, false) |> Equal \"\"\n\n let tag = stream.StateTag\n\n stream.SkipWhitespace() |> False; stream.Index |> Equal index0\n stream.SkipUnicodeWhitespace() |> False; stream.Index |> Equal index0\n stream.SkipNewline() |> False; stream.Index |> Equal index0\n stream.SkipUnicodeNewline() |> False; stream.Index |> Equal index0\n\n stream.SkipNewlineThenWhitespace(8, true) |> Equal -1; stream.Index |> Equal index0\n\n stream.SkipRestOfLine(false); stream.Index |> Equal index0\n stream.SkipRestOfLine(true); stream.Index |> Equal index0\n\n stream.ReadRestOfLine(false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadRestOfLine(true) |> Equal \"\"; stream.Index |> Equal index0\n\n stream.ReadCharOrNewline() |> Equal EOS\n\n stream.SkipCharsOrNewlines(1) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlines(System.Int32.MaxValue) |> Equal 0; stream.Index |> Equal index0\n\n stream.ReadCharsOrNewlines(1, false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlines(1, true) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlines(System.Int32.MaxValue, false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlines(System.Int32.MaxValue, true) |> Equal \"\"; stream.Index |> Equal index0\n\n stream.SkipCharsOrNewlinesWhile((fun c -> true)) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesWhile((fun c -> true), 0, 1) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesWhile((fun c -> true), 0, System.Int32.MaxValue) |> Equal 0; stream.Index |> Equal index0\n\n stream.ReadCharsOrNewlinesWhile((fun c -> true), false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlinesWhile((fun c -> true), true) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlinesWhile((fun c -> true), 0, 1, false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlinesWhile((fun c -> true), 0, 1, true) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlinesWhile((fun c -> true), 0, System.Int32.MaxValue, false) |> Equal \"\"; stream.Index |> Equal index0\n stream.ReadCharsOrNewlinesWhile((fun c -> true), 0, System.Int32.MaxValue, true) |> Equal \"\"; stream.Index |> Equal index0\n\n let mutable b = false\n stream.SkipCharsOrNewlinesUntilString(\"1\", 1, &b) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilString(\"1\", System.Int32.MaxValue, &b) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"1\", 1, &b) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"1\", System.Int32.MaxValue, &b) |> Equal 0; stream.Index |> Equal index0\n\n let mutable s = \"\"\n stream.SkipCharsOrNewlinesUntilString(\"1\", 1, false, &s) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilString(\"1\", System.Int32.MaxValue, false, &s) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"1\", 1, false, &s) |> Equal 0; stream.Index |> Equal index0\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"1\", System.Int32.MaxValue, false, &s) |> Equal 0; stream.Index |> Equal index0\n\n stream.StateTag |> Equal tag\n\n #if LOW_TRUST\n #else\n handle.Free()\n #endif\n\nlet testBasicCharStreamMethods (stream: CharStream) (refString: string) blockSize blockOverlap minRegexSpace =\n let index0 = stream.IndexOfFirstChar\n let dollarString = new string('$', refString.Length)\n let N = refString.Length\n\n let state0 = stream.State\n\n let seekStreamTo (i: int) =\n stream.BacktrackTo(state0)\n stream.Skip(uint32 i)\n\n let testProperties() =\n stream.BacktrackTo(state0)\n let tag = stream.StateTag\n\n stream.Name <- \"Name2\"\n stream.Name |> Equal \"Name2\"\n stream.StateTag |> Equal (tag + _1)\n\n stream.UserState <- -333\n stream.UserState |> Equal -333\n stream.StateTag |> Equal (tag + _1 + _1)\n\n stream.Skip()\n stream.Column |> Equal (stream.Index - stream.LineBegin + 1L)\n\n stream.StateTag <- tag\n stream.StateTag |> Equal tag\n\n let l = stream.Line\n stream.SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(l + 2L)\n stream.Line |> Equal 3L\n stream.StateTag |> Equal tag\n\n let lb = stream.LineBegin\n stream.SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(lb + 1L)\n stream.LineBegin |> Equal (lb + 1L)\n stream.StateTag |> Equal tag\n\n (stream.BlockOverlap >= 0) |> True\n let minRegexSpace = stream.MinRegexSpace\n if minRegexSpace <> 0 then\n stream.MinRegexSpace <- minRegexSpace - 1\n stream.MinRegexSpace |> Equal (minRegexSpace - 1)\n try stream.MinRegexSpace <- -1; Fail()\n with :? System.ArgumentException -> ()\n try stream.MinRegexSpace <- stream.BlockOverlap + 1; Fail()\n with :? System.ArgumentException -> ()\n stream.MinRegexSpace <- minRegexSpace\n\n testProperties()\n\n let testRegisterNewlines() =\n stream.BacktrackTo(state0)\n let line0 = stream.Line\n let lineBegin0 = stream.LineBegin\n let tag0 = stream.StateTag\n\n stream.Skip()\n stream.RegisterNewline()\n stream.StateTag |> Equal (tag0 + _1 + _1)\n stream.Line |> Equal (line0 + 1L)\n stream.LineBegin |> Equal stream.Index\n\n stream.BacktrackTo(state0)\n stream.Skip(3)\n stream.RegisterNewlines(2, 1)\n stream.StateTag |> Equal (tag0 + _1 + _1)\n stream.Line |> Equal (line0 + 2L)\n stream.LineBegin |> Equal (stream.Index - 1L)\n\n stream.BacktrackTo(state0)\n stream.Skip(3)\n stream.RegisterNewlines(2L, 1L)\n stream.StateTag |> Equal (tag0 + _1 + _1)\n stream.Line |> Equal (line0 + 2L)\n stream.LineBegin |> Equal (stream.Index - 1L)\n\n testRegisterNewlines()\n\n let testMove i1 i2 =\n let tag1 = state0.Tag + _1\n let tag2 = tag1 + _1\n\n let index1 = index0 + int64 (min i1 N)\n let index2 = index0 + int64 (min i2 N)\n\n let c1 = if i1 < N then refString[i1] else EOS\n let c2 = if i2 < N then refString[i2] else EOS\n\n let d = i2 - min i1 N\n\n stream.BacktrackTo(state0)\n stream.Seek(index0 + int64 i1)\n stream.Index |> Equal index1\n let indexToken1 = stream.IndexToken\n stream.StateTag |> Equal tag1\n stream.Peek() |> Equal c1\n stream.IsBeginOfStream |> Equal (i1 = 0)\n stream.IsEndOfStream |> Equal (i1 >= N)\n\n stream.Seek(index0 + int64 i2)\n stream.Index |> Equal index2\n stream.StateTag |> Equal tag2\n stream.Peek() |> Equal c2\n stream.IsBeginOfStream |> Equal (i2 = 0)\n stream.IsEndOfStream |> Equal (i2 >= N)\n\n indexToken1.GetIndex(stream) |> Equal index1\n stream.Seek(indexToken1)\n stream.Index |> Equal index1\n stream.Peek() |> Equal c1\n\n seekStreamTo i1\n stream.Peek(d) |> Equal c2\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n\n if d >= 0 then\n seekStreamTo i1\n stream.Peek(uint32 d) |> Equal c2\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n\n let checkStream() =\n stream.Index |> Equal index2\n if index1 <> index2 then\n stream.StateTag |> Equal tag2\n else\n (stream.StateTag = tag2 - _1 || stream.StateTag = tag2) |> True\n\n seekStreamTo i1\n stream.Skip(d)\n checkStream()\n\n seekStreamTo i1\n stream.Skip(int64 d)\n checkStream()\n\n seekStreamTo i1\n stream.SkipAndPeek(d) |> Equal c2\n checkStream()\n\n if d >= 0 then\n if d = 1 then\n seekStreamTo i1\n stream.Skip()\n stream.Index |> Equal index2\n if index2 <> index1 then\n stream.StateTag |> Equal tag2\n else\n stream.StateTag |> Equal tag1\n\n seekStreamTo i1\n stream.SkipAndPeek() |> Equal c2\n stream.Index |> Equal index2\n if index2 <> index1 then\n stream.StateTag |> Equal tag2\n else\n stream.StateTag |> Equal tag1\n\n seekStreamTo i1\n stream.Skip(uint32 d)\n checkStream()\n\n seekStreamTo i1\n stream.SkipAndPeek(uint32 d) |> Equal c2\n checkStream()\n\n elif i2 = 0 then // d <= 0\n seekStreamTo i1\n stream.Peek(d - 1) |> Equal EOS\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n stream.Peek(System.Int32.MinValue) |> Equal EOS\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n\n seekStreamTo i1\n stream.SkipAndPeek(d - 1) |> Equal EOS\n checkStream()\n\n seekStreamTo i1\n stream.SkipAndPeek(System.Int32.MinValue) |> Equal EOS\n checkStream()\n\n for i1 = 0 to N + 2 do\n for i2 = 0 to N + 2 do\n testMove i1 i2\n\n\n let testMoveException() =\n let endIndex = index0 + int64 N\n stream.Seek(System.Int64.MaxValue)\n stream.Index |> Equal endIndex\n stream.IsEndOfStream |> True\n try stream.Seek(-1L) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Seek(index0 - 1L) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Seek(System.Int64.MinValue) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n stream.Seek(index0)\n\n let indexToken = CharStreamIndexToken()\n try stream.Seek(indexToken) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try indexToken.GetIndex(stream) |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n let state = CharStreamState()\n try stream.BacktrackTo(state); Fail()\n with :? System.ArgumentException -> ()\n try state.GetIndex(stream) |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n try state.GetPosition(stream) |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n try stream.ReadFrom(state, true) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.CreateSubstream(state) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try state.IndexToken |> ignore; Fail()\n with :? System.InvalidOperationException -> ()\n\n\n\n for i = 0 to N do\n seekStreamTo i;\n stream.Skip(System.Int32.MaxValue)\n stream.Index |> Equal endIndex\n\n seekStreamTo i;\n stream.Skip(System.UInt32.MaxValue)\n stream.Index |> Equal endIndex\n\n seekStreamTo i;\n stream.Skip(System.Int64.MaxValue)\n stream.Index |> Equal endIndex\n\n seekStreamTo i;\n stream.SkipAndPeek(System.Int32.MaxValue) |> Equal EOS\n stream.Index |> Equal endIndex\n\n seekStreamTo i;\n stream.SkipAndPeek(System.UInt32.MaxValue) |> Equal EOS\n stream.Index |> Equal endIndex\n\n seekStreamTo i;\n stream.Peek(System.Int32.MaxValue) |> Equal EOS\n\n seekStreamTo i;\n stream.Peek(System.UInt32.MaxValue) |> Equal EOS\n\n // MinValue behaviour is checked in testMove\n\n try seekStreamTo i; stream.Seek(index0 - 1L) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try seekStreamTo i; stream.Seek(-1L) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try seekStreamTo i; stream.Seek(System.Int64.MinValue) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n\n try seekStreamTo i; stream.Skip(-i - 1) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try seekStreamTo i; stream.Skip(int64 (-i - 1)) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try seekStreamTo i; stream.Skip(System.Int32.MinValue) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n try seekStreamTo i; stream.Skip(System.Int64.MinValue) |> ignore; Fail ()\n with :? System.ArgumentOutOfRangeException -> ()\n\n testMoveException()\n\n let regex = new Regex(\".*\", RegexOptions.Singleline)\n\n let testMatch i n =\n let test (str: string) (result: bool) =\n assert (str.Length = n)\n let strA = str.ToCharArray()\n let cfStr = FParsec.Text.FoldCase(str)\n let cfStrA = cfStr.ToCharArray()\n\n seekStreamTo i\n let tag1 = stream.StateTag\n let index1 = stream.Index\n let tag2 = tag1 + _1\n let index2 = min (stream.Index + int64 n) stream.IndexOfLastCharPlus1\n\n let checkStream charsAreSkipped =\n if not charsAreSkipped then\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n else\n stream.Index |> Equal index2\n if n <> 0 then\n stream.StateTag |> Equal tag2\n else\n (stream.StateTag = tag1 || stream.StateTag = tag2) |> True\n\n stream.Match(str) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.Skip(str) |> Equal result\n checkStream result\n\n seekStreamTo i\n stream.MatchCaseFolded(cfStr) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.SkipCaseFolded(cfStr) |> Equal result\n checkStream result\n\n seekStreamTo i\n stream.Match(strA, 0, n) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.Skip(strA, 0, n) |> Equal result\n checkStream result\n\n if n = 1 then\n seekStreamTo i\n stream.Match(str[0]) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.Skip(str[0]) |> Equal result\n checkStream result\n\n seekStreamTo i\n stream.MatchCaseFolded(cfStr[0]) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.SkipCaseFolded(cfStr[0]) |> Equal result\n checkStream result\n\n elif n = 2 then\n seekStreamTo i\n if stream.Skip(FParsec.TwoChars(str[0], str[1])) <> result then\n stream.Skip(FParsec.TwoChars(str[0], str[1])) |> Equal result\n //stream.Skip(FParsec.TwoChars(str[0], str[1])) |> Equal result\n checkStream result\n\n elif n > 1 then\n seekStreamTo i\n let restIsEqual = stream.Peek(n - 1) = str[n - 1] // str only differs in first or last char\n seekStreamTo (i + 1)\n let index11 = stream.Index\n stream.Match(strA, 1, n - 1) |> Equal restIsEqual\n stream.Index |> Equal index11\n stream.StateTag |> Equal tag1\n\n seekStreamTo (i + 1)\n stream.Skip(strA, 1, n - 1) |> Equal restIsEqual\n if restIsEqual then\n stream.Index |> Equal index2\n stream.StateTag |> Equal tag2\n else\n stream.Index |> Equal index11\n stream.StateTag |> Equal tag1\n #if LOW_TRUST\n #else\n if n > 0 then\n let handle = System.Runtime.InteropServices.GCHandle.Alloc(str, System.Runtime.InteropServices.GCHandleType.Pinned)\n let strPtr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n let cfHandle = System.Runtime.InteropServices.GCHandle.Alloc(cfStr, System.Runtime.InteropServices.GCHandleType.Pinned)\n let cfStrPtr = NativePtr.ofNativeInt (cfHandle.AddrOfPinnedObject())\n\n seekStreamTo i\n stream.Match(strPtr, n) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.Skip(strPtr, n) |> Equal result\n checkStream result\n\n seekStreamTo i\n stream.MatchCaseFolded(cfStrPtr, n) |> Equal result\n checkStream false\n\n seekStreamTo i\n stream.SkipCaseFolded(cfStrPtr, n) |> Equal result\n checkStream result\n\n handle.Free()\n cfHandle.Free()\n else\n let handle = System.Runtime.InteropServices.GCHandle.Alloc(\"$\", System.Runtime.InteropServices.GCHandleType.Pinned)\n let ptr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n let mutable c = '$'\n seekStreamTo i\n stream.Match(ptr, n) |> Equal true\n checkStream false\n\n seekStreamTo i\n stream.Skip(ptr, n) |> Equal true\n checkStream true\n\n seekStreamTo i\n stream.MatchCaseFolded(ptr, n) |> Equal true\n checkStream false\n\n seekStreamTo i\n stream.SkipCaseFolded(ptr, n) |> Equal true\n checkStream true\n handle.Free()\n #endif\n\n if n = 0 then\n test \"\" true\n elif i < N then\n let ci1 = char (int refString[i] + 1)\n if n > 0 && i + n <= N then\n test (refString.Substring(i, n)) true\n if n = 1 then\n test (ci1.ToString()) false\n else\n test (ci1.ToString() + refString.Substring(i + 1, n - 1)) false\n test (refString.Substring(i, n - 1) + ((char (int (refString[i + n - 1]) + 1)).ToString())) false\n else\n test (refString.Substring(i, N - i) + (new string(refString[N - 1], n - (N - i)))) false\n\n seekStreamTo i\n let index = stream.Index\n let tag = stream.StateTag\n let mstr = stream.Match(regex).Value\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n let minLength = if blockOverlap = 0 then N - i else min minRegexSpace (N - i)\n (mstr.Length >= minLength) |> True\n mstr |> Equal (refString.Substring(i, mstr.Length))\n else\n let str = new string(refString[N - 1], n)\n test str false\n\n seekStreamTo i\n let index = stream.Index\n let tag = stream.StateTag\n stream.Match(regex).Value |> Equal \"\"\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n\n for i = 0 to N do\n for n = 0 to N + 15 - i do\n testMatch i n\n\n\n let testMatchException() =\n let str = \"$$$\"\n let a = str.ToCharArray()\n\n for i in [0; 1; N - 1; N] do\n seekStreamTo i\n\n try stream.Match(null: string) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.Skip(null: string) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n\n try stream.MatchCaseFolded(null) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.SkipCaseFolded(null) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n\n try stream.Match((null: char[]), 0, 0) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.Match(a, 0, 4) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, 2, 2) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, 3, 1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, -1, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, 0, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, 0, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, System.Int32.MinValue, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, System.Int32.MinValue, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(a, System.Int32.MinValue, System.Int32.MaxValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n #if LOW_TRUST\n #else\n if i <> N then\n try stream.Match(NativePtr.ofNativeInt 0n, 1) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n\n let handle = System.Runtime.InteropServices.GCHandle.Alloc([|'$'|], System.Runtime.InteropServices.GCHandleType.Pinned)\n let ptr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n try stream.Match(ptr, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Match(ptr, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n if i <> N then\n try stream.MatchCaseFolded(NativePtr.ofNativeInt 0n, 1) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.MatchCaseFolded(ptr, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.MatchCaseFolded(ptr, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n handle.Free()\n #endif\n\n try stream.Match(null: Regex) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n\n testMatchException()\n\n let testRead i n =\n\n seekStreamTo i\n let tag1 = stream.StateTag\n let index1 = stream.Index\n let tag2 = tag1 + _1\n let index2 = min (index1 + int64 n) stream.IndexOfLastCharPlus1\n\n let str = if i < N then refString.Substring(i, min n (N - i)) else \"\"\n\n let checkStream charsAreSkipped =\n if not charsAreSkipped then\n stream.Index |> Equal index1\n stream.StateTag |> Equal tag1\n else\n stream.Index |> Equal index2\n if str.Length <> 0 then\n stream.StateTag |> Equal tag2\n else\n (stream.StateTag = tag1 || stream.StateTag = tag2) |> True\n\n\n stream.Read(n) |> Equal str\n checkStream true\n\n seekStreamTo i\n stream.PeekString(n) |> Equal str\n checkStream false\n\n seekStreamTo i\n stream.PeekString(n) |> Equal str\n checkStream false\n\n let cs = Array.create (N + 3) '$'\n seekStreamTo i\n stream.Read(cs, i%3, min n N) |> Equal str.Length\n checkStream true\n new string(cs, i%3, str.Length) |> Equal str\n for j = 0 to i%3 - 1 do cs[j] |> Equal '$'\n for j = i%3 + str.Length to N - 2 do cs[j] |> Equal '$'\n\n Array.fill cs 0 (N + 3) '$'\n seekStreamTo i\n stream.PeekString(cs, i%3, min n N) |> Equal str.Length\n checkStream false\n new string(cs, i%3, str.Length) |> Equal str\n for j = 0 to i%3 - 1 do cs[j] |> Equal '$'\n for j = i%3 + str.Length to N - 2 do cs[j] |> Equal '$'\n\n #if LOW_TRUST\n #else\n let handle = System.Runtime.InteropServices.GCHandle.Alloc(cs, System.Runtime.InteropServices.GCHandleType.Pinned)\n let ptr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n Array.fill cs 0 (N + 3) '$'\n seekStreamTo i\n stream.Read(NativePtr.add ptr (i%3), min n N) |> Equal str.Length\n checkStream true\n new string(cs, i%3, str.Length) |> Equal str\n for j = 0 to i%3 - 1 do cs[j] |> Equal '$'\n for j = i%3 + str.Length to N - 2 do cs[j] |> Equal '$'\n\n Array.fill cs 0 (N + 3) '$'\n seekStreamTo i\n stream.PeekString(NativePtr.add ptr (i%3), min n N) |> Equal str.Length\n new string(cs, i%3, str.Length) |> Equal str\n for j = 0 to i%3 - 1 do cs[j] |> Equal '$'\n for j = i%3 + str.Length to N - 2 do cs[j] |> Equal '$'\n\n handle.Free()\n #endif\n\n if n = 1 then\n seekStreamTo i\n stream.Read() |> Equal (if str.Length > 0 then str[0] else EOS)\n checkStream true\n elif n = 2 then\n let c2 = new FParsec.TwoChars((if str.Length > 0 then str[0] else EOS), (if str.Length = 2 then str[1] else EOS))\n seekStreamTo i\n stream.Peek2() |> Equal c2\n checkStream false\n\n seekStreamTo i\n let indexToken = stream.IndexToken\n stream.Skip(n)\n stream.ReadFrom(indexToken) |> Equal str\n checkStream true\n\n seekStreamTo i\n let pos1 = stream.Position\n let state = stream.State\n stream.Skip(n)\n state.GetIndex(stream) |> Equal index1\n state.IndexToken.GetIndex(stream) |> Equal index1\n state.GetPosition(stream) |> Equal pos1\n stream.ReadFrom(state, false) |> Equal str\n checkStream true\n\n for i = 0 to N do\n for n = 0 to N + 15 - i do\n testRead i n\n\n let testReadException() =\n for i in [0; 1; N - 1; N] do\n seekStreamTo i\n\n let str2 = if i < N then refString[i..] else \"\"\n try\n let str = stream.Read(System.Int32.MaxValue)\n str |> Equal str2\n seekStreamTo i\n with :? System.OutOfMemoryException -> ()\n try stream.Read(-1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n try\n let str = stream.PeekString(System.Int32.MaxValue)\n str |> Equal str2\n stream.Index |> Equal (index0 + int64 i)\n with :? System.OutOfMemoryException -> ()\n try stream.PeekString(-1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.PeekString(System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n let a = Array.create 3 '$'\n\n try stream.Read(null, 0, 1) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.Read(a, 0, 4) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, 2, 2) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, 3, 1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, -1, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, 0, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, 0, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, System.Int32.MinValue, 0) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, System.Int32.MinValue, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(a, System.Int32.MinValue, System.Int32.MaxValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n #if LOW_TRUST\n #else\n if i <> N then\n try stream.Read(NativePtr.ofNativeInt 0n, 1) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n\n let handle = System.Runtime.InteropServices.GCHandle.Alloc([|'_'|], System.Runtime.InteropServices.GCHandleType.Pinned)\n let ptr = NativePtr.ofNativeInt (handle.AddrOfPinnedObject())\n let mutable c = '_'\n try stream.Read(ptr, -1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.Read(ptr, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n handle.Free()\n #endif\n\n let mutable indexToken = CharStreamIndexToken()\n try stream.ReadFrom(indexToken) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n let mutable state = CharStreamState<_>()\n try stream.ReadFrom(&state, false) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n if not stream.IsEndOfStream then\n stream.Skip()\n state <- stream.State\n indexToken <- stream.IndexToken\n stream.Skip(-1)\n\n try stream.ReadFrom(indexToken) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.ReadFrom(&state, false) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n testReadException()\n\n/// Cross verify the CharStream string wrapper version against the normal stream\n/// version. This is done by generating a random string and then checking\n/// randomly generated access sequences on CharStream instances with random parameters.\nlet xTest() =\n let rand = System.Random(43563456)\n\n let generateRandomUnicodeChars size =\n let cs = Array.zeroCreate size\n let mutable i = 0\n while i < cs.Length do\n let r = rand.Next()\n // see http://www.unicode.org/Public/UNIDATA/Blocks.txt\n if r &&& 0xffff < 0xfffe then\n if r < (System.Int32.MaxValue/3)*2 then\n // generate a char from the BMP with about a prob. of 2/3\n let c = r % 0xffff\n if (c < 0xd800 || c > 0xdfff) then\n cs[i] <- char c\n i <- i + 1\n else\n let c_ = 0x10000 + (r % 0x25000)\n let c = if c_ < 0x30000 then c_\n else 0xe0000 ||| (c_ &&& 0xfff)\n let v = c - 0x10000\n let h = char (0xd800 ||| (c >>> 10))\n let l = char (0xdc00 ||| (c &&& 0x3ff))\n if i + 1 < cs.Length then\n cs[i] <- h\n cs[i + 1] <- l\n i <- i + 2\n cs\n\n let maxBlockSize = 100 // extremely small size for testing purposes only\n let maxReadSize = 120\n let maxJumpSize = 200\n\n let readBuffer = Array.create maxReadSize '_'\n\n\n let utf8 = System.Text.Encoding.GetEncoding(65001, System.Text.EncoderFallback.ReplacementFallback,\n new System.Text.DecoderReplacementFallback(\"XYZ\"))\n\n // The handling of invalid input is buggy in the .NET decoder routines for UTF16 and GB18030,\n // so we can only use them for valid input (and we use the ExceptionFallback to verify that the input is valid).\n let utf16 = System.Text.Encoding.GetEncoding(1200, System.Text.EncoderFallback.ExceptionFallback,\n System.Text.DecoderFallback.ExceptionFallback) // utf16 litte endian\n let gb18030 = System.Text.Encoding.GetEncoding(54936, System.Text.EncoderFallback.ExceptionFallback,\n System.Text.DecoderFallback.ExceptionFallback)\n\n for j = 1 to 50 do\n let encoding, bytes, chars =\n match rand.Next()%4 with // 0 = utf8, 1 = utf16, 2-3 = gb18030\n | 0 ->\n let bytes = Array.zeroCreate (1 <<< 16)\n rand.NextBytes(bytes)\n let chars = utf8.GetChars(bytes)\n utf8, bytes, chars\n | r ->\n let encoding = utf16//if r = 1 then utf16 else gb18030\n let chars = generateRandomUnicodeChars (1 <<< 17)\n let bytes = encoding.GetBytes(chars)\n encoding, bytes, chars\n\n #if LOW_TRUST\n let chars = new string(chars)\n #endif\n\n use stringStream = new CharStream(chars, 0, chars.Length)\n\n let blockSize = 16 + rand.Next(maxBlockSize - 16)\n let maxCharsForOneByte = encoding.GetMaxCharCount(1)\n let blockOverlap = maxCharsForOneByte + rand.Next(blockSize/2 - maxCharsForOneByte)\n let byteBufferLength = 8 + rand.Next(maxBlockSize)\n let blockSizeMinusOverlap = blockSize - blockOverlap\n use charStream = new CharStream(new System.IO.MemoryStream(bytes), false,\n encoding, true,\n #if LOW_TRUST\n #else\n #if DISABLE_STREAM_BACKTRACKING_TESTS\n chars.Length, blockOverlap,\n #else \n blockSize, blockOverlap,\n #endif\n #endif\n byteBufferLength)\n\n if j%10 = 1 then\n charStream.Seek(0L)\n stringStream.Seek(0L)\n\n for i = 1 to chars.Length do\n Equal (stringStream.Read()) (charStream.Read())\n if i%blockSizeMinusOverlap = blockOverlap && i >= blockSize then\n Equal (stringStream.Peek(-blockSize)) (charStream.Peek(-blockSize))\n\n let mutable index = int32 (rand.Next(chars.Length))\n charStream.Seek(int64 index)\n stringStream.Seek(int64 index)\n // a random walk with jumps...\n let mutable i = 0;\n let mutable resetCounter = 0\n while i < 10000 do\n // biased towards jumping backwards\n let jumpSize = rand.Next(maxJumpSize) * if rand.Next(4) = 0 then 1 else -1\n index <- index + jumpSize\n if 0 < index && index < chars.Length then\n charStream.Skip(jumpSize)\n stringStream.Skip(jumpSize)\n else\n resetCounter <- resetCounter + 1\n index <- rand.Next(chars.Length)\n charStream.Seek(int64 index)\n stringStream.Seek(int64 index)\n\n let mutable doContinue = true\n while doContinue do\n if i % 500 = 0 then\n index <- rand.Next(chars.Length)\n charStream.Seek(int64 index)\n stringStream.Seek(int64 index)\n charStream.PeekString(3) |> Equal (stringStream.PeekString(3))\n\n stringStream.Index |> Equal (int64 index)\n charStream.Index |> Equal (int64 index)\n match rand.Next(4) with\n | 0 -> let c = stringStream.Read()\n if c <> EOS || not stringStream.IsEndOfStream then\n charStream.Skip(c) |> True\n index <- index + 1\n elif index >= 7 then\n charStream.Peek(-7) |> Equal (stringStream.Peek(-7))\n\n | 1 -> let n = 1 + rand.Next(maxReadSize)\n let str = stringStream.Read(n)\n charStream.Skip(str) |> True\n index <- index + str.Length\n\n | 2 -> let n = 1 + rand.Next(maxReadSize)\n let str = stringStream.Read(n)\n let cfStr = FParsec.Text.FoldCase(str)\n charStream.SkipCaseFolded(cfStr) |> True\n index <- index + str.Length\n\n | _ -> let n = 1 + rand.Next(maxReadSize)\n let str = charStream.Read(n)\n stringStream.Skip(str) |> True\n index <- index + str.Length\n\n doContinue <- rand.Next(4) <> 0 // jump every 4th iteration on average\n i <- i + 1\n\nlet testSkipWhitespace() =\n // check fast path\n\n let testChars = [|'\\t'; '\\n'; '\\r'; ' '; '\\u0008'; '\\f'; '\\u0021'; |]\n\n // returns with the stream state unchanged\n let checkSkipWhitespace (cs: char[]) iBegin (stream: CharStream<_>) =\n let mutable line = 1\n let mutable lineBegin = 0\n let mutable i = iBegin\n let mutable indentation = System.Int32.MinValue\n let mutable containsFormFeed = false\n let mutable fLine = line\n let mutable fLineBegin = lineBegin\n let mutable fI = i\n let mutable fIndentation = indentation\n\n let tabStopDistance = 8\n while i < cs.Length\n && (match cs[i] with\n | ' ' ->\n indentation <- indentation + 1\n true\n | '\\t' ->\n indentation <- indentation + (tabStopDistance - indentation%tabStopDistance)\n true\n | '\\r' ->\n line <- line + 1; lineBegin <- i + 1;\n indentation <- 0\n true\n | '\\n' ->\n if i = iBegin || cs[i - 1] <> '\\r' then\n line <- line + 1\n lineBegin <- i + 1\n indentation <- 0\n true\n | '\\f' ->\n if not containsFormFeed then\n containsFormFeed <- true\n fLine <- line\n fLineBegin <- lineBegin\n fI <- i\n fIndentation <- indentation\n indentation <- 0\n true\n | _ ->\n false)\n do i <- i + 1\n\n if not containsFormFeed then\n fLine <- line\n fLineBegin <- lineBegin\n fI <- i\n fIndentation <- indentation\n\n let state0 = stream.State\n let index0 = stream.Index\n let indexOffset = int32 index0 - iBegin\n\n let tag = if i <> iBegin then state0.Tag + _1 else state0.Tag\n let fTag = if fI <> iBegin then state0.Tag + _1 else state0.Tag\n let index = int64 ( i + indexOffset)\n let fIndex = int64 (fI + indexOffset)\n let lineBegin = if line <> 1 then int64 ( lineBegin + indexOffset) else state0.LineBegin\n let fLineBegin = if fLine <> 1 then int64 (fLineBegin + indexOffset) else state0.LineBegin\n let line = int64 line\n let fLine = int64 fLine\n\n try\n stream.SkipWhitespace() |> Equal (fI <> iBegin)\n stream.StateTag |> Equal fTag\n stream.Index |> Equal fIndex\n stream.Line |> Equal fLine\n stream.LineBegin |> Equal fLineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n with :? TestFailed ->\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n stream.SkipWhitespace() |> Equal (fI <> iBegin)\n\n stream.SkipUnicodeWhitespace() |> Equal (i <> iBegin)\n stream.StateTag |> Equal tag\n stream.Index |> Equal index\n stream.Line |> Equal line\n stream.LineBegin |> Equal lineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n let mutable ind = -3\n if cs[iBegin] = '\\r' || cs[iBegin] = '\\n' then\n stream.SkipNewlineThenWhitespace(tabStopDistance, false) |> Equal fIndentation\n stream.StateTag |> Equal fTag\n stream.Index |> Equal fIndex\n stream.Line |> Equal fLine\n stream.LineBegin |> Equal fLineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n stream.SkipNewlineThenWhitespace(tabStopDistance, true) |> Equal indentation\n stream.StateTag |> Equal tag\n stream.Index |> Equal index\n stream.Line |> Equal line\n stream.LineBegin |> Equal lineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n else\n stream.SkipNewlineThenWhitespace(tabStopDistance, true) |> Equal -1\n stream.StateTag |> Equal state0.Tag\n stream.Index |> Equal index0\n stream.Line |> Equal state0.Line\n stream.LineBegin |> Equal state0.LineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n\n let testFastPath() =\n let cs = Array.create 11 '_'\n #if LOW_TRUST\n #else\n use stream = new CharStream(cs, 1, 10, 100L)\n #endif\n for c1 in testChars do\n cs[1] <- c1\n for c2 in testChars do\n cs[2] <- c2\n for c3 in testChars do\n cs[3] <- c3\n for c4 in testChars do\n cs[4] <- c4\n for c5 in testChars do\n cs[5] <- c5\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n #if LOW_TRUST\n let stream = new CharStream(new string(cs), 1, 10, 100L)\n #endif\n checkSkipWhitespace cs 1 stream\n\n // check end of block/stream handling\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n for c9 in testChars do\n cs[9] <- c9\n for c10 in testChars do\n cs[10] <- c10\n #if LOW_TRUST\n let stream = new CharStream(new string(cs), 1, 10, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(6)\n checkSkipWhitespace cs 7 stream\n stream.Skip(1)\n checkSkipWhitespace cs 8 stream\n stream.Skip(1)\n checkSkipWhitespace cs 9 stream\n stream.Skip(1)\n checkSkipWhitespace cs 10 stream\n\n #if LOW_TRUST\n let stream = new CharStream(new string(cs), 1, 10, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(10)\n let tag = stream.StateTag\n stream.SkipWhitespace() |> False\n stream.StateTag |> Equal tag\n stream.Index |> Equal (stream.IndexOfFirstChar + 10L)\n stream.Line |> Equal 1L\n stream.LineBegin |> Equal stream.IndexOfFirstChar\n\n\n let testSlowPath() =\n let cs = Array.create 17 '_'\n // check end of block handling with multi-block CharStream (blockSize = 8, blockOverlap = 3)\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n for c9 in testChars do\n cs[9] <- c9\n for c10 in testChars do\n cs[10] <- c10\n use stream = createMultiBlockUtf8TestStream cs\n stream.Skip(6)\n checkSkipWhitespace cs 6 stream // will start in the fast path\n stream.Skip(1)\n checkSkipWhitespace cs 7 stream // will start in the slow path\n\n testFastPath()\n testSlowPath()\n\n\n\nlet testSkipUnicodeWhitespace() =\n // We've already tested the the basic skipping logic in testSkipWhitespace.\n // Here we only test that the additional newline chars are correctly recognized.\n\n let testChars = [|'\\u0008'; '\\t'; '\\n'; '\\u000B'; '\\f'; '\\r'; ' '; '\\u0021'; '\\u0085'; '\\u200a'; '\\u2028'; '\\u2029'; '\\u205f'|]\n\n let checkSkipUnicodeWhitespace (cs: char[]) iBegin (stream: CharStream<_>) =\n let mutable line = 1\n let mutable lineBegin = 0\n let mutable i = iBegin\n while i < cs.Length\n && (match cs[i] with\n | '\\r' | '\\u0085' | '\\u2028' | '\\u2029' ->\n line <- line + 1; lineBegin <- i + 1; true\n | '\\n' ->\n if i = iBegin || cs[i - 1] <> '\\r' then\n line <- line + 1\n lineBegin <- i + 1\n true\n | c ->\n System.Char.IsWhiteSpace(c))\n do i <- i + 1\n\n let state0 = stream.State\n let index0 = stream.Index\n let indexOffset = int32 index0 - iBegin\n\n let tag = if i <> iBegin then state0.Tag + _1 else state0.Tag\n let index = int64 (i + indexOffset)\n let lineBegin = if line <> 1 then int64 (lineBegin + indexOffset) else state0.LineBegin\n let line = int64 line + (state0.Line - 1L)\n\n stream.SkipUnicodeWhitespace() |> Equal (i <> iBegin)\n stream.StateTag |> Equal tag\n stream.Index |> Equal index\n stream.Line |> Equal line\n stream.LineBegin |> Equal lineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n let isNewline c = match c with\n |'\\n' | '\\r' | '\\u0085' | '\\u2028' | '\\u2029' -> true\n | _ -> false\n\n if iBegin + 2 >= i then\n if iBegin < cs.Length && isNewline cs[iBegin] then\n let index = if cs[iBegin] = '\\r' && iBegin + 1 < cs.Length && cs[iBegin + 1] = '\\n' then\n index0 + 2L\n else\n index0 + 1L\n stream.SkipUnicodeNewline() |> True\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n stream.Line |> Equal (state0.Line + 1L)\n stream.LineBegin |> Equal index\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n else\n stream.SkipUnicodeNewline() |> False\n stream.Index |> Equal index0\n stream.StateTag |> Equal state0.Tag\n stream.Line |> Equal state0.Line\n stream.LineBegin |> Equal state0.LineBegin\n\n let testFastPath() =\n let cs = Array.create 11 '_'\n #if LOW_TRUST\n #else\n use stream = new CharStream(cs, 1, 10, 100L)\n #endif\n for c1 in testChars do\n cs[1] <- c1\n for c2 in testChars do\n cs[2] <- c2\n for c3 in testChars do\n cs[3] <- c3\n for c4 in testChars do\n cs[4] <- c4\n for c5 in testChars do\n cs[5] <- c5\n #if LOW_TRUST\n let stream = new CharStream(new string(cs), 1, 10, 100L)\n #endif\n checkSkipUnicodeWhitespace cs 1 stream\n\n let testSlowPath() =\n let cs = Array.create 17 '_'\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n for c9 in testChars do\n cs[9] <- c9\n for c10 in testChars do\n cs[10] <- c10\n use stream = createMultiBlockUtf8TestStream cs\n stream.Skip(7)\n checkSkipUnicodeWhitespace cs 7 stream\n\n testFastPath()\n testSlowPath()\n\n\nlet testSkipNewlineWhitespace() =\n // Most of the testing for SkipNewlineWhitespace is done in testSkipWhitespace,\n // here we mostly test the error handling.\n\n let testArgumentChecking() =\n use stream = new CharStream(\"\\r\\n \")\n let state0 = stream.State\n let mutable indentation = 0\n try stream.SkipNewlineThenWhitespace(0, true) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.SkipNewlineThenWhitespace(-1, true) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.SkipNewlineThenWhitespace(-8, true) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.SkipNewlineThenWhitespace(System.Int32.MinValue, true) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try stream.SkipNewlineThenWhitespace(7, true) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n let testIndentationComputation() =\n // some additional checks complementing those in testSkipWhitespace\n use stream = new CharStream(\"\\r\\n\\t \\t \\t \\t \")\n let state0 = stream.State\n stream.SkipNewlineThenWhitespace(1, true) |> Equal 11\n stream.BacktrackTo(state0)\n stream.SkipNewlineThenWhitespace(2, true) |> Equal 13\n stream.BacktrackTo(state0)\n stream.SkipNewlineThenWhitespace(4, true) |> Equal 17\n\n let testIndentationOverflowHandling() =\n let mutable indentation = 0\n\n let test (str: string) (tabStopDistance: int) (index: int) (indentation: int) =\n use stream = new CharStream(str)\n\n stream.SkipNewlineThenWhitespace(tabStopDistance, true) |> Equal indentation\n stream.Index |> Equal (int64 index)\n\n test \"\\r\\n\\t\" (1 <<< 30) 3 (1 <<< 30)\n test \"\\r\\n\\t \" (1 <<< 30) 4 ((1 <<< 30) + 1)\n test \"\\r\\n\\t\\t\" (1 <<< 30) 3 (1 <<< 30)\n test \"\\r\\n\\t \\t\" (1 <<< 30) 4 ((1 <<< 30) + 1)\n test \"\\r\\n\\t\\t \" (1 <<< 30) 3 (1 <<< 30)\n test \"\\r\\n\\t \\t \" (1 <<< 30) 4 ((1 <<< 30) + 1)\n\n let cs = Array.zeroCreate (1 + (1 <<< 15) + (1 <<< 16)) // 1 + 2^15 + 2^16\n cs[0] <- '\\n'\n // (2^15 - 1)*2^16 + 2^16 = 2^31\n for i = 1 to (1 <<< 15) - 1 do\n cs[i] <- '\\t'\n for i = (1 <<< 15) to ((1 <<< 15)) + (1 <<< 16) do\n cs[i] <- ' '\n\n use stream = new CharStream(new string(cs))\n stream.SkipNewlineThenWhitespace((1 <<< 16), true) |> Equal System.Int32.MaxValue\n stream.Index |> Equal (int64 (cs.Length - 2))\n stream.Read(3) |> Equal \" \"\n\n // test slow path\n let e = System.Text.Encoding.Unicode\n let bs = e.GetBytes(cs)\n use stream = new CharStream(new System.IO.MemoryStream(bs, false), false, e, false,\n #if LOW_TRUST\n #else\n 6144, 2048,\n #endif\n 2048)\n stream.SkipNewlineThenWhitespace((1 <<< 16), true) |> Equal System.Int32.MaxValue\n stream.Index |> Equal (int64 (cs.Length - 2))\n stream.Read(3) |> Equal \" \"\n\n testArgumentChecking()\n testIndentationComputation()\n testIndentationOverflowHandling()\n\nlet testSkipRestOfLine() =\n let testChars = [|'\\n'; '\\r'; '\\t'; '\\u000C'; '\\u000E'|]\n\n let checkSkipRestOfLine (cs: char[]) iBegin (stream: CharStream<_>) =\n let state0 = stream.State\n\n let indexOffset = stream.Index - int64 iBegin\n let mutable i = iBegin\n while i < cs.Length && (cs[i] <> '\\r' && cs[i] <> '\\n') do i <- i + 1\n\n let index = indexOffset + int64 i\n let tag = if i <> iBegin then state0.Tag + _1 else state0.Tag\n let str = new string(cs, iBegin, i - iBegin)\n\n stream.SkipRestOfLine(false)\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n stream.Line |> Equal state0.Line\n stream.LineBegin |> Equal state0.LineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n stream.ReadRestOfLine(false) |> Equal str\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n stream.Line |> Equal state0.Line\n stream.LineBegin |> Equal state0.LineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n let mutable line2 = state0.Line\n if i < cs.Length then\n let c = cs[i]\n if c = '\\r' || c = '\\n' then\n i <- i + if c = '\\r' && i + 1 < cs.Length && cs[i + 1] = '\\n' then 2 else 1\n line2 <- line2 + 1L\n\n let index2 = indexOffset + int64 i\n let tag2 = if i <> iBegin then state0.Tag + _1 else state0.Tag\n let lineBegin2 = if line2 <> state0.Line then index2 else state0.LineBegin\n\n try\n stream.SkipRestOfLine(true)\n stream.Index |> Equal index2\n stream.StateTag |> Equal tag2\n stream.Line |> Equal line2\n stream.LineBegin |> Equal lineBegin2\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n with TestFailed str ->\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n stream.SkipRestOfLine(true)\n stream.Index |> Equal index2\n\n stream.ReadRestOfLine(true) |> Equal str\n stream.Index |> Equal index2\n stream.StateTag |> Equal tag2\n stream.Line |> Equal line2\n stream.LineBegin |> Equal lineBegin2\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n let testFastPath() =\n let cs = Array.create 8 '_'\n #if LOW_TRUST\n #else\n use stream = new CharStream(cs, 1, 7, 100L)\n #endif\n for c1 in testChars do\n cs[1] <- c1\n for c2 in testChars do\n cs[2] <- c2\n for c3 in testChars do\n cs[3] <- c3\n for c4 in testChars do\n cs[4] <- c4\n for c5 in testChars do\n cs[5] <- c5\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 7, 100L)\n #endif\n checkSkipRestOfLine cs 1 stream\n\n // check end of block/stream handling\n for c5 in testChars do\n cs[5] <- c5\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 7, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(4)\n checkSkipRestOfLine cs 5 stream\n stream.Skip(1)\n checkSkipRestOfLine cs 6 stream\n stream.Skip(1)\n checkSkipRestOfLine cs 7 stream\n\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 7, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(7)\n checkSkipRestOfLine cs 8 stream\n\n let testSlowPath() =\n let cs = Array.create 17 '_'\n // check end of block handling with multi-block CharStream (blockSize = 8, blockOverlap = 3)\n for c5 in testChars do\n cs[5] <- c5\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n use stream = createMultiBlockUtf8TestStream cs\n let s5 = stream.Skip(5)\n checkSkipRestOfLine cs 5 stream // will start in the fast path\n stream.Skip()\n checkSkipRestOfLine cs 6 stream // will start in the slow path\n\n testFastPath()\n testSlowPath()\n\nlet testSkipCharsOrNewlines() =\n let mutable counter = 0\n\n let check (stream: CharStream<_>) (cs: char[]) iBegin nMax =\n let state0 = stream.State\n let tag0 = state0.Tag\n let line0 = state0.Line\n let lineBegin0 = state0.LineBegin\n let index0 = stream.Index\n let indexOffset = int32 index0 - iBegin\n let lineOffset = int32 state0.Line - 1\n let alwaysTrue = fun (c: char) -> true\n let alwaysFalse = fun (c: char) -> false\n let mutable nTrueN = 0\n let nTrue = fun (c: char) -> if nTrueN > 0 then nTrueN <- nTrueN - 1; true else false\n for n = 0 to nMax do\n counter <- counter + 1\n let mutable line = 1\n let mutable lineBegin = 0\n let mutable i = iBegin\n let mutable c = 0\n while c < n && i < cs.Length do\n match cs[i] with\n | '\\r' | '\\n' ->\n i <- i + if cs[i] = '\\r' && i + 1 < cs.Length && cs[i + 1] = '\\n' then 2 else 1\n line <- line + 1\n lineBegin <- i\n | _ -> i <- i + 1\n c <- c + 1\n\n let consumed = c <> 0\n let tag = if consumed then tag0 + _1 else tag0\n let index = int64 (i + indexOffset)\n let containsNewline = line <> 1\n let lineBegin = if containsNewline then int64 (lineBegin + indexOffset) else lineBegin0\n let line = int64 (line + lineOffset)\n\n let checkStreamAndReset() = // this function needs to be fast\n stream.Index |> Equal index\n stream.StateTag |> Equal tag\n stream.Line |> Equal line\n stream.LineBegin |> Equal lineBegin\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n\n let str = new string(cs, iBegin, i - iBegin)\n let normalizedStr = FParsec.Text.NormalizeNewlines(str)\n\n if n = 1 then\n stream.ReadCharOrNewline() |> Equal (if c = 0 then EOS else normalizedStr[0])\n checkStreamAndReset()\n\n stream.SkipNewline() |> Equal containsNewline\n if containsNewline then\n checkStreamAndReset()\n else\n stream.Index |> Equal index0\n stream.StateTag |> Equal tag0\n stream.Line |> Equal line0\n stream.LineBegin |> Equal lineBegin0\n\n stream.SkipUnicodeNewline() |> Equal containsNewline\n if containsNewline then\n checkStreamAndReset()\n else\n stream.Index |> Equal index0\n stream.StateTag |> Equal tag0\n stream.Line |> Equal line0\n stream.LineBegin |> Equal lineBegin0\n\n stream.SkipCharsOrNewlinesWhile(alwaysTrue, alwaysFalse) |> Equal c\n checkStreamAndReset()\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, alwaysFalse, false) |> Equal str\n checkStreamAndReset()\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, alwaysFalse, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesWhile(alwaysTrue, alwaysFalse, 0, System.Int32.MaxValue) |> Equal c\n checkStreamAndReset()\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, alwaysFalse, 0, System.Int32.MaxValue, false) |> Equal str\n checkStreamAndReset()\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, alwaysFalse, 0, System.Int32.MaxValue, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlines(n) |> Equal c\n checkStreamAndReset()\n stream.ReadCharsOrNewlines(n, false) |> Equal str\n stream.ReadFrom(state0, false) |> Equal str\n checkStreamAndReset()\n stream.ReadCharsOrNewlines(n, true) |> Equal normalizedStr\n stream.ReadFrom(state0, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n try\n nTrueN <- n\n stream.SkipCharsOrNewlinesWhile(nTrue) |> Equal c\n checkStreamAndReset()\n with TestFailed _ ->\n stream.Seek(stream.IndexOfFirstChar); stream.BacktrackTo(state0)\n nTrueN <- n\n stream.SkipCharsOrNewlinesWhile(nTrue) |> Equal c\n\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(nTrue, false) |> Equal str\n checkStreamAndReset()\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(nTrue, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n nTrueN <- n\n stream.SkipCharsOrNewlinesWhile(nTrue, 0, System.Int32.MaxValue) |> Equal c\n checkStreamAndReset()\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(nTrue, 0, System.Int32.MaxValue, false) |> Equal str\n checkStreamAndReset()\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(nTrue, 0, System.Int32.MaxValue, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n nTrueN <- n\n stream.SkipCharsOrNewlinesWhile(alwaysTrue, 0, n) |> Equal c\n checkStreamAndReset()\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, 0, n, false) |> Equal str\n checkStreamAndReset()\n nTrueN <- n\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, 0, n, true) |> Equal normalizedStr\n checkStreamAndReset()\n\n let mutable foundString = false;\n stream.SkipCharsOrNewlinesUntilString(\"\\u0000\", n, &foundString) |> Equal c // the stream contains no '\\u0000'\n foundString |> Equal false\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"\\u0000\", n, &foundString) |> Equal c\n foundString |> Equal false\n checkStreamAndReset()\n\n let mutable str2 = null : string\n stream.SkipCharsOrNewlinesUntilString(\"\\u0000\", n, false, &str2) |> Equal c\n str2 |> IsNull\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilString(\"\\u0000\", n, true, &str2) |> Equal c\n str2 |> IsNull\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"\\u0000\", n, false, &str2) |> Equal c\n str2 |> IsNull\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"\\u0000\", n, true, &str2) |> Equal c\n str2 |> IsNull\n checkStreamAndReset()\n\n if i < cs.Length && str.IndexOf(cs[i]) = -1 then\n let cis = string cs[i]\n\n stream.SkipCharsOrNewlinesUntilString(cis, n, &foundString) |> Equal c // the stream contains no '\\u0000'\n foundString |> Equal true\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(cis, n, &foundString) |> Equal c\n foundString |> Equal true\n checkStreamAndReset()\n\n let mutable str2 = null : string\n stream.SkipCharsOrNewlinesUntilString(cis, n, false, &str2) |> Equal c\n str2 |> Equal str\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilString(cis, n, true, &str2) |> Equal c\n str2 |> Equal normalizedStr\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(cis, n, false, &str2) |> Equal c\n str2 |> Equal str\n checkStreamAndReset()\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(cis, n, true, &str2) |> Equal c\n str2 |> Equal normalizedStr\n checkStreamAndReset()\n\n let testChars = [|'\\n'; '\\r'; '\\t'; '\\u000C'; '\\u000E'|]\n\n let testFastPath() =\n let cs = Array.create 11 '_'\n #if LOW_TRUST\n #else\n use stream = new CharStream(cs, 1, 10, 100L)\n #endif\n for c1 in testChars do\n cs[1] <- c1\n for c2 in testChars do\n cs[2] <- c2\n for c3 in testChars do\n cs[3] <- c3\n for c4 in testChars do\n cs[4] <- c4\n for c5 in testChars do\n cs[5] <- c5\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 10, 100L)\n #endif\n check stream cs 1 7\n\n // check end of block/stream handling\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n for c9 in testChars do\n cs[9] <- c9\n for c10 in testChars do\n cs[10] <- c10\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 10, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(6)\n check stream cs 7 5\n stream.Skip()\n check stream cs 8 4\n stream.Skip()\n check stream cs 9 3\n stream.Skip()\n check stream cs 10 2\n\n #if LOW_TRUST\n use stream = new CharStream(new string(cs), 1, 10, 100L)\n #else\n stream.Seek(stream.IndexOfFirstChar)\n #endif\n stream.Skip(10)\n check stream cs 11 1\n\n let testSlowPath() =\n let cs = Array.create 17 '_'\n // check end of block handling with multi-block CharStream (blockSize = 8, blockOverlap = 3)\n for c6 in testChars do\n cs[6] <- c6\n for c7 in testChars do\n cs[7] <- c7\n for c8 in testChars do\n cs[8] <- c8\n for c9 in testChars do\n cs[9] <- c9\n for c10 in testChars do\n cs[10] <- c10\n use stream = createMultiBlockUtf8TestStream cs\n stream.Skip(6)\n check stream cs 6 4\n stream.Skip()\n check stream cs 7 4\n stream.Skip()\n check stream cs 8 4\n\n let testArgumentChecking() =\n let N = 10\n let cs = Array.create N '_'\n let css = new string(cs)\n use stream = new CharStream(css, 0, cs.Length)\n\n let alwaysTrue = fun c -> true\n\n for i in [0; 1; N - 1; N] do\n let str = if i < N then css[i..] else \"\"\n let n = N - i\n stream.Seek(int64 i)\n stream.SkipCharsOrNewlines(System.Int32.MaxValue) |> Equal n\n stream.Index |> Equal (int64 N)\n\n stream.Seek(int64 i)\n stream.ReadCharsOrNewlines(System.Int32.MaxValue, true) |> Equal str\n stream.Index |> Equal (int64 N)\n\n stream.Seek(int64 i)\n try stream.SkipCharsOrNewlines(-1) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlines(System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.ReadCharsOrNewlines(-1, true) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.ReadCharsOrNewlines(System.Int32.MinValue, true) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n stream.Seek(int64 i)\n stream.SkipCharsOrNewlinesWhile(alwaysTrue, -1, System.Int32.MaxValue) |> Equal n\n stream.Index |> Equal (int64 N)\n\n stream.Seek(int64 i)\n stream.ReadCharsOrNewlinesWhile(alwaysTrue, -1, System.Int32.MaxValue, true) |> Equal str\n stream.Index |> Equal (int64 N)\n\n stream.Seek(int64 i)\n try stream.SkipCharsOrNewlinesWhile(alwaysTrue, -1, -1) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesWhile(alwaysTrue, -1, System.Int32.MinValue) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.ReadCharsOrNewlinesWhile(alwaysTrue, -1, -1, true) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.ReadCharsOrNewlinesWhile(alwaysTrue, -1, System.Int32.MinValue, true) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n let mutable found = false\n let mutable str = null\n try stream.SkipCharsOrNewlinesUntilString(null, 10, &found) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.SkipCharsOrNewlinesUntilString(null, 10, false, &str) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"\", 10, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"\", 10, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"_\", -1, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"_\", -1, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"_\", System.Int32.MinValue, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilString(\"_\", System.Int32.MinValue, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(null, 10, &found) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(null, 10, false, &str) |> ignore; Fail()\n with :? System.NullReferenceException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"\", 10, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"\", 10, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"_\", -1, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"_\", -1, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"_\", System.Int32.MinValue, &found) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.SkipCharsOrNewlinesUntilCaseFoldedString(\"_\", System.Int32.MinValue, false, &str) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n\n let SkipCharsOrNewlinesWhileMinChars() =\n let cs = \"0123456789\"\n use stream = new CharStream(cs, 0, cs.Length)\n\n let smaller n = fun c -> int c < int '0' + n\n for n = 0 to 10 do\n stream.SkipCharsOrNewlinesWhile(smaller n, n, System.Int32.MaxValue) |> Equal n\n stream.Index |> Equal (int64 n)\n stream.Seek(0L)\n\n stream.ReadCharsOrNewlinesWhile(smaller n, n, System.Int32.MaxValue, true) |> ignore\n stream.Index |> Equal (int64 n)\n stream.Seek(0L)\n\n let tag = stream.StateTag\n stream.SkipCharsOrNewlinesWhile(smaller n, n + 1, System.Int32.MaxValue) |> Equal 0\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n stream.ReadCharsOrNewlinesWhile(smaller n, n + 1, System.Int32.MaxValue, true) |> ignore\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n stream.SkipCharsOrNewlinesWhile((fun c -> true), n + 1, n) |> Equal 0\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n stream.ReadCharsOrNewlinesWhile((fun c -> true), n + 1, n, true) |> ignore\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n stream.SkipCharsOrNewlinesWhile((fun c -> true), System.Int32.MaxValue, n) |> Equal 0\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n stream.ReadCharsOrNewlinesWhile((fun c -> true), System.Int32.MaxValue, n, true) |> ignore\n stream.Index |> Equal 0L\n stream.StateTag |> Equal tag\n\n\n testFastPath()\n testSlowPath()\n testArgumentChecking()\n SkipCharsOrNewlinesWhileMinChars()\n\n\nlet SkipCharsOrNewlinesUntilString() =\n // most of the testing has already been done in testSkipCharsOrNewlines\n let cs = \"ABCDEFGHI\\tJKLMNOPQRST\".ToCharArray()\n use stream = createMultiBlockUtf8TestStream cs // blockSize = 8, blockOverlap = 3\n for i0 = 0 to cs.Length - 1 do\n stream.Seek(0L); stream.Seek(int64 i0)\n for i1 = i0 to cs.Length - 1 do\n for n = 1 to cs.Length - i1 do\n\n let check strToFind maxChars isPresent =\n let iEnd = if isPresent then i1\n elif maxChars < cs.Length - i0 then i0 + maxChars\n else cs.Length\n\n let skippedString = if not isPresent then null\n else new string(cs[i0..i1 - 1])\n\n let state0 = stream.State\n let mutable found = false\n stream.SkipCharsOrNewlinesUntilString(strToFind, maxChars, &found) |> Equal (iEnd - i0)\n stream.Index |> Equal (int64 iEnd)\n found |> Equal isPresent\n stream.Skip(0L); stream.BacktrackTo(state0)\n\n let mutable str = null\n stream.SkipCharsOrNewlinesUntilString(strToFind, maxChars, false, &str) |> Equal (iEnd - i0)\n stream.Index |> Equal (int64 iEnd)\n str |> Equal skippedString\n stream.Skip(0L); stream.BacktrackTo(state0)\n\n let strToFindCI = FParsec.Text.FoldCase(strToFind)\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(strToFindCI, maxChars, &found) |> Equal (iEnd - i0)\n stream.Index |> Equal (int64 iEnd)\n found |> Equal isPresent\n stream.Skip(0L); stream.BacktrackTo(state0)\n\n stream.SkipCharsOrNewlinesUntilCaseFoldedString(strToFindCI, maxChars, false, &str) |> Equal (iEnd - i0)\n stream.Index |> Equal (int64 iEnd)\n str |> Equal skippedString\n stream.Skip(0L); stream.BacktrackTo(state0)\n\n let strToFind = new string(cs, i1, n)\n check strToFind System.Int32.MaxValue true\n check strToFind (i1 - i0) true\n if i1 - i0 > 0 then\n check strToFind (i1 - i0 - 1) false\n if n > 1 then\n let strToNotFind = string (char (int strToFind[0] + 1)) + (if n > 1 then strToFind.Substring(1) else \"\")\n check strToNotFind System.Int32.MaxValue false\n let strToNotFind2 = strToFind.Substring(0, n - 1) + string (char (int strToFind[n - 1] + 1))\n check strToNotFind2 System.Int32.MaxValue false\n\n\nlet testCreateSubstream() =\n\n let test (stream: CharStream<_>) =\n let off = stream.IndexOfFirstChar\n let state0 = stream.State\n stream.ReadCharsOrNewlines(3, false) |> Equal \" \\r\\n0\"\n stream.Index |> Equal (off + 4L)\n stream.Line |> Equal 2L\n stream.LineBegin |> Equal (off + 3L)\n let state1 = stream.State\n stream.ReadCharsOrNewlines(5, false) |> Equal \"1\\n345\"\n use substream = stream.CreateSubstream(state1)\n stream.BacktrackTo(state0)\n let substreamState0 = substream.State\n substream.IndexOfFirstChar |> Equal (off + 4L)\n substream.IndexOfLastCharPlus1 |> Equal (off + 9L)\n substream.Line |> Equal 2L\n substream.LineBegin |> Equal(off + 3L)\n substream.ReadCharsOrNewlines(10, false) |> Equal \"1\\n345\"\n substream.Index |> Equal (off + 9L)\n substream.IsEndOfStream |> True\n substream.Line |> Equal 3L\n substream.LineBegin |> Equal (off + 6L)\n stream.ReadCharsOrNewlines(10, false) |> Equal \" \\r\\n01\\n3456\"\n stream.Index |> Equal (off + 10L)\n stream.Line |> Equal 3L\n substream.BacktrackTo(substreamState0)\n substream.Read() |> Equal '1'\n\n let substreamState1 = substream.State\n substream.ReadCharsOrNewlines(3, false) |> Equal \"\\n34\"\n use subSubstream = substream.CreateSubstream(substreamState1)\n subSubstream.IndexOfFirstChar |> Equal (off + 5L)\n subSubstream.IndexOfLastCharPlus1 |> Equal (off + 8L)\n subSubstream.Line |> Equal 2L\n subSubstream.LineBegin |> Equal (off + 3L)\n subSubstream.ReadCharsOrNewlines(10, false) |> Equal \"\\n34\"\n subSubstream.Index |> Equal (off + 8L)\n subSubstream.IsEndOfStream |> True\n subSubstream.Line |> Equal 3L\n subSubstream.LineBegin |> Equal (off + 6L)\n\n substream.BacktrackTo(substreamState1)\n use subSubstream2 = substream.CreateSubstream(substreamState1)\n subSubstream2.IndexOfFirstChar |> Equal (off + 5L)\n subSubstream2.IndexOfLastCharPlus1 |> Equal (off + 5L)\n subSubstream2.Line |> Equal 2L\n subSubstream2.LineBegin |> Equal (off + 3L)\n subSubstream2.IsEndOfStream |> True\n substream.ReadCharsOrNewlines(10, false) |> Equal \"\\n345\"\n substream.IsEndOfStream |> True\n\n let substreamStateEnd = substream.State\n use subSubstream3 = substream.CreateSubstream(substreamStateEnd)\n subSubstream3.IndexOfFirstChar |> Equal (off + 9L)\n subSubstream3.IndexOfLastCharPlus1 |> Equal (off + 9L)\n subSubstream3.Line |> Equal 3L\n subSubstream3.LineBegin |> Equal (off + 6L)\n subSubstream3.IsEndOfStream |> True\n\n substream.BacktrackTo(substreamState0)\n try substream.CreateSubstream(substreamState1) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try substream.CreateSubstream(substreamStateEnd) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n\n stream.Skip(-1)\n let state2 = stream.State\n stream.Read() |> Equal '6'\n stream.IsEndOfStream |> True\n\n use substream2 = stream.CreateSubstream(state2)\n substream2.IndexOfFirstChar |> Equal (off + 9L)\n substream2.IndexOfLastCharPlus1 |> Equal (off + 10L)\n substream2.Index |> Equal(off + 9L)\n substream2.Line |> Equal 3L\n substream2.LineBegin |> Equal (off + 6L)\n substream2.Read() |> Equal '6'\n substream2.IsEndOfStream |> True\n\n let stateEnd = stream.State\n use substream3 = stream.CreateSubstream(stateEnd)\n substream3.IndexOfFirstChar |> Equal (off + 10L)\n substream3.IndexOfLastCharPlus1 |> Equal (off + 10L)\n substream3.Index |> Equal(off + 10L)\n substream3.Line |> Equal 3L\n substream3.LineBegin |> Equal (off + 6L)\n substream3.IsEndOfStream |> True\n\n stream.BacktrackTo(state1)\n use substream4 = stream.CreateSubstream(state1)\n substream4.IndexOfFirstChar |> Equal (off + 4L)\n substream4.IndexOfLastCharPlus1 |> Equal (off + 4L)\n substream4.Index |> Equal (off + 4L)\n substream4.Line |> Equal 2L\n substream4.LineBegin |> Equal (off + 3L)\n substream4.IsEndOfStream |> True\n\n stream.BacktrackTo(state0)\n try stream.CreateSubstream(state1) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try stream.CreateSubstream(stateEnd) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n let str = \" \\r\\n01\\n3456\"\n use stream1 = new CharStream(\"!\" + str + \"!\", 1, str.Length, 100L)\n test stream1\n\n#if !LOW_TRUST\n #if DEBUG\n let state = stream1.State\n use substream = stream1.CreateSubstream(state)\n try stream1.Dispose(); Fail()\n with :? System.InvalidOperationException -> ()\n substream.Read() |> ignore\n #endif\n#endif\n\n#if !LOW_TRUST\n use stream2 = new CharStream((\"!\" + str + \"!\").ToCharArray(), 1, str.Length, 100L)\n test stream2\n#endif\n\n use stream3 = createMultiBlockUtf8TestStream (str.ToCharArray()) // blockSize = 8, blockOverlap = 3\n test stream3\n\nlet testTwoChars() =\n let cs = new TwoChars('\\u0001', '\\u0002')\n cs.Char0 |> Equal '\\u0001'\n cs.Char1 |> Equal '\\u0002'\n cs.Equals(TwoChars('\\u0001', '\\u0002')) |> True\n (box cs).Equals(TwoChars('\\u0001', '\\u0002')) |> True\n TwoChars.op_Equality(cs, TwoChars((2u <<< 16) ||| 1u)) |> True\n TwoChars.op_Inequality(cs, TwoChars('\\u0001', '\\u0002')) |> False\n cs.GetHashCode() |> Equal ((2 <<< 16) ||| 1)\n let cs2 = new TwoChars('\\uffff', '\\uffff')\n cs2.Char0 |> Equal '\\uffff'\n cs2.Char1 |> Equal '\\uffff'\n\n\nlet run() =\n#if !LOW_TRUST\n setStaticField typeof \"MinimumByteBufferLength\" 10\n setStaticField typeof \"DoNotRoundUpBlockSizeToSimplifyTesting\" true\n#endif\n\n testNonStreamConstructors()\n testStreamConstructorArgumentChecking()\n testEncodingDetection()\n\n#if !LOW_TRUST\n #if !DISABLE_STREAM_BACKTRACKING_TESTS\n testNonSeekableCharStreamHandling()\n #endif\n#endif\n testDecoderFallbackExceptionHandling()\n\n let testStreams() =\n let refString = \"1234567890ABCDEF\"\n use stringStream = new CharStream(\" \" + refString, 1, refString.Length, 100L)\n testBasicCharStreamMethods stringStream refString refString.Length 0 0\n\n let be = new System.Text.UTF32Encoding(true, true)\n let bs = Array.append (be.GetPreamble()) (be.GetBytes(refString))\n use fileStream = createMultiBlockTestStream (new System.IO.MemoryStream(bs, false)) System.Text.Encoding.Unicode\n fileStream.MinRegexSpace <- 3\n testBasicCharStreamMethods fileStream refString 8 3 3\n\n let refString2 = \"1234567890ABCDEFGH\" // exactly three blocks + 1 overlap\n let bs2 = System.Text.Encoding.Unicode.GetBytes(refString2)\n use fileStream = createMultiBlockTestStream (new System.IO.MemoryStream(bs2, false)) System.Text.Encoding.Unicode\n fileStream.MinRegexSpace <- 3\n testBasicCharStreamMethods fileStream refString2 8 3 3\n\n use emptyStringStream = new CharStream(\"x\", 1, 0, 1000L)\n testEmptyStream emptyStringStream\n\n use emptyStringStream2 = new CharStream(\"\")\n testEmptyStream emptyStringStream2\n\n use emptyFileStream = createMultiBlockTestStream (new System.IO.MemoryStream(be.GetPreamble(), false)) System.Text.Encoding.Unicode\n testEmptyStream emptyFileStream\n\n use emptyFileStream2 = createMultiBlockTestStream (new System.IO.MemoryStream([||], false)) System.Text.Encoding.Unicode\n testEmptyStream emptyFileStream2\n\n#if NETCORE\n Encoding.RegisterProvider CodePagesEncodingProvider.Instance\n#endif\n testStreams()\n xTest()\n testSkipWhitespace()\n testSkipUnicodeWhitespace()\n testSkipNewlineWhitespace()\n testSkipRestOfLine()\n testSkipCharsOrNewlines()\n SkipCharsOrNewlinesUntilString()\n testCreateSubstream()\n testTwoChars()\n" }, { "path": "Test/CloningTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.CloningTests\n\n#if !LOW_TRUST\nopen FParsec.Test.Test\n\nopen FParsec.Cloning\n\n// The organization of this test module is a bit messy currently,\n// so \"Go to definition\" is your friend here.\n// The important point is that the code coverage is close to 100%.\n\ntype SerializationInfo = System.Runtime.Serialization.SerializationInfo\ntype StreamingContext = System.Runtime.Serialization.StreamingContext\ntype OnSerializingAttribute = System.Runtime.Serialization.OnSerializingAttribute\ntype OnSerializedAttribute = System.Runtime.Serialization.OnSerializedAttribute\ntype OnDeserializingAttribute = System.Runtime.Serialization.OnDeserializingAttribute\ntype OnDeserializedAttribute = System.Runtime.Serialization.OnDeserializedAttribute\ntype ISerializable = System.Runtime.Serialization.ISerializable\ntype IObjectReference = System.Runtime.Serialization.IObjectReference\ntype IDeserializationCallback = System.Runtime.Serialization.IDeserializationCallback\ntype SerializationException = System.Runtime.Serialization.SerializationException\n\ntype CloneEvents = FParsec.Cloning.CloneEvents\ntype CloneEventHandlers = FParsec.Cloning.CloneEventHandlers\n\ntype BindingFlags = System.Reflection.BindingFlags\n\ntype KeyValuePair<'k,'v> = System.Collections.Generic.KeyValuePair<'k,'v>\n\n\n[]\ntype TestState(objectIndices: int[]) =\n inherit Cloner.State(null, objectIndices)\n\n override t.Type =\n raise (System.NotImplementedException())\n\n override t.CreateUninitializedObject() =\n raise (System.NotImplementedException())\n\n override t.WriteToUninitializedObject(instance, objectGraph) =\n raise (System.NotImplementedException())\n\n// a reference implementation\nlet findStronglyConnectedComponents (states: Cloner.State[]) =\n let stack = new System.Collections.Generic.Stack()\n let mutable index = 1\n let indices = Array.zeroCreate states.Length\n let lowlinks = Array.zeroCreate states.Length\n\n let rec tarjan (v: int) =\n indices[v] <- index\n lowlinks[v] <- index\n index <- index + 1\n stack.Push(v)\n let objectIndices = states[v].ObjectIndices\n if objectIndices <> null then\n for w in objectIndices do\n if w <> 0 then\n if indices[w] = 0 then\n tarjan w\n lowlinks[v] <- min lowlinks[v] lowlinks[w]\n else if stack.Contains(w) then\n lowlinks[v] <- min lowlinks[v] indices[w]\n if lowlinks[v] = indices[v] then\n let mutable last = stack.Pop()\n if last <> v then\n let scc = new System.Collections.Generic.List()\n scc.Add(last)\n while last <> v do\n last <- stack.Pop()\n scc.Add(last)\n let scc = scc.ToArray()\n for i in scc do\n states[i].StronglyConnectedComponent <- scc\n tarjan 1\n\n\nlet copyTestStates (states: TestState[]) =\n states |> Array.map (fun s -> if s <> null then new TestState(s.ObjectIndices) else null)\n\nlet upcastTestStates (states: TestState[]) = box states :?> Cloner.State[]\n\nlet createRandomTestStateGraph (rand: System.Random) n =\n let states = Array.zeroCreate (n + 1)\n let p = rand.NextDouble()\n let p = p*p\n for i = 2 to n do\n let indices = ResizeArray()\n for j = 1 to n do\n if rand.NextDouble() < p then indices.Add(j)\n if indices.Count = 0 && rand.NextDouble() < 0.5 then\n states[i] <- new TestState(null)\n else\n if rand.NextDouble() < 0.5 then indices.Add(0) // 0 values should be ignored\n let indices = indices.ToArray()\n shuffleArray rand indices\n states[i] <- new TestState(indices)\n\n // find all roots\n let visited = Array.zeroCreate (n + 1)\n let unvisitedIndices = [|2..n|]\n let mutable unvisitedCount = n - 1\n let roots = ResizeArray()\n while unvisitedCount <> 0 do\n let oldUnvisitedCount = unvisitedCount\n let rec mark index =\n visited[index] <- 1uy\n unvisitedCount <- unvisitedCount - 1\n let indices = states[index].ObjectIndices\n if indices <> null then\n for idx in indices do\n if idx > 1 && visited[idx] = 0uy then\n mark idx\n let index = unvisitedIndices[rand.Next(oldUnvisitedCount)]\n roots.Add(index)\n mark index\n // remove newly marked indices from unvisitedIndices\n let mutable lag = 0\n for i = 0 to oldUnvisitedCount - 1 do\n if visited[unvisitedIndices[i]] <> 0uy then lag <- lag + 1\n elif lag <> 0 then\n unvisitedIndices[i - lag] <- unvisitedIndices[i]\n\n if rand.NextDouble() < p then roots.Add(1)\n states[1] <- TestState(roots.ToArray())\n states\n\n\n\nlet testStronglyConnectedComponents() =\n let test (states: TestState[]) =\n states[0] |> Equal null\n\n let states2 = upcastTestStates (copyTestStates states)\n let states = upcastTestStates states\n let components = Cloner.FindStronglyConnectedComponents(states)\n\n components[0] |> Equal 0\n components[0] <- System.Int32.MaxValue\n (components |> Array.tryFind (fun c -> c <= 0)).IsNone |> True\n // basic consistency checks\n for i = 1 to states.Length - 1 do\n let c = components[i]\n if c <> 0 then\n let state = states[i]\n let scc = state.StronglyConnectedComponent\n if scc = null then\n components[i] <- 0\n else\n for j in scc do\n states[j].StronglyConnectedComponent |> ReferenceEqual scc\n components[j] |> Equal c\n components[j] <- 0\n System.Array.IndexOf(components, c) |> Equal -1\n for i = 1 to states.Length - 1 do\n components[i] |> Equal 0\n\n // compare result with reference implementation\n findStronglyConnectedComponents states2\n for i = 1 to states.Length - 1 do\n let state1 = states[i]\n let state2 = states2[i]\n let scc1 = state1.StronglyConnectedComponent\n let scc2 = state2.StronglyConnectedComponent\n if scc2 = null then\n scc1 |> IsNull\n elif not (obj.ReferenceEquals(scc1, scc2)) then\n System.Array.Sort(scc1)\n System.Array.Sort(scc2)\n scc1 |> Equal scc2\n // speed up future comparisons\n for j in state2.StronglyConnectedComponent do\n states2[j].StronglyConnectedComponent <- scc1\n\n test [|null; TestState(null)|]\n test [|null; TestState([|1|])|]\n\n let rand = System.Random(1234)\n for i = 0 to 1000 do\n let graph = createRandomTestStateGraph rand 7\n test graph\n for i = 8 to 300 do\n let graph = createRandomTestStateGraph rand i\n test graph\n\nlet testComputeTopologicalOrder() =\n let test (states: Cloner.State[]) =\n let order = Cloner.ComputeTopologicalOrder(states)\n let marked = Array.zeroCreate states.Length\n\n // check that each index only occurs once\n for index in order do\n marked[index] |> Equal 0uy\n marked[index] <- 1uy\n System.Array.IndexOf(marked, 0uy) |> Equal -1\n System.Array.Clear(marked, 0, marked.Length)\n\n // check dependency order\n marked[0] <- 1uy // states[0] is ignored\n for i = order.Length - 1 downto 1 do\n let index = order[i]\n if marked[index] = 0uy then\n let state = states[index]\n if state.StronglyConnectedComponent = null then\n marked[index] <- 1uy\n // all dependencies must be marked\n if state.ObjectIndices <> null then\n for j in state.ObjectIndices do\n marked[j] |> Equal 1uy\n else\n // objects within a strongly connected components have no defined order\n for j in state.StronglyConnectedComponent do\n marked[j] <- 1uy\n for j in state.StronglyConnectedComponent do\n for k in states[j].ObjectIndices do\n marked[k] |> Equal 1uy\n\n\n test [|null; TestState(null)|]\n test [|null; TestState([|1|])|]\n\n let rand = System.Random(1234)\n for i = 0 to 1000 do\n let graph = createRandomTestStateGraph rand 7\n test (upcastTestStates graph)\n for i = 8 to 300 do\n let graph = createRandomTestStateGraph rand i\n test (upcastTestStates graph)\n\n\nlet callStreamingContextCallback (context: StreamingContext) =\n\n (context.Context :?> (unit -> unit))()\n\nlet addToContextList (context: StreamingContext) s =\n let r = context.Context :?> string list ref\n r.Value <- s::r.Value\n\n[]\ntype NonSerializableBase() = class end\n\ntype ClassWithNonSerializableBase() =\n inherit NonSerializableBase()\n\ntype ClassWithSingleOnSerializingHandler() =\n member t.AnotherMethod() = ()\n []\n member private t.OnSerializing(context) =\n addToContextList context \"OnSerializing\"\n\ntype ClassWithSingleOnSerializingHandlerAndNonSerializableBase() =\n inherit NonSerializableBase()\n member t.AnotherMethod() = ()\n []\n member private t.OnSerializing(context) =\n addToContextList context \"OnSerializing\"\n\n\ntype ClassWithSingleOnSerializingHandlerInBase() =\n inherit ClassWithSingleOnSerializingHandler()\n\ntype ClassWithSingleOnSerializingHandlerInBaseBase() =\n inherit ClassWithSingleOnSerializingHandlerInBase()\n\ntype ClassWithSingleOnSerializedHandler() =\n member t.AnotherMethod() = ()\n []\n member private t.OnSerialized(context) =\n addToContextList context \"OnSerialized\"\n\ntype ClassWithSingleOnDeserializingHandler() =\n member t.AnotherMethod() = ()\n []\n member private t.OnDeserializing(context) =\n addToContextList context \"OnDeserializing\"\n\ntype ClassWithSingleOnDeserializedHandler() =\n member t.AnotherMethod() = ()\n []\n member private t.OnDeserialized(context) =\n addToContextList context \"OnDeserialized\"\n\ntype ClassThatHasItAllBaseBase() =\n []\n member private t.OnSerializing(context) = addToContextList context \"OnSerializingBaseBase\"\n []\n member private t.OnDeserializing(context) = addToContextList context \"OnDeserializingBaseBase\"\n\ntype ClassThatHasItAllBase() =\n inherit ClassThatHasItAllBaseBase()\n []\n member private t.OnSerializing(context) = addToContextList context \"OnSerializingBase\"\n []\n member private t.OnDeserializing(context) = addToContextList context \"OnDeserializingBase\"\n\ntype ClassThatHasItAll() =\n inherit ClassThatHasItAllBase()\n member private t.OnSerializin(context) = raise (System.NotImplementedException())\n []\n member private t.OnSerializing(context) = addToContextList context \"OnSerializing\"\n []\n member private t.OnSerialized(context) = addToContextList context \"OnSerialized\"\n []\n member private t.OnDeserializing(context) = addToContextList context \"OnDeserializing\"\n []\n member private t.OnDeserialized(context) = addToContextList context \"OnDeserialized\"\n\n\n interface ISerializable with\n member t.GetObjectData(info, context) = raise (System.NotImplementedException())\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) = raise (System.NotImplementedException())\n\n interface IObjectReference with\n member t.GetRealObject(context) = raise (System.NotImplementedException())\n\n\nlet testCloningEventHandlers() =\n let contextList = ref []\n let context = StreamingContext(System.Runtime.Serialization.StreamingContextStates.Clone, contextList)\n\n let () =\n CloneEventHandlers.Create(typeof) |> IsNull\n CloneEventHandlers.Create(typeof) |> IsNull\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnSerializingHandler()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnSerializing\n handlers.InvokeOnSerializing(instance, context)\n contextList.Value |> Equal [\"OnSerializing\"]\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnSerializingHandlerInBase()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnSerializing\n handlers.InvokeOnSerializing(instance, context)\n contextList.Value |> Equal [\"OnSerializing\"]\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnSerializingHandlerInBaseBase()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnSerializing\n handlers.InvokeOnSerializing(instance, context)\n contextList.Value |> Equal [\"OnSerializing\"]\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnSerializedHandler()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnSerialized\n handlers.InvokeOnSerialized(instance, context)\n contextList.Value |> Equal [\"OnSerialized\"]\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnDeserializingHandler()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnDeserializing\n handlers.InvokeOnDeserializing(instance, context)\n contextList.Value |> Equal [\"OnDeserializing\"]\n\n let () =\n contextList.Value <- []\n let instance = ClassWithSingleOnDeserializedHandler()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.OnDeserialized\n handlers.InvokeOnDeserialized(instance, context)\n contextList.Value |> Equal [\"OnDeserialized\"]\n\n let () =\n let instance = {new ISerializable with\n member t.GetObjectData(info, context) = raise (System.NotImplementedException())}\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.ISerializable\n\n let () =\n let instance = {new IDeserializationCallback with\n member t.OnDeserialization(sender) = raise (System.NotImplementedException())}\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.IDeserializationCallback\n\n let () =\n let instance = {new ISerializable with\n member t.GetObjectData(info, context) = raise (System.NotImplementedException())\n interface IObjectReference with\n member t.GetRealObject(context) = raise (System.NotImplementedException())}\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal (CloneEvents.ISerializable ||| CloneEvents.IObjectReference)\n\n let () =\n let instance = {new IObjectReference with\n member t.GetRealObject(context) = raise (System.NotImplementedException())}\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal CloneEvents.IObjectReference\n\n let () =\n let instance = new ClassThatHasItAll()\n let handlers = CloneEventHandlers.Create(instance.GetType())\n handlers.Events |> Equal ( CloneEvents.OnSerializing\n ||| CloneEvents.OnSerialized\n ||| CloneEvents.OnDeserializing\n ||| CloneEvents.OnDeserialized\n ||| CloneEvents.ISerializable\n ||| CloneEvents.IDeserializationCallback\n ||| CloneEvents.IObjectReference)\n contextList.Value <- []\n handlers.InvokeOnSerializing(instance, context)\n contextList.Value |> Equal [\"OnSerializing\"; \"OnSerializingBase\"; \"OnSerializingBaseBase\"]\n contextList.Value <- []\n handlers.InvokeOnSerialized(instance, context)\n contextList.Value |> Equal [\"OnSerialized\"]\n contextList.Value <- []\n handlers.InvokeOnDeserializing(instance, context)\n contextList.Value |> Equal [\"OnDeserializing\"; \"OnDeserializingBase\"; \"OnDeserializingBaseBase\"]\n contextList.Value <- []\n handlers.InvokeOnDeserialized(instance, context)\n contextList.Value |> Equal [\"OnDeserialized\"]\n\n try\n CloneEventHandlers.Create(typeof) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n CloneEventHandlers.Create(typeof) |> ignore\n Fail()\n with :? SerializationException -> ()\n\ntype TypeWithNoSerializedField() =\n []\n val mutable Value: int\n []\n val mutable OnSerializedWasCalled: bool\n []\n val mutable OnDeserializedWasCalled: bool\n\n []\n member private t.OnSerialized(context: StreamingContext) =\n t.OnSerializedWasCalled |> False\n t.OnSerializedWasCalled <- true\n\n []\n member private t.OnDeserialized(context: StreamingContext) =\n t.OnDeserializedWasCalled |> False\n t.OnDeserializedWasCalled <- true\n\ntype BlittableType(val1: int, val2: string, val3: KeyValuePair) =\n member t.Value1 = val1\n member t.Value2 = val2\n member t.Value3 = val3\n\n[]\ntype BlittableStructType(val1: int, val2: string, val3: KeyValuePair) =\n member t.Value1 = val1\n member t.Value2 = val2\n member t.Value3 = val3\n\ntype NonBlittableType1 =\n val Value1: int\n val Value2: int[]\n val Value3: KeyValuePair\n\ntype NonBlittableType2 =\n val Value1: int\n []\n val Value2: string\n val Value3: KeyValuePair\n\ntype NonBlittableType3 =\n val Value1: int\n val Value2: string\n val Value3: KeyValuePair\n\ntype BlittableTypeWithBase =\n inherit BlittableType\n val Value1: int\n val Value2: string\n val Value3: KeyValuePair\n\ntype BlittableTypeWithNonBlittableBase1 =\n inherit NonBlittableType1\n val Value1: int\n\ntype BlittableTypeWithNonBlittableBase2 =\n inherit NonBlittableType2\n val Value1: int\n\ntype BlittableTypeWithNonBlittableBase3 =\n inherit NonBlittableType3\n val Value1: int\n []\n val Value2: int\n\ntype BlittableTypeWithNonBlittableBase4 =\n inherit BlittableTypeWithNonBlittableBase3\n val Value1: int\n\nlet getFieldValues (fields: System.Reflection.FieldInfo[]) (instance: obj) =\n let values = Array.zeroCreate fields.Length\n for i = 0 to fields.Length - 1 do\n let f = fields[i]\n values[i] <- f.GetValue(instance)\n values\n\nlet testGetSerializedFields() =\n let mutable blittable = false\n Cloner.GetSerializedFields(typeof, &blittable).Length |> Equal 0\n blittable |> True\n Cloner.GetSerializedFields(typeof, &blittable).Length |> Equal 0\n blittable |> True\n let test (ty: System.Type) =\n let rec getFields (ty: System.Type) =\n let fields =\n ty.GetFields(BindingFlags.Public ||| BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.DeclaredOnly)\n |> Array.filter (fun field -> not field.IsNotSerialized)\n let baseType = ty.BaseType\n if baseType = null then fields\n else Array.append fields (getFields baseType)\n\n let rec isBlittableField (field: System.Reflection.FieldInfo) =\n let ty = field.FieldType\n ty.IsPrimitive || ty = typeof || (ty.IsValueType && getFields ty |> Array.forall isBlittableField)\n\n let mutable isBlittable = false\n let fields = Cloner.GetSerializedFields(ty, &isBlittable)\n let fields2 = getFields ty\n let isBlittable2 = fields2 |> Array.forall isBlittableField\n fields.Length |> Equal fields2.Length\n for i = 0 to fields2.Length - 1 do\n let f1, f2 = fields[i], fields2[i]\n f1.Name |> Equal f2.Name\n f1.FieldType |> Equal f2.FieldType\n f1.DeclaringType |> Equal f2.DeclaringType\n isBlittable |> Equal isBlittable\n\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n test typeof\n\n try\n test typeof\n Fail()\n with :? SerializationException -> ()\n\nlet testCreateFieldValuesGetter() =\n let test (instance: obj) =\n let mutable isBlittable = false\n let fields = Cloner.GetSerializedFields(instance.GetType(), &isBlittable)\n let fieldValuesGetter = Cloner.CreateFieldValuesGetter(instance.GetType(), fields)\n let values = getFieldValues fields instance\n fieldValuesGetter.Invoke(instance) |> Equal values\n\n test (BlittableType(1, \"2\", KeyValuePair(3, \"4\")))\n test (BlittableStructType(1, \"2\", KeyValuePair(3, \"4\")))\n\nlet invokeSetter (setter: System.Action) (instance: obj) (values: obj[]) (objectIndices: int[]) (objectGraph: obj[]) =\n setter.Invoke(instance, values, objectIndices, objectGraph)\n\nlet testCreateFieldValuesSetter() =\n let test (setter: System.Action) (instance: obj) (values: obj[]) (objectIndices: int[]) (objectGraph: obj[]) result =\n invokeSetter setter instance values objectIndices objectGraph\n instance |> Equal result\n\n let r1 = (1, \"2\", KeyValuePair(3, \"4\"))\n let ty1 = r1.GetType()\n let fields1 = ty1.GetFields(BindingFlags.NonPublic ||| BindingFlags.Instance)\n let setter1 = Cloner.CreateFieldValuesSetter(ty1, fields1)\n test setter1 (-1, \"-1\", KeyValuePair(-1, \"-1\")) [|1; \"2\"; null|] [|0; 0; 1|] [|null; KeyValuePair(3, \"4\")|] r1\n test setter1 (-1, \"-1\", KeyValuePair(-1, \"-1\")) [|1; \"2\"; KeyValuePair(3, \"4\")|] [|0; 0; 0|] [||] r1\n test setter1 (-1, \"-1\", KeyValuePair(-1, \"-1\")) [|null; null; null|] [|3; 2; 1|] [|null; KeyValuePair(3, \"4\"); \"2\"; 1;|] r1\n\n let r2 = (BlittableStructType(1, \"2\", KeyValuePair(3, \"4\")))\n let ty2 = r2.GetType()\n let fields2 = ty2.GetFields(BindingFlags.NonPublic ||| BindingFlags.Instance)\n let setter2 = Cloner.CreateFieldValuesSetter(ty2, fields2)\n test setter2 (BlittableStructType(-1, \"-1\", KeyValuePair(-1, \"-1\"))) [|1; \"2\"; null|] [|0; 0; 1|] [|null; KeyValuePair(3, \"4\")|] r2\n test setter2 (BlittableStructType(-1, \"-1\", KeyValuePair(-1, \"-1\"))) [|1; \"2\"; KeyValuePair(3, \"4\")|] [|0; 0; 0|] [||] r2\n test setter2 (BlittableStructType(-1, \"-1\", KeyValuePair(-1, \"-1\"))) [|null; null; null|] [|3; 2; 1|] [|null; KeyValuePair(3, \"4\"); \"2\"; 1;|] r2\n\n\ntype SerializableConstructorTestClass(calledFromISerializableConstructor: bool) =\n member t.CalledFromISerializableConstructor = calledFromISerializableConstructor\n\n private new (info: SerializationInfo, context: StreamingContext) =\n info.GetBoolean(\"ok\") |> True\n new SerializableConstructorTestClass(true)\n\n\ntype SerializableConstructorTestStruct = struct\n val mutable CalledFromISerializableConstructor: bool\n\n private new (info: SerializationInfo, context: StreamingContext) =\n info.GetBoolean(\"ok\") |> True\n {CalledFromISerializableConstructor = true}\nend\n\nlet testCreateISerializableConstructorCaller() =\n let context = new StreamingContext(System.Runtime.Serialization.StreamingContextStates.Clone)\n let info = new SerializationInfo(typeof, new System.Runtime.Serialization.FormatterConverter())\n info.AddValue(\"ok\", true)\n\n let instance1 = SerializableConstructorTestClass(false)\n instance1.CalledFromISerializableConstructor |> False\n let constructor1 = instance1.GetType().GetConstructor(BindingFlags.NonPublic ||| BindingFlags.Instance, null, [|typeof; typeof|], null)\n let constructorCaller1 = Cloner.CreateISerializableConstructorCaller(constructor1)\n constructorCaller1.Invoke(instance1, info, context)\n instance1.CalledFromISerializableConstructor |> True\n\n let mutable instance2 = SerializableConstructorTestStruct()\n instance2.CalledFromISerializableConstructor |> False\n let constructor2 = instance2.GetType().GetConstructor(BindingFlags.NonPublic ||| BindingFlags.Instance, null, [|typeof; typeof|], null)\n let constructorCaller2 = Cloner.CreateISerializableConstructorCaller(constructor2)\n let boxedInstance2 = box instance2\n constructorCaller2.Invoke(boxedInstance2, info, context)\n instance2 <- unbox boxedInstance2\n instance2.CalledFromISerializableConstructor |> True\n\n\ntype NativeSerializationTestType(val1: int, val2: string, val3: obj, val4: obj[]) =\n member t.Value1 = val1\n member t.Value2 = val2\n member t.Value3 = val3\n member t.Value4 = val4\n\nlet equalityCacheComparer =\n {new System.Collections.Generic.EqualityComparer() with\n override t.Equals((x1, x2), (y1, y2)) =\n obj.ReferenceEquals(x1, y1) && obj.ReferenceEquals(x2, y2)\n\n override t.GetHashCode((x1, x2)) =\n System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(x1)}\n\nlet equalityCache = System.Collections.Generic.Dictionary(equalityCacheComparer)\n\n/// can deal with recursive values, i.e. cyclic object graphs\nlet recEquals (value1: obj) (value2: obj) =\n let vv = (value1, value2)\n let mutable b = false\n if equalityCache.TryGetValue(vv, &b) then b\n else\n equalityCache.Add(vv, true)\n b <- value1 = value2\n equalityCache[vv] <- b\n b\n\nlet mutable onDeserializedList = [] : int list\n\n[]\ntype NativeSerializationTestClassWithUnorderedEvents<'t>(id: int, value: 't) =\n let mutable value = value\n member t.Id = id\n member t.Value with get() = value\n and set v = value <- v\n\n []\n val mutable OnSerializingWasCalled: bool\n []\n val mutable OnSerializedWasCalled: bool\n []\n val mutable OnDeserializingWasCalled: bool\n []\n val mutable DeserializationCallbackWasCalled: bool\n\n []\n member private t.OnSerializing(context: StreamingContext) =\n t.OnSerializingWasCalled |> False\n t.OnSerializingWasCalled <- true\n\n []\n member private t.OnSerialized(context: StreamingContext) =\n t.OnSerializingWasCalled |> True\n t.OnSerializedWasCalled |> False\n t.OnSerializedWasCalled <- true\n\n []\n member private t.OnDeserializing(context: StreamingContext) =\n t.OnSerializedWasCalled |> False\n t.OnDeserializingWasCalled |> False\n t.OnDeserializingWasCalled <- true\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializingWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n t.DeserializationCallbackWasCalled <- true\n\n override t.Equals(obj) =\n match obj with\n | :? NativeSerializationTestClassWithUnorderedEvents<'t> as o ->\n id = o.Id && recEquals value o.Value\n | _ -> false\n\n override t.GetHashCode() = raise (System.NotImplementedException())\n\n[]\ntype NativeSerializationTestClass<'t>(id_, value_) =\n inherit NativeSerializationTestClassWithUnorderedEvents<'t>(id_, value_)\n\n []\n val mutable OnDeserializedWasCalled: bool\n []\n val mutable DeserializationCallbackWasCalled: bool\n\n []\n member private t.OnDeserialized(context: StreamingContext) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializingWasCalled |> True\n t.OnDeserializedWasCalled |> False\n t.OnDeserializedWasCalled <- true\n onDeserializedList <- t.Id::onDeserializedList\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializingWasCalled |> True\n t.OnDeserializedWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n t.DeserializationCallbackWasCalled <- true\n\n override t.Equals(o) =\n match o with\n | :? NativeSerializationTestClass<'t> as o ->\n t.Id = o.Id && recEquals t.Value o.Value\n | _ -> false\n\n override t.GetHashCode() = raise (System.NotImplementedException())\n\n[]\ntype NativeSerializationTestClass2<'t, 't2>(id_, value_, value2: 't2) =\n inherit NativeSerializationTestClass<'t>(id_, value_)\n\n member t.Value2 = value2\n\n override t.Equals(o) =\n match o with\n | :? NativeSerializationTestClass2<'t,'t2> as o ->\n t.Id = o.Id && recEquals t.Value o.Value && recEquals t.Value2 o.Value2\n | _ -> false\n\n override t.GetHashCode() = raise (System.NotImplementedException())\n\n[]\ntype CustomSerializationTestClass<'t> private (id: int, value: 't, isConstructedFromSerializationInfo: bool) =\n let mutable value = value\n\n member t.Id = id\n member t.Value with get() = value\n and set v = value <- v\n\n []\n static val mutable private GetObjectDataCounter: int\n\n interface ISerializable with\n member t.GetObjectData(info, context) =\n let c = CustomSerializationTestClass<'t>.GetObjectDataCounter\n CustomSerializationTestClass<'t>.GetObjectDataCounter <- c + 1\n if c%3 <> 0 then\n info.AddValue(\"id\", id)\n info.AddValue(\"value\", value)\n else\n info.AddValue(\"value\", value)\n info.AddValue(\"id\", id)\n\n public new (id, value) = CustomSerializationTestClass(id, value, false)\n\n private new (info: SerializationInfo, context: StreamingContext) =\n CustomSerializationTestClass(info.GetValue(\"id\", typeof) :?> int,\n info.GetValue(\"value\", typeof<'t>) :?> 't,\n true)\n\n member t.IsConstructedFromSerializationInfo = isConstructedFromSerializationInfo\n\n []\n val mutable OnSerializingWasCalled: bool\n []\n val mutable OnSerializedWasCalled: bool\n []\n val mutable OnDeserializingWasCalled: bool\n []\n val mutable OnDeserializedWasCalled: bool\n []\n val mutable DeserializationCallbackWasCalled: bool\n\n []\n member private t.OnSerializing(context: StreamingContext) =\n t.OnSerializingWasCalled |> False\n t.OnSerializingWasCalled <- true\n\n []\n member private t.OnSerialized(context: StreamingContext) =\n t.OnSerializingWasCalled |> True\n t.OnSerializedWasCalled |> False\n t.OnSerializedWasCalled <- true\n\n []\n member private t.OnDeserializing(context: StreamingContext) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializingWasCalled |> False\n t.OnDeserializingWasCalled <- true\n\n []\n member private t.OnDeserialized(context: StreamingContext) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializingWasCalled |> True\n t.OnDeserializedWasCalled |> False\n t.OnDeserializedWasCalled <- true\n onDeserializedList <- id::onDeserializedList\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) =\n t.OnSerializingWasCalled |> False\n t.OnDeserializedWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n t.DeserializationCallbackWasCalled <- true\n\n override t.Equals(o) =\n match o with\n | :? CustomSerializationTestClass<'t> as o ->\n id = o.Id && recEquals value o.Value\n | _ -> false\n\n override t.GetHashCode() = raise (System.NotImplementedException())\n\ntype CustomSerializationTestClassProxyProxy<'t>(id: int, value: 't) =\n interface IObjectReference with\n member t.GetRealObject(context) =\n box (CustomSerializationTestClassWithProxy(id, value))\n\nand CustomSerializationTestClassProxy<'t>(id: int, value: 't) =\n private new (info: SerializationInfo, context: StreamingContext) =\n CustomSerializationTestClassProxy(info.GetValue(\"id\", typeof) :?> int,\n info.GetValue(\"value\", typeof<'t>) :?> 't)\n\n []\n val mutable OnDeserializingWasCalled: bool\n []\n val mutable DeserializationCallbackWasCalled: bool\n\n []\n member private t.OnDeserializing(context: StreamingContext) =\n t.OnDeserializingWasCalled |> False\n t.OnDeserializingWasCalled <- true\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) =\n t.OnDeserializingWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n t.DeserializationCallbackWasCalled <- true\n\n interface ISerializable with\n member t.GetObjectData(info, context) =\n raise (System.NotImplementedException())\n\n interface IObjectReference with\n member t.GetRealObject(context) =\n t.OnDeserializingWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n box (CustomSerializationTestClassProxyProxy(id, value))\n\nand CustomSerializationTestClassProxyWithOnDeserialized<'t>(id: int, value: 't) =\n []\n val mutable OnDeserializingWasCalled: bool\n []\n val mutable OnDeserializedWasCalled: bool\n []\n val mutable DeserializationCallbackWasCalled: bool\n\n []\n member private t.OnDeserializing(context: StreamingContext) =\n t.OnDeserializingWasCalled |> False\n t.OnDeserializingWasCalled <- true\n\n []\n member private t.OnDeserialized(context: StreamingContext) =\n t.OnDeserializingWasCalled |> True\n t.OnDeserializedWasCalled |> False\n t.OnDeserializedWasCalled <- true\n\n interface IDeserializationCallback with\n member t.OnDeserialization(sender) =\n t.OnDeserializedWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n t.DeserializationCallbackWasCalled <- true\n\n interface IObjectReference with\n member t.GetRealObject(context) =\n t.OnDeserializedWasCalled |> True\n t.DeserializationCallbackWasCalled |> False\n box (CustomSerializationTestClassProxyProxy(id, value))\n\nand CustomSerializationTestClassProxy2<'t> = struct\n val mutable id: int\n val mutable value: 't\n\n member t.Id with get() = t.id and set v = t.id <- v\n member t.Value with get() = t.value and set v = t.value <- v\n\n interface IObjectReference with\n member t.GetRealObject(context) =\n box (CustomSerializationTestClassProxyProxy(t.id, t.value))\nend\n\nand CustomSerializationTestClassWithProxy12<'t>(id_, value_) =\n inherit CustomSerializationTestClass<'t>(id_, value_)\n\n []\n static val mutable private ProxyCounter: int\n\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.AddValue(\"id\", t.Id)\n info.AddValue(\"value\", t.Value)\n info.AddValue(\"unused\", \"data\")\n let c = CustomSerializationTestClassWithProxy12<'t>.ProxyCounter\n CustomSerializationTestClassWithProxy12<'t>.ProxyCounter <- c + 1\n if c%3 <> 0 then\n info.FullTypeName <- typeof>.FullName\n info.AssemblyName <- typeof>.Assembly.FullName\n else\n info.FullTypeName <- typeof>.FullName\n info.AssemblyName <- typeof>.Assembly.FullName\n\nand CustomSerializationTestClassWithProxy<'t>(id_, value_) =\n inherit CustomSerializationTestClass<'t>(id_, value_)\n\n []\n static val mutable private ProxyCounter: int\n\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.AddValue(\"id\", t.Id)\n info.AddValue(\"value\", t.Value)\n info.AddValue(\"unused\", \"data\")\n let c = CustomSerializationTestClassWithProxy<'t>.ProxyCounter\n CustomSerializationTestClassWithProxy<'t>.ProxyCounter <- c + 1\n info.FullTypeName <- typeof>.FullName\n info.AssemblyName <- typeof>.Assembly.FullName\n\n\ntype CustomSerializationTestClassWithSimpleProxyBase() = class end\ntype CustomSerializationTestClassWithSimpleProxy() =\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.FullTypeName <- typeof.FullName\n info.AssemblyName <- typeof.Assembly.FullName\n\n\ntype CustomSerializationTestClassWithInvalidProxy<'t>(id_: int, value_: 't) =\n inherit CustomSerializationTestClass<'t>(id_, value_)\n // misses a deserialization constructor\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.FullTypeName <- typeof>.FullName\n info.AssemblyName <- typeof>.Assembly.FullName\n\ntype CustomSerializationTestClassInvalidProxy() =\n interface ISerializable with\n member t.GetObjectData(info, context) =\n raise (System.NotImplementedException())\n\ntype CustomSerializationTestClassWithInvalidProxy2<'t>(id: int, value: 't) =\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.FullTypeName <- typeof.FullName\n info.AssemblyName <- typeof.Assembly.FullName\n\ntype CustomSerializationTestClassWithInvalidProxy3<'t>(id: int, value: 't) =\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.FullTypeName <- typeof.FullName\n info.AssemblyName <- typeof.Assembly.FullName\n\n\ntype ClassWithNonExistentSerializationProxyClass() =\n interface ISerializable with\n override t.GetObjectData(info, context) =\n info.FullTypeName <- \"_NonExistentType\"\n info.AssemblyName <- \"_NonExistentAssembly\"\n\ntype ClassWithBuggyObjectReferenceImplementation1() =\n interface IObjectReference with\n member t.GetRealObject(context) =\n null\n\ntype ClassWithBuggyObjectReferenceImplementation2() =\n interface IObjectReference with\n member t.GetRealObject(context) =\n box (ClassWithBuggyObjectReferenceImplementation2())\n\ntype ClassWithObjectReferenceImplementationThatReturnsThis() =\n interface IObjectReference with\n member t.GetRealObject(context) =\n box t\n\ntype ClassWithObjectReferenceImplementationThatReturnsThis2() =\n interface IObjectReference with\n member t.GetRealObject(context) =\n box (ClassWithObjectReferenceImplementationThatReturnsThis())\n\n\nlet testCloners() =\n let testBlittableCloner() =\n let v1 = (BlittableType(1, \"2\", KeyValuePair(3, \"4\")))\n let v2 = Cloner.Create(v1.GetType()).Clone(v1) :?> BlittableType\n obj.ReferenceEquals(v1, v2) |> False\n v1.Value1 |> Equal v2.Value1\n v1.Value2 |> Equal v2.Value2\n v1.Value3 |> Equal v2.Value3\n let v3 = Cloner.Create(v1.GetType()).CaptureImage(v1).CreateClone() :?> BlittableType\n obj.ReferenceEquals(v1, v3) |> False\n v1.Value1 |> Equal v3.Value1\n v1.Value2 |> Equal v3.Value2\n v1.Value3 |> Equal v3.Value3\n\n let v1 = (BlittableStructType(1, \"2\", KeyValuePair(3, \"4\")))\n let v2 = Cloner.Create(v1.GetType()).Clone(v1) :?> BlittableStructType\n v1.Value1 |> Equal v2.Value1\n v1.Value2 |> Equal v2.Value2\n v1.Value3 |> Equal v2.Value3\n let v3 = Cloner.Create(v1.GetType()).CaptureImage(v1).CreateClone() :?> BlittableStructType\n v1.Value1 |> Equal v3.Value1\n v1.Value2 |> Equal v3.Value2\n v1.Value3 |> Equal v3.Value3\n\n testBlittableCloner()\n\n let testArrayCloners() =\n let EqualArray (a: System.Array) (b: System.Array) =\n let r = b.Rank\n r |> Equal a.Rank\n if r = 1 then // the F# equality comparison is bugged for rank-1 arrays with non-zero lower bound\n let off = a.GetLowerBound(0)\n (b.GetLowerBound(0)) |> Equal off\n for i = 0 to a.GetLength(0) - 1 do\n b.GetValue(off + i) |> Equal (a.GetValue(off + i))\n else\n a |> Equal b\n\n let v1 = box ([||] : int[])\n Cloner.Create(v1.GetType()).Clone(v1) |> Equal v1\n let v2 = box [|0; 1; 2|]\n Cloner.Create(v1.GetType()).Clone(v2) |> Equal v2\n let v2b = System.Array.CreateInstance(typeof, [|3|], [|1|])\n v2b.SetValue(1, 1)\n v2b.SetValue(2, 2)\n v2b.SetValue(3, 3)\n Cloner.Create(v2b.GetType()).Clone(v2b) :?> System.Array |> EqualArray v2b\n let v3 = box [|null; \"1\"; \"2\"|]\n Cloner.Create(v3.GetType()).Clone(v3) |> Equal v3\n let v4 = box [|KeyValuePair(1,2); KeyValuePair(3,4)|]\n Cloner.Create(v4.GetType()).Clone(v4) |> Equal v4\n let v5 = box ([||] : option[])\n Cloner.Create(v5.GetType()).Clone(v5) |> Equal v5\n let v6 = box [|None; Some 1; Some 2|]\n Cloner.Create(v6.GetType()).Clone(v6) |> Equal v6\n let v6b = System.Array.CreateInstance(typeof>, [|3|], [|1|])\n v6b.SetValue(None, 1)\n v6b.SetValue(Some 2, 2)\n v6b.SetValue(Some 3, 3)\n Cloner.Create(v6b.GetType()).Clone(v6b) :?> System.Array |> EqualArray v6b\n let v7 = [|box (Some 1); null; box [|Some 2; Some 3|]|]\n let cloner = Cloner.Create(v7.GetType())\n cloner.Clone(v7) |> Equal (box v7)\n v7[0] <- box [|Some 4; Some 5|]\n v7[1] <- box (Some 6)\n v7[2] <- null\n cloner.Clone(v7) |> Equal (box v7)\n\n let v8 = Array3D.zeroCreate 0 0 0\n Cloner.Create(v8.GetType()).Clone(v8) |> Equal (box v8)\n\n let v9 = Array3D.zeroCreate 3 4 5\n for i = 0 to 2 do\n for j = 0 to 3 do\n for k = 0 to 4 do\n v9[i,j,k] <- i*3*4 + j*5 + k\n Cloner.Create(v9.GetType()).Clone(v9) |> Equal (box v9)\n\n let v10 = System.Array.CreateInstance(typeof, [|3; 5; 7|], [|1; 2; 3|])\n for i = 0 to 2 do\n for j = 0 to 4 do\n for k = 0 to 6 do\n let c = i*5*7 + j*7 + k\n v10.SetValue((if c%3 = 0 then box [|Some c|]\n elif c%5 = 0 then null\n else box (Some c)), [|1 + i; 2 + j; 3 + k|])\n Cloner.Create(v10.GetType()).Clone(v10) |> Equal (box v10)\n\n testArrayCloners()\n\n let testNativeSerializationCloner() =\n let () =\n let v = TypeWithNoSerializedField()\n let v2 = Cloner.Create(v.GetType()).Clone(v) :?> TypeWithNoSerializedField\n v2.GetType() |> Equal (v.GetType())\n obj.ReferenceEquals(v, v2) |> False\n v.OnSerializedWasCalled |> True\n v.OnDeserializedWasCalled |> False\n v2.OnSerializedWasCalled |> False\n v2.OnDeserializedWasCalled |> True\n\n let () =\n let v = NativeSerializationTestClassWithUnorderedEvents(1, (2, \"3\"))\n let v2 = Cloner.Create(v.GetType()).Clone(v) :?> _\n v2 |> Equal v\n\n v.OnSerializedWasCalled |> True\n v.OnSerializingWasCalled |> True\n v.OnDeserializingWasCalled |> False\n\n v2.OnSerializingWasCalled |> False\n v2.OnSerializedWasCalled |> False\n v2.DeserializationCallbackWasCalled |> True\n\n let () =\n let v = NativeSerializationTestClass2(1, box \"2\", (3, \"4\"))\n let cloner = Cloner.Create(v.GetType())\n let v2 = cloner.Clone(v) :?> _\n v2 |> Equal v\n\n v.OnSerializedWasCalled |> True\n v.OnSerializingWasCalled |> True\n v.OnDeserializingWasCalled |> False\n v.OnDeserializedWasCalled |> False\n v.DeserializationCallbackWasCalled |> False\n\n v2.OnSerializingWasCalled |> False\n v2.OnDeserializedWasCalled |> True\n v2.DeserializationCallbackWasCalled |> True\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v3 = cloner.Clone(v) :?> _\n v3 |> Equal v\n\n v.Value <- Some 2\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v4 = cloner.Clone(v) :?> _\n v4 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v5 = cloner.Clone(v) :?> _\n v5 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- ref 2\n let v6 = cloner.Clone(v) :?> _\n v6 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- null\n let v7 = cloner.Clone(v) :?> _\n v7 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- ref 2\n let v8 = cloner.Clone(v) :?> _\n v8 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- \"2\"\n let v9 = cloner.Clone(v) :?> _\n v9 |> Equal v\n ()\n testNativeSerializationCloner()\n\n let testCustomSerializationCloner() =\n let () =\n let v = CustomSerializationTestClass(1, box \"2\")\n let cloner = Cloner.Create(v.GetType())\n let v2 = cloner.Clone(v) :?> _\n v2 |> Equal v\n\n v.OnSerializedWasCalled |> True\n v.OnSerializingWasCalled |> True\n v.OnDeserializingWasCalled |> False\n v.OnDeserializedWasCalled |> False\n v.DeserializationCallbackWasCalled |> False\n\n v2.OnSerializingWasCalled |> False\n v2.OnDeserializedWasCalled |> True\n v2.DeserializationCallbackWasCalled |> True\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v3 = cloner.Clone(v) :?> _\n v3 |> Equal v\n\n v.Value <- Some 2\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v4 = cloner.Clone(v) :?> _\n v4 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v5 = cloner.Clone(v) :?> _\n v5 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- ref 2\n let v6 = cloner.Clone(v) :?> _\n v6 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- null\n let v7 = cloner.Clone(v) :?> _\n v7 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- ref 2\n let v8 = cloner.Clone(v) :?> _\n v8 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n v.Value <- \"2\"\n let v9 = cloner.Clone(v) :?> _\n v9 |> Equal v\n\n let () =\n let v = box (CustomSerializationTestClassWithProxy12(1, KeyValuePair(\"2\", 3))) :?> CustomSerializationTestClass>\n let cloner = Cloner.Create(v.GetType())\n let v2 = cloner.Clone(v) :?> _\n v2 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v3 = cloner.Clone(v) :?> _\n v3 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v4 = cloner.Clone(v) :?> _\n v4 |> Equal v\n\n let () =\n let v = box (CustomSerializationTestClassWithProxy12(1, Some 2)) :?> CustomSerializationTestClass\n let cloner = Cloner.Create(v.GetType())\n let v2 = cloner.Clone(v) :?> _\n v2 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v3 = cloner.Clone(v) :?> _\n v3 |> Equal v\n\n v.OnSerializingWasCalled <- false; v.OnSerializedWasCalled <- false\n let v4 = cloner.Clone(v) :?> _\n v4 |> Equal v\n\n let () =\n let v = CustomSerializationTestClassWithSimpleProxy()\n let cloner = Cloner.Create(v.GetType())\n let v2 = cloner.Clone(v)\n obj.ReferenceEquals(v, v2) |> False\n v2.GetType() |> Equal (typeof)\n\n let () =\n let v1 = CustomSerializationTestClassWithProxy12(1, null)\n let v2 = CustomSerializationTestClassWithProxy12(2, null)\n v1.Value <- v2\n Cloner.Create(v1.GetType()).Clone(v1) |> Equal (box v1)\n v1.OnSerializingWasCalled <- false; v1.OnSerializedWasCalled <- false\n v2.OnSerializingWasCalled <- false; v2.OnSerializedWasCalled <- false\n v1.Value <- null\n v2.Value <- v1\n Cloner.Create(v2.GetType()).Clone(v2) |> Equal (box v2)\n v1.OnSerializingWasCalled <- false; v1.OnSerializedWasCalled <- false\n v2.OnSerializingWasCalled <- false; v2.OnSerializedWasCalled <- false\n v1.Value <- v2 // creates object graph cycle involving IObjectReferences\n try\n Cloner.Create(v2.GetType()).Clone(v2) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n let () =\n let v1 = NativeSerializationTestClass>(1, KeyValuePair())\n let v2 = CustomSerializationTestClassWithProxy(2, v1)\n v1.Value <- KeyValuePair(null, v2)\n try\n Cloner.Create(v1.GetType()).Clone(v1) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n let () =\n let v1 = NativeSerializationTestClass(1, null)\n let v2 = CustomSerializationTestClassProxyWithOnDeserialized(2, v1)\n v1.Value <- v2\n try\n Cloner.Create(v1.GetType()).Clone(v1) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n let () =\n let v1 = NativeSerializationTestClass(1, null)\n let mutable v2 = CustomSerializationTestClassProxy2()\n v2.Id <- 2\n v2.Value <- v1\n v1.Value <- v2\n try\n Cloner.Create(v1.GetType()).Clone(v1) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n let instance = CustomSerializationTestClassWithInvalidProxy(1, \"2\")\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n let instance = CustomSerializationTestClassWithInvalidProxy2(1, \"2\")\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n let instance = CustomSerializationTestClassWithInvalidProxy3(1, \"2\")\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n let instance = ClassWithNonExistentSerializationProxyClass()\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n testCustomSerializationCloner()\n\n let testObjectReferenceHandling() =\n let () =\n let instance = ClassWithObjectReferenceImplementationThatReturnsThis()\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n let instance = ClassWithObjectReferenceImplementationThatReturnsThis2()\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n\n try\n let instance = ClassWithBuggyObjectReferenceImplementation1()\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n\n try\n let instance = ClassWithBuggyObjectReferenceImplementation2()\n Cloner.Create(instance.GetType()).Clone(instance) |> ignore\n Fail()\n with :? SerializationException -> ()\n testObjectReferenceHandling()\n\n\nlet testCloning() =\n let () =\n let o2 = NativeSerializationTestClassWithUnorderedEvents(2, null)\n let o1 = NativeSerializationTestClassWithUnorderedEvents(1, o2)\n let os = [|o2; o1|]\n let cloner = Cloner.Create(os.GetType())\n let os2 = unbox (cloner.Clone(os))\n os2 |> Equal os\n o1.OnSerializedWasCalled |> True\n o1.OnDeserializingWasCalled |> False\n o2.OnSerializedWasCalled |> True\n o2.OnDeserializingWasCalled |> False\n\n let () =\n let o2 = NativeSerializationTestClass(1, null)\n let o1 = NativeSerializationTestClass(2, o2)\n let os = [|o2; o1|]\n let cloner = Cloner.Create(os.GetType())\n onDeserializedList <- []\n let os2 = unbox (cloner.Clone(os))\n os2 |> Equal os\n onDeserializedList |> Equal [2; 1]\n o1.OnSerializedWasCalled |> True\n o1.OnDeserializingWasCalled |> False\n o2.OnSerializedWasCalled |> True\n o2.OnDeserializingWasCalled |> False\n\n let () =\n // cycle 1\n let o6_2 = NativeSerializationTestClass(6, null)\n let o5_2 = NativeSerializationTestClass(5, o6_2)\n let o4_2 = CustomSerializationTestClassWithProxy(4, o5_2)\n o6_2.Value <- o4_2\n\n // cycle 2\n let o3_1 = NativeSerializationTestClass2(3, null, o4_2)\n let o2_1 = NativeSerializationTestClass(2, o3_1)\n let o1_1 = CustomSerializationTestClassWithProxy(1, o2_1)\n o3_1.Value <- o1_1\n\n onDeserializedList <- []\n let cloner = Cloner.Create(o1_1.GetType())\n let o = cloner.Clone(o1_1) :?> CustomSerializationTestClassWithProxy\n o |> Equal o1_1\n onDeserializedList |> Equal [2;3;5;6] // the order within the strongly connected components\n // is implementation defined\n\n let () =\n let o8 = CustomSerializationTestClassWithProxy(8, Some(2))\n let o7 = CustomSerializationTestClassWithProxy(7, Some(1))\n\n // cycle 1\n let o6_2 = CustomSerializationTestClass(6, null)\n let o5_2 = CustomSerializationTestClass(5, (o6_2, box o7, box o8))\n let o4_2 = CustomSerializationTestClass(4, o5_2)\n o6_2.Value <- o4_2\n\n // cycle 2\n let o3_1 = NativeSerializationTestClass2(3, null, o4_2)\n let o2_1 = NativeSerializationTestClass(2, o3_1)\n let o1_1 = NativeSerializationTestClass(1, o2_1)\n o3_1.Value <- o1_1\n\n let o0 = CustomSerializationTestClass(0, o1_1)\n\n onDeserializedList <- []\n let os = [|o8; o7; o5_2; o1_1; o0|] : obj[]\n let cloner = Cloner.Create(os.GetType())\n let os2 = cloner.Clone(os) :?> obj[]\n os2[4] |> Equal os[4]\n onDeserializedList |> Equal [0;1;2;3;5;6;4] // the order within the strongly connected components\n // is implementation defined\n\n let reset() =\n onDeserializedList <- []\n o0.OnSerializingWasCalled <- false; o0.OnSerializedWasCalled <- false\n o1_1.OnSerializingWasCalled <- false; o1_1.OnSerializedWasCalled <- false\n o2_1.OnSerializingWasCalled <- false; o2_1.OnSerializedWasCalled <- false\n o3_1.OnSerializingWasCalled <- false; o3_1.OnSerializedWasCalled <- false\n o4_2.OnSerializingWasCalled <- false; o4_2.OnSerializedWasCalled <- false\n o5_2.OnSerializingWasCalled <- false; o5_2.OnSerializedWasCalled <- false\n o6_2.OnSerializingWasCalled <- false; o6_2.OnSerializedWasCalled <- false\n o7.OnSerializingWasCalled <- false; o7.OnSerializedWasCalled <- false\n o8.OnSerializingWasCalled <- false; o8.OnSerializedWasCalled <- false\n\n reset()\n let os3 = cloner.Clone(os) :?> obj[]\n os3[4] |> Equal os[4]\n onDeserializedList |> Equal [0;1;2;3;5;6;4]\n\n reset()\n let o = Cloner.Create(o1_1.GetType()).Clone(o1_1)\n onDeserializedList |> Equal [1;2;3;4;5;6]\n o |> Equal (box o1_1)\n\n try Cloner.Create(typeof) |> ignore\n with :? System.Runtime.Serialization.SerializationException -> ()\n try Cloner.Create(typeof).Clone(Some \"\") |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\nlet encodingTests() =\n for e in System.Text.Encoding.GetEncodings() do\n let encoding = e.GetEncoding()\n let bs = encoding.GetBytes(\"test test\")\n let decoder = encoding.GetDecoder()\n let cs = Array.zeroCreate 20\n new string(cs, 0, decoder.GetChars(bs, 0, bs.Length, cs, 0)) |> Equal \"test test\"\n let cloner = FParsec.Cloning.Cloner.Create(decoder.GetType())\n let image = cloner.CaptureImage(decoder)\n let decoder2 = image.CreateClone() :?> System.Text.Decoder\n new string(cs, 0, decoder2.GetChars(bs, 0, bs.Length, cs, 0)) |> Equal \"test test\"\n\n\nlet run() =\n testStronglyConnectedComponents()\n testComputeTopologicalOrder()\n testCloningEventHandlers()\n testGetSerializedFields()\n testCreateFieldValuesGetter()\n testCreateFieldValuesSetter()\n testCreateISerializableConstructorCaller()\n testCloners()\n testCloning()\n encodingTests()\n\n#endif" }, { "path": "Test/HexFloatTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.HexFloatTests\n\nopen FParsec.Test.Test\n\nlet floatToHexString = FParsec.CharParsers.floatToHexString\nlet floatOfHexString = FParsec.CharParsers.floatOfHexString\nlet float32ToHexString = FParsec.CharParsers.float32ToHexString\nlet float32OfHexString = FParsec.CharParsers.float32OfHexString\n\nlet testDoubleHexFloat() =\n /// bitwise equal\n let BEqual a b =\n Equal (System.BitConverter.DoubleToInt64Bits(a)) (System.BitConverter.DoubleToInt64Bits(b))\n\n // float32ToHexString\n ///////////////////\n\n let max = System.Double.MaxValue\n let eps = System.Math.Pow(2.0, -53.0)\n let min = System.Math.Pow(2.0, -1022.0) // smallest normal number\n let minmin = System.Math.Pow(2.0, -1074.0) // smallest subnormal number\n\n floatToHexString 0.0 |> Equal \"0x0.0p0\"\n floatToHexString -0.0 |> Equal \"-0x0.0p0\"\n floatToHexString 1.0 |> Equal \"0x1.0p0\"\n floatToHexString -1.0 |> Equal \"-0x1.0p0\"\n floatToHexString (1.0 + 4.*eps) |> Equal \"0x1.0000000000002p0\"\n floatToHexString (1.0 + 2.*eps) |> Equal \"0x1.0000000000001p0\"\n floatToHexString (1.0 - eps) |> Equal \"0x1.fffffffffffffp-1\"\n floatToHexString (1.0 - 2.*eps) |> Equal \"0x1.ffffffffffffep-1\"\n floatToHexString min |> Equal \"0x1.0p-1022\"\n floatToHexString (min + minmin) |> Equal \"0x1.0000000000001p-1022\"\n floatToHexString (min - minmin) |> Equal \"0x0.fffffffffffffp-1022\"\n floatToHexString (min - 2.*minmin) |> Equal \"0x0.ffffffffffffep-1022\"\n floatToHexString (minmin) |> Equal \"0x0.0000000000001p-1022\"\n floatToHexString max |> Equal \"0x1.fffffffffffffp1023\"\n\n floatToHexString System.Double.PositiveInfinity |> Equal \"Infinity\"\n floatToHexString System.Double.NegativeInfinity |> Equal \"-Infinity\"\n floatToHexString System.Double.NaN |> Equal \"NaN\"\n\n\n // floatOfHexString\n ///////////////////\n\n try floatOfHexString null |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n\n let checkFormatError s =\n try floatOfHexString s |> ignore; Fail ()\n with :? System.FormatException -> ()\n\n checkFormatError \"\"\n checkFormatError \".\"\n checkFormatError \"p1\"\n checkFormatError \".p1\"\n checkFormatError \"1x1\"\n checkFormatError \"x1\"\n checkFormatError \"0xx1\"\n checkFormatError \"0x/\"\n checkFormatError \"0x:\"\n checkFormatError \"0x@\"\n checkFormatError \"0xG\"\n checkFormatError \"0x`\"\n checkFormatError \"0xg\"\n checkFormatError \"0.1pp1\"\n checkFormatError \"0.1p+\"\n checkFormatError \"0.1p-\"\n checkFormatError \"0.fg\"\n checkFormatError \"1.0 \"\n checkFormatError \"1..\"\n checkFormatError \"1.0.\"\n\n floatOfHexString \"Inf\" |> Equal System.Double.PositiveInfinity\n\n floatOfHexString \"iNf\" |> Equal System.Double.PositiveInfinity\n floatOfHexString \"Infinity\" |> Equal System.Double.PositiveInfinity\n floatOfHexString \"+InFinITy\" |> Equal System.Double.PositiveInfinity\n floatOfHexString \"-Inf\" |> Equal (-System.Double.PositiveInfinity)\n floatOfHexString \"-InFinITy\" |> Equal (-System.Double.PositiveInfinity)\n floatOfHexString \"NaN\" |> BEqual System.Double.NaN\n floatOfHexString \"-nAn\" |> BEqual System.Double.NaN\n floatOfHexString \"+Nan\" |> BEqual System.Double.NaN\n\n floatOfHexString \"001\" |> Equal 1.0\n floatOfHexString \"1.\" |> Equal 1.0\n floatOfHexString \"1.0\" |> Equal 1.0\n floatOfHexString \"0x1\" |> Equal 1.0\n floatOfHexString \"0X1\" |> Equal 1.0\n floatOfHexString \"0x0001\" |> Equal 1.0\n floatOfHexString \"0x1.\" |> Equal 1.0\n floatOfHexString \"0x1.0\" |> Equal 1.0\n floatOfHexString \"0x001.0\" |> Equal 1.0\n floatOfHexString \"1.0p0\" |> Equal 1.0\n floatOfHexString \"1.0P0\" |> Equal 1.0\n floatOfHexString \"001.00p+000\" |> Equal 1.0\n floatOfHexString \".100p+004\" |> Equal 1.0\n floatOfHexString \".0100p+008\" |> Equal 1.0\n floatOfHexString \"00.100p+004\" |> Equal 1.0\n floatOfHexString \"00.0100p+008\" |> Equal 1.0\n floatOfHexString \"0010.0p-004\" |> Equal 1.0\n floatOfHexString \"0x1.0p0\" |> Equal 1.0\n floatOfHexString \"0X1.0P0\" |> Equal 1.0\n floatOfHexString \"0x001.00p+000\" |> Equal 1.0\n floatOfHexString \"0x00.100p+004\" |> Equal 1.0\n floatOfHexString \"0x.100p+004\" |> Equal 1.0\n floatOfHexString \"0x0010.0p-004\" |> Equal 1.0\n\n floatOfHexString \"-001\" |> Equal -1.0\n floatOfHexString \"-1.\" |> Equal -1.0\n floatOfHexString \"-1.0\" |> Equal -1.0\n floatOfHexString \"-0x1\" |> Equal -1.0\n floatOfHexString \"-0X1\" |> Equal -1.0\n floatOfHexString \"-0x0001\" |> Equal -1.0\n floatOfHexString \"-0x1.\" |> Equal -1.0\n floatOfHexString \"-0x1.0\" |> Equal -1.0\n floatOfHexString \"-0x001.0\" |> Equal -1.0\n floatOfHexString \"-1.0p0\" |> Equal -1.0\n floatOfHexString \"-1.0P0\" |> Equal -1.0\n floatOfHexString \"-001.00p+000\" |> Equal -1.0\n floatOfHexString \"-.100p+004\" |> Equal -1.0\n floatOfHexString \"-.0100p+008\" |> Equal -1.0\n floatOfHexString \"-00.100p+004\" |> Equal -1.0\n floatOfHexString \"-00.0100p+008\" |> Equal -1.0\n floatOfHexString \"-0010.0p-004\" |> Equal -1.0\n floatOfHexString \"-0x1.0p0\" |> Equal -1.0\n floatOfHexString \"-0X1.0P0\" |> Equal -1.0\n floatOfHexString \"-0x001.00p+000\" |> Equal -1.0\n floatOfHexString \"-0x00.100p+004\" |> Equal -1.0\n floatOfHexString \"-0x.100p+004\" |> Equal -1.0\n floatOfHexString \"-0x0010.0p-004\" |> Equal -1.0\n\n floatOfHexString \"+001\" |> Equal 1.0\n floatOfHexString \"+1.\" |> Equal 1.0\n floatOfHexString \"+1.0\" |> Equal 1.0\n floatOfHexString \"+.100p+004\" |> Equal 1.0\n floatOfHexString \"+0x0010.0p-004\" |> Equal 1.0\n\n floatOfHexString \"0\" |> BEqual 0.\n floatOfHexString \"0.\" |> BEqual 0.\n floatOfHexString \"0.0\" |> BEqual 0.\n floatOfHexString \"00.0\" |> BEqual 0.\n floatOfHexString \"00.000\" |> BEqual 0.\n floatOfHexString \"00.000p0\" |> BEqual 0.\n floatOfHexString \"00.000p99999999\" |> BEqual 0.\n floatOfHexString \"0x0\" |> BEqual 0.\n floatOfHexString \"0x0.\" |> BEqual 0.\n floatOfHexString \"0x0.0\" |> BEqual 0.\n floatOfHexString \"0x00.0\" |> BEqual 0.\n floatOfHexString \"0x00.000\" |> BEqual 0.\n floatOfHexString \"0x00.000p0\" |> BEqual 0.\n floatOfHexString \"0x00.000p99999999\" |> BEqual 0.\n floatOfHexString \"100P-2147483639\" |> BEqual 0.\n floatOfHexString \"100P-2147483640\" |> BEqual 0.\n floatOfHexString \"100P-2147483647\" |> BEqual 0.\n floatOfHexString \"100P-9999999999999999999999999\" |> BEqual 0.\n floatOfHexString \"0.001P-2147483639\" |> BEqual 0.\n floatOfHexString \"0.001P-2147483640\" |> BEqual 0.\n floatOfHexString \"0.001P-2147483647\" |> BEqual 0.\n floatOfHexString \"0.001P-9999999999999999999999999\" |> BEqual 0.\n\n floatOfHexString \"-0\" |> BEqual -0.0\n floatOfHexString \"-0.\" |> BEqual -0.0\n floatOfHexString \"-0.0\" |> BEqual -0.0\n floatOfHexString \"-00.0\" |> BEqual -0.0\n floatOfHexString \"-00.000\" |> BEqual -0.0\n floatOfHexString \"-00.000p0\" |> BEqual -0.\n floatOfHexString \"-00.000p99999999\" |> BEqual -0.\n floatOfHexString \"-0x0\" |> BEqual -0.0\n floatOfHexString \"-0x0.\" |> BEqual -0.0\n floatOfHexString \"-0x0.0\" |> BEqual -0.0\n floatOfHexString \"-0x00.0\" |> BEqual -0.0\n floatOfHexString \"-0x00.000\" |> BEqual -0.0\n floatOfHexString \"-0x00.000p0\" |> BEqual -0.0\n floatOfHexString \"-0x00.000p0\" |> BEqual -0.\n floatOfHexString \"-0x00.000p99999999\" |> BEqual -0.\n floatOfHexString \"-100P-2147483639\" |> BEqual -0.\n floatOfHexString \"-100P-2147483640\" |> BEqual -0.\n floatOfHexString \"-100P-2147483647\" |> BEqual -0.\n floatOfHexString \"-100P-9999999999999999999999999\" |> BEqual -0.\n floatOfHexString \"-0.001P-2147483639\" |> BEqual -0.\n floatOfHexString \"-0.001P-2147483640\" |> BEqual -0.\n floatOfHexString \"-0.001P-2147483647\" |> BEqual -0.\n floatOfHexString \"-0.001P-9999999999999999999999999\" |> BEqual -0.\n\n floatOfHexString \"0x0123\" |> Equal (double 0x0123)\n floatOfHexString \"0x4567\" |> Equal (double 0x4567)\n floatOfHexString \"0x89ab\" |> Equal (double 0x89ab)\n floatOfHexString \"0x89AB\" |> Equal (double 0x89ab)\n floatOfHexString \"0xcdef\" |> Equal (double 0xcdef)\n floatOfHexString \"0xCDEF\" |> Equal (double 0xcdef)\n\n let v = floatOfHexString \"0x1.23456789abcde\"\n\n floatOfHexString \"0x123.456789abcde00p-8\" |> Equal v\n floatOfHexString \"0x91.a2b3c4d5e6f00p-7\" |> Equal v\n floatOfHexString \"0x48.d159e26af3780p-6\" |> Equal v\n floatOfHexString \"0x24.68acf13579bc0p-5\" |> Equal v\n floatOfHexString \"0x12.3456789abcde0p-4\" |> Equal v\n floatOfHexString \"0x9.1a2b3c4d5e6fp-3\" |> Equal v\n floatOfHexString \"0x4.8d159e26af378p-2\" |> Equal v\n floatOfHexString \"0x2.468acf13579bcp-1\" |> Equal v\n floatOfHexString \"0x.91a2b3c4d5e6f0p+1\" |> Equal v\n floatOfHexString \"0x.48d159e26af378p+2\" |> Equal v\n floatOfHexString \"0x.2468acf13579bcp+3\" |> Equal v\n floatOfHexString \"0x.123456789abcdep+4\" |> Equal v\n floatOfHexString \"0x.091a2b3c4d5e6f0p+5\" |> Equal v\n floatOfHexString \"0x.048d159e26af378p+6\" |> Equal v\n floatOfHexString \"0x.02468acf13579bcp+7\" |> Equal v\n floatOfHexString \"0x.0123456789abcdep+8\" |> Equal v\n\n // near max\n floatOfHexString \"0x1.fffffffffffffp1023\" |> Equal max\n floatOfHexString \"0x.1fffffffffffffp1027\" |> Equal max\n floatOfHexString \"0x.01fffffffffffffp1031\" |> Equal max\n floatOfHexString \"0x1f.ffffffffffffp1019\" |> Equal max\n floatOfHexString \"0x1fffffffffffff.p971\" |> Equal max\n floatOfHexString \"-0x1fffffffffffff000.p959\" |> Equal (-max)\n\n floatOfHexString \"0x1.fffffffffffff5p1023\" |> Equal max\n floatOfHexString \"0x1.fffffffffffff6p1023\" |> Equal max\n floatOfHexString \"0x1.fffffffffffff7p1023\" |> Equal max\n floatOfHexString \"0x1.fffffffffffff7ffffffffp1023\" |> Equal max\n\n let checkOverflow s =\n try floatOfHexString s |> ignore; Fail ()\n with :? System.OverflowException -> ()\n\n checkOverflow \"0x1.fffffffffffff8p1023\"\n checkOverflow \"0x1.fffffffffffff800000p1023\"\n checkOverflow \"0x1.ffffffffffffffp1023\"\n checkOverflow \"0x1p1024\"\n checkOverflow \"100P2147483639\"\n checkOverflow \"100P2147483640\"\n checkOverflow \"100P2147483647\"\n checkOverflow \"100P9999999999999999999999999\"\n checkOverflow \"0.001P2147483639\"\n checkOverflow \"0.001P2147483640\"\n checkOverflow \"0.001P2147483647\"\n checkOverflow \"0.001P9999999999999999999999999\"\n\n // near 1\n floatOfHexString \"0x1.0000000000000f\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x1.0000000000000e\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x1.0000000000000d\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.0000000000000c\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.00000000000018p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.0000000000003p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.0000000000006p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.0000000000000b\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.0000000000000a\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.00000000000014p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.00000000000028p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.0000000000005p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.00000000000009\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.00000000000012p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.00000000000024p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.00000000000048p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.000000000000088\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.00000000000011p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.00000000000022p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.00000000000044p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.000000000000084\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.000000000000108p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.00000000000021p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.00000000000042p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.000000000000082\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x2.000000000000104p-1\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x4.000000000000208p-2\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x8.00000000000041p-3\" |> Equal (1.0 + 2.*eps)\n\n floatOfHexString \"0x1.00000000000008\" |> Equal (1.0) // round towards even\n floatOfHexString \"0x2.0000000000001p-1\" |> Equal (1.0)\n floatOfHexString \"0x4.0000000000002p-2\" |> Equal (1.0)\n floatOfHexString \"0x8.0000000000004p-3\" |> Equal (1.0)\n\n floatOfHexString \"0x1.0000000000000800000010000\" |> Equal (1.0 + 2.*eps)\n floatOfHexString \"0x1.00000000000007ffffffffff\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000007\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000006\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000005\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000004\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000003\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000002\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000001\" |> Equal (1.0)\n floatOfHexString \"0x1.00000000000000\" |> Equal (1.0)\n floatOfHexString \"0x1.ffffffffffffffffP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffffeP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffffdP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffffcP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffffbP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffffaP-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffff9P-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffff8P-1\" |> Equal (1.0) // round towards even\n floatOfHexString \"0x1.fffffffffffff800P-1\" |> Equal (1.0)\n floatOfHexString \"0x1.fffffffffffff7ffffffP-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff7000001P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff7P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff6P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff5P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff4P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff3P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff2P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff1P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.fffffffffffff0P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.ffffffffffffefP-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.ffffffffffffe8001P-1\" |> Equal (1.0 - eps)\n floatOfHexString \"0x1.ffffffffffffe8P-1\" |> Equal (1.0 - 2.*eps) // round towards even\n floatOfHexString \"0x1.ffffffffffffe80P-1\" |> Equal (1.0 - 2.*eps)\n floatOfHexString \"0x1.ffffffffffffe7ffffP-1\" |> Equal (1.0 - 2.*eps)\n floatOfHexString \"0x1.ffffffffffffe70P-1\" |> Equal (1.0 - 2.*eps)\n floatOfHexString \"0x1.ffffffffffffe1P-1\" |> Equal (1.0 - 2.*eps)\n\n floatOfHexString \"0x1.0000000000000fP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.0000000000000eP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.0000000000000dP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.0000000000000cP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.0000000000000bP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.0000000000000aP-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.00000000000009P-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.00000000000008001P-1022\" |> Equal (min + minmin)\n floatOfHexString \"0x1.00000000000008P-1022\" |> Equal (min) // round towards even\n floatOfHexString \"0x1.000000000000080P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000007ffffP-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000007P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000006P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000005P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000004P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000003P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000002P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000001P-1022\" |> Equal (min)\n floatOfHexString \"0x1.00000000000000P-1022\" |> Equal (min)\n floatOfHexString \"0x0.ffffffffffffffP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffffeP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffffdP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffffcP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffffbP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffffaP-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffff9P-1022\" |> Equal (min)\n floatOfHexString \"0x0.fffffffffffff8P-1022\" |> Equal (min) // round towards even\n floatOfHexString \"0x0.fffffffffffff7fffP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff7P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff6P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff5P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff4P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff3P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff2P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff1P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.fffffffffffff0P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffefP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffeeP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffedP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffecP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffebP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffeaP-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffe9P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffe8001P-1022\" |> Equal (min - minmin)\n floatOfHexString \"0x0.ffffffffffffe8P-1022\" |> Equal (min - 2.*minmin) // round towards even\n floatOfHexString \"0x0.ffffffffffffe80P-1022\" |> Equal (min - 2.*minmin) // round towards even\n floatOfHexString \"0x0.ffffffffffffe7ffP-1022\" |> Equal (min - 2.*minmin)\n\n floatOfHexString \"0x0.00000000000019P-1022\" |> Equal (2.*minmin)\n floatOfHexString \"0x0.00000000000018P-1022\" |> Equal (2.*minmin) // round towards even\n floatOfHexString \"0x0.00000000000017ffP-1022\" |> Equal (minmin)\n floatOfHexString \"0x0.0000000000001P-1022\" |> Equal (minmin)\n floatOfHexString \"0x0.00000000000010P-1022\" |> Equal (minmin)\n floatOfHexString \"0x0.00000000000008001P-1022\" |> Equal (minmin)\n floatOfHexString \"0x0.00000000000008P-1022\" |> Equal (0.) // round towards even\n floatOfHexString \"0x0.00000000000007ffffP-1022\" |> Equal (0.)\n floatOfHexString \"0x1.P-1075\" |> Equal (0.)\n\n // round trip checking\n //////////////////////\n let rand = System.Random(123)\n let buffer = Array.zeroCreate 8\n let randomFloat() =\n rand.NextBytes(buffer)\n System.BitConverter.ToDouble(buffer, 0)\n\n for i = 0 to 100000 do\n let f = randomFloat()\n let s = floatToHexString f\n let f2 = floatOfHexString s\n True (f = f2 || f <> f)\n\n\nlet testSingleHexFloat() =\n\n /// bitwise equal\n let BEqual (a: float32) (b: float32) =\n Equal (System.BitConverter.GetBytes(a)) (System.BitConverter.GetBytes(b))\n\n\n // float32ToHexString\n ///////////////////\n\n let max = System.Single.MaxValue\n let eps = (float32) (System.Math.Pow(2.0, -24.0))\n let min = (float32) (System.Math.Pow(2.0, -126.0)) // smallest normal number\n let minmin = (float32) (System.Math.Pow(2.0, -149.0)) // smallest subnormal number\n\n float32ToHexString 0.0f |> Equal \"0x0.0p0\"\n float32ToHexString -0.0f |> Equal \"-0x0.0p0\"\n float32ToHexString 1.0f |> Equal \"0x1.0p0\"\n float32ToHexString -1.0f |> Equal \"-0x1.0p0\"\n float32ToHexString (1.0f + 4.f*eps) |> Equal \"0x1.000004p0\"\n float32ToHexString (1.0f + 2.f*eps) |> Equal \"0x1.000002p0\"\n float32ToHexString (1.0f - eps) |> Equal \"0x1.fffffep-1\"\n float32ToHexString (1.0f - 2.f*eps) |> Equal \"0x1.fffffcp-1\"\n float32ToHexString min |> Equal \"0x1.0p-126\"\n float32ToHexString (min + minmin) |> Equal \"0x1.000002p-126\"\n float32ToHexString (min - minmin) |> Equal \"0x0.fffffep-126\"\n float32ToHexString (min - 2.f*minmin) |> Equal \"0x0.fffffcp-126\"\n float32ToHexString (minmin) |> Equal \"0x0.000002p-126\"\n float32ToHexString max |> Equal \"0x1.fffffep127\"\n\n float32ToHexString System.Single.PositiveInfinity |> Equal \"Infinity\"\n float32ToHexString System.Single.NegativeInfinity |> Equal \"-Infinity\"\n float32ToHexString System.Single.NaN |> Equal \"NaN\"\n\n\n // float32OfHexString\n ///////////////////\n\n try float32OfHexString null |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n\n let checkFormatError s =\n try float32OfHexString s |> ignore; Fail ()\n with :? System.FormatException -> ()\n\n checkFormatError \"\"\n checkFormatError \".\"\n checkFormatError \"p1\"\n checkFormatError \".p1\"\n checkFormatError \"1x1\"\n checkFormatError \"x1\"\n checkFormatError \"0xx1\"\n checkFormatError \"0x/\"\n checkFormatError \"0x:\"\n checkFormatError \"0x@\"\n checkFormatError \"0xG\"\n checkFormatError \"0x`\"\n checkFormatError \"0xg\"\n checkFormatError \"0.1pp1\"\n checkFormatError \"0.1p+\"\n checkFormatError \"0.1p-\"\n checkFormatError \"0.fg\"\n checkFormatError \"1.0 \"\n checkFormatError \"1..\"\n checkFormatError \"1.0.\"\n\n float32OfHexString \"Inf\" |> Equal System.Single.PositiveInfinity\n\n float32OfHexString \"iNf\" |> Equal System.Single.PositiveInfinity\n float32OfHexString \"Infinity\" |> Equal System.Single.PositiveInfinity\n float32OfHexString \"+InFinITy\" |> Equal System.Single.PositiveInfinity\n float32OfHexString \"-Inf\" |> Equal (-System.Single.PositiveInfinity)\n float32OfHexString \"-InFinITy\" |> Equal (-System.Single.PositiveInfinity)\n float32OfHexString \"NaN\" |> BEqual System.Single.NaN\n float32OfHexString \"-nAn\" |> BEqual System.Single.NaN\n float32OfHexString \"+Nan\" |> BEqual System.Single.NaN\n\n float32OfHexString \"001\" |> Equal 1.0f\n float32OfHexString \"1.\" |> Equal 1.0f\n float32OfHexString \"1.0\" |> Equal 1.0f\n float32OfHexString \"0x1\" |> Equal 1.0f\n float32OfHexString \"0X1\" |> Equal 1.0f\n float32OfHexString \"0x0001\" |> Equal 1.0f\n float32OfHexString \"0x1.\" |> Equal 1.0f\n float32OfHexString \"0x1.0\" |> Equal 1.0f\n float32OfHexString \"0x001.0\" |> Equal 1.0f\n float32OfHexString \"1.0p0\" |> Equal 1.0f\n float32OfHexString \"1.0P0\" |> Equal 1.0f\n float32OfHexString \"001.00p+000\" |> Equal 1.0f\n float32OfHexString \".100p+004\" |> Equal 1.0f\n float32OfHexString \".0100p+008\" |> Equal 1.0f\n float32OfHexString \"00.100p+004\" |> Equal 1.0f\n float32OfHexString \"00.0100p+008\" |> Equal 1.0f\n float32OfHexString \"0010.0p-004\" |> Equal 1.0f\n float32OfHexString \"0x1.0p0\" |> Equal 1.0f\n float32OfHexString \"0X1.0P0\" |> Equal 1.0f\n float32OfHexString \"0x001.00p+000\" |> Equal 1.0f\n float32OfHexString \"0x00.100p+004\" |> Equal 1.0f\n float32OfHexString \"0x.100p+004\" |> Equal 1.0f\n float32OfHexString \"0x0010.0p-004\" |> Equal 1.0f\n\n float32OfHexString \"-001\" |> Equal -1.0f\n float32OfHexString \"-1.\" |> Equal -1.0f\n float32OfHexString \"-1.0\" |> Equal -1.0f\n float32OfHexString \"-0x1\" |> Equal -1.0f\n float32OfHexString \"-0X1\" |> Equal -1.0f\n float32OfHexString \"-0x0001\" |> Equal -1.0f\n float32OfHexString \"-0x1.\" |> Equal -1.0f\n float32OfHexString \"-0x1.0\" |> Equal -1.0f\n float32OfHexString \"-0x001.0\" |> Equal -1.0f\n float32OfHexString \"-1.0p0\" |> Equal -1.0f\n float32OfHexString \"-1.0P0\" |> Equal -1.0f\n float32OfHexString \"-001.00p+000\" |> Equal -1.0f\n float32OfHexString \"-.100p+004\" |> Equal -1.0f\n float32OfHexString \"-.0100p+008\" |> Equal -1.0f\n float32OfHexString \"-00.100p+004\" |> Equal -1.0f\n float32OfHexString \"-00.0100p+008\" |> Equal -1.0f\n float32OfHexString \"-0010.0p-004\" |> Equal -1.0f\n float32OfHexString \"-0x1.0p0\" |> Equal -1.0f\n float32OfHexString \"-0X1.0P0\" |> Equal -1.0f\n float32OfHexString \"-0x001.00p+000\" |> Equal -1.0f\n float32OfHexString \"-0x00.100p+004\" |> Equal -1.0f\n float32OfHexString \"-0x.100p+004\" |> Equal -1.0f\n float32OfHexString \"-0x0010.0p-004\" |> Equal -1.0f\n\n float32OfHexString \"+001\" |> Equal 1.0f\n float32OfHexString \"+1.\" |> Equal 1.0f\n float32OfHexString \"+1.0\" |> Equal 1.0f\n float32OfHexString \"+.100p+004\" |> Equal 1.0f\n float32OfHexString \"+0x0010.0p-004\" |> Equal 1.0f\n\n float32OfHexString \"0\" |> BEqual 0.f\n float32OfHexString \"0.\" |> BEqual 0.f\n float32OfHexString \"0.0\" |> BEqual 0.f\n float32OfHexString \"00.0\" |> BEqual 0.f\n float32OfHexString \"00.000\" |> BEqual 0.f\n float32OfHexString \"00.000p0\" |> BEqual 0.f\n float32OfHexString \"00.000p99999999\" |> BEqual 0.f\n float32OfHexString \"0x0\" |> BEqual 0.f\n float32OfHexString \"0x0.\" |> BEqual 0.f\n float32OfHexString \"0x0.0\" |> BEqual 0.f\n float32OfHexString \"0x00.0\" |> BEqual 0.f\n float32OfHexString \"0x00.000\" |> BEqual 0.f\n float32OfHexString \"0x00.000p0\" |> BEqual 0.f\n float32OfHexString \"0x00.000p99999999\" |> BEqual 0.f\n float32OfHexString \"100P-2147483639\" |> BEqual 0.f\n float32OfHexString \"100P-2147483640\" |> BEqual 0.f\n float32OfHexString \"100P-2147483647\" |> BEqual 0.f\n float32OfHexString \"100P-9999999999999999999999999\" |> BEqual 0.f\n float32OfHexString \"0.001P-2147483639\" |> BEqual 0.f\n float32OfHexString \"0.001P-2147483640\" |> BEqual 0.f\n float32OfHexString \"0.001P-2147483647\" |> BEqual 0.f\n float32OfHexString \"0.001P-9999999999999999999999999\" |> BEqual 0.f\n\n float32OfHexString \"-0\" |> BEqual -0.0f\n float32OfHexString \"-0.\" |> BEqual -0.0f\n float32OfHexString \"-0.0\" |> BEqual -0.0f\n float32OfHexString \"-00.0\" |> BEqual -0.0f\n float32OfHexString \"-00.000\" |> BEqual -0.0f\n float32OfHexString \"-00.000p0\" |> BEqual -0.f\n float32OfHexString \"-00.000p99999999\" |> BEqual -0.f\n float32OfHexString \"-0x0\" |> BEqual -0.0f\n float32OfHexString \"-0x0.\" |> BEqual -0.0f\n float32OfHexString \"-0x0.0\" |> BEqual -0.0f\n float32OfHexString \"-0x00.0\" |> BEqual -0.0f\n float32OfHexString \"-0x00.000\" |> BEqual -0.0f\n float32OfHexString \"-0x00.000p0\" |> BEqual -0.0f\n float32OfHexString \"-0x00.000p0\" |> BEqual -0.f\n float32OfHexString \"-0x00.000p99999999\" |> BEqual -0.f\n float32OfHexString \"-100P-2147483639\" |> BEqual -0.f\n float32OfHexString \"-100P-2147483640\" |> BEqual -0.f\n float32OfHexString \"-100P-2147483647\" |> BEqual -0.f\n float32OfHexString \"-100P-9999999999999999999999999\" |> BEqual -0.f\n float32OfHexString \"-0.001P-2147483639\" |> BEqual -0.f\n float32OfHexString \"-0.001P-2147483640\" |> BEqual -0.f\n float32OfHexString \"-0.001P-2147483647\" |> BEqual -0.f\n float32OfHexString \"-0.001P-9999999999999999999999999\" |> BEqual -0.f\n\n float32OfHexString \"0x0123\" |> Equal (single 0x0123)\n float32OfHexString \"0x4567\" |> Equal (single 0x4567)\n float32OfHexString \"0x89ab\" |> Equal (single 0x89ab)\n float32OfHexString \"0x89AB\" |> Equal (single 0x89ab)\n float32OfHexString \"0xcdef\" |> Equal (single 0xcdef)\n float32OfHexString \"0xCDEF\" |> Equal (single 0xcdef)\n\n let v = float32OfHexString \"0x1.23456e\"\n\n float32OfHexString \"0x123.456e00p-8\" |> Equal v\n float32OfHexString \"0x91.a2b700p-7\" |> Equal v\n float32OfHexString \"0x48.d15b80p-6\" |> Equal v\n float32OfHexString \"0x24.68adc0p-5\" |> Equal v\n float32OfHexString \"0x12.3456e0p-4\" |> Equal v\n float32OfHexString \"0x9.1a2b70p-3\" |> Equal v\n float32OfHexString \"0x4.8d15b8p-2\" |> Equal v\n float32OfHexString \"0x2.468adcp-1\" |> Equal v\n float32OfHexString \"0x.91a2b70p+1\" |> Equal v\n float32OfHexString \"0x.48d15b8p+2\" |> Equal v\n float32OfHexString \"0x.2468adcp+3\" |> Equal v\n float32OfHexString \"0x.123456ep+4\" |> Equal v\n float32OfHexString \"0x.091a2b70p+5\" |> Equal v\n float32OfHexString \"0x.048d15b8p+6\" |> Equal v\n float32OfHexString \"0x.02468adcp+7\" |> Equal v\n float32OfHexString \"0x.0123456ep+8\" |> Equal v\n\n\n // near max\n float32OfHexString \"0x1.fffffep127\" |> Equal max\n float32OfHexString \"0x.1fffffep131\" |> Equal max\n float32OfHexString \"0x.01fffffep135\" |> Equal max\n float32OfHexString \"0x1f.ffffep123\" |> Equal max\n float32OfHexString \"0x1fffffe.p103\" |> Equal max\n float32OfHexString \"-0x1fffffe000.p91\" |> Equal (-max)\n\n float32OfHexString \"0x0.ffffff5p128\" |> Equal max\n float32OfHexString \"0x0.ffffff6p128\" |> Equal max\n float32OfHexString \"0x0.ffffff7p128\" |> Equal max\n float32OfHexString \"0x0.ffffff7ffffffffp128\" |> Equal max\n\n let checkOverflow s =\n try float32OfHexString s |> ignore; Fail ()\n with :? System.OverflowException -> ()\n\n checkOverflow \"0x0.ffffff8p128\"\n checkOverflow \"0x0.ffffff800000p128\"\n checkOverflow \"0x0.fffffffp128\"\n checkOverflow \"0x1p128\"\n checkOverflow \"100P2147483639\"\n checkOverflow \"100P2147483640\"\n checkOverflow \"100P2147483647\"\n checkOverflow \"100P9999999999999999999999999\"\n checkOverflow \"0.001P2147483639\"\n checkOverflow \"0.001P2147483640\"\n checkOverflow \"0.001P2147483647\"\n checkOverflow \"0.001P9999999999999999999999999\"\n\n // near 1\n float32OfHexString \"0x1.000001e\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x1.000001c\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x1.000001a\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.0000018\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.0000024p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.0000048p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.0000090p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.0000016\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.0000014\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.0000028p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.0000050p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.00000a0p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.0000012\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.0000024p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.0000048p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.0000090p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.00000110\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.00000220p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.00000440p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.00000880p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.00000108\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.00000210p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.00000420p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.00000840p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.00000104\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x2.00000208p-1\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x4.0000041p-2\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x8.0000082p-3\" |> Equal (1.f + 2.f*eps)\n\n float32OfHexString \"0x1.000001\" |> Equal (1.f) // round towards even\n float32OfHexString \"0x2.000002p-1\" |> Equal (1.f)\n float32OfHexString \"0x4.000004p-2\" |> Equal (1.f)\n float32OfHexString \"0x8.00000p-3\" |> Equal (1.f)\n\n float32OfHexString \"0x1.00000100000010000\" |> Equal (1.f + 2.f*eps)\n float32OfHexString \"0x1.000000ffffffffff\" |> Equal (1.f)\n float32OfHexString \"0x1.000000e\" |> Equal (1.f)\n float32OfHexString \"0x1.000000c\" |> Equal (1.f)\n float32OfHexString \"0x1.000000a\" |> Equal (1.f)\n float32OfHexString \"0x1.0000008\" |> Equal (1.f)\n float32OfHexString \"0x1.0000006\" |> Equal (1.f)\n float32OfHexString \"0x1.0000004\" |> Equal (1.f)\n float32OfHexString \"0x1.0000002\" |> Equal (1.f)\n float32OfHexString \"0x1.0000000\" |> Equal (1.f)\n float32OfHexString \"0x1.fffffffffP-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffffeP-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffffcP-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffffaP-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffff8P-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffff6P-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffff4P-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffff2P-1\" |> Equal (1.f)\n float32OfHexString \"0x1.ffffffP-1\" |> Equal (1.f) // round towards even\n float32OfHexString \"0x1.ffffff0000P-1\" |> Equal (1.f)\n float32OfHexString \"0x1.fffffefffffP-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffee0001P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffecP-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffeaP-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffe8P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffe6P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffe4P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffe2P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffe0P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffd001P-1\" |> Equal (1.f - eps)\n float32OfHexString \"0x1.fffffdP-1\" |> Equal (1.f - 2.f*eps) // round towards zero\n float32OfHexString \"0x1.fffffd0P-1\" |> Equal (1.f - 2.f*eps)\n float32OfHexString \"0x1.fffffcfffP-1\" |> Equal (1.f - 2.f*eps)\n float32OfHexString \"0x1.fffffce0P-1\" |> Equal (1.f - 2.f*eps)\n float32OfHexString \"0x1.fffffc20P-1\" |> Equal (1.f - 2.f*eps)\n\n float32OfHexString \"0x0.800000fP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.800000eP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.800000dP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.800000cP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.800000bP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.800000aP-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.8000009P-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.8000008001P-125\" |> Equal (min + minmin)\n float32OfHexString \"0x0.8000008P-125\" |> Equal (min) // round towards even\n float32OfHexString \"0x0.80000080P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000007ffffP-125\" |> Equal (min)\n float32OfHexString \"0x0.8000007P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000006P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000005P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000004P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000003P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000002P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000001P-125\" |> Equal (min)\n float32OfHexString \"0x0.8000000P-125\" |> Equal (min)\n float32OfHexString \"0x0.7ffffffP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffffeP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffffdP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffffcP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffffbP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffffaP-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffff9P-125\" |> Equal (min)\n float32OfHexString \"0x0.7fffff8P-125\" |> Equal (min) // round towards even\n float32OfHexString \"0x0.7fffff7fffP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff7P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff6P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff5P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff4P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff3P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff2P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff1P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7fffff0P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffefP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffeeP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffedP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffecP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffebP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffeaP-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffe9P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffe8001P-125\" |> Equal (min - minmin)\n float32OfHexString \"0x0.7ffffe8P-125\" |> Equal (min - 2.f*minmin) // round towards even\n float32OfHexString \"0x0.7ffffe80P-125\" |> Equal (min - 2.f*minmin) // round towards even\n float32OfHexString \"0x0.7ffffe7ffP-125\" |> Equal (min - 2.f*minmin)\n\n float32OfHexString \"0x0.0000019P-125\" |> Equal (2.f*minmin)\n float32OfHexString \"0x0.0000018P-125\" |> Equal (2.f*minmin) // round towards even\n float32OfHexString \"0x0.0000017ffP-125\" |> Equal (minmin)\n float32OfHexString \"0x0.000001P-125\" |> Equal (minmin)\n float32OfHexString \"0x0.0000010P-125\" |> Equal (minmin)\n float32OfHexString \"0x0.0000008001P-125\" |> Equal (minmin)\n float32OfHexString \"0x0.0000008P-125\" |> Equal (0.f) // round towards even\n float32OfHexString \"0x0.0000007ffffP-125\" |> Equal (0.f)\n float32OfHexString \"0x1P-150\" |> Equal (0.f)\n\n // round trip checking\n //////////////////////\n let rand = System.Random(123)\n let buffer = Array.zeroCreate 4\n let randomFloat32() =\n rand.NextBytes(buffer)\n System.BitConverter.ToSingle(buffer, 0)\n\n for i = 0 to 100000 do\n let f = randomFloat32()\n let s = float32ToHexString f\n let f2 = float32OfHexString s\n True (f = f2 || f <> f)\n\nlet run() =\n testDoubleHexFloat()\n testSingleHexFloat()" }, { "path": "Test/IdentifierValidatorTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2012\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.IdentifierValidatorTests\n\n#if NETCORE\nopen System\n#endif\n\nopen FParsec.Test.Test\n\n// the following string contains parts of http://www.unicode.org/Public/8.0.0/ucd/DerivedCoreProperties.txt\nlet xidProperties = @\"\n# DerivedCoreProperties-8.0.0.txt\n# Date: 2015-03-11, 22:29:21 GMT [MD]\n#\n# Unicode Character Database\n# Copyright (c) 1991-2015 Unicode, Inc.\n# For terms of use, see http://www.unicode.org/terms_of_use.html\n# For documentation, see http://www.unicode.org/reports/tr44/\n\n# ================================================\n\n# Derived Property: XID_Start\n# ID_Start modified for closure under NFKx\n# Modified as described in UAX #15\n# NOTE: Does NOT remove the non-NFKx characters.\n# Merely ensures that if isIdentifer(string) then isIdentifier(NFKx(string))\n# NOTE: See UAX #31 for more information\n\n0041..005A ; XID_Start # L& [26] LATIN CAPITAL LETTER A..LATIN CAPITAL LETTER Z\n0061..007A ; XID_Start # L& [26] LATIN SMALL LETTER A..LATIN SMALL LETTER Z\n00AA ; XID_Start # Lo FEMININE ORDINAL INDICATOR\n00B5 ; XID_Start # L& MICRO SIGN\n00BA ; XID_Start # Lo MASCULINE ORDINAL INDICATOR\n00C0..00D6 ; XID_Start # L& [23] LATIN CAPITAL LETTER A WITH GRAVE..LATIN CAPITAL LETTER O WITH DIAERESIS\n00D8..00F6 ; XID_Start # L& [31] LATIN CAPITAL LETTER O WITH STROKE..LATIN SMALL LETTER O WITH DIAERESIS\n00F8..01BA ; XID_Start # L& [195] LATIN SMALL LETTER O WITH STROKE..LATIN SMALL LETTER EZH WITH TAIL\n01BB ; XID_Start # Lo LATIN LETTER TWO WITH STROKE\n01BC..01BF ; XID_Start # L& [4] LATIN CAPITAL LETTER TONE FIVE..LATIN LETTER WYNN\n01C0..01C3 ; XID_Start # Lo [4] LATIN LETTER DENTAL CLICK..LATIN LETTER RETROFLEX CLICK\n01C4..0293 ; XID_Start # L& [208] LATIN CAPITAL LETTER DZ WITH CARON..LATIN SMALL LETTER EZH WITH CURL\n0294 ; XID_Start # Lo LATIN LETTER GLOTTAL STOP\n0295..02AF ; XID_Start # L& [27] LATIN LETTER PHARYNGEAL VOICED FRICATIVE..LATIN SMALL LETTER TURNED H WITH FISHHOOK AND TAIL\n02B0..02C1 ; XID_Start # Lm [18] MODIFIER LETTER SMALL H..MODIFIER LETTER REVERSED GLOTTAL STOP\n02C6..02D1 ; XID_Start # Lm [12] MODIFIER LETTER CIRCUMFLEX ACCENT..MODIFIER LETTER HALF TRIANGULAR COLON\n02E0..02E4 ; XID_Start # Lm [5] MODIFIER LETTER SMALL GAMMA..MODIFIER LETTER SMALL REVERSED GLOTTAL STOP\n02EC ; XID_Start # Lm MODIFIER LETTER VOICING\n02EE ; XID_Start # Lm MODIFIER LETTER DOUBLE APOSTROPHE\n0370..0373 ; XID_Start # L& [4] GREEK CAPITAL LETTER HETA..GREEK SMALL LETTER ARCHAIC SAMPI\n0374 ; XID_Start # Lm GREEK NUMERAL SIGN\n0376..0377 ; XID_Start # L& [2] GREEK CAPITAL LETTER PAMPHYLIAN DIGAMMA..GREEK SMALL LETTER PAMPHYLIAN DIGAMMA\n037B..037D ; XID_Start # L& [3] GREEK SMALL REVERSED LUNATE SIGMA SYMBOL..GREEK SMALL REVERSED DOTTED LUNATE SIGMA SYMBOL\n037F ; XID_Start # L& GREEK CAPITAL LETTER YOT\n0386 ; XID_Start # L& GREEK CAPITAL LETTER ALPHA WITH TONOS\n0388..038A ; XID_Start # L& [3] GREEK CAPITAL LETTER EPSILON WITH TONOS..GREEK CAPITAL LETTER IOTA WITH TONOS\n038C ; XID_Start # L& GREEK CAPITAL LETTER OMICRON WITH TONOS\n038E..03A1 ; XID_Start # L& [20] GREEK CAPITAL LETTER UPSILON WITH TONOS..GREEK CAPITAL LETTER RHO\n03A3..03F5 ; XID_Start # L& [83] GREEK CAPITAL LETTER SIGMA..GREEK LUNATE EPSILON SYMBOL\n03F7..0481 ; XID_Start # L& [139] GREEK CAPITAL LETTER SHO..CYRILLIC SMALL LETTER KOPPA\n048A..052F ; XID_Start # L& [166] CYRILLIC CAPITAL LETTER SHORT I WITH TAIL..CYRILLIC SMALL LETTER EL WITH DESCENDER\n0531..0556 ; XID_Start # L& [38] ARMENIAN CAPITAL LETTER AYB..ARMENIAN CAPITAL LETTER FEH\n0559 ; XID_Start # Lm ARMENIAN MODIFIER LETTER LEFT HALF RING\n0561..0587 ; XID_Start # L& [39] ARMENIAN SMALL LETTER AYB..ARMENIAN SMALL LIGATURE ECH YIWN\n05D0..05EA ; XID_Start # Lo [27] HEBREW LETTER ALEF..HEBREW LETTER TAV\n05F0..05F2 ; XID_Start # Lo [3] HEBREW LIGATURE YIDDISH DOUBLE VAV..HEBREW LIGATURE YIDDISH DOUBLE YOD\n0620..063F ; XID_Start # Lo [32] ARABIC LETTER KASHMIRI YEH..ARABIC LETTER FARSI YEH WITH THREE DOTS ABOVE\n0640 ; XID_Start # Lm ARABIC TATWEEL\n0641..064A ; XID_Start # Lo [10] ARABIC LETTER FEH..ARABIC LETTER YEH\n066E..066F ; XID_Start # Lo [2] ARABIC LETTER DOTLESS BEH..ARABIC LETTER DOTLESS QAF\n0671..06D3 ; XID_Start # Lo [99] ARABIC LETTER ALEF WASLA..ARABIC LETTER YEH BARREE WITH HAMZA ABOVE\n06D5 ; XID_Start # Lo ARABIC LETTER AE\n06E5..06E6 ; XID_Start # Lm [2] ARABIC SMALL WAW..ARABIC SMALL YEH\n06EE..06EF ; XID_Start # Lo [2] ARABIC LETTER DAL WITH INVERTED V..ARABIC LETTER REH WITH INVERTED V\n06FA..06FC ; XID_Start # Lo [3] ARABIC LETTER SHEEN WITH DOT BELOW..ARABIC LETTER GHAIN WITH DOT BELOW\n06FF ; XID_Start # Lo ARABIC LETTER HEH WITH INVERTED V\n0710 ; XID_Start # Lo SYRIAC LETTER ALAPH\n0712..072F ; XID_Start # Lo [30] SYRIAC LETTER BETH..SYRIAC LETTER PERSIAN DHALATH\n074D..07A5 ; XID_Start # Lo [89] SYRIAC LETTER SOGDIAN ZHAIN..THAANA LETTER WAAVU\n07B1 ; XID_Start # Lo THAANA LETTER NAA\n07CA..07EA ; XID_Start # Lo [33] NKO LETTER A..NKO LETTER JONA RA\n07F4..07F5 ; XID_Start # Lm [2] NKO HIGH TONE APOSTROPHE..NKO LOW TONE APOSTROPHE\n07FA ; XID_Start # Lm NKO LAJANYALAN\n0800..0815 ; XID_Start # Lo [22] SAMARITAN LETTER ALAF..SAMARITAN LETTER TAAF\n081A ; XID_Start # Lm SAMARITAN MODIFIER LETTER EPENTHETIC YUT\n0824 ; XID_Start # Lm SAMARITAN MODIFIER LETTER SHORT A\n0828 ; XID_Start # Lm SAMARITAN MODIFIER LETTER I\n0840..0858 ; XID_Start # Lo [25] MANDAIC LETTER HALQA..MANDAIC LETTER AIN\n08A0..08B4 ; XID_Start # Lo [21] ARABIC LETTER BEH WITH SMALL V BELOW..ARABIC LETTER KAF WITH DOT BELOW\n0904..0939 ; XID_Start # Lo [54] DEVANAGARI LETTER SHORT A..DEVANAGARI LETTER HA\n093D ; XID_Start # Lo DEVANAGARI SIGN AVAGRAHA\n0950 ; XID_Start # Lo DEVANAGARI OM\n0958..0961 ; XID_Start # Lo [10] DEVANAGARI LETTER QA..DEVANAGARI LETTER VOCALIC LL\n0971 ; XID_Start # Lm DEVANAGARI SIGN HIGH SPACING DOT\n0972..0980 ; XID_Start # Lo [15] DEVANAGARI LETTER CANDRA A..BENGALI ANJI\n0985..098C ; XID_Start # Lo [8] BENGALI LETTER A..BENGALI LETTER VOCALIC L\n098F..0990 ; XID_Start # Lo [2] BENGALI LETTER E..BENGALI LETTER AI\n0993..09A8 ; XID_Start # Lo [22] BENGALI LETTER O..BENGALI LETTER NA\n09AA..09B0 ; XID_Start # Lo [7] BENGALI LETTER PA..BENGALI LETTER RA\n09B2 ; XID_Start # Lo BENGALI LETTER LA\n09B6..09B9 ; XID_Start # Lo [4] BENGALI LETTER SHA..BENGALI LETTER HA\n09BD ; XID_Start # Lo BENGALI SIGN AVAGRAHA\n09CE ; XID_Start # Lo BENGALI LETTER KHANDA TA\n09DC..09DD ; XID_Start # Lo [2] BENGALI LETTER RRA..BENGALI LETTER RHA\n09DF..09E1 ; XID_Start # Lo [3] BENGALI LETTER YYA..BENGALI LETTER VOCALIC LL\n09F0..09F1 ; XID_Start # Lo [2] BENGALI LETTER RA WITH MIDDLE DIAGONAL..BENGALI LETTER RA WITH LOWER DIAGONAL\n0A05..0A0A ; XID_Start # Lo [6] GURMUKHI LETTER A..GURMUKHI LETTER UU\n0A0F..0A10 ; XID_Start # Lo [2] GURMUKHI LETTER EE..GURMUKHI LETTER AI\n0A13..0A28 ; XID_Start # Lo [22] GURMUKHI LETTER OO..GURMUKHI LETTER NA\n0A2A..0A30 ; XID_Start # Lo [7] GURMUKHI LETTER PA..GURMUKHI LETTER RA\n0A32..0A33 ; XID_Start # Lo [2] GURMUKHI LETTER LA..GURMUKHI LETTER LLA\n0A35..0A36 ; XID_Start # Lo [2] GURMUKHI LETTER VA..GURMUKHI LETTER SHA\n0A38..0A39 ; XID_Start # Lo [2] GURMUKHI LETTER SA..GURMUKHI LETTER HA\n0A59..0A5C ; XID_Start # Lo [4] GURMUKHI LETTER KHHA..GURMUKHI LETTER RRA\n0A5E ; XID_Start # Lo GURMUKHI LETTER FA\n0A72..0A74 ; XID_Start # Lo [3] GURMUKHI IRI..GURMUKHI EK ONKAR\n0A85..0A8D ; XID_Start # Lo [9] GUJARATI LETTER A..GUJARATI VOWEL CANDRA E\n0A8F..0A91 ; XID_Start # Lo [3] GUJARATI LETTER E..GUJARATI VOWEL CANDRA O\n0A93..0AA8 ; XID_Start # Lo [22] GUJARATI LETTER O..GUJARATI LETTER NA\n0AAA..0AB0 ; XID_Start # Lo [7] GUJARATI LETTER PA..GUJARATI LETTER RA\n0AB2..0AB3 ; XID_Start # Lo [2] GUJARATI LETTER LA..GUJARATI LETTER LLA\n0AB5..0AB9 ; XID_Start # Lo [5] GUJARATI LETTER VA..GUJARATI LETTER HA\n0ABD ; XID_Start # Lo GUJARATI SIGN AVAGRAHA\n0AD0 ; XID_Start # Lo GUJARATI OM\n0AE0..0AE1 ; XID_Start # Lo [2] GUJARATI LETTER VOCALIC RR..GUJARATI LETTER VOCALIC LL\n0AF9 ; XID_Start # Lo GUJARATI LETTER ZHA\n0B05..0B0C ; XID_Start # Lo [8] ORIYA LETTER A..ORIYA LETTER VOCALIC L\n0B0F..0B10 ; XID_Start # Lo [2] ORIYA LETTER E..ORIYA LETTER AI\n0B13..0B28 ; XID_Start # Lo [22] ORIYA LETTER O..ORIYA LETTER NA\n0B2A..0B30 ; XID_Start # Lo [7] ORIYA LETTER PA..ORIYA LETTER RA\n0B32..0B33 ; XID_Start # Lo [2] ORIYA LETTER LA..ORIYA LETTER LLA\n0B35..0B39 ; XID_Start # Lo [5] ORIYA LETTER VA..ORIYA LETTER HA\n0B3D ; XID_Start # Lo ORIYA SIGN AVAGRAHA\n0B5C..0B5D ; XID_Start # Lo [2] ORIYA LETTER RRA..ORIYA LETTER RHA\n0B5F..0B61 ; XID_Start # Lo [3] ORIYA LETTER YYA..ORIYA LETTER VOCALIC LL\n0B71 ; XID_Start # Lo ORIYA LETTER WA\n0B83 ; XID_Start # Lo TAMIL SIGN VISARGA\n0B85..0B8A ; XID_Start # Lo [6] TAMIL LETTER A..TAMIL LETTER UU\n0B8E..0B90 ; XID_Start # Lo [3] TAMIL LETTER E..TAMIL LETTER AI\n0B92..0B95 ; XID_Start # Lo [4] TAMIL LETTER O..TAMIL LETTER KA\n0B99..0B9A ; XID_Start # Lo [2] TAMIL LETTER NGA..TAMIL LETTER CA\n0B9C ; XID_Start # Lo TAMIL LETTER JA\n0B9E..0B9F ; XID_Start # Lo [2] TAMIL LETTER NYA..TAMIL LETTER TTA\n0BA3..0BA4 ; XID_Start # Lo [2] TAMIL LETTER NNA..TAMIL LETTER TA\n0BA8..0BAA ; XID_Start # Lo [3] TAMIL LETTER NA..TAMIL LETTER PA\n0BAE..0BB9 ; XID_Start # Lo [12] TAMIL LETTER MA..TAMIL LETTER HA\n0BD0 ; XID_Start # Lo TAMIL OM\n0C05..0C0C ; XID_Start # Lo [8] TELUGU LETTER A..TELUGU LETTER VOCALIC L\n0C0E..0C10 ; XID_Start # Lo [3] TELUGU LETTER E..TELUGU LETTER AI\n0C12..0C28 ; XID_Start # Lo [23] TELUGU LETTER O..TELUGU LETTER NA\n0C2A..0C39 ; XID_Start # Lo [16] TELUGU LETTER PA..TELUGU LETTER HA\n0C3D ; XID_Start # Lo TELUGU SIGN AVAGRAHA\n0C58..0C5A ; XID_Start # Lo [3] TELUGU LETTER TSA..TELUGU LETTER RRRA\n0C60..0C61 ; XID_Start # Lo [2] TELUGU LETTER VOCALIC RR..TELUGU LETTER VOCALIC LL\n0C85..0C8C ; XID_Start # Lo [8] KANNADA LETTER A..KANNADA LETTER VOCALIC L\n0C8E..0C90 ; XID_Start # Lo [3] KANNADA LETTER E..KANNADA LETTER AI\n0C92..0CA8 ; XID_Start # Lo [23] KANNADA LETTER O..KANNADA LETTER NA\n0CAA..0CB3 ; XID_Start # Lo [10] KANNADA LETTER PA..KANNADA LETTER LLA\n0CB5..0CB9 ; XID_Start # Lo [5] KANNADA LETTER VA..KANNADA LETTER HA\n0CBD ; XID_Start # Lo KANNADA SIGN AVAGRAHA\n0CDE ; XID_Start # Lo KANNADA LETTER FA\n0CE0..0CE1 ; XID_Start # Lo [2] KANNADA LETTER VOCALIC RR..KANNADA LETTER VOCALIC LL\n0CF1..0CF2 ; XID_Start # Lo [2] KANNADA SIGN JIHVAMULIYA..KANNADA SIGN UPADHMANIYA\n0D05..0D0C ; XID_Start # Lo [8] MALAYALAM LETTER A..MALAYALAM LETTER VOCALIC L\n0D0E..0D10 ; XID_Start # Lo [3] MALAYALAM LETTER E..MALAYALAM LETTER AI\n0D12..0D3A ; XID_Start # Lo [41] MALAYALAM LETTER O..MALAYALAM LETTER TTTA\n0D3D ; XID_Start # Lo MALAYALAM SIGN AVAGRAHA\n0D4E ; XID_Start # Lo MALAYALAM LETTER DOT REPH\n0D5F..0D61 ; XID_Start # Lo [3] MALAYALAM LETTER ARCHAIC II..MALAYALAM LETTER VOCALIC LL\n0D7A..0D7F ; XID_Start # Lo [6] MALAYALAM LETTER CHILLU NN..MALAYALAM LETTER CHILLU K\n0D85..0D96 ; XID_Start # Lo [18] SINHALA LETTER AYANNA..SINHALA LETTER AUYANNA\n0D9A..0DB1 ; XID_Start # Lo [24] SINHALA LETTER ALPAPRAANA KAYANNA..SINHALA LETTER DANTAJA NAYANNA\n0DB3..0DBB ; XID_Start # Lo [9] SINHALA LETTER SANYAKA DAYANNA..SINHALA LETTER RAYANNA\n0DBD ; XID_Start # Lo SINHALA LETTER DANTAJA LAYANNA\n0DC0..0DC6 ; XID_Start # Lo [7] SINHALA LETTER VAYANNA..SINHALA LETTER FAYANNA\n0E01..0E30 ; XID_Start # Lo [48] THAI CHARACTER KO KAI..THAI CHARACTER SARA A\n0E32 ; XID_Start # Lo THAI CHARACTER SARA AA\n0E40..0E45 ; XID_Start # Lo [6] THAI CHARACTER SARA E..THAI CHARACTER LAKKHANGYAO\n0E46 ; XID_Start # Lm THAI CHARACTER MAIYAMOK\n0E81..0E82 ; XID_Start # Lo [2] LAO LETTER KO..LAO LETTER KHO SUNG\n0E84 ; XID_Start # Lo LAO LETTER KHO TAM\n0E87..0E88 ; XID_Start # Lo [2] LAO LETTER NGO..LAO LETTER CO\n0E8A ; XID_Start # Lo LAO LETTER SO TAM\n0E8D ; XID_Start # Lo LAO LETTER NYO\n0E94..0E97 ; XID_Start # Lo [4] LAO LETTER DO..LAO LETTER THO TAM\n0E99..0E9F ; XID_Start # Lo [7] LAO LETTER NO..LAO LETTER FO SUNG\n0EA1..0EA3 ; XID_Start # Lo [3] LAO LETTER MO..LAO LETTER LO LING\n0EA5 ; XID_Start # Lo LAO LETTER LO LOOT\n0EA7 ; XID_Start # Lo LAO LETTER WO\n0EAA..0EAB ; XID_Start # Lo [2] LAO LETTER SO SUNG..LAO LETTER HO SUNG\n0EAD..0EB0 ; XID_Start # Lo [4] LAO LETTER O..LAO VOWEL SIGN A\n0EB2 ; XID_Start # Lo LAO VOWEL SIGN AA\n0EBD ; XID_Start # Lo LAO SEMIVOWEL SIGN NYO\n0EC0..0EC4 ; XID_Start # Lo [5] LAO VOWEL SIGN E..LAO VOWEL SIGN AI\n0EC6 ; XID_Start # Lm LAO KO LA\n0EDC..0EDF ; XID_Start # Lo [4] LAO HO NO..LAO LETTER KHMU NYO\n0F00 ; XID_Start # Lo TIBETAN SYLLABLE OM\n0F40..0F47 ; XID_Start # Lo [8] TIBETAN LETTER KA..TIBETAN LETTER JA\n0F49..0F6C ; XID_Start # Lo [36] TIBETAN LETTER NYA..TIBETAN LETTER RRA\n0F88..0F8C ; XID_Start # Lo [5] TIBETAN SIGN LCE TSA CAN..TIBETAN SIGN INVERTED MCHU CAN\n1000..102A ; XID_Start # Lo [43] MYANMAR LETTER KA..MYANMAR LETTER AU\n103F ; XID_Start # Lo MYANMAR LETTER GREAT SA\n1050..1055 ; XID_Start # Lo [6] MYANMAR LETTER SHA..MYANMAR LETTER VOCALIC LL\n105A..105D ; XID_Start # Lo [4] MYANMAR LETTER MON NGA..MYANMAR LETTER MON BBE\n1061 ; XID_Start # Lo MYANMAR LETTER SGAW KAREN SHA\n1065..1066 ; XID_Start # Lo [2] MYANMAR LETTER WESTERN PWO KAREN THA..MYANMAR LETTER WESTERN PWO KAREN PWA\n106E..1070 ; XID_Start # Lo [3] MYANMAR LETTER EASTERN PWO KAREN NNA..MYANMAR LETTER EASTERN PWO KAREN GHWA\n1075..1081 ; XID_Start # Lo [13] MYANMAR LETTER SHAN KA..MYANMAR LETTER SHAN HA\n108E ; XID_Start # Lo MYANMAR LETTER RUMAI PALAUNG FA\n10A0..10C5 ; XID_Start # L& [38] GEORGIAN CAPITAL LETTER AN..GEORGIAN CAPITAL LETTER HOE\n10C7 ; XID_Start # L& GEORGIAN CAPITAL LETTER YN\n10CD ; XID_Start # L& GEORGIAN CAPITAL LETTER AEN\n10D0..10FA ; XID_Start # Lo [43] GEORGIAN LETTER AN..GEORGIAN LETTER AIN\n10FC ; XID_Start # Lm MODIFIER LETTER GEORGIAN NAR\n10FD..1248 ; XID_Start # Lo [332] GEORGIAN LETTER AEN..ETHIOPIC SYLLABLE QWA\n124A..124D ; XID_Start # Lo [4] ETHIOPIC SYLLABLE QWI..ETHIOPIC SYLLABLE QWE\n1250..1256 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE QHA..ETHIOPIC SYLLABLE QHO\n1258 ; XID_Start # Lo ETHIOPIC SYLLABLE QHWA\n125A..125D ; XID_Start # Lo [4] ETHIOPIC SYLLABLE QHWI..ETHIOPIC SYLLABLE QHWE\n1260..1288 ; XID_Start # Lo [41] ETHIOPIC SYLLABLE BA..ETHIOPIC SYLLABLE XWA\n128A..128D ; XID_Start # Lo [4] ETHIOPIC SYLLABLE XWI..ETHIOPIC SYLLABLE XWE\n1290..12B0 ; XID_Start # Lo [33] ETHIOPIC SYLLABLE NA..ETHIOPIC SYLLABLE KWA\n12B2..12B5 ; XID_Start # Lo [4] ETHIOPIC SYLLABLE KWI..ETHIOPIC SYLLABLE KWE\n12B8..12BE ; XID_Start # Lo [7] ETHIOPIC SYLLABLE KXA..ETHIOPIC SYLLABLE KXO\n12C0 ; XID_Start # Lo ETHIOPIC SYLLABLE KXWA\n12C2..12C5 ; XID_Start # Lo [4] ETHIOPIC SYLLABLE KXWI..ETHIOPIC SYLLABLE KXWE\n12C8..12D6 ; XID_Start # Lo [15] ETHIOPIC SYLLABLE WA..ETHIOPIC SYLLABLE PHARYNGEAL O\n12D8..1310 ; XID_Start # Lo [57] ETHIOPIC SYLLABLE ZA..ETHIOPIC SYLLABLE GWA\n1312..1315 ; XID_Start # Lo [4] ETHIOPIC SYLLABLE GWI..ETHIOPIC SYLLABLE GWE\n1318..135A ; XID_Start # Lo [67] ETHIOPIC SYLLABLE GGA..ETHIOPIC SYLLABLE FYA\n1380..138F ; XID_Start # Lo [16] ETHIOPIC SYLLABLE SEBATBEIT MWA..ETHIOPIC SYLLABLE PWE\n13A0..13F5 ; XID_Start # L& [86] CHEROKEE LETTER A..CHEROKEE LETTER MV\n13F8..13FD ; XID_Start # L& [6] CHEROKEE SMALL LETTER YE..CHEROKEE SMALL LETTER MV\n1401..166C ; XID_Start # Lo [620] CANADIAN SYLLABICS E..CANADIAN SYLLABICS CARRIER TTSA\n166F..167F ; XID_Start # Lo [17] CANADIAN SYLLABICS QAI..CANADIAN SYLLABICS BLACKFOOT W\n1681..169A ; XID_Start # Lo [26] OGHAM LETTER BEITH..OGHAM LETTER PEITH\n16A0..16EA ; XID_Start # Lo [75] RUNIC LETTER FEHU FEOH FE F..RUNIC LETTER X\n16EE..16F0 ; XID_Start # Nl [3] RUNIC ARLAUG SYMBOL..RUNIC BELGTHOR SYMBOL\n16F1..16F8 ; XID_Start # Lo [8] RUNIC LETTER K..RUNIC LETTER FRANKS CASKET AESC\n1700..170C ; XID_Start # Lo [13] TAGALOG LETTER A..TAGALOG LETTER YA\n170E..1711 ; XID_Start # Lo [4] TAGALOG LETTER LA..TAGALOG LETTER HA\n1720..1731 ; XID_Start # Lo [18] HANUNOO LETTER A..HANUNOO LETTER HA\n1740..1751 ; XID_Start # Lo [18] BUHID LETTER A..BUHID LETTER HA\n1760..176C ; XID_Start # Lo [13] TAGBANWA LETTER A..TAGBANWA LETTER YA\n176E..1770 ; XID_Start # Lo [3] TAGBANWA LETTER LA..TAGBANWA LETTER SA\n1780..17B3 ; XID_Start # Lo [52] KHMER LETTER KA..KHMER INDEPENDENT VOWEL QAU\n17D7 ; XID_Start # Lm KHMER SIGN LEK TOO\n17DC ; XID_Start # Lo KHMER SIGN AVAKRAHASANYA\n1820..1842 ; XID_Start # Lo [35] MONGOLIAN LETTER A..MONGOLIAN LETTER CHI\n1843 ; XID_Start # Lm MONGOLIAN LETTER TODO LONG VOWEL SIGN\n1844..1877 ; XID_Start # Lo [52] MONGOLIAN LETTER TODO E..MONGOLIAN LETTER MANCHU ZHA\n1880..18A8 ; XID_Start # Lo [41] MONGOLIAN LETTER ALI GALI ANUSVARA ONE..MONGOLIAN LETTER MANCHU ALI GALI BHA\n18AA ; XID_Start # Lo MONGOLIAN LETTER MANCHU ALI GALI LHA\n18B0..18F5 ; XID_Start # Lo [70] CANADIAN SYLLABICS OY..CANADIAN SYLLABICS CARRIER DENTAL S\n1900..191E ; XID_Start # Lo [31] LIMBU VOWEL-CARRIER LETTER..LIMBU LETTER TRA\n1950..196D ; XID_Start # Lo [30] TAI LE LETTER KA..TAI LE LETTER AI\n1970..1974 ; XID_Start # Lo [5] TAI LE LETTER TONE-2..TAI LE LETTER TONE-6\n1980..19AB ; XID_Start # Lo [44] NEW TAI LUE LETTER HIGH QA..NEW TAI LUE LETTER LOW SUA\n19B0..19C9 ; XID_Start # Lo [26] NEW TAI LUE VOWEL SIGN VOWEL SHORTENER..NEW TAI LUE TONE MARK-2\n1A00..1A16 ; XID_Start # Lo [23] BUGINESE LETTER KA..BUGINESE LETTER HA\n1A20..1A54 ; XID_Start # Lo [53] TAI THAM LETTER HIGH KA..TAI THAM LETTER GREAT SA\n1AA7 ; XID_Start # Lm TAI THAM SIGN MAI YAMOK\n1B05..1B33 ; XID_Start # Lo [47] BALINESE LETTER AKARA..BALINESE LETTER HA\n1B45..1B4B ; XID_Start # Lo [7] BALINESE LETTER KAF SASAK..BALINESE LETTER ASYURA SASAK\n1B83..1BA0 ; XID_Start # Lo [30] SUNDANESE LETTER A..SUNDANESE LETTER HA\n1BAE..1BAF ; XID_Start # Lo [2] SUNDANESE LETTER KHA..SUNDANESE LETTER SYA\n1BBA..1BE5 ; XID_Start # Lo [44] SUNDANESE AVAGRAHA..BATAK LETTER U\n1C00..1C23 ; XID_Start # Lo [36] LEPCHA LETTER KA..LEPCHA LETTER A\n1C4D..1C4F ; XID_Start # Lo [3] LEPCHA LETTER TTA..LEPCHA LETTER DDA\n1C5A..1C77 ; XID_Start # Lo [30] OL CHIKI LETTER LA..OL CHIKI LETTER OH\n1C78..1C7D ; XID_Start # Lm [6] OL CHIKI MU TTUDDAG..OL CHIKI AHAD\n1CE9..1CEC ; XID_Start # Lo [4] VEDIC SIGN ANUSVARA ANTARGOMUKHA..VEDIC SIGN ANUSVARA VAMAGOMUKHA WITH TAIL\n1CEE..1CF1 ; XID_Start # Lo [4] VEDIC SIGN HEXIFORM LONG ANUSVARA..VEDIC SIGN ANUSVARA UBHAYATO MUKHA\n1CF5..1CF6 ; XID_Start # Lo [2] VEDIC SIGN JIHVAMULIYA..VEDIC SIGN UPADHMANIYA\n1D00..1D2B ; XID_Start # L& [44] LATIN LETTER SMALL CAPITAL A..CYRILLIC LETTER SMALL CAPITAL EL\n1D2C..1D6A ; XID_Start # Lm [63] MODIFIER LETTER CAPITAL A..GREEK SUBSCRIPT SMALL LETTER CHI\n1D6B..1D77 ; XID_Start # L& [13] LATIN SMALL LETTER UE..LATIN SMALL LETTER TURNED G\n1D78 ; XID_Start # Lm MODIFIER LETTER CYRILLIC EN\n1D79..1D9A ; XID_Start # L& [34] LATIN SMALL LETTER INSULAR G..LATIN SMALL LETTER EZH WITH RETROFLEX HOOK\n1D9B..1DBF ; XID_Start # Lm [37] MODIFIER LETTER SMALL TURNED ALPHA..MODIFIER LETTER SMALL THETA\n1E00..1F15 ; XID_Start # L& [278] LATIN CAPITAL LETTER A WITH RING BELOW..GREEK SMALL LETTER EPSILON WITH DASIA AND OXIA\n1F18..1F1D ; XID_Start # L& [6] GREEK CAPITAL LETTER EPSILON WITH PSILI..GREEK CAPITAL LETTER EPSILON WITH DASIA AND OXIA\n1F20..1F45 ; XID_Start # L& [38] GREEK SMALL LETTER ETA WITH PSILI..GREEK SMALL LETTER OMICRON WITH DASIA AND OXIA\n1F48..1F4D ; XID_Start # L& [6] GREEK CAPITAL LETTER OMICRON WITH PSILI..GREEK CAPITAL LETTER OMICRON WITH DASIA AND OXIA\n1F50..1F57 ; XID_Start # L& [8] GREEK SMALL LETTER UPSILON WITH PSILI..GREEK SMALL LETTER UPSILON WITH DASIA AND PERISPOMENI\n1F59 ; XID_Start # L& GREEK CAPITAL LETTER UPSILON WITH DASIA\n1F5B ; XID_Start # L& GREEK CAPITAL LETTER UPSILON WITH DASIA AND VARIA\n1F5D ; XID_Start # L& GREEK CAPITAL LETTER UPSILON WITH DASIA AND OXIA\n1F5F..1F7D ; XID_Start # L& [31] GREEK CAPITAL LETTER UPSILON WITH DASIA AND PERISPOMENI..GREEK SMALL LETTER OMEGA WITH OXIA\n1F80..1FB4 ; XID_Start # L& [53] GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI..GREEK SMALL LETTER ALPHA WITH OXIA AND YPOGEGRAMMENI\n1FB6..1FBC ; XID_Start # L& [7] GREEK SMALL LETTER ALPHA WITH PERISPOMENI..GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI\n1FBE ; XID_Start # L& GREEK PROSGEGRAMMENI\n1FC2..1FC4 ; XID_Start # L& [3] GREEK SMALL LETTER ETA WITH VARIA AND YPOGEGRAMMENI..GREEK SMALL LETTER ETA WITH OXIA AND YPOGEGRAMMENI\n1FC6..1FCC ; XID_Start # L& [7] GREEK SMALL LETTER ETA WITH PERISPOMENI..GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI\n1FD0..1FD3 ; XID_Start # L& [4] GREEK SMALL LETTER IOTA WITH VRACHY..GREEK SMALL LETTER IOTA WITH DIALYTIKA AND OXIA\n1FD6..1FDB ; XID_Start # L& [6] GREEK SMALL LETTER IOTA WITH PERISPOMENI..GREEK CAPITAL LETTER IOTA WITH OXIA\n1FE0..1FEC ; XID_Start # L& [13] GREEK SMALL LETTER UPSILON WITH VRACHY..GREEK CAPITAL LETTER RHO WITH DASIA\n1FF2..1FF4 ; XID_Start # L& [3] GREEK SMALL LETTER OMEGA WITH VARIA AND YPOGEGRAMMENI..GREEK SMALL LETTER OMEGA WITH OXIA AND YPOGEGRAMMENI\n1FF6..1FFC ; XID_Start # L& [7] GREEK SMALL LETTER OMEGA WITH PERISPOMENI..GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI\n2071 ; XID_Start # Lm SUPERSCRIPT LATIN SMALL LETTER I\n207F ; XID_Start # Lm SUPERSCRIPT LATIN SMALL LETTER N\n2090..209C ; XID_Start # Lm [13] LATIN SUBSCRIPT SMALL LETTER A..LATIN SUBSCRIPT SMALL LETTER T\n2102 ; XID_Start # L& DOUBLE-STRUCK CAPITAL C\n2107 ; XID_Start # L& EULER CONSTANT\n210A..2113 ; XID_Start # L& [10] SCRIPT SMALL G..SCRIPT SMALL L\n2115 ; XID_Start # L& DOUBLE-STRUCK CAPITAL N\n2118 ; XID_Start # Sm SCRIPT CAPITAL P\n2119..211D ; XID_Start # L& [5] DOUBLE-STRUCK CAPITAL P..DOUBLE-STRUCK CAPITAL R\n2124 ; XID_Start # L& DOUBLE-STRUCK CAPITAL Z\n2126 ; XID_Start # L& OHM SIGN\n2128 ; XID_Start # L& BLACK-LETTER CAPITAL Z\n212A..212D ; XID_Start # L& [4] KELVIN SIGN..BLACK-LETTER CAPITAL C\n212E ; XID_Start # So ESTIMATED SYMBOL\n212F..2134 ; XID_Start # L& [6] SCRIPT SMALL E..SCRIPT SMALL O\n2135..2138 ; XID_Start # Lo [4] ALEF SYMBOL..DALET SYMBOL\n2139 ; XID_Start # L& INFORMATION SOURCE\n213C..213F ; XID_Start # L& [4] DOUBLE-STRUCK SMALL PI..DOUBLE-STRUCK CAPITAL PI\n2145..2149 ; XID_Start # L& [5] DOUBLE-STRUCK ITALIC CAPITAL D..DOUBLE-STRUCK ITALIC SMALL J\n214E ; XID_Start # L& TURNED SMALL F\n2160..2182 ; XID_Start # Nl [35] ROMAN NUMERAL ONE..ROMAN NUMERAL TEN THOUSAND\n2183..2184 ; XID_Start # L& [2] ROMAN NUMERAL REVERSED ONE HUNDRED..LATIN SMALL LETTER REVERSED C\n2185..2188 ; XID_Start # Nl [4] ROMAN NUMERAL SIX LATE FORM..ROMAN NUMERAL ONE HUNDRED THOUSAND\n2C00..2C2E ; XID_Start # L& [47] GLAGOLITIC CAPITAL LETTER AZU..GLAGOLITIC CAPITAL LETTER LATINATE MYSLITE\n2C30..2C5E ; XID_Start # L& [47] GLAGOLITIC SMALL LETTER AZU..GLAGOLITIC SMALL LETTER LATINATE MYSLITE\n2C60..2C7B ; XID_Start # L& [28] LATIN CAPITAL LETTER L WITH DOUBLE BAR..LATIN LETTER SMALL CAPITAL TURNED E\n2C7C..2C7D ; XID_Start # Lm [2] LATIN SUBSCRIPT SMALL LETTER J..MODIFIER LETTER CAPITAL V\n2C7E..2CE4 ; XID_Start # L& [103] LATIN CAPITAL LETTER S WITH SWASH TAIL..COPTIC SYMBOL KAI\n2CEB..2CEE ; XID_Start # L& [4] COPTIC CAPITAL LETTER CRYPTOGRAMMIC SHEI..COPTIC SMALL LETTER CRYPTOGRAMMIC GANGIA\n2CF2..2CF3 ; XID_Start # L& [2] COPTIC CAPITAL LETTER BOHAIRIC KHEI..COPTIC SMALL LETTER BOHAIRIC KHEI\n2D00..2D25 ; XID_Start # L& [38] GEORGIAN SMALL LETTER AN..GEORGIAN SMALL LETTER HOE\n2D27 ; XID_Start # L& GEORGIAN SMALL LETTER YN\n2D2D ; XID_Start # L& GEORGIAN SMALL LETTER AEN\n2D30..2D67 ; XID_Start # Lo [56] TIFINAGH LETTER YA..TIFINAGH LETTER YO\n2D6F ; XID_Start # Lm TIFINAGH MODIFIER LETTER LABIALIZATION MARK\n2D80..2D96 ; XID_Start # Lo [23] ETHIOPIC SYLLABLE LOA..ETHIOPIC SYLLABLE GGWE\n2DA0..2DA6 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE SSA..ETHIOPIC SYLLABLE SSO\n2DA8..2DAE ; XID_Start # Lo [7] ETHIOPIC SYLLABLE CCA..ETHIOPIC SYLLABLE CCO\n2DB0..2DB6 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE ZZA..ETHIOPIC SYLLABLE ZZO\n2DB8..2DBE ; XID_Start # Lo [7] ETHIOPIC SYLLABLE CCHA..ETHIOPIC SYLLABLE CCHO\n2DC0..2DC6 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE QYA..ETHIOPIC SYLLABLE QYO\n2DC8..2DCE ; XID_Start # Lo [7] ETHIOPIC SYLLABLE KYA..ETHIOPIC SYLLABLE KYO\n2DD0..2DD6 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE XYA..ETHIOPIC SYLLABLE XYO\n2DD8..2DDE ; XID_Start # Lo [7] ETHIOPIC SYLLABLE GYA..ETHIOPIC SYLLABLE GYO\n3005 ; XID_Start # Lm IDEOGRAPHIC ITERATION MARK\n3006 ; XID_Start # Lo IDEOGRAPHIC CLOSING MARK\n3007 ; XID_Start # Nl IDEOGRAPHIC NUMBER ZERO\n3021..3029 ; XID_Start # Nl [9] HANGZHOU NUMERAL ONE..HANGZHOU NUMERAL NINE\n3031..3035 ; XID_Start # Lm [5] VERTICAL KANA REPEAT MARK..VERTICAL KANA REPEAT MARK LOWER HALF\n3038..303A ; XID_Start # Nl [3] HANGZHOU NUMERAL TEN..HANGZHOU NUMERAL THIRTY\n303B ; XID_Start # Lm VERTICAL IDEOGRAPHIC ITERATION MARK\n303C ; XID_Start # Lo MASU MARK\n3041..3096 ; XID_Start # Lo [86] HIRAGANA LETTER SMALL A..HIRAGANA LETTER SMALL KE\n309D..309E ; XID_Start # Lm [2] HIRAGANA ITERATION MARK..HIRAGANA VOICED ITERATION MARK\n309F ; XID_Start # Lo HIRAGANA DIGRAPH YORI\n30A1..30FA ; XID_Start # Lo [90] KATAKANA LETTER SMALL A..KATAKANA LETTER VO\n30FC..30FE ; XID_Start # Lm [3] KATAKANA-HIRAGANA PROLONGED SOUND MARK..KATAKANA VOICED ITERATION MARK\n30FF ; XID_Start # Lo KATAKANA DIGRAPH KOTO\n3105..312D ; XID_Start # Lo [41] BOPOMOFO LETTER B..BOPOMOFO LETTER IH\n3131..318E ; XID_Start # Lo [94] HANGUL LETTER KIYEOK..HANGUL LETTER ARAEAE\n31A0..31BA ; XID_Start # Lo [27] BOPOMOFO LETTER BU..BOPOMOFO LETTER ZY\n31F0..31FF ; XID_Start # Lo [16] KATAKANA LETTER SMALL KU..KATAKANA LETTER SMALL RO\n3400..4DB5 ; XID_Start # Lo [6582] CJK UNIFIED IDEOGRAPH-3400..CJK UNIFIED IDEOGRAPH-4DB5\n4E00..9FD5 ; XID_Start # Lo [20950] CJK UNIFIED IDEOGRAPH-4E00..CJK UNIFIED IDEOGRAPH-9FD5\nA000..A014 ; XID_Start # Lo [21] YI SYLLABLE IT..YI SYLLABLE E\nA015 ; XID_Start # Lm YI SYLLABLE WU\nA016..A48C ; XID_Start # Lo [1143] YI SYLLABLE BIT..YI SYLLABLE YYR\nA4D0..A4F7 ; XID_Start # Lo [40] LISU LETTER BA..LISU LETTER OE\nA4F8..A4FD ; XID_Start # Lm [6] LISU LETTER TONE MYA TI..LISU LETTER TONE MYA JEU\nA500..A60B ; XID_Start # Lo [268] VAI SYLLABLE EE..VAI SYLLABLE NG\nA60C ; XID_Start # Lm VAI SYLLABLE LENGTHENER\nA610..A61F ; XID_Start # Lo [16] VAI SYLLABLE NDOLE FA..VAI SYMBOL JONG\nA62A..A62B ; XID_Start # Lo [2] VAI SYLLABLE NDOLE MA..VAI SYLLABLE NDOLE DO\nA640..A66D ; XID_Start # L& [46] CYRILLIC CAPITAL LETTER ZEMLYA..CYRILLIC SMALL LETTER DOUBLE MONOCULAR O\nA66E ; XID_Start # Lo CYRILLIC LETTER MULTIOCULAR O\nA67F ; XID_Start # Lm CYRILLIC PAYEROK\nA680..A69B ; XID_Start # L& [28] CYRILLIC CAPITAL LETTER DWE..CYRILLIC SMALL LETTER CROSSED O\nA69C..A69D ; XID_Start # Lm [2] MODIFIER LETTER CYRILLIC HARD SIGN..MODIFIER LETTER CYRILLIC SOFT SIGN\nA6A0..A6E5 ; XID_Start # Lo [70] BAMUM LETTER A..BAMUM LETTER KI\nA6E6..A6EF ; XID_Start # Nl [10] BAMUM LETTER MO..BAMUM LETTER KOGHOM\nA717..A71F ; XID_Start # Lm [9] MODIFIER LETTER DOT VERTICAL BAR..MODIFIER LETTER LOW INVERTED EXCLAMATION MARK\nA722..A76F ; XID_Start # L& [78] LATIN CAPITAL LETTER EGYPTOLOGICAL ALEF..LATIN SMALL LETTER CON\nA770 ; XID_Start # Lm MODIFIER LETTER US\nA771..A787 ; XID_Start # L& [23] LATIN SMALL LETTER DUM..LATIN SMALL LETTER INSULAR T\nA788 ; XID_Start # Lm MODIFIER LETTER LOW CIRCUMFLEX ACCENT\nA78B..A78E ; XID_Start # L& [4] LATIN CAPITAL LETTER SALTILLO..LATIN SMALL LETTER L WITH RETROFLEX HOOK AND BELT\nA78F ; XID_Start # Lo LATIN LETTER SINOLOGICAL DOT\nA790..A7AD ; XID_Start # L& [30] LATIN CAPITAL LETTER N WITH DESCENDER..LATIN CAPITAL LETTER L WITH BELT\nA7B0..A7B7 ; XID_Start # L& [8] LATIN CAPITAL LETTER TURNED K..LATIN SMALL LETTER OMEGA\nA7F7 ; XID_Start # Lo LATIN EPIGRAPHIC LETTER SIDEWAYS I\nA7F8..A7F9 ; XID_Start # Lm [2] MODIFIER LETTER CAPITAL H WITH STROKE..MODIFIER LETTER SMALL LIGATURE OE\nA7FA ; XID_Start # L& LATIN LETTER SMALL CAPITAL TURNED M\nA7FB..A801 ; XID_Start # Lo [7] LATIN EPIGRAPHIC LETTER REVERSED F..SYLOTI NAGRI LETTER I\nA803..A805 ; XID_Start # Lo [3] SYLOTI NAGRI LETTER U..SYLOTI NAGRI LETTER O\nA807..A80A ; XID_Start # Lo [4] SYLOTI NAGRI LETTER KO..SYLOTI NAGRI LETTER GHO\nA80C..A822 ; XID_Start # Lo [23] SYLOTI NAGRI LETTER CO..SYLOTI NAGRI LETTER HO\nA840..A873 ; XID_Start # Lo [52] PHAGS-PA LETTER KA..PHAGS-PA LETTER CANDRABINDU\nA882..A8B3 ; XID_Start # Lo [50] SAURASHTRA LETTER A..SAURASHTRA LETTER LLA\nA8F2..A8F7 ; XID_Start # Lo [6] DEVANAGARI SIGN SPACING CANDRABINDU..DEVANAGARI SIGN CANDRABINDU AVAGRAHA\nA8FB ; XID_Start # Lo DEVANAGARI HEADSTROKE\nA8FD ; XID_Start # Lo DEVANAGARI JAIN OM\nA90A..A925 ; XID_Start # Lo [28] KAYAH LI LETTER KA..KAYAH LI LETTER OO\nA930..A946 ; XID_Start # Lo [23] REJANG LETTER KA..REJANG LETTER A\nA960..A97C ; XID_Start # Lo [29] HANGUL CHOSEONG TIKEUT-MIEUM..HANGUL CHOSEONG SSANGYEORINHIEUH\nA984..A9B2 ; XID_Start # Lo [47] JAVANESE LETTER A..JAVANESE LETTER HA\nA9CF ; XID_Start # Lm JAVANESE PANGRANGKEP\nA9E0..A9E4 ; XID_Start # Lo [5] MYANMAR LETTER SHAN GHA..MYANMAR LETTER SHAN BHA\nA9E6 ; XID_Start # Lm MYANMAR MODIFIER LETTER SHAN REDUPLICATION\nA9E7..A9EF ; XID_Start # Lo [9] MYANMAR LETTER TAI LAING NYA..MYANMAR LETTER TAI LAING NNA\nA9FA..A9FE ; XID_Start # Lo [5] MYANMAR LETTER TAI LAING LLA..MYANMAR LETTER TAI LAING BHA\nAA00..AA28 ; XID_Start # Lo [41] CHAM LETTER A..CHAM LETTER HA\nAA40..AA42 ; XID_Start # Lo [3] CHAM LETTER FINAL K..CHAM LETTER FINAL NG\nAA44..AA4B ; XID_Start # Lo [8] CHAM LETTER FINAL CH..CHAM LETTER FINAL SS\nAA60..AA6F ; XID_Start # Lo [16] MYANMAR LETTER KHAMTI GA..MYANMAR LETTER KHAMTI FA\nAA70 ; XID_Start # Lm MYANMAR MODIFIER LETTER KHAMTI REDUPLICATION\nAA71..AA76 ; XID_Start # Lo [6] MYANMAR LETTER KHAMTI XA..MYANMAR LOGOGRAM KHAMTI HM\nAA7A ; XID_Start # Lo MYANMAR LETTER AITON RA\nAA7E..AAAF ; XID_Start # Lo [50] MYANMAR LETTER SHWE PALAUNG CHA..TAI VIET LETTER HIGH O\nAAB1 ; XID_Start # Lo TAI VIET VOWEL AA\nAAB5..AAB6 ; XID_Start # Lo [2] TAI VIET VOWEL E..TAI VIET VOWEL O\nAAB9..AABD ; XID_Start # Lo [5] TAI VIET VOWEL UEA..TAI VIET VOWEL AN\nAAC0 ; XID_Start # Lo TAI VIET TONE MAI NUENG\nAAC2 ; XID_Start # Lo TAI VIET TONE MAI SONG\nAADB..AADC ; XID_Start # Lo [2] TAI VIET SYMBOL KON..TAI VIET SYMBOL NUENG\nAADD ; XID_Start # Lm TAI VIET SYMBOL SAM\nAAE0..AAEA ; XID_Start # Lo [11] MEETEI MAYEK LETTER E..MEETEI MAYEK LETTER SSA\nAAF2 ; XID_Start # Lo MEETEI MAYEK ANJI\nAAF3..AAF4 ; XID_Start # Lm [2] MEETEI MAYEK SYLLABLE REPETITION MARK..MEETEI MAYEK WORD REPETITION MARK\nAB01..AB06 ; XID_Start # Lo [6] ETHIOPIC SYLLABLE TTHU..ETHIOPIC SYLLABLE TTHO\nAB09..AB0E ; XID_Start # Lo [6] ETHIOPIC SYLLABLE DDHU..ETHIOPIC SYLLABLE DDHO\nAB11..AB16 ; XID_Start # Lo [6] ETHIOPIC SYLLABLE DZU..ETHIOPIC SYLLABLE DZO\nAB20..AB26 ; XID_Start # Lo [7] ETHIOPIC SYLLABLE CCHHA..ETHIOPIC SYLLABLE CCHHO\nAB28..AB2E ; XID_Start # Lo [7] ETHIOPIC SYLLABLE BBA..ETHIOPIC SYLLABLE BBO\nAB30..AB5A ; XID_Start # L& [43] LATIN SMALL LETTER BARRED ALPHA..LATIN SMALL LETTER Y WITH SHORT RIGHT LEG\nAB5C..AB5F ; XID_Start # Lm [4] MODIFIER LETTER SMALL HENG..MODIFIER LETTER SMALL U WITH LEFT HOOK\nAB60..AB65 ; XID_Start # L& [6] LATIN SMALL LETTER SAKHA YAT..GREEK LETTER SMALL CAPITAL OMEGA\nAB70..ABBF ; XID_Start # L& [80] CHEROKEE SMALL LETTER A..CHEROKEE SMALL LETTER YA\nABC0..ABE2 ; XID_Start # Lo [35] MEETEI MAYEK LETTER KOK..MEETEI MAYEK LETTER I LONSUM\nAC00..D7A3 ; XID_Start # Lo [11172] HANGUL SYLLABLE GA..HANGUL SYLLABLE HIH\nD7B0..D7C6 ; XID_Start # Lo [23] HANGUL JUNGSEONG O-YEO..HANGUL JUNGSEONG ARAEA-E\nD7CB..D7FB ; XID_Start # Lo [49] HANGUL JONGSEONG NIEUN-RIEUL..HANGUL JONGSEONG PHIEUPH-THIEUTH\nF900..FA6D ; XID_Start # Lo [366] CJK COMPATIBILITY IDEOGRAPH-F900..CJK COMPATIBILITY IDEOGRAPH-FA6D\nFA70..FAD9 ; XID_Start # Lo [106] CJK COMPATIBILITY IDEOGRAPH-FA70..CJK COMPATIBILITY IDEOGRAPH-FAD9\nFB00..FB06 ; XID_Start # L& [7] LATIN SMALL LIGATURE FF..LATIN SMALL LIGATURE ST\nFB13..FB17 ; XID_Start # L& [5] ARMENIAN SMALL LIGATURE MEN NOW..ARMENIAN SMALL LIGATURE MEN XEH\nFB1D ; XID_Start # Lo HEBREW LETTER YOD WITH HIRIQ\nFB1F..FB28 ; XID_Start # Lo [10] HEBREW LIGATURE YIDDISH YOD YOD PATAH..HEBREW LETTER WIDE TAV\nFB2A..FB36 ; XID_Start # Lo [13] HEBREW LETTER SHIN WITH SHIN DOT..HEBREW LETTER ZAYIN WITH DAGESH\nFB38..FB3C ; XID_Start # Lo [5] HEBREW LETTER TET WITH DAGESH..HEBREW LETTER LAMED WITH DAGESH\nFB3E ; XID_Start # Lo HEBREW LETTER MEM WITH DAGESH\nFB40..FB41 ; XID_Start # Lo [2] HEBREW LETTER NUN WITH DAGESH..HEBREW LETTER SAMEKH WITH DAGESH\nFB43..FB44 ; XID_Start # Lo [2] HEBREW LETTER FINAL PE WITH DAGESH..HEBREW LETTER PE WITH DAGESH\nFB46..FBB1 ; XID_Start # Lo [108] HEBREW LETTER TSADI WITH DAGESH..ARABIC LETTER YEH BARREE WITH HAMZA ABOVE FINAL FORM\nFBD3..FC5D ; XID_Start # Lo [139] ARABIC LETTER NG ISOLATED FORM..ARABIC LIGATURE ALEF MAKSURA WITH SUPERSCRIPT ALEF ISOLATED FORM\nFC64..FD3D ; XID_Start # Lo [218] ARABIC LIGATURE YEH WITH HAMZA ABOVE WITH REH FINAL FORM..ARABIC LIGATURE ALEF WITH FATHATAN ISOLATED FORM\nFD50..FD8F ; XID_Start # Lo [64] ARABIC LIGATURE TEH WITH JEEM WITH MEEM INITIAL FORM..ARABIC LIGATURE MEEM WITH KHAH WITH MEEM INITIAL FORM\nFD92..FDC7 ; XID_Start # Lo [54] ARABIC LIGATURE MEEM WITH JEEM WITH KHAH INITIAL FORM..ARABIC LIGATURE NOON WITH JEEM WITH YEH FINAL FORM\nFDF0..FDF9 ; XID_Start # Lo [10] ARABIC LIGATURE SALLA USED AS KORANIC STOP SIGN ISOLATED FORM..ARABIC LIGATURE SALLA ISOLATED FORM\nFE71 ; XID_Start # Lo ARABIC TATWEEL WITH FATHATAN ABOVE\nFE73 ; XID_Start # Lo ARABIC TAIL FRAGMENT\nFE77 ; XID_Start # Lo ARABIC FATHA MEDIAL FORM\nFE79 ; XID_Start # Lo ARABIC DAMMA MEDIAL FORM\nFE7B ; XID_Start # Lo ARABIC KASRA MEDIAL FORM\nFE7D ; XID_Start # Lo ARABIC SHADDA MEDIAL FORM\nFE7F..FEFC ; XID_Start # Lo [126] ARABIC SUKUN MEDIAL FORM..ARABIC LIGATURE LAM WITH ALEF FINAL FORM\nFF21..FF3A ; XID_Start # L& [26] FULLWIDTH LATIN CAPITAL LETTER A..FULLWIDTH LATIN CAPITAL LETTER Z\nFF41..FF5A ; XID_Start # L& [26] FULLWIDTH LATIN SMALL LETTER A..FULLWIDTH LATIN SMALL LETTER Z\nFF66..FF6F ; XID_Start # Lo [10] HALFWIDTH KATAKANA LETTER WO..HALFWIDTH KATAKANA LETTER SMALL TU\nFF70 ; XID_Start # Lm HALFWIDTH KATAKANA-HIRAGANA PROLONGED SOUND MARK\nFF71..FF9D ; XID_Start # Lo [45] HALFWIDTH KATAKANA LETTER A..HALFWIDTH KATAKANA LETTER N\nFFA0..FFBE ; XID_Start # Lo [31] HALFWIDTH HANGUL FILLER..HALFWIDTH HANGUL LETTER HIEUH\nFFC2..FFC7 ; XID_Start # Lo [6] HALFWIDTH HANGUL LETTER A..HALFWIDTH HANGUL LETTER E\nFFCA..FFCF ; XID_Start # Lo [6] HALFWIDTH HANGUL LETTER YEO..HALFWIDTH HANGUL LETTER OE\nFFD2..FFD7 ; XID_Start # Lo [6] HALFWIDTH HANGUL LETTER YO..HALFWIDTH HANGUL LETTER YU\nFFDA..FFDC ; XID_Start # Lo [3] HALFWIDTH HANGUL LETTER EU..HALFWIDTH HANGUL LETTER I\n10000..1000B ; XID_Start # Lo [12] LINEAR B SYLLABLE B008 A..LINEAR B SYLLABLE B046 JE\n1000D..10026 ; XID_Start # Lo [26] LINEAR B SYLLABLE B036 JO..LINEAR B SYLLABLE B032 QO\n10028..1003A ; XID_Start # Lo [19] LINEAR B SYLLABLE B060 RA..LINEAR B SYLLABLE B042 WO\n1003C..1003D ; XID_Start # Lo [2] LINEAR B SYLLABLE B017 ZA..LINEAR B SYLLABLE B074 ZE\n1003F..1004D ; XID_Start # Lo [15] LINEAR B SYLLABLE B020 ZO..LINEAR B SYLLABLE B091 TWO\n10050..1005D ; XID_Start # Lo [14] LINEAR B SYMBOL B018..LINEAR B SYMBOL B089\n10080..100FA ; XID_Start # Lo [123] LINEAR B IDEOGRAM B100 MAN..LINEAR B IDEOGRAM VESSEL B305\n10140..10174 ; XID_Start # Nl [53] GREEK ACROPHONIC ATTIC ONE QUARTER..GREEK ACROPHONIC STRATIAN FIFTY MNAS\n10280..1029C ; XID_Start # Lo [29] LYCIAN LETTER A..LYCIAN LETTER X\n102A0..102D0 ; XID_Start # Lo [49] CARIAN LETTER A..CARIAN LETTER UUU3\n10300..1031F ; XID_Start # Lo [32] OLD ITALIC LETTER A..OLD ITALIC LETTER ESS\n10330..10340 ; XID_Start # Lo [17] GOTHIC LETTER AHSA..GOTHIC LETTER PAIRTHRA\n10341 ; XID_Start # Nl GOTHIC LETTER NINETY\n10342..10349 ; XID_Start # Lo [8] GOTHIC LETTER RAIDA..GOTHIC LETTER OTHAL\n1034A ; XID_Start # Nl GOTHIC LETTER NINE HUNDRED\n10350..10375 ; XID_Start # Lo [38] OLD PERMIC LETTER AN..OLD PERMIC LETTER IA\n10380..1039D ; XID_Start # Lo [30] UGARITIC LETTER ALPA..UGARITIC LETTER SSU\n103A0..103C3 ; XID_Start # Lo [36] OLD PERSIAN SIGN A..OLD PERSIAN SIGN HA\n103C8..103CF ; XID_Start # Lo [8] OLD PERSIAN SIGN AURAMAZDAA..OLD PERSIAN SIGN BUUMISH\n103D1..103D5 ; XID_Start # Nl [5] OLD PERSIAN NUMBER ONE..OLD PERSIAN NUMBER HUNDRED\n10400..1044F ; XID_Start # L& [80] DESERET CAPITAL LETTER LONG I..DESERET SMALL LETTER EW\n10450..1049D ; XID_Start # Lo [78] SHAVIAN LETTER PEEP..OSMANYA LETTER OO\n10500..10527 ; XID_Start # Lo [40] ELBASAN LETTER A..ELBASAN LETTER KHE\n10530..10563 ; XID_Start # Lo [52] CAUCASIAN ALBANIAN LETTER ALT..CAUCASIAN ALBANIAN LETTER KIW\n10600..10736 ; XID_Start # Lo [311] LINEAR A SIGN AB001..LINEAR A SIGN A664\n10740..10755 ; XID_Start # Lo [22] LINEAR A SIGN A701 A..LINEAR A SIGN A732 JE\n10760..10767 ; XID_Start # Lo [8] LINEAR A SIGN A800..LINEAR A SIGN A807\n10800..10805 ; XID_Start # Lo [6] CYPRIOT SYLLABLE A..CYPRIOT SYLLABLE JA\n10808 ; XID_Start # Lo CYPRIOT SYLLABLE JO\n1080A..10835 ; XID_Start # Lo [44] CYPRIOT SYLLABLE KA..CYPRIOT SYLLABLE WO\n10837..10838 ; XID_Start # Lo [2] CYPRIOT SYLLABLE XA..CYPRIOT SYLLABLE XE\n1083C ; XID_Start # Lo CYPRIOT SYLLABLE ZA\n1083F..10855 ; XID_Start # Lo [23] CYPRIOT SYLLABLE ZO..IMPERIAL ARAMAIC LETTER TAW\n10860..10876 ; XID_Start # Lo [23] PALMYRENE LETTER ALEPH..PALMYRENE LETTER TAW\n10880..1089E ; XID_Start # Lo [31] NABATAEAN LETTER FINAL ALEPH..NABATAEAN LETTER TAW\n108E0..108F2 ; XID_Start # Lo [19] HATRAN LETTER ALEPH..HATRAN LETTER QOPH\n108F4..108F5 ; XID_Start # Lo [2] HATRAN LETTER SHIN..HATRAN LETTER TAW\n10900..10915 ; XID_Start # Lo [22] PHOENICIAN LETTER ALF..PHOENICIAN LETTER TAU\n10920..10939 ; XID_Start # Lo [26] LYDIAN LETTER A..LYDIAN LETTER C\n10980..109B7 ; XID_Start # Lo [56] MEROITIC HIEROGLYPHIC LETTER A..MEROITIC CURSIVE LETTER DA\n109BE..109BF ; XID_Start # Lo [2] MEROITIC CURSIVE LOGOGRAM RMT..MEROITIC CURSIVE LOGOGRAM IMN\n10A00 ; XID_Start # Lo KHAROSHTHI LETTER A\n10A10..10A13 ; XID_Start # Lo [4] KHAROSHTHI LETTER KA..KHAROSHTHI LETTER GHA\n10A15..10A17 ; XID_Start # Lo [3] KHAROSHTHI LETTER CA..KHAROSHTHI LETTER JA\n10A19..10A33 ; XID_Start # Lo [27] KHAROSHTHI LETTER NYA..KHAROSHTHI LETTER TTTHA\n10A60..10A7C ; XID_Start # Lo [29] OLD SOUTH ARABIAN LETTER HE..OLD SOUTH ARABIAN LETTER THETH\n10A80..10A9C ; XID_Start # Lo [29] OLD NORTH ARABIAN LETTER HEH..OLD NORTH ARABIAN LETTER ZAH\n10AC0..10AC7 ; XID_Start # Lo [8] MANICHAEAN LETTER ALEPH..MANICHAEAN LETTER WAW\n10AC9..10AE4 ; XID_Start # Lo [28] MANICHAEAN LETTER ZAYIN..MANICHAEAN LETTER TAW\n10B00..10B35 ; XID_Start # Lo [54] AVESTAN LETTER A..AVESTAN LETTER HE\n10B40..10B55 ; XID_Start # Lo [22] INSCRIPTIONAL PARTHIAN LETTER ALEPH..INSCRIPTIONAL PARTHIAN LETTER TAW\n10B60..10B72 ; XID_Start # Lo [19] INSCRIPTIONAL PAHLAVI LETTER ALEPH..INSCRIPTIONAL PAHLAVI LETTER TAW\n10B80..10B91 ; XID_Start # Lo [18] PSALTER PAHLAVI LETTER ALEPH..PSALTER PAHLAVI LETTER TAW\n10C00..10C48 ; XID_Start # Lo [73] OLD TURKIC LETTER ORKHON A..OLD TURKIC LETTER ORKHON BASH\n10C80..10CB2 ; XID_Start # L& [51] OLD HUNGARIAN CAPITAL LETTER A..OLD HUNGARIAN CAPITAL LETTER US\n10CC0..10CF2 ; XID_Start # L& [51] OLD HUNGARIAN SMALL LETTER A..OLD HUNGARIAN SMALL LETTER US\n11003..11037 ; XID_Start # Lo [53] BRAHMI SIGN JIHVAMULIYA..BRAHMI LETTER OLD TAMIL NNNA\n11083..110AF ; XID_Start # Lo [45] KAITHI LETTER A..KAITHI LETTER HA\n110D0..110E8 ; XID_Start # Lo [25] SORA SOMPENG LETTER SAH..SORA SOMPENG LETTER MAE\n11103..11126 ; XID_Start # Lo [36] CHAKMA LETTER AA..CHAKMA LETTER HAA\n11150..11172 ; XID_Start # Lo [35] MAHAJANI LETTER A..MAHAJANI LETTER RRA\n11176 ; XID_Start # Lo MAHAJANI LIGATURE SHRI\n11183..111B2 ; XID_Start # Lo [48] SHARADA LETTER A..SHARADA LETTER HA\n111C1..111C4 ; XID_Start # Lo [4] SHARADA SIGN AVAGRAHA..SHARADA OM\n111DA ; XID_Start # Lo SHARADA EKAM\n111DC ; XID_Start # Lo SHARADA HEADSTROKE\n11200..11211 ; XID_Start # Lo [18] KHOJKI LETTER A..KHOJKI LETTER JJA\n11213..1122B ; XID_Start # Lo [25] KHOJKI LETTER NYA..KHOJKI LETTER LLA\n11280..11286 ; XID_Start # Lo [7] MULTANI LETTER A..MULTANI LETTER GA\n11288 ; XID_Start # Lo MULTANI LETTER GHA\n1128A..1128D ; XID_Start # Lo [4] MULTANI LETTER CA..MULTANI LETTER JJA\n1128F..1129D ; XID_Start # Lo [15] MULTANI LETTER NYA..MULTANI LETTER BA\n1129F..112A8 ; XID_Start # Lo [10] MULTANI LETTER BHA..MULTANI LETTER RHA\n112B0..112DE ; XID_Start # Lo [47] KHUDAWADI LETTER A..KHUDAWADI LETTER HA\n11305..1130C ; XID_Start # Lo [8] GRANTHA LETTER A..GRANTHA LETTER VOCALIC L\n1130F..11310 ; XID_Start # Lo [2] GRANTHA LETTER EE..GRANTHA LETTER AI\n11313..11328 ; XID_Start # Lo [22] GRANTHA LETTER OO..GRANTHA LETTER NA\n1132A..11330 ; XID_Start # Lo [7] GRANTHA LETTER PA..GRANTHA LETTER RA\n11332..11333 ; XID_Start # Lo [2] GRANTHA LETTER LA..GRANTHA LETTER LLA\n11335..11339 ; XID_Start # Lo [5] GRANTHA LETTER VA..GRANTHA LETTER HA\n1133D ; XID_Start # Lo GRANTHA SIGN AVAGRAHA\n11350 ; XID_Start # Lo GRANTHA OM\n1135D..11361 ; XID_Start # Lo [5] GRANTHA SIGN PLUTA..GRANTHA LETTER VOCALIC LL\n11480..114AF ; XID_Start # Lo [48] TIRHUTA ANJI..TIRHUTA LETTER HA\n114C4..114C5 ; XID_Start # Lo [2] TIRHUTA SIGN AVAGRAHA..TIRHUTA GVANG\n114C7 ; XID_Start # Lo TIRHUTA OM\n11580..115AE ; XID_Start # Lo [47] SIDDHAM LETTER A..SIDDHAM LETTER HA\n115D8..115DB ; XID_Start # Lo [4] SIDDHAM LETTER THREE-CIRCLE ALTERNATE I..SIDDHAM LETTER ALTERNATE U\n11600..1162F ; XID_Start # Lo [48] MODI LETTER A..MODI LETTER LLA\n11644 ; XID_Start # Lo MODI SIGN HUVA\n11680..116AA ; XID_Start # Lo [43] TAKRI LETTER A..TAKRI LETTER RRA\n11700..11719 ; XID_Start # Lo [26] AHOM LETTER KA..AHOM LETTER JHA\n118A0..118DF ; XID_Start # L& [64] WARANG CITI CAPITAL LETTER NGAA..WARANG CITI SMALL LETTER VIYO\n118FF ; XID_Start # Lo WARANG CITI OM\n11AC0..11AF8 ; XID_Start # Lo [57] PAU CIN HAU LETTER PA..PAU CIN HAU GLOTTAL STOP FINAL\n12000..12399 ; XID_Start # Lo [922] CUNEIFORM SIGN A..CUNEIFORM SIGN U U\n12400..1246E ; XID_Start # Nl [111] CUNEIFORM NUMERIC SIGN TWO ASH..CUNEIFORM NUMERIC SIGN NINE U VARIANT FORM\n12480..12543 ; XID_Start # Lo [196] CUNEIFORM SIGN AB TIMES NUN TENU..CUNEIFORM SIGN ZU5 TIMES THREE DISH TENU\n13000..1342E ; XID_Start # Lo [1071] EGYPTIAN HIEROGLYPH A001..EGYPTIAN HIEROGLYPH AA032\n14400..14646 ; XID_Start # Lo [583] ANATOLIAN HIEROGLYPH A001..ANATOLIAN HIEROGLYPH A530\n16800..16A38 ; XID_Start # Lo [569] BAMUM LETTER PHASE-A NGKUE MFON..BAMUM LETTER PHASE-F VUEQ\n16A40..16A5E ; XID_Start # Lo [31] MRO LETTER TA..MRO LETTER TEK\n16AD0..16AED ; XID_Start # Lo [30] BASSA VAH LETTER ENNI..BASSA VAH LETTER I\n16B00..16B2F ; XID_Start # Lo [48] PAHAWH HMONG VOWEL KEEB..PAHAWH HMONG CONSONANT CAU\n16B40..16B43 ; XID_Start # Lm [4] PAHAWH HMONG SIGN VOS SEEV..PAHAWH HMONG SIGN IB YAM\n16B63..16B77 ; XID_Start # Lo [21] PAHAWH HMONG SIGN VOS LUB..PAHAWH HMONG SIGN CIM NRES TOS\n16B7D..16B8F ; XID_Start # Lo [19] PAHAWH HMONG CLAN SIGN TSHEEJ..PAHAWH HMONG CLAN SIGN VWJ\n16F00..16F44 ; XID_Start # Lo [69] MIAO LETTER PA..MIAO LETTER HHA\n16F50 ; XID_Start # Lo MIAO LETTER NASALIZATION\n16F93..16F9F ; XID_Start # Lm [13] MIAO LETTER TONE-2..MIAO LETTER REFORMED TONE-8\n1B000..1B001 ; XID_Start # Lo [2] KATAKANA LETTER ARCHAIC E..HIRAGANA LETTER ARCHAIC YE\n1BC00..1BC6A ; XID_Start # Lo [107] DUPLOYAN LETTER H..DUPLOYAN LETTER VOCALIC M\n1BC70..1BC7C ; XID_Start # Lo [13] DUPLOYAN AFFIX LEFT HORIZONTAL SECANT..DUPLOYAN AFFIX ATTACHED TANGENT HOOK\n1BC80..1BC88 ; XID_Start # Lo [9] DUPLOYAN AFFIX HIGH ACUTE..DUPLOYAN AFFIX HIGH VERTICAL\n1BC90..1BC99 ; XID_Start # Lo [10] DUPLOYAN AFFIX LOW ACUTE..DUPLOYAN AFFIX LOW ARROW\n1D400..1D454 ; XID_Start # L& [85] MATHEMATICAL BOLD CAPITAL A..MATHEMATICAL ITALIC SMALL G\n1D456..1D49C ; XID_Start # L& [71] MATHEMATICAL ITALIC SMALL I..MATHEMATICAL SCRIPT CAPITAL A\n1D49E..1D49F ; XID_Start # L& [2] MATHEMATICAL SCRIPT CAPITAL C..MATHEMATICAL SCRIPT CAPITAL D\n1D4A2 ; XID_Start # L& MATHEMATICAL SCRIPT CAPITAL G\n1D4A5..1D4A6 ; XID_Start # L& [2] MATHEMATICAL SCRIPT CAPITAL J..MATHEMATICAL SCRIPT CAPITAL K\n1D4A9..1D4AC ; XID_Start # L& [4] MATHEMATICAL SCRIPT CAPITAL N..MATHEMATICAL SCRIPT CAPITAL Q\n1D4AE..1D4B9 ; XID_Start # L& [12] MATHEMATICAL SCRIPT CAPITAL S..MATHEMATICAL SCRIPT SMALL D\n1D4BB ; XID_Start # L& MATHEMATICAL SCRIPT SMALL F\n1D4BD..1D4C3 ; XID_Start # L& [7] MATHEMATICAL SCRIPT SMALL H..MATHEMATICAL SCRIPT SMALL N\n1D4C5..1D505 ; XID_Start # L& [65] MATHEMATICAL SCRIPT SMALL P..MATHEMATICAL FRAKTUR CAPITAL B\n1D507..1D50A ; XID_Start # L& [4] MATHEMATICAL FRAKTUR CAPITAL D..MATHEMATICAL FRAKTUR CAPITAL G\n1D50D..1D514 ; XID_Start # L& [8] MATHEMATICAL FRAKTUR CAPITAL J..MATHEMATICAL FRAKTUR CAPITAL Q\n1D516..1D51C ; XID_Start # L& [7] MATHEMATICAL FRAKTUR CAPITAL S..MATHEMATICAL FRAKTUR CAPITAL Y\n1D51E..1D539 ; XID_Start # L& [28] MATHEMATICAL FRAKTUR SMALL A..MATHEMATICAL DOUBLE-STRUCK CAPITAL B\n1D53B..1D53E ; XID_Start # L& [4] MATHEMATICAL DOUBLE-STRUCK CAPITAL D..MATHEMATICAL DOUBLE-STRUCK CAPITAL G\n1D540..1D544 ; XID_Start # L& [5] MATHEMATICAL DOUBLE-STRUCK CAPITAL I..MATHEMATICAL DOUBLE-STRUCK CAPITAL M\n1D546 ; XID_Start # L& MATHEMATICAL DOUBLE-STRUCK CAPITAL O\n1D54A..1D550 ; XID_Start # L& [7] MATHEMATICAL DOUBLE-STRUCK CAPITAL S..MATHEMATICAL DOUBLE-STRUCK CAPITAL Y\n1D552..1D6A5 ; XID_Start # L& [340] MATHEMATICAL DOUBLE-STRUCK SMALL A..MATHEMATICAL ITALIC SMALL DOTLESS J\n1D6A8..1D6C0 ; XID_Start # L& [25] MATHEMATICAL BOLD CAPITAL ALPHA..MATHEMATICAL BOLD CAPITAL OMEGA\n1D6C2..1D6DA ; XID_Start # L& [25] MATHEMATICAL BOLD SMALL ALPHA..MATHEMATICAL BOLD SMALL OMEGA\n1D6DC..1D6FA ; XID_Start # L& [31] MATHEMATICAL BOLD EPSILON SYMBOL..MATHEMATICAL ITALIC CAPITAL OMEGA\n1D6FC..1D714 ; XID_Start # L& [25] MATHEMATICAL ITALIC SMALL ALPHA..MATHEMATICAL ITALIC SMALL OMEGA\n1D716..1D734 ; XID_Start # L& [31] MATHEMATICAL ITALIC EPSILON SYMBOL..MATHEMATICAL BOLD ITALIC CAPITAL OMEGA\n1D736..1D74E ; XID_Start # L& [25] MATHEMATICAL BOLD ITALIC SMALL ALPHA..MATHEMATICAL BOLD ITALIC SMALL OMEGA\n1D750..1D76E ; XID_Start # L& [31] MATHEMATICAL BOLD ITALIC EPSILON SYMBOL..MATHEMATICAL SANS-SERIF BOLD CAPITAL OMEGA\n1D770..1D788 ; XID_Start # L& [25] MATHEMATICAL SANS-SERIF BOLD SMALL ALPHA..MATHEMATICAL SANS-SERIF BOLD SMALL OMEGA\n1D78A..1D7A8 ; XID_Start # L& [31] MATHEMATICAL SANS-SERIF BOLD EPSILON SYMBOL..MATHEMATICAL SANS-SERIF BOLD ITALIC CAPITAL OMEGA\n1D7AA..1D7C2 ; XID_Start # L& [25] MATHEMATICAL SANS-SERIF BOLD ITALIC SMALL ALPHA..MATHEMATICAL SANS-SERIF BOLD ITALIC SMALL OMEGA\n1D7C4..1D7CB ; XID_Start # L& [8] MATHEMATICAL SANS-SERIF BOLD ITALIC EPSILON SYMBOL..MATHEMATICAL BOLD SMALL DIGAMMA\n1E800..1E8C4 ; XID_Start # Lo [197] MENDE KIKAKUI SYLLABLE M001 KI..MENDE KIKAKUI SYLLABLE M060 NYON\n1EE00..1EE03 ; XID_Start # Lo [4] ARABIC MATHEMATICAL ALEF..ARABIC MATHEMATICAL DAL\n1EE05..1EE1F ; XID_Start # Lo [27] ARABIC MATHEMATICAL WAW..ARABIC MATHEMATICAL DOTLESS QAF\n1EE21..1EE22 ; XID_Start # Lo [2] ARABIC MATHEMATICAL INITIAL BEH..ARABIC MATHEMATICAL INITIAL JEEM\n1EE24 ; XID_Start # Lo ARABIC MATHEMATICAL INITIAL HEH\n1EE27 ; XID_Start # Lo ARABIC MATHEMATICAL INITIAL HAH\n1EE29..1EE32 ; XID_Start # Lo [10] ARABIC MATHEMATICAL INITIAL YEH..ARABIC MATHEMATICAL INITIAL QAF\n1EE34..1EE37 ; XID_Start # Lo [4] ARABIC MATHEMATICAL INITIAL SHEEN..ARABIC MATHEMATICAL INITIAL KHAH\n1EE39 ; XID_Start # Lo ARABIC MATHEMATICAL INITIAL DAD\n1EE3B ; XID_Start # Lo ARABIC MATHEMATICAL INITIAL GHAIN\n1EE42 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED JEEM\n1EE47 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED HAH\n1EE49 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED YEH\n1EE4B ; XID_Start # Lo ARABIC MATHEMATICAL TAILED LAM\n1EE4D..1EE4F ; XID_Start # Lo [3] ARABIC MATHEMATICAL TAILED NOON..ARABIC MATHEMATICAL TAILED AIN\n1EE51..1EE52 ; XID_Start # Lo [2] ARABIC MATHEMATICAL TAILED SAD..ARABIC MATHEMATICAL TAILED QAF\n1EE54 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED SHEEN\n1EE57 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED KHAH\n1EE59 ; XID_Start # Lo ARABIC MATHEMATICAL TAILED DAD\n1EE5B ; XID_Start # Lo ARABIC MATHEMATICAL TAILED GHAIN\n1EE5D ; XID_Start # Lo ARABIC MATHEMATICAL TAILED DOTLESS NOON\n1EE5F ; XID_Start # Lo ARABIC MATHEMATICAL TAILED DOTLESS QAF\n1EE61..1EE62 ; XID_Start # Lo [2] ARABIC MATHEMATICAL STRETCHED BEH..ARABIC MATHEMATICAL STRETCHED JEEM\n1EE64 ; XID_Start # Lo ARABIC MATHEMATICAL STRETCHED HEH\n1EE67..1EE6A ; XID_Start # Lo [4] ARABIC MATHEMATICAL STRETCHED HAH..ARABIC MATHEMATICAL STRETCHED KAF\n1EE6C..1EE72 ; XID_Start # Lo [7] ARABIC MATHEMATICAL STRETCHED MEEM..ARABIC MATHEMATICAL STRETCHED QAF\n1EE74..1EE77 ; XID_Start # Lo [4] ARABIC MATHEMATICAL STRETCHED SHEEN..ARABIC MATHEMATICAL STRETCHED KHAH\n1EE79..1EE7C ; XID_Start # Lo [4] ARABIC MATHEMATICAL STRETCHED DAD..ARABIC MATHEMATICAL STRETCHED DOTLESS BEH\n1EE7E ; XID_Start # Lo ARABIC MATHEMATICAL STRETCHED DOTLESS FEH\n1EE80..1EE89 ; XID_Start # Lo [10] ARABIC MATHEMATICAL LOOPED ALEF..ARABIC MATHEMATICAL LOOPED YEH\n1EE8B..1EE9B ; XID_Start # Lo [17] ARABIC MATHEMATICAL LOOPED LAM..ARABIC MATHEMATICAL LOOPED GHAIN\n1EEA1..1EEA3 ; XID_Start # Lo [3] ARABIC MATHEMATICAL DOUBLE-STRUCK BEH..ARABIC MATHEMATICAL DOUBLE-STRUCK DAL\n1EEA5..1EEA9 ; XID_Start # Lo [5] ARABIC MATHEMATICAL DOUBLE-STRUCK WAW..ARABIC MATHEMATICAL DOUBLE-STRUCK YEH\n1EEAB..1EEBB ; XID_Start # Lo [17] ARABIC MATHEMATICAL DOUBLE-STRUCK LAM..ARABIC MATHEMATICAL DOUBLE-STRUCK GHAIN\n20000..2A6D6 ; XID_Start # Lo [42711] CJK UNIFIED IDEOGRAPH-20000..CJK UNIFIED IDEOGRAPH-2A6D6\n2A700..2B734 ; XID_Start # Lo [4149] CJK UNIFIED IDEOGRAPH-2A700..CJK UNIFIED IDEOGRAPH-2B734\n2B740..2B81D ; XID_Start # Lo [222] CJK UNIFIED IDEOGRAPH-2B740..CJK UNIFIED IDEOGRAPH-2B81D\n2B820..2CEA1 ; XID_Start # Lo [5762] CJK UNIFIED IDEOGRAPH-2B820..CJK UNIFIED IDEOGRAPH-2CEA1\n2F800..2FA1D ; XID_Start # Lo [542] CJK COMPATIBILITY IDEOGRAPH-2F800..CJK COMPATIBILITY IDEOGRAPH-2FA1D\n\n# Total code points: 109807\n\n# ================================================\n\n# Derived Property: XID_Continue\n# Mod_ID_Continue modified for closure under NFKx\n# Modified as described in UAX #15\n# NOTE: Does NOT remove the non-NFKx characters.\n# Merely ensures that if isIdentifer(string) then isIdentifier(NFKx(string))\n# NOTE: See UAX #31 for more information\n\n0030..0039 ; XID_Continue # Nd [10] DIGIT ZERO..DIGIT NINE\n0041..005A ; XID_Continue # L& [26] LATIN CAPITAL LETTER A..LATIN CAPITAL LETTER Z\n005F ; XID_Continue # Pc LOW LINE\n0061..007A ; XID_Continue # L& [26] LATIN SMALL LETTER A..LATIN SMALL LETTER Z\n00AA ; XID_Continue # Lo FEMININE ORDINAL INDICATOR\n00B5 ; XID_Continue # L& MICRO SIGN\n00B7 ; XID_Continue # Po MIDDLE DOT\n00BA ; XID_Continue # Lo MASCULINE ORDINAL INDICATOR\n00C0..00D6 ; XID_Continue # L& [23] LATIN CAPITAL LETTER A WITH GRAVE..LATIN CAPITAL LETTER O WITH DIAERESIS\n00D8..00F6 ; XID_Continue # L& [31] LATIN CAPITAL LETTER O WITH STROKE..LATIN SMALL LETTER O WITH DIAERESIS\n00F8..01BA ; XID_Continue # L& [195] LATIN SMALL LETTER O WITH STROKE..LATIN SMALL LETTER EZH WITH TAIL\n01BB ; XID_Continue # Lo LATIN LETTER TWO WITH STROKE\n01BC..01BF ; XID_Continue # L& [4] LATIN CAPITAL LETTER TONE FIVE..LATIN LETTER WYNN\n01C0..01C3 ; XID_Continue # Lo [4] LATIN LETTER DENTAL CLICK..LATIN LETTER RETROFLEX CLICK\n01C4..0293 ; XID_Continue # L& [208] LATIN CAPITAL LETTER DZ WITH CARON..LATIN SMALL LETTER EZH WITH CURL\n0294 ; XID_Continue # Lo LATIN LETTER GLOTTAL STOP\n0295..02AF ; XID_Continue # L& [27] LATIN LETTER PHARYNGEAL VOICED FRICATIVE..LATIN SMALL LETTER TURNED H WITH FISHHOOK AND TAIL\n02B0..02C1 ; XID_Continue # Lm [18] MODIFIER LETTER SMALL H..MODIFIER LETTER REVERSED GLOTTAL STOP\n02C6..02D1 ; XID_Continue # Lm [12] MODIFIER LETTER CIRCUMFLEX ACCENT..MODIFIER LETTER HALF TRIANGULAR COLON\n02E0..02E4 ; XID_Continue # Lm [5] MODIFIER LETTER SMALL GAMMA..MODIFIER LETTER SMALL REVERSED GLOTTAL STOP\n02EC ; XID_Continue # Lm MODIFIER LETTER VOICING\n02EE ; XID_Continue # Lm MODIFIER LETTER DOUBLE APOSTROPHE\n0300..036F ; XID_Continue # Mn [112] COMBINING GRAVE ACCENT..COMBINING LATIN SMALL LETTER X\n0370..0373 ; XID_Continue # L& [4] GREEK CAPITAL LETTER HETA..GREEK SMALL LETTER ARCHAIC SAMPI\n0374 ; XID_Continue # Lm GREEK NUMERAL SIGN\n0376..0377 ; XID_Continue # L& [2] GREEK CAPITAL LETTER PAMPHYLIAN DIGAMMA..GREEK SMALL LETTER PAMPHYLIAN DIGAMMA\n037B..037D ; XID_Continue # L& [3] GREEK SMALL REVERSED LUNATE SIGMA SYMBOL..GREEK SMALL REVERSED DOTTED LUNATE SIGMA SYMBOL\n037F ; XID_Continue # L& GREEK CAPITAL LETTER YOT\n0386 ; XID_Continue # L& GREEK CAPITAL LETTER ALPHA WITH TONOS\n0387 ; XID_Continue # Po GREEK ANO TELEIA\n0388..038A ; XID_Continue # L& [3] GREEK CAPITAL LETTER EPSILON WITH TONOS..GREEK CAPITAL LETTER IOTA WITH TONOS\n038C ; XID_Continue # L& GREEK CAPITAL LETTER OMICRON WITH TONOS\n038E..03A1 ; XID_Continue # L& [20] GREEK CAPITAL LETTER UPSILON WITH TONOS..GREEK CAPITAL LETTER RHO\n03A3..03F5 ; XID_Continue # L& [83] GREEK CAPITAL LETTER SIGMA..GREEK LUNATE EPSILON SYMBOL\n03F7..0481 ; XID_Continue # L& [139] GREEK CAPITAL LETTER SHO..CYRILLIC SMALL LETTER KOPPA\n0483..0487 ; XID_Continue # Mn [5] COMBINING CYRILLIC TITLO..COMBINING CYRILLIC POKRYTIE\n048A..052F ; XID_Continue # L& [166] CYRILLIC CAPITAL LETTER SHORT I WITH TAIL..CYRILLIC SMALL LETTER EL WITH DESCENDER\n0531..0556 ; XID_Continue # L& [38] ARMENIAN CAPITAL LETTER AYB..ARMENIAN CAPITAL LETTER FEH\n0559 ; XID_Continue # Lm ARMENIAN MODIFIER LETTER LEFT HALF RING\n0561..0587 ; XID_Continue # L& [39] ARMENIAN SMALL LETTER AYB..ARMENIAN SMALL LIGATURE ECH YIWN\n0591..05BD ; XID_Continue # Mn [45] HEBREW ACCENT ETNAHTA..HEBREW POINT METEG\n05BF ; XID_Continue # Mn HEBREW POINT RAFE\n05C1..05C2 ; XID_Continue # Mn [2] HEBREW POINT SHIN DOT..HEBREW POINT SIN DOT\n05C4..05C5 ; XID_Continue # Mn [2] HEBREW MARK UPPER DOT..HEBREW MARK LOWER DOT\n05C7 ; XID_Continue # Mn HEBREW POINT QAMATS QATAN\n05D0..05EA ; XID_Continue # Lo [27] HEBREW LETTER ALEF..HEBREW LETTER TAV\n05F0..05F2 ; XID_Continue # Lo [3] HEBREW LIGATURE YIDDISH DOUBLE VAV..HEBREW LIGATURE YIDDISH DOUBLE YOD\n0610..061A ; XID_Continue # Mn [11] ARABIC SIGN SALLALLAHOU ALAYHE WASSALLAM..ARABIC SMALL KASRA\n0620..063F ; XID_Continue # Lo [32] ARABIC LETTER KASHMIRI YEH..ARABIC LETTER FARSI YEH WITH THREE DOTS ABOVE\n0640 ; XID_Continue # Lm ARABIC TATWEEL\n0641..064A ; XID_Continue # Lo [10] ARABIC LETTER FEH..ARABIC LETTER YEH\n064B..065F ; XID_Continue # Mn [21] ARABIC FATHATAN..ARABIC WAVY HAMZA BELOW\n0660..0669 ; XID_Continue # Nd [10] ARABIC-INDIC DIGIT ZERO..ARABIC-INDIC DIGIT NINE\n066E..066F ; XID_Continue # Lo [2] ARABIC LETTER DOTLESS BEH..ARABIC LETTER DOTLESS QAF\n0670 ; XID_Continue # Mn ARABIC LETTER SUPERSCRIPT ALEF\n0671..06D3 ; XID_Continue # Lo [99] ARABIC LETTER ALEF WASLA..ARABIC LETTER YEH BARREE WITH HAMZA ABOVE\n06D5 ; XID_Continue # Lo ARABIC LETTER AE\n06D6..06DC ; XID_Continue # Mn [7] ARABIC SMALL HIGH LIGATURE SAD WITH LAM WITH ALEF MAKSURA..ARABIC SMALL HIGH SEEN\n06DF..06E4 ; XID_Continue # Mn [6] ARABIC SMALL HIGH ROUNDED ZERO..ARABIC SMALL HIGH MADDA\n06E5..06E6 ; XID_Continue # Lm [2] ARABIC SMALL WAW..ARABIC SMALL YEH\n06E7..06E8 ; XID_Continue # Mn [2] ARABIC SMALL HIGH YEH..ARABIC SMALL HIGH NOON\n06EA..06ED ; XID_Continue # Mn [4] ARABIC EMPTY CENTRE LOW STOP..ARABIC SMALL LOW MEEM\n06EE..06EF ; XID_Continue # Lo [2] ARABIC LETTER DAL WITH INVERTED V..ARABIC LETTER REH WITH INVERTED V\n06F0..06F9 ; XID_Continue # Nd [10] EXTENDED ARABIC-INDIC DIGIT ZERO..EXTENDED ARABIC-INDIC DIGIT NINE\n06FA..06FC ; XID_Continue # Lo [3] ARABIC LETTER SHEEN WITH DOT BELOW..ARABIC LETTER GHAIN WITH DOT BELOW\n06FF ; XID_Continue # Lo ARABIC LETTER HEH WITH INVERTED V\n0710 ; XID_Continue # Lo SYRIAC LETTER ALAPH\n0711 ; XID_Continue # Mn SYRIAC LETTER SUPERSCRIPT ALAPH\n0712..072F ; XID_Continue # Lo [30] SYRIAC LETTER BETH..SYRIAC LETTER PERSIAN DHALATH\n0730..074A ; XID_Continue # Mn [27] SYRIAC PTHAHA ABOVE..SYRIAC BARREKH\n074D..07A5 ; XID_Continue # Lo [89] SYRIAC LETTER SOGDIAN ZHAIN..THAANA LETTER WAAVU\n07A6..07B0 ; XID_Continue # Mn [11] THAANA ABAFILI..THAANA SUKUN\n07B1 ; XID_Continue # Lo THAANA LETTER NAA\n07C0..07C9 ; XID_Continue # Nd [10] NKO DIGIT ZERO..NKO DIGIT NINE\n07CA..07EA ; XID_Continue # Lo [33] NKO LETTER A..NKO LETTER JONA RA\n07EB..07F3 ; XID_Continue # Mn [9] NKO COMBINING SHORT HIGH TONE..NKO COMBINING DOUBLE DOT ABOVE\n07F4..07F5 ; XID_Continue # Lm [2] NKO HIGH TONE APOSTROPHE..NKO LOW TONE APOSTROPHE\n07FA ; XID_Continue # Lm NKO LAJANYALAN\n0800..0815 ; XID_Continue # Lo [22] SAMARITAN LETTER ALAF..SAMARITAN LETTER TAAF\n0816..0819 ; XID_Continue # Mn [4] SAMARITAN MARK IN..SAMARITAN MARK DAGESH\n081A ; XID_Continue # Lm SAMARITAN MODIFIER LETTER EPENTHETIC YUT\n081B..0823 ; XID_Continue # Mn [9] SAMARITAN MARK EPENTHETIC YUT..SAMARITAN VOWEL SIGN A\n0824 ; XID_Continue # Lm SAMARITAN MODIFIER LETTER SHORT A\n0825..0827 ; XID_Continue # Mn [3] SAMARITAN VOWEL SIGN SHORT A..SAMARITAN VOWEL SIGN U\n0828 ; XID_Continue # Lm SAMARITAN MODIFIER LETTER I\n0829..082D ; XID_Continue # Mn [5] SAMARITAN VOWEL SIGN LONG I..SAMARITAN MARK NEQUDAA\n0840..0858 ; XID_Continue # Lo [25] MANDAIC LETTER HALQA..MANDAIC LETTER AIN\n0859..085B ; XID_Continue # Mn [3] MANDAIC AFFRICATION MARK..MANDAIC GEMINATION MARK\n08A0..08B4 ; XID_Continue # Lo [21] ARABIC LETTER BEH WITH SMALL V BELOW..ARABIC LETTER KAF WITH DOT BELOW\n08E3..0902 ; XID_Continue # Mn [32] ARABIC TURNED DAMMA BELOW..DEVANAGARI SIGN ANUSVARA\n0903 ; XID_Continue # Mc DEVANAGARI SIGN VISARGA\n0904..0939 ; XID_Continue # Lo [54] DEVANAGARI LETTER SHORT A..DEVANAGARI LETTER HA\n093A ; XID_Continue # Mn DEVANAGARI VOWEL SIGN OE\n093B ; XID_Continue # Mc DEVANAGARI VOWEL SIGN OOE\n093C ; XID_Continue # Mn DEVANAGARI SIGN NUKTA\n093D ; XID_Continue # Lo DEVANAGARI SIGN AVAGRAHA\n093E..0940 ; XID_Continue # Mc [3] DEVANAGARI VOWEL SIGN AA..DEVANAGARI VOWEL SIGN II\n0941..0948 ; XID_Continue # Mn [8] DEVANAGARI VOWEL SIGN U..DEVANAGARI VOWEL SIGN AI\n0949..094C ; XID_Continue # Mc [4] DEVANAGARI VOWEL SIGN CANDRA O..DEVANAGARI VOWEL SIGN AU\n094D ; XID_Continue # Mn DEVANAGARI SIGN VIRAMA\n094E..094F ; XID_Continue # Mc [2] DEVANAGARI VOWEL SIGN PRISHTHAMATRA E..DEVANAGARI VOWEL SIGN AW\n0950 ; XID_Continue # Lo DEVANAGARI OM\n0951..0957 ; XID_Continue # Mn [7] DEVANAGARI STRESS SIGN UDATTA..DEVANAGARI VOWEL SIGN UUE\n0958..0961 ; XID_Continue # Lo [10] DEVANAGARI LETTER QA..DEVANAGARI LETTER VOCALIC LL\n0962..0963 ; XID_Continue # Mn [2] DEVANAGARI VOWEL SIGN VOCALIC L..DEVANAGARI VOWEL SIGN VOCALIC LL\n0966..096F ; XID_Continue # Nd [10] DEVANAGARI DIGIT ZERO..DEVANAGARI DIGIT NINE\n0971 ; XID_Continue # Lm DEVANAGARI SIGN HIGH SPACING DOT\n0972..0980 ; XID_Continue # Lo [15] DEVANAGARI LETTER CANDRA A..BENGALI ANJI\n0981 ; XID_Continue # Mn BENGALI SIGN CANDRABINDU\n0982..0983 ; XID_Continue # Mc [2] BENGALI SIGN ANUSVARA..BENGALI SIGN VISARGA\n0985..098C ; XID_Continue # Lo [8] BENGALI LETTER A..BENGALI LETTER VOCALIC L\n098F..0990 ; XID_Continue # Lo [2] BENGALI LETTER E..BENGALI LETTER AI\n0993..09A8 ; XID_Continue # Lo [22] BENGALI LETTER O..BENGALI LETTER NA\n09AA..09B0 ; XID_Continue # Lo [7] BENGALI LETTER PA..BENGALI LETTER RA\n09B2 ; XID_Continue # Lo BENGALI LETTER LA\n09B6..09B9 ; XID_Continue # Lo [4] BENGALI LETTER SHA..BENGALI LETTER HA\n09BC ; XID_Continue # Mn BENGALI SIGN NUKTA\n09BD ; XID_Continue # Lo BENGALI SIGN AVAGRAHA\n09BE..09C0 ; XID_Continue # Mc [3] BENGALI VOWEL SIGN AA..BENGALI VOWEL SIGN II\n09C1..09C4 ; XID_Continue # Mn [4] BENGALI VOWEL SIGN U..BENGALI VOWEL SIGN VOCALIC RR\n09C7..09C8 ; XID_Continue # Mc [2] BENGALI VOWEL SIGN E..BENGALI VOWEL SIGN AI\n09CB..09CC ; XID_Continue # Mc [2] BENGALI VOWEL SIGN O..BENGALI VOWEL SIGN AU\n09CD ; XID_Continue # Mn BENGALI SIGN VIRAMA\n09CE ; XID_Continue # Lo BENGALI LETTER KHANDA TA\n09D7 ; XID_Continue # Mc BENGALI AU LENGTH MARK\n09DC..09DD ; XID_Continue # Lo [2] BENGALI LETTER RRA..BENGALI LETTER RHA\n09DF..09E1 ; XID_Continue # Lo [3] BENGALI LETTER YYA..BENGALI LETTER VOCALIC LL\n09E2..09E3 ; XID_Continue # Mn [2] BENGALI VOWEL SIGN VOCALIC L..BENGALI VOWEL SIGN VOCALIC LL\n09E6..09EF ; XID_Continue # Nd [10] BENGALI DIGIT ZERO..BENGALI DIGIT NINE\n09F0..09F1 ; XID_Continue # Lo [2] BENGALI LETTER RA WITH MIDDLE DIAGONAL..BENGALI LETTER RA WITH LOWER DIAGONAL\n0A01..0A02 ; XID_Continue # Mn [2] GURMUKHI SIGN ADAK BINDI..GURMUKHI SIGN BINDI\n0A03 ; XID_Continue # Mc GURMUKHI SIGN VISARGA\n0A05..0A0A ; XID_Continue # Lo [6] GURMUKHI LETTER A..GURMUKHI LETTER UU\n0A0F..0A10 ; XID_Continue # Lo [2] GURMUKHI LETTER EE..GURMUKHI LETTER AI\n0A13..0A28 ; XID_Continue # Lo [22] GURMUKHI LETTER OO..GURMUKHI LETTER NA\n0A2A..0A30 ; XID_Continue # Lo [7] GURMUKHI LETTER PA..GURMUKHI LETTER RA\n0A32..0A33 ; XID_Continue # Lo [2] GURMUKHI LETTER LA..GURMUKHI LETTER LLA\n0A35..0A36 ; XID_Continue # Lo [2] GURMUKHI LETTER VA..GURMUKHI LETTER SHA\n0A38..0A39 ; XID_Continue # Lo [2] GURMUKHI LETTER SA..GURMUKHI LETTER HA\n0A3C ; XID_Continue # Mn GURMUKHI SIGN NUKTA\n0A3E..0A40 ; XID_Continue # Mc [3] GURMUKHI VOWEL SIGN AA..GURMUKHI VOWEL SIGN II\n0A41..0A42 ; XID_Continue # Mn [2] GURMUKHI VOWEL SIGN U..GURMUKHI VOWEL SIGN UU\n0A47..0A48 ; XID_Continue # Mn [2] GURMUKHI VOWEL SIGN EE..GURMUKHI VOWEL SIGN AI\n0A4B..0A4D ; XID_Continue # Mn [3] GURMUKHI VOWEL SIGN OO..GURMUKHI SIGN VIRAMA\n0A51 ; XID_Continue # Mn GURMUKHI SIGN UDAAT\n0A59..0A5C ; XID_Continue # Lo [4] GURMUKHI LETTER KHHA..GURMUKHI LETTER RRA\n0A5E ; XID_Continue # Lo GURMUKHI LETTER FA\n0A66..0A6F ; XID_Continue # Nd [10] GURMUKHI DIGIT ZERO..GURMUKHI DIGIT NINE\n0A70..0A71 ; XID_Continue # Mn [2] GURMUKHI TIPPI..GURMUKHI ADDAK\n0A72..0A74 ; XID_Continue # Lo [3] GURMUKHI IRI..GURMUKHI EK ONKAR\n0A75 ; XID_Continue # Mn GURMUKHI SIGN YAKASH\n0A81..0A82 ; XID_Continue # Mn [2] GUJARATI SIGN CANDRABINDU..GUJARATI SIGN ANUSVARA\n0A83 ; XID_Continue # Mc GUJARATI SIGN VISARGA\n0A85..0A8D ; XID_Continue # Lo [9] GUJARATI LETTER A..GUJARATI VOWEL CANDRA E\n0A8F..0A91 ; XID_Continue # Lo [3] GUJARATI LETTER E..GUJARATI VOWEL CANDRA O\n0A93..0AA8 ; XID_Continue # Lo [22] GUJARATI LETTER O..GUJARATI LETTER NA\n0AAA..0AB0 ; XID_Continue # Lo [7] GUJARATI LETTER PA..GUJARATI LETTER RA\n0AB2..0AB3 ; XID_Continue # Lo [2] GUJARATI LETTER LA..GUJARATI LETTER LLA\n0AB5..0AB9 ; XID_Continue # Lo [5] GUJARATI LETTER VA..GUJARATI LETTER HA\n0ABC ; XID_Continue # Mn GUJARATI SIGN NUKTA\n0ABD ; XID_Continue # Lo GUJARATI SIGN AVAGRAHA\n0ABE..0AC0 ; XID_Continue # Mc [3] GUJARATI VOWEL SIGN AA..GUJARATI VOWEL SIGN II\n0AC1..0AC5 ; XID_Continue # Mn [5] GUJARATI VOWEL SIGN U..GUJARATI VOWEL SIGN CANDRA E\n0AC7..0AC8 ; XID_Continue # Mn [2] GUJARATI VOWEL SIGN E..GUJARATI VOWEL SIGN AI\n0AC9 ; XID_Continue # Mc GUJARATI VOWEL SIGN CANDRA O\n0ACB..0ACC ; XID_Continue # Mc [2] GUJARATI VOWEL SIGN O..GUJARATI VOWEL SIGN AU\n0ACD ; XID_Continue # Mn GUJARATI SIGN VIRAMA\n0AD0 ; XID_Continue # Lo GUJARATI OM\n0AE0..0AE1 ; XID_Continue # Lo [2] GUJARATI LETTER VOCALIC RR..GUJARATI LETTER VOCALIC LL\n0AE2..0AE3 ; XID_Continue # Mn [2] GUJARATI VOWEL SIGN VOCALIC L..GUJARATI VOWEL SIGN VOCALIC LL\n0AE6..0AEF ; XID_Continue # Nd [10] GUJARATI DIGIT ZERO..GUJARATI DIGIT NINE\n0AF9 ; XID_Continue # Lo GUJARATI LETTER ZHA\n0B01 ; XID_Continue # Mn ORIYA SIGN CANDRABINDU\n0B02..0B03 ; XID_Continue # Mc [2] ORIYA SIGN ANUSVARA..ORIYA SIGN VISARGA\n0B05..0B0C ; XID_Continue # Lo [8] ORIYA LETTER A..ORIYA LETTER VOCALIC L\n0B0F..0B10 ; XID_Continue # Lo [2] ORIYA LETTER E..ORIYA LETTER AI\n0B13..0B28 ; XID_Continue # Lo [22] ORIYA LETTER O..ORIYA LETTER NA\n0B2A..0B30 ; XID_Continue # Lo [7] ORIYA LETTER PA..ORIYA LETTER RA\n0B32..0B33 ; XID_Continue # Lo [2] ORIYA LETTER LA..ORIYA LETTER LLA\n0B35..0B39 ; XID_Continue # Lo [5] ORIYA LETTER VA..ORIYA LETTER HA\n0B3C ; XID_Continue # Mn ORIYA SIGN NUKTA\n0B3D ; XID_Continue # Lo ORIYA SIGN AVAGRAHA\n0B3E ; XID_Continue # Mc ORIYA VOWEL SIGN AA\n0B3F ; XID_Continue # Mn ORIYA VOWEL SIGN I\n0B40 ; XID_Continue # Mc ORIYA VOWEL SIGN II\n0B41..0B44 ; XID_Continue # Mn [4] ORIYA VOWEL SIGN U..ORIYA VOWEL SIGN VOCALIC RR\n0B47..0B48 ; XID_Continue # Mc [2] ORIYA VOWEL SIGN E..ORIYA VOWEL SIGN AI\n0B4B..0B4C ; XID_Continue # Mc [2] ORIYA VOWEL SIGN O..ORIYA VOWEL SIGN AU\n0B4D ; XID_Continue # Mn ORIYA SIGN VIRAMA\n0B56 ; XID_Continue # Mn ORIYA AI LENGTH MARK\n0B57 ; XID_Continue # Mc ORIYA AU LENGTH MARK\n0B5C..0B5D ; XID_Continue # Lo [2] ORIYA LETTER RRA..ORIYA LETTER RHA\n0B5F..0B61 ; XID_Continue # Lo [3] ORIYA LETTER YYA..ORIYA LETTER VOCALIC LL\n0B62..0B63 ; XID_Continue # Mn [2] ORIYA VOWEL SIGN VOCALIC L..ORIYA VOWEL SIGN VOCALIC LL\n0B66..0B6F ; XID_Continue # Nd [10] ORIYA DIGIT ZERO..ORIYA DIGIT NINE\n0B71 ; XID_Continue # Lo ORIYA LETTER WA\n0B82 ; XID_Continue # Mn TAMIL SIGN ANUSVARA\n0B83 ; XID_Continue # Lo TAMIL SIGN VISARGA\n0B85..0B8A ; XID_Continue # Lo [6] TAMIL LETTER A..TAMIL LETTER UU\n0B8E..0B90 ; XID_Continue # Lo [3] TAMIL LETTER E..TAMIL LETTER AI\n0B92..0B95 ; XID_Continue # Lo [4] TAMIL LETTER O..TAMIL LETTER KA\n0B99..0B9A ; XID_Continue # Lo [2] TAMIL LETTER NGA..TAMIL LETTER CA\n0B9C ; XID_Continue # Lo TAMIL LETTER JA\n0B9E..0B9F ; XID_Continue # Lo [2] TAMIL LETTER NYA..TAMIL LETTER TTA\n0BA3..0BA4 ; XID_Continue # Lo [2] TAMIL LETTER NNA..TAMIL LETTER TA\n0BA8..0BAA ; XID_Continue # Lo [3] TAMIL LETTER NA..TAMIL LETTER PA\n0BAE..0BB9 ; XID_Continue # Lo [12] TAMIL LETTER MA..TAMIL LETTER HA\n0BBE..0BBF ; XID_Continue # Mc [2] TAMIL VOWEL SIGN AA..TAMIL VOWEL SIGN I\n0BC0 ; XID_Continue # Mn TAMIL VOWEL SIGN II\n0BC1..0BC2 ; XID_Continue # Mc [2] TAMIL VOWEL SIGN U..TAMIL VOWEL SIGN UU\n0BC6..0BC8 ; XID_Continue # Mc [3] TAMIL VOWEL SIGN E..TAMIL VOWEL SIGN AI\n0BCA..0BCC ; XID_Continue # Mc [3] TAMIL VOWEL SIGN O..TAMIL VOWEL SIGN AU\n0BCD ; XID_Continue # Mn TAMIL SIGN VIRAMA\n0BD0 ; XID_Continue # Lo TAMIL OM\n0BD7 ; XID_Continue # Mc TAMIL AU LENGTH MARK\n0BE6..0BEF ; XID_Continue # Nd [10] TAMIL DIGIT ZERO..TAMIL DIGIT NINE\n0C00 ; XID_Continue # Mn TELUGU SIGN COMBINING CANDRABINDU ABOVE\n0C01..0C03 ; XID_Continue # Mc [3] TELUGU SIGN CANDRABINDU..TELUGU SIGN VISARGA\n0C05..0C0C ; XID_Continue # Lo [8] TELUGU LETTER A..TELUGU LETTER VOCALIC L\n0C0E..0C10 ; XID_Continue # Lo [3] TELUGU LETTER E..TELUGU LETTER AI\n0C12..0C28 ; XID_Continue # Lo [23] TELUGU LETTER O..TELUGU LETTER NA\n0C2A..0C39 ; XID_Continue # Lo [16] TELUGU LETTER PA..TELUGU LETTER HA\n0C3D ; XID_Continue # Lo TELUGU SIGN AVAGRAHA\n0C3E..0C40 ; XID_Continue # Mn [3] TELUGU VOWEL SIGN AA..TELUGU VOWEL SIGN II\n0C41..0C44 ; XID_Continue # Mc [4] TELUGU VOWEL SIGN U..TELUGU VOWEL SIGN VOCALIC RR\n0C46..0C48 ; XID_Continue # Mn [3] TELUGU VOWEL SIGN E..TELUGU VOWEL SIGN AI\n0C4A..0C4D ; XID_Continue # Mn [4] TELUGU VOWEL SIGN O..TELUGU SIGN VIRAMA\n0C55..0C56 ; XID_Continue # Mn [2] TELUGU LENGTH MARK..TELUGU AI LENGTH MARK\n0C58..0C5A ; XID_Continue # Lo [3] TELUGU LETTER TSA..TELUGU LETTER RRRA\n0C60..0C61 ; XID_Continue # Lo [2] TELUGU LETTER VOCALIC RR..TELUGU LETTER VOCALIC LL\n0C62..0C63 ; XID_Continue # Mn [2] TELUGU VOWEL SIGN VOCALIC L..TELUGU VOWEL SIGN VOCALIC LL\n0C66..0C6F ; XID_Continue # Nd [10] TELUGU DIGIT ZERO..TELUGU DIGIT NINE\n0C81 ; XID_Continue # Mn KANNADA SIGN CANDRABINDU\n0C82..0C83 ; XID_Continue # Mc [2] KANNADA SIGN ANUSVARA..KANNADA SIGN VISARGA\n0C85..0C8C ; XID_Continue # Lo [8] KANNADA LETTER A..KANNADA LETTER VOCALIC L\n0C8E..0C90 ; XID_Continue # Lo [3] KANNADA LETTER E..KANNADA LETTER AI\n0C92..0CA8 ; XID_Continue # Lo [23] KANNADA LETTER O..KANNADA LETTER NA\n0CAA..0CB3 ; XID_Continue # Lo [10] KANNADA LETTER PA..KANNADA LETTER LLA\n0CB5..0CB9 ; XID_Continue # Lo [5] KANNADA LETTER VA..KANNADA LETTER HA\n0CBC ; XID_Continue # Mn KANNADA SIGN NUKTA\n0CBD ; XID_Continue # Lo KANNADA SIGN AVAGRAHA\n0CBE ; XID_Continue # Mc KANNADA VOWEL SIGN AA\n0CBF ; XID_Continue # Mn KANNADA VOWEL SIGN I\n0CC0..0CC4 ; XID_Continue # Mc [5] KANNADA VOWEL SIGN II..KANNADA VOWEL SIGN VOCALIC RR\n0CC6 ; XID_Continue # Mn KANNADA VOWEL SIGN E\n0CC7..0CC8 ; XID_Continue # Mc [2] KANNADA VOWEL SIGN EE..KANNADA VOWEL SIGN AI\n0CCA..0CCB ; XID_Continue # Mc [2] KANNADA VOWEL SIGN O..KANNADA VOWEL SIGN OO\n0CCC..0CCD ; XID_Continue # Mn [2] KANNADA VOWEL SIGN AU..KANNADA SIGN VIRAMA\n0CD5..0CD6 ; XID_Continue # Mc [2] KANNADA LENGTH MARK..KANNADA AI LENGTH MARK\n0CDE ; XID_Continue # Lo KANNADA LETTER FA\n0CE0..0CE1 ; XID_Continue # Lo [2] KANNADA LETTER VOCALIC RR..KANNADA LETTER VOCALIC LL\n0CE2..0CE3 ; XID_Continue # Mn [2] KANNADA VOWEL SIGN VOCALIC L..KANNADA VOWEL SIGN VOCALIC LL\n0CE6..0CEF ; XID_Continue # Nd [10] KANNADA DIGIT ZERO..KANNADA DIGIT NINE\n0CF1..0CF2 ; XID_Continue # Lo [2] KANNADA SIGN JIHVAMULIYA..KANNADA SIGN UPADHMANIYA\n0D01 ; XID_Continue # Mn MALAYALAM SIGN CANDRABINDU\n0D02..0D03 ; XID_Continue # Mc [2] MALAYALAM SIGN ANUSVARA..MALAYALAM SIGN VISARGA\n0D05..0D0C ; XID_Continue # Lo [8] MALAYALAM LETTER A..MALAYALAM LETTER VOCALIC L\n0D0E..0D10 ; XID_Continue # Lo [3] MALAYALAM LETTER E..MALAYALAM LETTER AI\n0D12..0D3A ; XID_Continue # Lo [41] MALAYALAM LETTER O..MALAYALAM LETTER TTTA\n0D3D ; XID_Continue # Lo MALAYALAM SIGN AVAGRAHA\n0D3E..0D40 ; XID_Continue # Mc [3] MALAYALAM VOWEL SIGN AA..MALAYALAM VOWEL SIGN II\n0D41..0D44 ; XID_Continue # Mn [4] MALAYALAM VOWEL SIGN U..MALAYALAM VOWEL SIGN VOCALIC RR\n0D46..0D48 ; XID_Continue # Mc [3] MALAYALAM VOWEL SIGN E..MALAYALAM VOWEL SIGN AI\n0D4A..0D4C ; XID_Continue # Mc [3] MALAYALAM VOWEL SIGN O..MALAYALAM VOWEL SIGN AU\n0D4D ; XID_Continue # Mn MALAYALAM SIGN VIRAMA\n0D4E ; XID_Continue # Lo MALAYALAM LETTER DOT REPH\n0D57 ; XID_Continue # Mc MALAYALAM AU LENGTH MARK\n0D5F..0D61 ; XID_Continue # Lo [3] MALAYALAM LETTER ARCHAIC II..MALAYALAM LETTER VOCALIC LL\n0D62..0D63 ; XID_Continue # Mn [2] MALAYALAM VOWEL SIGN VOCALIC L..MALAYALAM VOWEL SIGN VOCALIC LL\n0D66..0D6F ; XID_Continue # Nd [10] MALAYALAM DIGIT ZERO..MALAYALAM DIGIT NINE\n0D7A..0D7F ; XID_Continue # Lo [6] MALAYALAM LETTER CHILLU NN..MALAYALAM LETTER CHILLU K\n0D82..0D83 ; XID_Continue # Mc [2] SINHALA SIGN ANUSVARAYA..SINHALA SIGN VISARGAYA\n0D85..0D96 ; XID_Continue # Lo [18] SINHALA LETTER AYANNA..SINHALA LETTER AUYANNA\n0D9A..0DB1 ; XID_Continue # Lo [24] SINHALA LETTER ALPAPRAANA KAYANNA..SINHALA LETTER DANTAJA NAYANNA\n0DB3..0DBB ; XID_Continue # Lo [9] SINHALA LETTER SANYAKA DAYANNA..SINHALA LETTER RAYANNA\n0DBD ; XID_Continue # Lo SINHALA LETTER DANTAJA LAYANNA\n0DC0..0DC6 ; XID_Continue # Lo [7] SINHALA LETTER VAYANNA..SINHALA LETTER FAYANNA\n0DCA ; XID_Continue # Mn SINHALA SIGN AL-LAKUNA\n0DCF..0DD1 ; XID_Continue # Mc [3] SINHALA VOWEL SIGN AELA-PILLA..SINHALA VOWEL SIGN DIGA AEDA-PILLA\n0DD2..0DD4 ; XID_Continue # Mn [3] SINHALA VOWEL SIGN KETTI IS-PILLA..SINHALA VOWEL SIGN KETTI PAA-PILLA\n0DD6 ; XID_Continue # Mn SINHALA VOWEL SIGN DIGA PAA-PILLA\n0DD8..0DDF ; XID_Continue # Mc [8] SINHALA VOWEL SIGN GAETTA-PILLA..SINHALA VOWEL SIGN GAYANUKITTA\n0DE6..0DEF ; XID_Continue # Nd [10] SINHALA LITH DIGIT ZERO..SINHALA LITH DIGIT NINE\n0DF2..0DF3 ; XID_Continue # Mc [2] SINHALA VOWEL SIGN DIGA GAETTA-PILLA..SINHALA VOWEL SIGN DIGA GAYANUKITTA\n0E01..0E30 ; XID_Continue # Lo [48] THAI CHARACTER KO KAI..THAI CHARACTER SARA A\n0E31 ; XID_Continue # Mn THAI CHARACTER MAI HAN-AKAT\n0E32..0E33 ; XID_Continue # Lo [2] THAI CHARACTER SARA AA..THAI CHARACTER SARA AM\n0E34..0E3A ; XID_Continue # Mn [7] THAI CHARACTER SARA I..THAI CHARACTER PHINTHU\n0E40..0E45 ; XID_Continue # Lo [6] THAI CHARACTER SARA E..THAI CHARACTER LAKKHANGYAO\n0E46 ; XID_Continue # Lm THAI CHARACTER MAIYAMOK\n0E47..0E4E ; XID_Continue # Mn [8] THAI CHARACTER MAITAIKHU..THAI CHARACTER YAMAKKAN\n0E50..0E59 ; XID_Continue # Nd [10] THAI DIGIT ZERO..THAI DIGIT NINE\n0E81..0E82 ; XID_Continue # Lo [2] LAO LETTER KO..LAO LETTER KHO SUNG\n0E84 ; XID_Continue # Lo LAO LETTER KHO TAM\n0E87..0E88 ; XID_Continue # Lo [2] LAO LETTER NGO..LAO LETTER CO\n0E8A ; XID_Continue # Lo LAO LETTER SO TAM\n0E8D ; XID_Continue # Lo LAO LETTER NYO\n0E94..0E97 ; XID_Continue # Lo [4] LAO LETTER DO..LAO LETTER THO TAM\n0E99..0E9F ; XID_Continue # Lo [7] LAO LETTER NO..LAO LETTER FO SUNG\n0EA1..0EA3 ; XID_Continue # Lo [3] LAO LETTER MO..LAO LETTER LO LING\n0EA5 ; XID_Continue # Lo LAO LETTER LO LOOT\n0EA7 ; XID_Continue # Lo LAO LETTER WO\n0EAA..0EAB ; XID_Continue # Lo [2] LAO LETTER SO SUNG..LAO LETTER HO SUNG\n0EAD..0EB0 ; XID_Continue # Lo [4] LAO LETTER O..LAO VOWEL SIGN A\n0EB1 ; XID_Continue # Mn LAO VOWEL SIGN MAI KAN\n0EB2..0EB3 ; XID_Continue # Lo [2] LAO VOWEL SIGN AA..LAO VOWEL SIGN AM\n0EB4..0EB9 ; XID_Continue # Mn [6] LAO VOWEL SIGN I..LAO VOWEL SIGN UU\n0EBB..0EBC ; XID_Continue # Mn [2] LAO VOWEL SIGN MAI KON..LAO SEMIVOWEL SIGN LO\n0EBD ; XID_Continue # Lo LAO SEMIVOWEL SIGN NYO\n0EC0..0EC4 ; XID_Continue # Lo [5] LAO VOWEL SIGN E..LAO VOWEL SIGN AI\n0EC6 ; XID_Continue # Lm LAO KO LA\n0EC8..0ECD ; XID_Continue # Mn [6] LAO TONE MAI EK..LAO NIGGAHITA\n0ED0..0ED9 ; XID_Continue # Nd [10] LAO DIGIT ZERO..LAO DIGIT NINE\n0EDC..0EDF ; XID_Continue # Lo [4] LAO HO NO..LAO LETTER KHMU NYO\n0F00 ; XID_Continue # Lo TIBETAN SYLLABLE OM\n0F18..0F19 ; XID_Continue # Mn [2] TIBETAN ASTROLOGICAL SIGN -KHYUD PA..TIBETAN ASTROLOGICAL SIGN SDONG TSHUGS\n0F20..0F29 ; XID_Continue # Nd [10] TIBETAN DIGIT ZERO..TIBETAN DIGIT NINE\n0F35 ; XID_Continue # Mn TIBETAN MARK NGAS BZUNG NYI ZLA\n0F37 ; XID_Continue # Mn TIBETAN MARK NGAS BZUNG SGOR RTAGS\n0F39 ; XID_Continue # Mn TIBETAN MARK TSA -PHRU\n0F3E..0F3F ; XID_Continue # Mc [2] TIBETAN SIGN YAR TSHES..TIBETAN SIGN MAR TSHES\n0F40..0F47 ; XID_Continue # Lo [8] TIBETAN LETTER KA..TIBETAN LETTER JA\n0F49..0F6C ; XID_Continue # Lo [36] TIBETAN LETTER NYA..TIBETAN LETTER RRA\n0F71..0F7E ; XID_Continue # Mn [14] TIBETAN VOWEL SIGN AA..TIBETAN SIGN RJES SU NGA RO\n0F7F ; XID_Continue # Mc TIBETAN SIGN RNAM BCAD\n0F80..0F84 ; XID_Continue # Mn [5] TIBETAN VOWEL SIGN REVERSED I..TIBETAN MARK HALANTA\n0F86..0F87 ; XID_Continue # Mn [2] TIBETAN SIGN LCI RTAGS..TIBETAN SIGN YANG RTAGS\n0F88..0F8C ; XID_Continue # Lo [5] TIBETAN SIGN LCE TSA CAN..TIBETAN SIGN INVERTED MCHU CAN\n0F8D..0F97 ; XID_Continue # Mn [11] TIBETAN SUBJOINED SIGN LCE TSA CAN..TIBETAN SUBJOINED LETTER JA\n0F99..0FBC ; XID_Continue # Mn [36] TIBETAN SUBJOINED LETTER NYA..TIBETAN SUBJOINED LETTER FIXED-FORM RA\n0FC6 ; XID_Continue # Mn TIBETAN SYMBOL PADMA GDAN\n1000..102A ; XID_Continue # Lo [43] MYANMAR LETTER KA..MYANMAR LETTER AU\n102B..102C ; XID_Continue # Mc [2] MYANMAR VOWEL SIGN TALL AA..MYANMAR VOWEL SIGN AA\n102D..1030 ; XID_Continue # Mn [4] MYANMAR VOWEL SIGN I..MYANMAR VOWEL SIGN UU\n1031 ; XID_Continue # Mc MYANMAR VOWEL SIGN E\n1032..1037 ; XID_Continue # Mn [6] MYANMAR VOWEL SIGN AI..MYANMAR SIGN DOT BELOW\n1038 ; XID_Continue # Mc MYANMAR SIGN VISARGA\n1039..103A ; XID_Continue # Mn [2] MYANMAR SIGN VIRAMA..MYANMAR SIGN ASAT\n103B..103C ; XID_Continue # Mc [2] MYANMAR CONSONANT SIGN MEDIAL YA..MYANMAR CONSONANT SIGN MEDIAL RA\n103D..103E ; XID_Continue # Mn [2] MYANMAR CONSONANT SIGN MEDIAL WA..MYANMAR CONSONANT SIGN MEDIAL HA\n103F ; XID_Continue # Lo MYANMAR LETTER GREAT SA\n1040..1049 ; XID_Continue # Nd [10] MYANMAR DIGIT ZERO..MYANMAR DIGIT NINE\n1050..1055 ; XID_Continue # Lo [6] MYANMAR LETTER SHA..MYANMAR LETTER VOCALIC LL\n1056..1057 ; XID_Continue # Mc [2] MYANMAR VOWEL SIGN VOCALIC R..MYANMAR VOWEL SIGN VOCALIC RR\n1058..1059 ; XID_Continue # Mn [2] MYANMAR VOWEL SIGN VOCALIC L..MYANMAR VOWEL SIGN VOCALIC LL\n105A..105D ; XID_Continue # Lo [4] MYANMAR LETTER MON NGA..MYANMAR LETTER MON BBE\n105E..1060 ; XID_Continue # Mn [3] MYANMAR CONSONANT SIGN MON MEDIAL NA..MYANMAR CONSONANT SIGN MON MEDIAL LA\n1061 ; XID_Continue # Lo MYANMAR LETTER SGAW KAREN SHA\n1062..1064 ; XID_Continue # Mc [3] MYANMAR VOWEL SIGN SGAW KAREN EU..MYANMAR TONE MARK SGAW KAREN KE PHO\n1065..1066 ; XID_Continue # Lo [2] MYANMAR LETTER WESTERN PWO KAREN THA..MYANMAR LETTER WESTERN PWO KAREN PWA\n1067..106D ; XID_Continue # Mc [7] MYANMAR VOWEL SIGN WESTERN PWO KAREN EU..MYANMAR SIGN WESTERN PWO KAREN TONE-5\n106E..1070 ; XID_Continue # Lo [3] MYANMAR LETTER EASTERN PWO KAREN NNA..MYANMAR LETTER EASTERN PWO KAREN GHWA\n1071..1074 ; XID_Continue # Mn [4] MYANMAR VOWEL SIGN GEBA KAREN I..MYANMAR VOWEL SIGN KAYAH EE\n1075..1081 ; XID_Continue # Lo [13] MYANMAR LETTER SHAN KA..MYANMAR LETTER SHAN HA\n1082 ; XID_Continue # Mn MYANMAR CONSONANT SIGN SHAN MEDIAL WA\n1083..1084 ; XID_Continue # Mc [2] MYANMAR VOWEL SIGN SHAN AA..MYANMAR VOWEL SIGN SHAN E\n1085..1086 ; XID_Continue # Mn [2] MYANMAR VOWEL SIGN SHAN E ABOVE..MYANMAR VOWEL SIGN SHAN FINAL Y\n1087..108C ; XID_Continue # Mc [6] MYANMAR SIGN SHAN TONE-2..MYANMAR SIGN SHAN COUNCIL TONE-3\n108D ; XID_Continue # Mn MYANMAR SIGN SHAN COUNCIL EMPHATIC TONE\n108E ; XID_Continue # Lo MYANMAR LETTER RUMAI PALAUNG FA\n108F ; XID_Continue # Mc MYANMAR SIGN RUMAI PALAUNG TONE-5\n1090..1099 ; XID_Continue # Nd [10] MYANMAR SHAN DIGIT ZERO..MYANMAR SHAN DIGIT NINE\n109A..109C ; XID_Continue # Mc [3] MYANMAR SIGN KHAMTI TONE-1..MYANMAR VOWEL SIGN AITON A\n109D ; XID_Continue # Mn MYANMAR VOWEL SIGN AITON AI\n10A0..10C5 ; XID_Continue # L& [38] GEORGIAN CAPITAL LETTER AN..GEORGIAN CAPITAL LETTER HOE\n10C7 ; XID_Continue # L& GEORGIAN CAPITAL LETTER YN\n10CD ; XID_Continue # L& GEORGIAN CAPITAL LETTER AEN\n10D0..10FA ; XID_Continue # Lo [43] GEORGIAN LETTER AN..GEORGIAN LETTER AIN\n10FC ; XID_Continue # Lm MODIFIER LETTER GEORGIAN NAR\n10FD..1248 ; XID_Continue # Lo [332] GEORGIAN LETTER AEN..ETHIOPIC SYLLABLE QWA\n124A..124D ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE QWI..ETHIOPIC SYLLABLE QWE\n1250..1256 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE QHA..ETHIOPIC SYLLABLE QHO\n1258 ; XID_Continue # Lo ETHIOPIC SYLLABLE QHWA\n125A..125D ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE QHWI..ETHIOPIC SYLLABLE QHWE\n1260..1288 ; XID_Continue # Lo [41] ETHIOPIC SYLLABLE BA..ETHIOPIC SYLLABLE XWA\n128A..128D ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE XWI..ETHIOPIC SYLLABLE XWE\n1290..12B0 ; XID_Continue # Lo [33] ETHIOPIC SYLLABLE NA..ETHIOPIC SYLLABLE KWA\n12B2..12B5 ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE KWI..ETHIOPIC SYLLABLE KWE\n12B8..12BE ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE KXA..ETHIOPIC SYLLABLE KXO\n12C0 ; XID_Continue # Lo ETHIOPIC SYLLABLE KXWA\n12C2..12C5 ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE KXWI..ETHIOPIC SYLLABLE KXWE\n12C8..12D6 ; XID_Continue # Lo [15] ETHIOPIC SYLLABLE WA..ETHIOPIC SYLLABLE PHARYNGEAL O\n12D8..1310 ; XID_Continue # Lo [57] ETHIOPIC SYLLABLE ZA..ETHIOPIC SYLLABLE GWA\n1312..1315 ; XID_Continue # Lo [4] ETHIOPIC SYLLABLE GWI..ETHIOPIC SYLLABLE GWE\n1318..135A ; XID_Continue # Lo [67] ETHIOPIC SYLLABLE GGA..ETHIOPIC SYLLABLE FYA\n135D..135F ; XID_Continue # Mn [3] ETHIOPIC COMBINING GEMINATION AND VOWEL LENGTH MARK..ETHIOPIC COMBINING GEMINATION MARK\n1369..1371 ; XID_Continue # No [9] ETHIOPIC DIGIT ONE..ETHIOPIC DIGIT NINE\n1380..138F ; XID_Continue # Lo [16] ETHIOPIC SYLLABLE SEBATBEIT MWA..ETHIOPIC SYLLABLE PWE\n13A0..13F5 ; XID_Continue # L& [86] CHEROKEE LETTER A..CHEROKEE LETTER MV\n13F8..13FD ; XID_Continue # L& [6] CHEROKEE SMALL LETTER YE..CHEROKEE SMALL LETTER MV\n1401..166C ; XID_Continue # Lo [620] CANADIAN SYLLABICS E..CANADIAN SYLLABICS CARRIER TTSA\n166F..167F ; XID_Continue # Lo [17] CANADIAN SYLLABICS QAI..CANADIAN SYLLABICS BLACKFOOT W\n1681..169A ; XID_Continue # Lo [26] OGHAM LETTER BEITH..OGHAM LETTER PEITH\n16A0..16EA ; XID_Continue # Lo [75] RUNIC LETTER FEHU FEOH FE F..RUNIC LETTER X\n16EE..16F0 ; XID_Continue # Nl [3] RUNIC ARLAUG SYMBOL..RUNIC BELGTHOR SYMBOL\n16F1..16F8 ; XID_Continue # Lo [8] RUNIC LETTER K..RUNIC LETTER FRANKS CASKET AESC\n1700..170C ; XID_Continue # Lo [13] TAGALOG LETTER A..TAGALOG LETTER YA\n170E..1711 ; XID_Continue # Lo [4] TAGALOG LETTER LA..TAGALOG LETTER HA\n1712..1714 ; XID_Continue # Mn [3] TAGALOG VOWEL SIGN I..TAGALOG SIGN VIRAMA\n1720..1731 ; XID_Continue # Lo [18] HANUNOO LETTER A..HANUNOO LETTER HA\n1732..1734 ; XID_Continue # Mn [3] HANUNOO VOWEL SIGN I..HANUNOO SIGN PAMUDPOD\n1740..1751 ; XID_Continue # Lo [18] BUHID LETTER A..BUHID LETTER HA\n1752..1753 ; XID_Continue # Mn [2] BUHID VOWEL SIGN I..BUHID VOWEL SIGN U\n1760..176C ; XID_Continue # Lo [13] TAGBANWA LETTER A..TAGBANWA LETTER YA\n176E..1770 ; XID_Continue # Lo [3] TAGBANWA LETTER LA..TAGBANWA LETTER SA\n1772..1773 ; XID_Continue # Mn [2] TAGBANWA VOWEL SIGN I..TAGBANWA VOWEL SIGN U\n1780..17B3 ; XID_Continue # Lo [52] KHMER LETTER KA..KHMER INDEPENDENT VOWEL QAU\n17B4..17B5 ; XID_Continue # Mn [2] KHMER VOWEL INHERENT AQ..KHMER VOWEL INHERENT AA\n17B6 ; XID_Continue # Mc KHMER VOWEL SIGN AA\n17B7..17BD ; XID_Continue # Mn [7] KHMER VOWEL SIGN I..KHMER VOWEL SIGN UA\n17BE..17C5 ; XID_Continue # Mc [8] KHMER VOWEL SIGN OE..KHMER VOWEL SIGN AU\n17C6 ; XID_Continue # Mn KHMER SIGN NIKAHIT\n17C7..17C8 ; XID_Continue # Mc [2] KHMER SIGN REAHMUK..KHMER SIGN YUUKALEAPINTU\n17C9..17D3 ; XID_Continue # Mn [11] KHMER SIGN MUUSIKATOAN..KHMER SIGN BATHAMASAT\n17D7 ; XID_Continue # Lm KHMER SIGN LEK TOO\n17DC ; XID_Continue # Lo KHMER SIGN AVAKRAHASANYA\n17DD ; XID_Continue # Mn KHMER SIGN ATTHACAN\n17E0..17E9 ; XID_Continue # Nd [10] KHMER DIGIT ZERO..KHMER DIGIT NINE\n180B..180D ; XID_Continue # Mn [3] MONGOLIAN FREE VARIATION SELECTOR ONE..MONGOLIAN FREE VARIATION SELECTOR THREE\n1810..1819 ; XID_Continue # Nd [10] MONGOLIAN DIGIT ZERO..MONGOLIAN DIGIT NINE\n1820..1842 ; XID_Continue # Lo [35] MONGOLIAN LETTER A..MONGOLIAN LETTER CHI\n1843 ; XID_Continue # Lm MONGOLIAN LETTER TODO LONG VOWEL SIGN\n1844..1877 ; XID_Continue # Lo [52] MONGOLIAN LETTER TODO E..MONGOLIAN LETTER MANCHU ZHA\n1880..18A8 ; XID_Continue # Lo [41] MONGOLIAN LETTER ALI GALI ANUSVARA ONE..MONGOLIAN LETTER MANCHU ALI GALI BHA\n18A9 ; XID_Continue # Mn MONGOLIAN LETTER ALI GALI DAGALGA\n18AA ; XID_Continue # Lo MONGOLIAN LETTER MANCHU ALI GALI LHA\n18B0..18F5 ; XID_Continue # Lo [70] CANADIAN SYLLABICS OY..CANADIAN SYLLABICS CARRIER DENTAL S\n1900..191E ; XID_Continue # Lo [31] LIMBU VOWEL-CARRIER LETTER..LIMBU LETTER TRA\n1920..1922 ; XID_Continue # Mn [3] LIMBU VOWEL SIGN A..LIMBU VOWEL SIGN U\n1923..1926 ; XID_Continue # Mc [4] LIMBU VOWEL SIGN EE..LIMBU VOWEL SIGN AU\n1927..1928 ; XID_Continue # Mn [2] LIMBU VOWEL SIGN E..LIMBU VOWEL SIGN O\n1929..192B ; XID_Continue # Mc [3] LIMBU SUBJOINED LETTER YA..LIMBU SUBJOINED LETTER WA\n1930..1931 ; XID_Continue # Mc [2] LIMBU SMALL LETTER KA..LIMBU SMALL LETTER NGA\n1932 ; XID_Continue # Mn LIMBU SMALL LETTER ANUSVARA\n1933..1938 ; XID_Continue # Mc [6] LIMBU SMALL LETTER TA..LIMBU SMALL LETTER LA\n1939..193B ; XID_Continue # Mn [3] LIMBU SIGN MUKPHRENG..LIMBU SIGN SA-I\n1946..194F ; XID_Continue # Nd [10] LIMBU DIGIT ZERO..LIMBU DIGIT NINE\n1950..196D ; XID_Continue # Lo [30] TAI LE LETTER KA..TAI LE LETTER AI\n1970..1974 ; XID_Continue # Lo [5] TAI LE LETTER TONE-2..TAI LE LETTER TONE-6\n1980..19AB ; XID_Continue # Lo [44] NEW TAI LUE LETTER HIGH QA..NEW TAI LUE LETTER LOW SUA\n19B0..19C9 ; XID_Continue # Lo [26] NEW TAI LUE VOWEL SIGN VOWEL SHORTENER..NEW TAI LUE TONE MARK-2\n19D0..19D9 ; XID_Continue # Nd [10] NEW TAI LUE DIGIT ZERO..NEW TAI LUE DIGIT NINE\n19DA ; XID_Continue # No NEW TAI LUE THAM DIGIT ONE\n1A00..1A16 ; XID_Continue # Lo [23] BUGINESE LETTER KA..BUGINESE LETTER HA\n1A17..1A18 ; XID_Continue # Mn [2] BUGINESE VOWEL SIGN I..BUGINESE VOWEL SIGN U\n1A19..1A1A ; XID_Continue # Mc [2] BUGINESE VOWEL SIGN E..BUGINESE VOWEL SIGN O\n1A1B ; XID_Continue # Mn BUGINESE VOWEL SIGN AE\n1A20..1A54 ; XID_Continue # Lo [53] TAI THAM LETTER HIGH KA..TAI THAM LETTER GREAT SA\n1A55 ; XID_Continue # Mc TAI THAM CONSONANT SIGN MEDIAL RA\n1A56 ; XID_Continue # Mn TAI THAM CONSONANT SIGN MEDIAL LA\n1A57 ; XID_Continue # Mc TAI THAM CONSONANT SIGN LA TANG LAI\n1A58..1A5E ; XID_Continue # Mn [7] TAI THAM SIGN MAI KANG LAI..TAI THAM CONSONANT SIGN SA\n1A60 ; XID_Continue # Mn TAI THAM SIGN SAKOT\n1A61 ; XID_Continue # Mc TAI THAM VOWEL SIGN A\n1A62 ; XID_Continue # Mn TAI THAM VOWEL SIGN MAI SAT\n1A63..1A64 ; XID_Continue # Mc [2] TAI THAM VOWEL SIGN AA..TAI THAM VOWEL SIGN TALL AA\n1A65..1A6C ; XID_Continue # Mn [8] TAI THAM VOWEL SIGN I..TAI THAM VOWEL SIGN OA BELOW\n1A6D..1A72 ; XID_Continue # Mc [6] TAI THAM VOWEL SIGN OY..TAI THAM VOWEL SIGN THAM AI\n1A73..1A7C ; XID_Continue # Mn [10] TAI THAM VOWEL SIGN OA ABOVE..TAI THAM SIGN KHUEN-LUE KARAN\n1A7F ; XID_Continue # Mn TAI THAM COMBINING CRYPTOGRAMMIC DOT\n1A80..1A89 ; XID_Continue # Nd [10] TAI THAM HORA DIGIT ZERO..TAI THAM HORA DIGIT NINE\n1A90..1A99 ; XID_Continue # Nd [10] TAI THAM THAM DIGIT ZERO..TAI THAM THAM DIGIT NINE\n1AA7 ; XID_Continue # Lm TAI THAM SIGN MAI YAMOK\n1AB0..1ABD ; XID_Continue # Mn [14] COMBINING DOUBLED CIRCUMFLEX ACCENT..COMBINING PARENTHESES BELOW\n1B00..1B03 ; XID_Continue # Mn [4] BALINESE SIGN ULU RICEM..BALINESE SIGN SURANG\n1B04 ; XID_Continue # Mc BALINESE SIGN BISAH\n1B05..1B33 ; XID_Continue # Lo [47] BALINESE LETTER AKARA..BALINESE LETTER HA\n1B34 ; XID_Continue # Mn BALINESE SIGN REREKAN\n1B35 ; XID_Continue # Mc BALINESE VOWEL SIGN TEDUNG\n1B36..1B3A ; XID_Continue # Mn [5] BALINESE VOWEL SIGN ULU..BALINESE VOWEL SIGN RA REPA\n1B3B ; XID_Continue # Mc BALINESE VOWEL SIGN RA REPA TEDUNG\n1B3C ; XID_Continue # Mn BALINESE VOWEL SIGN LA LENGA\n1B3D..1B41 ; XID_Continue # Mc [5] BALINESE VOWEL SIGN LA LENGA TEDUNG..BALINESE VOWEL SIGN TALING REPA TEDUNG\n1B42 ; XID_Continue # Mn BALINESE VOWEL SIGN PEPET\n1B43..1B44 ; XID_Continue # Mc [2] BALINESE VOWEL SIGN PEPET TEDUNG..BALINESE ADEG ADEG\n1B45..1B4B ; XID_Continue # Lo [7] BALINESE LETTER KAF SASAK..BALINESE LETTER ASYURA SASAK\n1B50..1B59 ; XID_Continue # Nd [10] BALINESE DIGIT ZERO..BALINESE DIGIT NINE\n1B6B..1B73 ; XID_Continue # Mn [9] BALINESE MUSICAL SYMBOL COMBINING TEGEH..BALINESE MUSICAL SYMBOL COMBINING GONG\n1B80..1B81 ; XID_Continue # Mn [2] SUNDANESE SIGN PANYECEK..SUNDANESE SIGN PANGLAYAR\n1B82 ; XID_Continue # Mc SUNDANESE SIGN PANGWISAD\n1B83..1BA0 ; XID_Continue # Lo [30] SUNDANESE LETTER A..SUNDANESE LETTER HA\n1BA1 ; XID_Continue # Mc SUNDANESE CONSONANT SIGN PAMINGKAL\n1BA2..1BA5 ; XID_Continue # Mn [4] SUNDANESE CONSONANT SIGN PANYAKRA..SUNDANESE VOWEL SIGN PANYUKU\n1BA6..1BA7 ; XID_Continue # Mc [2] SUNDANESE VOWEL SIGN PANAELAENG..SUNDANESE VOWEL SIGN PANOLONG\n1BA8..1BA9 ; XID_Continue # Mn [2] SUNDANESE VOWEL SIGN PAMEPET..SUNDANESE VOWEL SIGN PANEULEUNG\n1BAA ; XID_Continue # Mc SUNDANESE SIGN PAMAAEH\n1BAB..1BAD ; XID_Continue # Mn [3] SUNDANESE SIGN VIRAMA..SUNDANESE CONSONANT SIGN PASANGAN WA\n1BAE..1BAF ; XID_Continue # Lo [2] SUNDANESE LETTER KHA..SUNDANESE LETTER SYA\n1BB0..1BB9 ; XID_Continue # Nd [10] SUNDANESE DIGIT ZERO..SUNDANESE DIGIT NINE\n1BBA..1BE5 ; XID_Continue # Lo [44] SUNDANESE AVAGRAHA..BATAK LETTER U\n1BE6 ; XID_Continue # Mn BATAK SIGN TOMPI\n1BE7 ; XID_Continue # Mc BATAK VOWEL SIGN E\n1BE8..1BE9 ; XID_Continue # Mn [2] BATAK VOWEL SIGN PAKPAK E..BATAK VOWEL SIGN EE\n1BEA..1BEC ; XID_Continue # Mc [3] BATAK VOWEL SIGN I..BATAK VOWEL SIGN O\n1BED ; XID_Continue # Mn BATAK VOWEL SIGN KARO O\n1BEE ; XID_Continue # Mc BATAK VOWEL SIGN U\n1BEF..1BF1 ; XID_Continue # Mn [3] BATAK VOWEL SIGN U FOR SIMALUNGUN SA..BATAK CONSONANT SIGN H\n1BF2..1BF3 ; XID_Continue # Mc [2] BATAK PANGOLAT..BATAK PANONGONAN\n1C00..1C23 ; XID_Continue # Lo [36] LEPCHA LETTER KA..LEPCHA LETTER A\n1C24..1C2B ; XID_Continue # Mc [8] LEPCHA SUBJOINED LETTER YA..LEPCHA VOWEL SIGN UU\n1C2C..1C33 ; XID_Continue # Mn [8] LEPCHA VOWEL SIGN E..LEPCHA CONSONANT SIGN T\n1C34..1C35 ; XID_Continue # Mc [2] LEPCHA CONSONANT SIGN NYIN-DO..LEPCHA CONSONANT SIGN KANG\n1C36..1C37 ; XID_Continue # Mn [2] LEPCHA SIGN RAN..LEPCHA SIGN NUKTA\n1C40..1C49 ; XID_Continue # Nd [10] LEPCHA DIGIT ZERO..LEPCHA DIGIT NINE\n1C4D..1C4F ; XID_Continue # Lo [3] LEPCHA LETTER TTA..LEPCHA LETTER DDA\n1C50..1C59 ; XID_Continue # Nd [10] OL CHIKI DIGIT ZERO..OL CHIKI DIGIT NINE\n1C5A..1C77 ; XID_Continue # Lo [30] OL CHIKI LETTER LA..OL CHIKI LETTER OH\n1C78..1C7D ; XID_Continue # Lm [6] OL CHIKI MU TTUDDAG..OL CHIKI AHAD\n1CD0..1CD2 ; XID_Continue # Mn [3] VEDIC TONE KARSHANA..VEDIC TONE PRENKHA\n1CD4..1CE0 ; XID_Continue # Mn [13] VEDIC SIGN YAJURVEDIC MIDLINE SVARITA..VEDIC TONE RIGVEDIC KASHMIRI INDEPENDENT SVARITA\n1CE1 ; XID_Continue # Mc VEDIC TONE ATHARVAVEDIC INDEPENDENT SVARITA\n1CE2..1CE8 ; XID_Continue # Mn [7] VEDIC SIGN VISARGA SVARITA..VEDIC SIGN VISARGA ANUDATTA WITH TAIL\n1CE9..1CEC ; XID_Continue # Lo [4] VEDIC SIGN ANUSVARA ANTARGOMUKHA..VEDIC SIGN ANUSVARA VAMAGOMUKHA WITH TAIL\n1CED ; XID_Continue # Mn VEDIC SIGN TIRYAK\n1CEE..1CF1 ; XID_Continue # Lo [4] VEDIC SIGN HEXIFORM LONG ANUSVARA..VEDIC SIGN ANUSVARA UBHAYATO MUKHA\n1CF2..1CF3 ; XID_Continue # Mc [2] VEDIC SIGN ARDHAVISARGA..VEDIC SIGN ROTATED ARDHAVISARGA\n1CF4 ; XID_Continue # Mn VEDIC TONE CANDRA ABOVE\n1CF5..1CF6 ; XID_Continue # Lo [2] VEDIC SIGN JIHVAMULIYA..VEDIC SIGN UPADHMANIYA\n1CF8..1CF9 ; XID_Continue # Mn [2] VEDIC TONE RING ABOVE..VEDIC TONE DOUBLE RING ABOVE\n1D00..1D2B ; XID_Continue # L& [44] LATIN LETTER SMALL CAPITAL A..CYRILLIC LETTER SMALL CAPITAL EL\n1D2C..1D6A ; XID_Continue # Lm [63] MODIFIER LETTER CAPITAL A..GREEK SUBSCRIPT SMALL LETTER CHI\n1D6B..1D77 ; XID_Continue # L& [13] LATIN SMALL LETTER UE..LATIN SMALL LETTER TURNED G\n1D78 ; XID_Continue # Lm MODIFIER LETTER CYRILLIC EN\n1D79..1D9A ; XID_Continue # L& [34] LATIN SMALL LETTER INSULAR G..LATIN SMALL LETTER EZH WITH RETROFLEX HOOK\n1D9B..1DBF ; XID_Continue # Lm [37] MODIFIER LETTER SMALL TURNED ALPHA..MODIFIER LETTER SMALL THETA\n1DC0..1DF5 ; XID_Continue # Mn [54] COMBINING DOTTED GRAVE ACCENT..COMBINING UP TACK ABOVE\n1DFC..1DFF ; XID_Continue # Mn [4] COMBINING DOUBLE INVERTED BREVE BELOW..COMBINING RIGHT ARROWHEAD AND DOWN ARROWHEAD BELOW\n1E00..1F15 ; XID_Continue # L& [278] LATIN CAPITAL LETTER A WITH RING BELOW..GREEK SMALL LETTER EPSILON WITH DASIA AND OXIA\n1F18..1F1D ; XID_Continue # L& [6] GREEK CAPITAL LETTER EPSILON WITH PSILI..GREEK CAPITAL LETTER EPSILON WITH DASIA AND OXIA\n1F20..1F45 ; XID_Continue # L& [38] GREEK SMALL LETTER ETA WITH PSILI..GREEK SMALL LETTER OMICRON WITH DASIA AND OXIA\n1F48..1F4D ; XID_Continue # L& [6] GREEK CAPITAL LETTER OMICRON WITH PSILI..GREEK CAPITAL LETTER OMICRON WITH DASIA AND OXIA\n1F50..1F57 ; XID_Continue # L& [8] GREEK SMALL LETTER UPSILON WITH PSILI..GREEK SMALL LETTER UPSILON WITH DASIA AND PERISPOMENI\n1F59 ; XID_Continue # L& GREEK CAPITAL LETTER UPSILON WITH DASIA\n1F5B ; XID_Continue # L& GREEK CAPITAL LETTER UPSILON WITH DASIA AND VARIA\n1F5D ; XID_Continue # L& GREEK CAPITAL LETTER UPSILON WITH DASIA AND OXIA\n1F5F..1F7D ; XID_Continue # L& [31] GREEK CAPITAL LETTER UPSILON WITH DASIA AND PERISPOMENI..GREEK SMALL LETTER OMEGA WITH OXIA\n1F80..1FB4 ; XID_Continue # L& [53] GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI..GREEK SMALL LETTER ALPHA WITH OXIA AND YPOGEGRAMMENI\n1FB6..1FBC ; XID_Continue # L& [7] GREEK SMALL LETTER ALPHA WITH PERISPOMENI..GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI\n1FBE ; XID_Continue # L& GREEK PROSGEGRAMMENI\n1FC2..1FC4 ; XID_Continue # L& [3] GREEK SMALL LETTER ETA WITH VARIA AND YPOGEGRAMMENI..GREEK SMALL LETTER ETA WITH OXIA AND YPOGEGRAMMENI\n1FC6..1FCC ; XID_Continue # L& [7] GREEK SMALL LETTER ETA WITH PERISPOMENI..GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI\n1FD0..1FD3 ; XID_Continue # L& [4] GREEK SMALL LETTER IOTA WITH VRACHY..GREEK SMALL LETTER IOTA WITH DIALYTIKA AND OXIA\n1FD6..1FDB ; XID_Continue # L& [6] GREEK SMALL LETTER IOTA WITH PERISPOMENI..GREEK CAPITAL LETTER IOTA WITH OXIA\n1FE0..1FEC ; XID_Continue # L& [13] GREEK SMALL LETTER UPSILON WITH VRACHY..GREEK CAPITAL LETTER RHO WITH DASIA\n1FF2..1FF4 ; XID_Continue # L& [3] GREEK SMALL LETTER OMEGA WITH VARIA AND YPOGEGRAMMENI..GREEK SMALL LETTER OMEGA WITH OXIA AND YPOGEGRAMMENI\n1FF6..1FFC ; XID_Continue # L& [7] GREEK SMALL LETTER OMEGA WITH PERISPOMENI..GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI\n203F..2040 ; XID_Continue # Pc [2] UNDERTIE..CHARACTER TIE\n2054 ; XID_Continue # Pc INVERTED UNDERTIE\n2071 ; XID_Continue # Lm SUPERSCRIPT LATIN SMALL LETTER I\n207F ; XID_Continue # Lm SUPERSCRIPT LATIN SMALL LETTER N\n2090..209C ; XID_Continue # Lm [13] LATIN SUBSCRIPT SMALL LETTER A..LATIN SUBSCRIPT SMALL LETTER T\n20D0..20DC ; XID_Continue # Mn [13] COMBINING LEFT HARPOON ABOVE..COMBINING FOUR DOTS ABOVE\n20E1 ; XID_Continue # Mn COMBINING LEFT RIGHT ARROW ABOVE\n20E5..20F0 ; XID_Continue # Mn [12] COMBINING REVERSE SOLIDUS OVERLAY..COMBINING ASTERISK ABOVE\n2102 ; XID_Continue # L& DOUBLE-STRUCK CAPITAL C\n2107 ; XID_Continue # L& EULER CONSTANT\n210A..2113 ; XID_Continue # L& [10] SCRIPT SMALL G..SCRIPT SMALL L\n2115 ; XID_Continue # L& DOUBLE-STRUCK CAPITAL N\n2118 ; XID_Continue # Sm SCRIPT CAPITAL P\n2119..211D ; XID_Continue # L& [5] DOUBLE-STRUCK CAPITAL P..DOUBLE-STRUCK CAPITAL R\n2124 ; XID_Continue # L& DOUBLE-STRUCK CAPITAL Z\n2126 ; XID_Continue # L& OHM SIGN\n2128 ; XID_Continue # L& BLACK-LETTER CAPITAL Z\n212A..212D ; XID_Continue # L& [4] KELVIN SIGN..BLACK-LETTER CAPITAL C\n212E ; XID_Continue # So ESTIMATED SYMBOL\n212F..2134 ; XID_Continue # L& [6] SCRIPT SMALL E..SCRIPT SMALL O\n2135..2138 ; XID_Continue # Lo [4] ALEF SYMBOL..DALET SYMBOL\n2139 ; XID_Continue # L& INFORMATION SOURCE\n213C..213F ; XID_Continue # L& [4] DOUBLE-STRUCK SMALL PI..DOUBLE-STRUCK CAPITAL PI\n2145..2149 ; XID_Continue # L& [5] DOUBLE-STRUCK ITALIC CAPITAL D..DOUBLE-STRUCK ITALIC SMALL J\n214E ; XID_Continue # L& TURNED SMALL F\n2160..2182 ; XID_Continue # Nl [35] ROMAN NUMERAL ONE..ROMAN NUMERAL TEN THOUSAND\n2183..2184 ; XID_Continue # L& [2] ROMAN NUMERAL REVERSED ONE HUNDRED..LATIN SMALL LETTER REVERSED C\n2185..2188 ; XID_Continue # Nl [4] ROMAN NUMERAL SIX LATE FORM..ROMAN NUMERAL ONE HUNDRED THOUSAND\n2C00..2C2E ; XID_Continue # L& [47] GLAGOLITIC CAPITAL LETTER AZU..GLAGOLITIC CAPITAL LETTER LATINATE MYSLITE\n2C30..2C5E ; XID_Continue # L& [47] GLAGOLITIC SMALL LETTER AZU..GLAGOLITIC SMALL LETTER LATINATE MYSLITE\n2C60..2C7B ; XID_Continue # L& [28] LATIN CAPITAL LETTER L WITH DOUBLE BAR..LATIN LETTER SMALL CAPITAL TURNED E\n2C7C..2C7D ; XID_Continue # Lm [2] LATIN SUBSCRIPT SMALL LETTER J..MODIFIER LETTER CAPITAL V\n2C7E..2CE4 ; XID_Continue # L& [103] LATIN CAPITAL LETTER S WITH SWASH TAIL..COPTIC SYMBOL KAI\n2CEB..2CEE ; XID_Continue # L& [4] COPTIC CAPITAL LETTER CRYPTOGRAMMIC SHEI..COPTIC SMALL LETTER CRYPTOGRAMMIC GANGIA\n2CEF..2CF1 ; XID_Continue # Mn [3] COPTIC COMBINING NI ABOVE..COPTIC COMBINING SPIRITUS LENIS\n2CF2..2CF3 ; XID_Continue # L& [2] COPTIC CAPITAL LETTER BOHAIRIC KHEI..COPTIC SMALL LETTER BOHAIRIC KHEI\n2D00..2D25 ; XID_Continue # L& [38] GEORGIAN SMALL LETTER AN..GEORGIAN SMALL LETTER HOE\n2D27 ; XID_Continue # L& GEORGIAN SMALL LETTER YN\n2D2D ; XID_Continue # L& GEORGIAN SMALL LETTER AEN\n2D30..2D67 ; XID_Continue # Lo [56] TIFINAGH LETTER YA..TIFINAGH LETTER YO\n2D6F ; XID_Continue # Lm TIFINAGH MODIFIER LETTER LABIALIZATION MARK\n2D7F ; XID_Continue # Mn TIFINAGH CONSONANT JOINER\n2D80..2D96 ; XID_Continue # Lo [23] ETHIOPIC SYLLABLE LOA..ETHIOPIC SYLLABLE GGWE\n2DA0..2DA6 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE SSA..ETHIOPIC SYLLABLE SSO\n2DA8..2DAE ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE CCA..ETHIOPIC SYLLABLE CCO\n2DB0..2DB6 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE ZZA..ETHIOPIC SYLLABLE ZZO\n2DB8..2DBE ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE CCHA..ETHIOPIC SYLLABLE CCHO\n2DC0..2DC6 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE QYA..ETHIOPIC SYLLABLE QYO\n2DC8..2DCE ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE KYA..ETHIOPIC SYLLABLE KYO\n2DD0..2DD6 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE XYA..ETHIOPIC SYLLABLE XYO\n2DD8..2DDE ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE GYA..ETHIOPIC SYLLABLE GYO\n2DE0..2DFF ; XID_Continue # Mn [32] COMBINING CYRILLIC LETTER BE..COMBINING CYRILLIC LETTER IOTIFIED BIG YUS\n3005 ; XID_Continue # Lm IDEOGRAPHIC ITERATION MARK\n3006 ; XID_Continue # Lo IDEOGRAPHIC CLOSING MARK\n3007 ; XID_Continue # Nl IDEOGRAPHIC NUMBER ZERO\n3021..3029 ; XID_Continue # Nl [9] HANGZHOU NUMERAL ONE..HANGZHOU NUMERAL NINE\n302A..302D ; XID_Continue # Mn [4] IDEOGRAPHIC LEVEL TONE MARK..IDEOGRAPHIC ENTERING TONE MARK\n302E..302F ; XID_Continue # Mc [2] HANGUL SINGLE DOT TONE MARK..HANGUL DOUBLE DOT TONE MARK\n3031..3035 ; XID_Continue # Lm [5] VERTICAL KANA REPEAT MARK..VERTICAL KANA REPEAT MARK LOWER HALF\n3038..303A ; XID_Continue # Nl [3] HANGZHOU NUMERAL TEN..HANGZHOU NUMERAL THIRTY\n303B ; XID_Continue # Lm VERTICAL IDEOGRAPHIC ITERATION MARK\n303C ; XID_Continue # Lo MASU MARK\n3041..3096 ; XID_Continue # Lo [86] HIRAGANA LETTER SMALL A..HIRAGANA LETTER SMALL KE\n3099..309A ; XID_Continue # Mn [2] COMBINING KATAKANA-HIRAGANA VOICED SOUND MARK..COMBINING KATAKANA-HIRAGANA SEMI-VOICED SOUND MARK\n309D..309E ; XID_Continue # Lm [2] HIRAGANA ITERATION MARK..HIRAGANA VOICED ITERATION MARK\n309F ; XID_Continue # Lo HIRAGANA DIGRAPH YORI\n30A1..30FA ; XID_Continue # Lo [90] KATAKANA LETTER SMALL A..KATAKANA LETTER VO\n30FC..30FE ; XID_Continue # Lm [3] KATAKANA-HIRAGANA PROLONGED SOUND MARK..KATAKANA VOICED ITERATION MARK\n30FF ; XID_Continue # Lo KATAKANA DIGRAPH KOTO\n3105..312D ; XID_Continue # Lo [41] BOPOMOFO LETTER B..BOPOMOFO LETTER IH\n3131..318E ; XID_Continue # Lo [94] HANGUL LETTER KIYEOK..HANGUL LETTER ARAEAE\n31A0..31BA ; XID_Continue # Lo [27] BOPOMOFO LETTER BU..BOPOMOFO LETTER ZY\n31F0..31FF ; XID_Continue # Lo [16] KATAKANA LETTER SMALL KU..KATAKANA LETTER SMALL RO\n3400..4DB5 ; XID_Continue # Lo [6582] CJK UNIFIED IDEOGRAPH-3400..CJK UNIFIED IDEOGRAPH-4DB5\n4E00..9FD5 ; XID_Continue # Lo [20950] CJK UNIFIED IDEOGRAPH-4E00..CJK UNIFIED IDEOGRAPH-9FD5\nA000..A014 ; XID_Continue # Lo [21] YI SYLLABLE IT..YI SYLLABLE E\nA015 ; XID_Continue # Lm YI SYLLABLE WU\nA016..A48C ; XID_Continue # Lo [1143] YI SYLLABLE BIT..YI SYLLABLE YYR\nA4D0..A4F7 ; XID_Continue # Lo [40] LISU LETTER BA..LISU LETTER OE\nA4F8..A4FD ; XID_Continue # Lm [6] LISU LETTER TONE MYA TI..LISU LETTER TONE MYA JEU\nA500..A60B ; XID_Continue # Lo [268] VAI SYLLABLE EE..VAI SYLLABLE NG\nA60C ; XID_Continue # Lm VAI SYLLABLE LENGTHENER\nA610..A61F ; XID_Continue # Lo [16] VAI SYLLABLE NDOLE FA..VAI SYMBOL JONG\nA620..A629 ; XID_Continue # Nd [10] VAI DIGIT ZERO..VAI DIGIT NINE\nA62A..A62B ; XID_Continue # Lo [2] VAI SYLLABLE NDOLE MA..VAI SYLLABLE NDOLE DO\nA640..A66D ; XID_Continue # L& [46] CYRILLIC CAPITAL LETTER ZEMLYA..CYRILLIC SMALL LETTER DOUBLE MONOCULAR O\nA66E ; XID_Continue # Lo CYRILLIC LETTER MULTIOCULAR O\nA66F ; XID_Continue # Mn COMBINING CYRILLIC VZMET\nA674..A67D ; XID_Continue # Mn [10] COMBINING CYRILLIC LETTER UKRAINIAN IE..COMBINING CYRILLIC PAYEROK\nA67F ; XID_Continue # Lm CYRILLIC PAYEROK\nA680..A69B ; XID_Continue # L& [28] CYRILLIC CAPITAL LETTER DWE..CYRILLIC SMALL LETTER CROSSED O\nA69C..A69D ; XID_Continue # Lm [2] MODIFIER LETTER CYRILLIC HARD SIGN..MODIFIER LETTER CYRILLIC SOFT SIGN\nA69E..A69F ; XID_Continue # Mn [2] COMBINING CYRILLIC LETTER EF..COMBINING CYRILLIC LETTER IOTIFIED E\nA6A0..A6E5 ; XID_Continue # Lo [70] BAMUM LETTER A..BAMUM LETTER KI\nA6E6..A6EF ; XID_Continue # Nl [10] BAMUM LETTER MO..BAMUM LETTER KOGHOM\nA6F0..A6F1 ; XID_Continue # Mn [2] BAMUM COMBINING MARK KOQNDON..BAMUM COMBINING MARK TUKWENTIS\nA717..A71F ; XID_Continue # Lm [9] MODIFIER LETTER DOT VERTICAL BAR..MODIFIER LETTER LOW INVERTED EXCLAMATION MARK\nA722..A76F ; XID_Continue # L& [78] LATIN CAPITAL LETTER EGYPTOLOGICAL ALEF..LATIN SMALL LETTER CON\nA770 ; XID_Continue # Lm MODIFIER LETTER US\nA771..A787 ; XID_Continue # L& [23] LATIN SMALL LETTER DUM..LATIN SMALL LETTER INSULAR T\nA788 ; XID_Continue # Lm MODIFIER LETTER LOW CIRCUMFLEX ACCENT\nA78B..A78E ; XID_Continue # L& [4] LATIN CAPITAL LETTER SALTILLO..LATIN SMALL LETTER L WITH RETROFLEX HOOK AND BELT\nA78F ; XID_Continue # Lo LATIN LETTER SINOLOGICAL DOT\nA790..A7AD ; XID_Continue # L& [30] LATIN CAPITAL LETTER N WITH DESCENDER..LATIN CAPITAL LETTER L WITH BELT\nA7B0..A7B7 ; XID_Continue # L& [8] LATIN CAPITAL LETTER TURNED K..LATIN SMALL LETTER OMEGA\nA7F7 ; XID_Continue # Lo LATIN EPIGRAPHIC LETTER SIDEWAYS I\nA7F8..A7F9 ; XID_Continue # Lm [2] MODIFIER LETTER CAPITAL H WITH STROKE..MODIFIER LETTER SMALL LIGATURE OE\nA7FA ; XID_Continue # L& LATIN LETTER SMALL CAPITAL TURNED M\nA7FB..A801 ; XID_Continue # Lo [7] LATIN EPIGRAPHIC LETTER REVERSED F..SYLOTI NAGRI LETTER I\nA802 ; XID_Continue # Mn SYLOTI NAGRI SIGN DVISVARA\nA803..A805 ; XID_Continue # Lo [3] SYLOTI NAGRI LETTER U..SYLOTI NAGRI LETTER O\nA806 ; XID_Continue # Mn SYLOTI NAGRI SIGN HASANTA\nA807..A80A ; XID_Continue # Lo [4] SYLOTI NAGRI LETTER KO..SYLOTI NAGRI LETTER GHO\nA80B ; XID_Continue # Mn SYLOTI NAGRI SIGN ANUSVARA\nA80C..A822 ; XID_Continue # Lo [23] SYLOTI NAGRI LETTER CO..SYLOTI NAGRI LETTER HO\nA823..A824 ; XID_Continue # Mc [2] SYLOTI NAGRI VOWEL SIGN A..SYLOTI NAGRI VOWEL SIGN I\nA825..A826 ; XID_Continue # Mn [2] SYLOTI NAGRI VOWEL SIGN U..SYLOTI NAGRI VOWEL SIGN E\nA827 ; XID_Continue # Mc SYLOTI NAGRI VOWEL SIGN OO\nA840..A873 ; XID_Continue # Lo [52] PHAGS-PA LETTER KA..PHAGS-PA LETTER CANDRABINDU\nA880..A881 ; XID_Continue # Mc [2] SAURASHTRA SIGN ANUSVARA..SAURASHTRA SIGN VISARGA\nA882..A8B3 ; XID_Continue # Lo [50] SAURASHTRA LETTER A..SAURASHTRA LETTER LLA\nA8B4..A8C3 ; XID_Continue # Mc [16] SAURASHTRA CONSONANT SIGN HAARU..SAURASHTRA VOWEL SIGN AU\nA8C4 ; XID_Continue # Mn SAURASHTRA SIGN VIRAMA\nA8D0..A8D9 ; XID_Continue # Nd [10] SAURASHTRA DIGIT ZERO..SAURASHTRA DIGIT NINE\nA8E0..A8F1 ; XID_Continue # Mn [18] COMBINING DEVANAGARI DIGIT ZERO..COMBINING DEVANAGARI SIGN AVAGRAHA\nA8F2..A8F7 ; XID_Continue # Lo [6] DEVANAGARI SIGN SPACING CANDRABINDU..DEVANAGARI SIGN CANDRABINDU AVAGRAHA\nA8FB ; XID_Continue # Lo DEVANAGARI HEADSTROKE\nA8FD ; XID_Continue # Lo DEVANAGARI JAIN OM\nA900..A909 ; XID_Continue # Nd [10] KAYAH LI DIGIT ZERO..KAYAH LI DIGIT NINE\nA90A..A925 ; XID_Continue # Lo [28] KAYAH LI LETTER KA..KAYAH LI LETTER OO\nA926..A92D ; XID_Continue # Mn [8] KAYAH LI VOWEL UE..KAYAH LI TONE CALYA PLOPHU\nA930..A946 ; XID_Continue # Lo [23] REJANG LETTER KA..REJANG LETTER A\nA947..A951 ; XID_Continue # Mn [11] REJANG VOWEL SIGN I..REJANG CONSONANT SIGN R\nA952..A953 ; XID_Continue # Mc [2] REJANG CONSONANT SIGN H..REJANG VIRAMA\nA960..A97C ; XID_Continue # Lo [29] HANGUL CHOSEONG TIKEUT-MIEUM..HANGUL CHOSEONG SSANGYEORINHIEUH\nA980..A982 ; XID_Continue # Mn [3] JAVANESE SIGN PANYANGGA..JAVANESE SIGN LAYAR\nA983 ; XID_Continue # Mc JAVANESE SIGN WIGNYAN\nA984..A9B2 ; XID_Continue # Lo [47] JAVANESE LETTER A..JAVANESE LETTER HA\nA9B3 ; XID_Continue # Mn JAVANESE SIGN CECAK TELU\nA9B4..A9B5 ; XID_Continue # Mc [2] JAVANESE VOWEL SIGN TARUNG..JAVANESE VOWEL SIGN TOLONG\nA9B6..A9B9 ; XID_Continue # Mn [4] JAVANESE VOWEL SIGN WULU..JAVANESE VOWEL SIGN SUKU MENDUT\nA9BA..A9BB ; XID_Continue # Mc [2] JAVANESE VOWEL SIGN TALING..JAVANESE VOWEL SIGN DIRGA MURE\nA9BC ; XID_Continue # Mn JAVANESE VOWEL SIGN PEPET\nA9BD..A9C0 ; XID_Continue # Mc [4] JAVANESE CONSONANT SIGN KERET..JAVANESE PANGKON\nA9CF ; XID_Continue # Lm JAVANESE PANGRANGKEP\nA9D0..A9D9 ; XID_Continue # Nd [10] JAVANESE DIGIT ZERO..JAVANESE DIGIT NINE\nA9E0..A9E4 ; XID_Continue # Lo [5] MYANMAR LETTER SHAN GHA..MYANMAR LETTER SHAN BHA\nA9E5 ; XID_Continue # Mn MYANMAR SIGN SHAN SAW\nA9E6 ; XID_Continue # Lm MYANMAR MODIFIER LETTER SHAN REDUPLICATION\nA9E7..A9EF ; XID_Continue # Lo [9] MYANMAR LETTER TAI LAING NYA..MYANMAR LETTER TAI LAING NNA\nA9F0..A9F9 ; XID_Continue # Nd [10] MYANMAR TAI LAING DIGIT ZERO..MYANMAR TAI LAING DIGIT NINE\nA9FA..A9FE ; XID_Continue # Lo [5] MYANMAR LETTER TAI LAING LLA..MYANMAR LETTER TAI LAING BHA\nAA00..AA28 ; XID_Continue # Lo [41] CHAM LETTER A..CHAM LETTER HA\nAA29..AA2E ; XID_Continue # Mn [6] CHAM VOWEL SIGN AA..CHAM VOWEL SIGN OE\nAA2F..AA30 ; XID_Continue # Mc [2] CHAM VOWEL SIGN O..CHAM VOWEL SIGN AI\nAA31..AA32 ; XID_Continue # Mn [2] CHAM VOWEL SIGN AU..CHAM VOWEL SIGN UE\nAA33..AA34 ; XID_Continue # Mc [2] CHAM CONSONANT SIGN YA..CHAM CONSONANT SIGN RA\nAA35..AA36 ; XID_Continue # Mn [2] CHAM CONSONANT SIGN LA..CHAM CONSONANT SIGN WA\nAA40..AA42 ; XID_Continue # Lo [3] CHAM LETTER FINAL K..CHAM LETTER FINAL NG\nAA43 ; XID_Continue # Mn CHAM CONSONANT SIGN FINAL NG\nAA44..AA4B ; XID_Continue # Lo [8] CHAM LETTER FINAL CH..CHAM LETTER FINAL SS\nAA4C ; XID_Continue # Mn CHAM CONSONANT SIGN FINAL M\nAA4D ; XID_Continue # Mc CHAM CONSONANT SIGN FINAL H\nAA50..AA59 ; XID_Continue # Nd [10] CHAM DIGIT ZERO..CHAM DIGIT NINE\nAA60..AA6F ; XID_Continue # Lo [16] MYANMAR LETTER KHAMTI GA..MYANMAR LETTER KHAMTI FA\nAA70 ; XID_Continue # Lm MYANMAR MODIFIER LETTER KHAMTI REDUPLICATION\nAA71..AA76 ; XID_Continue # Lo [6] MYANMAR LETTER KHAMTI XA..MYANMAR LOGOGRAM KHAMTI HM\nAA7A ; XID_Continue # Lo MYANMAR LETTER AITON RA\nAA7B ; XID_Continue # Mc MYANMAR SIGN PAO KAREN TONE\nAA7C ; XID_Continue # Mn MYANMAR SIGN TAI LAING TONE-2\nAA7D ; XID_Continue # Mc MYANMAR SIGN TAI LAING TONE-5\nAA7E..AAAF ; XID_Continue # Lo [50] MYANMAR LETTER SHWE PALAUNG CHA..TAI VIET LETTER HIGH O\nAAB0 ; XID_Continue # Mn TAI VIET MAI KANG\nAAB1 ; XID_Continue # Lo TAI VIET VOWEL AA\nAAB2..AAB4 ; XID_Continue # Mn [3] TAI VIET VOWEL I..TAI VIET VOWEL U\nAAB5..AAB6 ; XID_Continue # Lo [2] TAI VIET VOWEL E..TAI VIET VOWEL O\nAAB7..AAB8 ; XID_Continue # Mn [2] TAI VIET MAI KHIT..TAI VIET VOWEL IA\nAAB9..AABD ; XID_Continue # Lo [5] TAI VIET VOWEL UEA..TAI VIET VOWEL AN\nAABE..AABF ; XID_Continue # Mn [2] TAI VIET VOWEL AM..TAI VIET TONE MAI EK\nAAC0 ; XID_Continue # Lo TAI VIET TONE MAI NUENG\nAAC1 ; XID_Continue # Mn TAI VIET TONE MAI THO\nAAC2 ; XID_Continue # Lo TAI VIET TONE MAI SONG\nAADB..AADC ; XID_Continue # Lo [2] TAI VIET SYMBOL KON..TAI VIET SYMBOL NUENG\nAADD ; XID_Continue # Lm TAI VIET SYMBOL SAM\nAAE0..AAEA ; XID_Continue # Lo [11] MEETEI MAYEK LETTER E..MEETEI MAYEK LETTER SSA\nAAEB ; XID_Continue # Mc MEETEI MAYEK VOWEL SIGN II\nAAEC..AAED ; XID_Continue # Mn [2] MEETEI MAYEK VOWEL SIGN UU..MEETEI MAYEK VOWEL SIGN AAI\nAAEE..AAEF ; XID_Continue # Mc [2] MEETEI MAYEK VOWEL SIGN AU..MEETEI MAYEK VOWEL SIGN AAU\nAAF2 ; XID_Continue # Lo MEETEI MAYEK ANJI\nAAF3..AAF4 ; XID_Continue # Lm [2] MEETEI MAYEK SYLLABLE REPETITION MARK..MEETEI MAYEK WORD REPETITION MARK\nAAF5 ; XID_Continue # Mc MEETEI MAYEK VOWEL SIGN VISARGA\nAAF6 ; XID_Continue # Mn MEETEI MAYEK VIRAMA\nAB01..AB06 ; XID_Continue # Lo [6] ETHIOPIC SYLLABLE TTHU..ETHIOPIC SYLLABLE TTHO\nAB09..AB0E ; XID_Continue # Lo [6] ETHIOPIC SYLLABLE DDHU..ETHIOPIC SYLLABLE DDHO\nAB11..AB16 ; XID_Continue # Lo [6] ETHIOPIC SYLLABLE DZU..ETHIOPIC SYLLABLE DZO\nAB20..AB26 ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE CCHHA..ETHIOPIC SYLLABLE CCHHO\nAB28..AB2E ; XID_Continue # Lo [7] ETHIOPIC SYLLABLE BBA..ETHIOPIC SYLLABLE BBO\nAB30..AB5A ; XID_Continue # L& [43] LATIN SMALL LETTER BARRED ALPHA..LATIN SMALL LETTER Y WITH SHORT RIGHT LEG\nAB5C..AB5F ; XID_Continue # Lm [4] MODIFIER LETTER SMALL HENG..MODIFIER LETTER SMALL U WITH LEFT HOOK\nAB60..AB65 ; XID_Continue # L& [6] LATIN SMALL LETTER SAKHA YAT..GREEK LETTER SMALL CAPITAL OMEGA\nAB70..ABBF ; XID_Continue # L& [80] CHEROKEE SMALL LETTER A..CHEROKEE SMALL LETTER YA\nABC0..ABE2 ; XID_Continue # Lo [35] MEETEI MAYEK LETTER KOK..MEETEI MAYEK LETTER I LONSUM\nABE3..ABE4 ; XID_Continue # Mc [2] MEETEI MAYEK VOWEL SIGN ONAP..MEETEI MAYEK VOWEL SIGN INAP\nABE5 ; XID_Continue # Mn MEETEI MAYEK VOWEL SIGN ANAP\nABE6..ABE7 ; XID_Continue # Mc [2] MEETEI MAYEK VOWEL SIGN YENAP..MEETEI MAYEK VOWEL SIGN SOUNAP\nABE8 ; XID_Continue # Mn MEETEI MAYEK VOWEL SIGN UNAP\nABE9..ABEA ; XID_Continue # Mc [2] MEETEI MAYEK VOWEL SIGN CHEINAP..MEETEI MAYEK VOWEL SIGN NUNG\nABEC ; XID_Continue # Mc MEETEI MAYEK LUM IYEK\nABED ; XID_Continue # Mn MEETEI MAYEK APUN IYEK\nABF0..ABF9 ; XID_Continue # Nd [10] MEETEI MAYEK DIGIT ZERO..MEETEI MAYEK DIGIT NINE\nAC00..D7A3 ; XID_Continue # Lo [11172] HANGUL SYLLABLE GA..HANGUL SYLLABLE HIH\nD7B0..D7C6 ; XID_Continue # Lo [23] HANGUL JUNGSEONG O-YEO..HANGUL JUNGSEONG ARAEA-E\nD7CB..D7FB ; XID_Continue # Lo [49] HANGUL JONGSEONG NIEUN-RIEUL..HANGUL JONGSEONG PHIEUPH-THIEUTH\nF900..FA6D ; XID_Continue # Lo [366] CJK COMPATIBILITY IDEOGRAPH-F900..CJK COMPATIBILITY IDEOGRAPH-FA6D\nFA70..FAD9 ; XID_Continue # Lo [106] CJK COMPATIBILITY IDEOGRAPH-FA70..CJK COMPATIBILITY IDEOGRAPH-FAD9\nFB00..FB06 ; XID_Continue # L& [7] LATIN SMALL LIGATURE FF..LATIN SMALL LIGATURE ST\nFB13..FB17 ; XID_Continue # L& [5] ARMENIAN SMALL LIGATURE MEN NOW..ARMENIAN SMALL LIGATURE MEN XEH\nFB1D ; XID_Continue # Lo HEBREW LETTER YOD WITH HIRIQ\nFB1E ; XID_Continue # Mn HEBREW POINT JUDEO-SPANISH VARIKA\nFB1F..FB28 ; XID_Continue # Lo [10] HEBREW LIGATURE YIDDISH YOD YOD PATAH..HEBREW LETTER WIDE TAV\nFB2A..FB36 ; XID_Continue # Lo [13] HEBREW LETTER SHIN WITH SHIN DOT..HEBREW LETTER ZAYIN WITH DAGESH\nFB38..FB3C ; XID_Continue # Lo [5] HEBREW LETTER TET WITH DAGESH..HEBREW LETTER LAMED WITH DAGESH\nFB3E ; XID_Continue # Lo HEBREW LETTER MEM WITH DAGESH\nFB40..FB41 ; XID_Continue # Lo [2] HEBREW LETTER NUN WITH DAGESH..HEBREW LETTER SAMEKH WITH DAGESH\nFB43..FB44 ; XID_Continue # Lo [2] HEBREW LETTER FINAL PE WITH DAGESH..HEBREW LETTER PE WITH DAGESH\nFB46..FBB1 ; XID_Continue # Lo [108] HEBREW LETTER TSADI WITH DAGESH..ARABIC LETTER YEH BARREE WITH HAMZA ABOVE FINAL FORM\nFBD3..FC5D ; XID_Continue # Lo [139] ARABIC LETTER NG ISOLATED FORM..ARABIC LIGATURE ALEF MAKSURA WITH SUPERSCRIPT ALEF ISOLATED FORM\nFC64..FD3D ; XID_Continue # Lo [218] ARABIC LIGATURE YEH WITH HAMZA ABOVE WITH REH FINAL FORM..ARABIC LIGATURE ALEF WITH FATHATAN ISOLATED FORM\nFD50..FD8F ; XID_Continue # Lo [64] ARABIC LIGATURE TEH WITH JEEM WITH MEEM INITIAL FORM..ARABIC LIGATURE MEEM WITH KHAH WITH MEEM INITIAL FORM\nFD92..FDC7 ; XID_Continue # Lo [54] ARABIC LIGATURE MEEM WITH JEEM WITH KHAH INITIAL FORM..ARABIC LIGATURE NOON WITH JEEM WITH YEH FINAL FORM\nFDF0..FDF9 ; XID_Continue # Lo [10] ARABIC LIGATURE SALLA USED AS KORANIC STOP SIGN ISOLATED FORM..ARABIC LIGATURE SALLA ISOLATED FORM\nFE00..FE0F ; XID_Continue # Mn [16] VARIATION SELECTOR-1..VARIATION SELECTOR-16\nFE20..FE2F ; XID_Continue # Mn [16] COMBINING LIGATURE LEFT HALF..COMBINING CYRILLIC TITLO RIGHT HALF\nFE33..FE34 ; XID_Continue # Pc [2] PRESENTATION FORM FOR VERTICAL LOW LINE..PRESENTATION FORM FOR VERTICAL WAVY LOW LINE\nFE4D..FE4F ; XID_Continue # Pc [3] DASHED LOW LINE..WAVY LOW LINE\nFE71 ; XID_Continue # Lo ARABIC TATWEEL WITH FATHATAN ABOVE\nFE73 ; XID_Continue # Lo ARABIC TAIL FRAGMENT\nFE77 ; XID_Continue # Lo ARABIC FATHA MEDIAL FORM\nFE79 ; XID_Continue # Lo ARABIC DAMMA MEDIAL FORM\nFE7B ; XID_Continue # Lo ARABIC KASRA MEDIAL FORM\nFE7D ; XID_Continue # Lo ARABIC SHADDA MEDIAL FORM\nFE7F..FEFC ; XID_Continue # Lo [126] ARABIC SUKUN MEDIAL FORM..ARABIC LIGATURE LAM WITH ALEF FINAL FORM\nFF10..FF19 ; XID_Continue # Nd [10] FULLWIDTH DIGIT ZERO..FULLWIDTH DIGIT NINE\nFF21..FF3A ; XID_Continue # L& [26] FULLWIDTH LATIN CAPITAL LETTER A..FULLWIDTH LATIN CAPITAL LETTER Z\nFF3F ; XID_Continue # Pc FULLWIDTH LOW LINE\nFF41..FF5A ; XID_Continue # L& [26] FULLWIDTH LATIN SMALL LETTER A..FULLWIDTH LATIN SMALL LETTER Z\nFF66..FF6F ; XID_Continue # Lo [10] HALFWIDTH KATAKANA LETTER WO..HALFWIDTH KATAKANA LETTER SMALL TU\nFF70 ; XID_Continue # Lm HALFWIDTH KATAKANA-HIRAGANA PROLONGED SOUND MARK\nFF71..FF9D ; XID_Continue # Lo [45] HALFWIDTH KATAKANA LETTER A..HALFWIDTH KATAKANA LETTER N\nFF9E..FF9F ; XID_Continue # Lm [2] HALFWIDTH KATAKANA VOICED SOUND MARK..HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK\nFFA0..FFBE ; XID_Continue # Lo [31] HALFWIDTH HANGUL FILLER..HALFWIDTH HANGUL LETTER HIEUH\nFFC2..FFC7 ; XID_Continue # Lo [6] HALFWIDTH HANGUL LETTER A..HALFWIDTH HANGUL LETTER E\nFFCA..FFCF ; XID_Continue # Lo [6] HALFWIDTH HANGUL LETTER YEO..HALFWIDTH HANGUL LETTER OE\nFFD2..FFD7 ; XID_Continue # Lo [6] HALFWIDTH HANGUL LETTER YO..HALFWIDTH HANGUL LETTER YU\nFFDA..FFDC ; XID_Continue # Lo [3] HALFWIDTH HANGUL LETTER EU..HALFWIDTH HANGUL LETTER I\n10000..1000B ; XID_Continue # Lo [12] LINEAR B SYLLABLE B008 A..LINEAR B SYLLABLE B046 JE\n1000D..10026 ; XID_Continue # Lo [26] LINEAR B SYLLABLE B036 JO..LINEAR B SYLLABLE B032 QO\n10028..1003A ; XID_Continue # Lo [19] LINEAR B SYLLABLE B060 RA..LINEAR B SYLLABLE B042 WO\n1003C..1003D ; XID_Continue # Lo [2] LINEAR B SYLLABLE B017 ZA..LINEAR B SYLLABLE B074 ZE\n1003F..1004D ; XID_Continue # Lo [15] LINEAR B SYLLABLE B020 ZO..LINEAR B SYLLABLE B091 TWO\n10050..1005D ; XID_Continue # Lo [14] LINEAR B SYMBOL B018..LINEAR B SYMBOL B089\n10080..100FA ; XID_Continue # Lo [123] LINEAR B IDEOGRAM B100 MAN..LINEAR B IDEOGRAM VESSEL B305\n10140..10174 ; XID_Continue # Nl [53] GREEK ACROPHONIC ATTIC ONE QUARTER..GREEK ACROPHONIC STRATIAN FIFTY MNAS\n101FD ; XID_Continue # Mn PHAISTOS DISC SIGN COMBINING OBLIQUE STROKE\n10280..1029C ; XID_Continue # Lo [29] LYCIAN LETTER A..LYCIAN LETTER X\n102A0..102D0 ; XID_Continue # Lo [49] CARIAN LETTER A..CARIAN LETTER UUU3\n102E0 ; XID_Continue # Mn COPTIC EPACT THOUSANDS MARK\n10300..1031F ; XID_Continue # Lo [32] OLD ITALIC LETTER A..OLD ITALIC LETTER ESS\n10330..10340 ; XID_Continue # Lo [17] GOTHIC LETTER AHSA..GOTHIC LETTER PAIRTHRA\n10341 ; XID_Continue # Nl GOTHIC LETTER NINETY\n10342..10349 ; XID_Continue # Lo [8] GOTHIC LETTER RAIDA..GOTHIC LETTER OTHAL\n1034A ; XID_Continue # Nl GOTHIC LETTER NINE HUNDRED\n10350..10375 ; XID_Continue # Lo [38] OLD PERMIC LETTER AN..OLD PERMIC LETTER IA\n10376..1037A ; XID_Continue # Mn [5] COMBINING OLD PERMIC LETTER AN..COMBINING OLD PERMIC LETTER SII\n10380..1039D ; XID_Continue # Lo [30] UGARITIC LETTER ALPA..UGARITIC LETTER SSU\n103A0..103C3 ; XID_Continue # Lo [36] OLD PERSIAN SIGN A..OLD PERSIAN SIGN HA\n103C8..103CF ; XID_Continue # Lo [8] OLD PERSIAN SIGN AURAMAZDAA..OLD PERSIAN SIGN BUUMISH\n103D1..103D5 ; XID_Continue # Nl [5] OLD PERSIAN NUMBER ONE..OLD PERSIAN NUMBER HUNDRED\n10400..1044F ; XID_Continue # L& [80] DESERET CAPITAL LETTER LONG I..DESERET SMALL LETTER EW\n10450..1049D ; XID_Continue # Lo [78] SHAVIAN LETTER PEEP..OSMANYA LETTER OO\n104A0..104A9 ; XID_Continue # Nd [10] OSMANYA DIGIT ZERO..OSMANYA DIGIT NINE\n10500..10527 ; XID_Continue # Lo [40] ELBASAN LETTER A..ELBASAN LETTER KHE\n10530..10563 ; XID_Continue # Lo [52] CAUCASIAN ALBANIAN LETTER ALT..CAUCASIAN ALBANIAN LETTER KIW\n10600..10736 ; XID_Continue # Lo [311] LINEAR A SIGN AB001..LINEAR A SIGN A664\n10740..10755 ; XID_Continue # Lo [22] LINEAR A SIGN A701 A..LINEAR A SIGN A732 JE\n10760..10767 ; XID_Continue # Lo [8] LINEAR A SIGN A800..LINEAR A SIGN A807\n10800..10805 ; XID_Continue # Lo [6] CYPRIOT SYLLABLE A..CYPRIOT SYLLABLE JA\n10808 ; XID_Continue # Lo CYPRIOT SYLLABLE JO\n1080A..10835 ; XID_Continue # Lo [44] CYPRIOT SYLLABLE KA..CYPRIOT SYLLABLE WO\n10837..10838 ; XID_Continue # Lo [2] CYPRIOT SYLLABLE XA..CYPRIOT SYLLABLE XE\n1083C ; XID_Continue # Lo CYPRIOT SYLLABLE ZA\n1083F..10855 ; XID_Continue # Lo [23] CYPRIOT SYLLABLE ZO..IMPERIAL ARAMAIC LETTER TAW\n10860..10876 ; XID_Continue # Lo [23] PALMYRENE LETTER ALEPH..PALMYRENE LETTER TAW\n10880..1089E ; XID_Continue # Lo [31] NABATAEAN LETTER FINAL ALEPH..NABATAEAN LETTER TAW\n108E0..108F2 ; XID_Continue # Lo [19] HATRAN LETTER ALEPH..HATRAN LETTER QOPH\n108F4..108F5 ; XID_Continue # Lo [2] HATRAN LETTER SHIN..HATRAN LETTER TAW\n10900..10915 ; XID_Continue # Lo [22] PHOENICIAN LETTER ALF..PHOENICIAN LETTER TAU\n10920..10939 ; XID_Continue # Lo [26] LYDIAN LETTER A..LYDIAN LETTER C\n10980..109B7 ; XID_Continue # Lo [56] MEROITIC HIEROGLYPHIC LETTER A..MEROITIC CURSIVE LETTER DA\n109BE..109BF ; XID_Continue # Lo [2] MEROITIC CURSIVE LOGOGRAM RMT..MEROITIC CURSIVE LOGOGRAM IMN\n10A00 ; XID_Continue # Lo KHAROSHTHI LETTER A\n10A01..10A03 ; XID_Continue # Mn [3] KHAROSHTHI VOWEL SIGN I..KHAROSHTHI VOWEL SIGN VOCALIC R\n10A05..10A06 ; XID_Continue # Mn [2] KHAROSHTHI VOWEL SIGN E..KHAROSHTHI VOWEL SIGN O\n10A0C..10A0F ; XID_Continue # Mn [4] KHAROSHTHI VOWEL LENGTH MARK..KHAROSHTHI SIGN VISARGA\n10A10..10A13 ; XID_Continue # Lo [4] KHAROSHTHI LETTER KA..KHAROSHTHI LETTER GHA\n10A15..10A17 ; XID_Continue # Lo [3] KHAROSHTHI LETTER CA..KHAROSHTHI LETTER JA\n10A19..10A33 ; XID_Continue # Lo [27] KHAROSHTHI LETTER NYA..KHAROSHTHI LETTER TTTHA\n10A38..10A3A ; XID_Continue # Mn [3] KHAROSHTHI SIGN BAR ABOVE..KHAROSHTHI SIGN DOT BELOW\n10A3F ; XID_Continue # Mn KHAROSHTHI VIRAMA\n10A60..10A7C ; XID_Continue # Lo [29] OLD SOUTH ARABIAN LETTER HE..OLD SOUTH ARABIAN LETTER THETH\n10A80..10A9C ; XID_Continue # Lo [29] OLD NORTH ARABIAN LETTER HEH..OLD NORTH ARABIAN LETTER ZAH\n10AC0..10AC7 ; XID_Continue # Lo [8] MANICHAEAN LETTER ALEPH..MANICHAEAN LETTER WAW\n10AC9..10AE4 ; XID_Continue # Lo [28] MANICHAEAN LETTER ZAYIN..MANICHAEAN LETTER TAW\n10AE5..10AE6 ; XID_Continue # Mn [2] MANICHAEAN ABBREVIATION MARK ABOVE..MANICHAEAN ABBREVIATION MARK BELOW\n10B00..10B35 ; XID_Continue # Lo [54] AVESTAN LETTER A..AVESTAN LETTER HE\n10B40..10B55 ; XID_Continue # Lo [22] INSCRIPTIONAL PARTHIAN LETTER ALEPH..INSCRIPTIONAL PARTHIAN LETTER TAW\n10B60..10B72 ; XID_Continue # Lo [19] INSCRIPTIONAL PAHLAVI LETTER ALEPH..INSCRIPTIONAL PAHLAVI LETTER TAW\n10B80..10B91 ; XID_Continue # Lo [18] PSALTER PAHLAVI LETTER ALEPH..PSALTER PAHLAVI LETTER TAW\n10C00..10C48 ; XID_Continue # Lo [73] OLD TURKIC LETTER ORKHON A..OLD TURKIC LETTER ORKHON BASH\n10C80..10CB2 ; XID_Continue # L& [51] OLD HUNGARIAN CAPITAL LETTER A..OLD HUNGARIAN CAPITAL LETTER US\n10CC0..10CF2 ; XID_Continue # L& [51] OLD HUNGARIAN SMALL LETTER A..OLD HUNGARIAN SMALL LETTER US\n11000 ; XID_Continue # Mc BRAHMI SIGN CANDRABINDU\n11001 ; XID_Continue # Mn BRAHMI SIGN ANUSVARA\n11002 ; XID_Continue # Mc BRAHMI SIGN VISARGA\n11003..11037 ; XID_Continue # Lo [53] BRAHMI SIGN JIHVAMULIYA..BRAHMI LETTER OLD TAMIL NNNA\n11038..11046 ; XID_Continue # Mn [15] BRAHMI VOWEL SIGN AA..BRAHMI VIRAMA\n11066..1106F ; XID_Continue # Nd [10] BRAHMI DIGIT ZERO..BRAHMI DIGIT NINE\n1107F..11081 ; XID_Continue # Mn [3] BRAHMI NUMBER JOINER..KAITHI SIGN ANUSVARA\n11082 ; XID_Continue # Mc KAITHI SIGN VISARGA\n11083..110AF ; XID_Continue # Lo [45] KAITHI LETTER A..KAITHI LETTER HA\n110B0..110B2 ; XID_Continue # Mc [3] KAITHI VOWEL SIGN AA..KAITHI VOWEL SIGN II\n110B3..110B6 ; XID_Continue # Mn [4] KAITHI VOWEL SIGN U..KAITHI VOWEL SIGN AI\n110B7..110B8 ; XID_Continue # Mc [2] KAITHI VOWEL SIGN O..KAITHI VOWEL SIGN AU\n110B9..110BA ; XID_Continue # Mn [2] KAITHI SIGN VIRAMA..KAITHI SIGN NUKTA\n110D0..110E8 ; XID_Continue # Lo [25] SORA SOMPENG LETTER SAH..SORA SOMPENG LETTER MAE\n110F0..110F9 ; XID_Continue # Nd [10] SORA SOMPENG DIGIT ZERO..SORA SOMPENG DIGIT NINE\n11100..11102 ; XID_Continue # Mn [3] CHAKMA SIGN CANDRABINDU..CHAKMA SIGN VISARGA\n11103..11126 ; XID_Continue # Lo [36] CHAKMA LETTER AA..CHAKMA LETTER HAA\n11127..1112B ; XID_Continue # Mn [5] CHAKMA VOWEL SIGN A..CHAKMA VOWEL SIGN UU\n1112C ; XID_Continue # Mc CHAKMA VOWEL SIGN E\n1112D..11134 ; XID_Continue # Mn [8] CHAKMA VOWEL SIGN AI..CHAKMA MAAYYAA\n11136..1113F ; XID_Continue # Nd [10] CHAKMA DIGIT ZERO..CHAKMA DIGIT NINE\n11150..11172 ; XID_Continue # Lo [35] MAHAJANI LETTER A..MAHAJANI LETTER RRA\n11173 ; XID_Continue # Mn MAHAJANI SIGN NUKTA\n11176 ; XID_Continue # Lo MAHAJANI LIGATURE SHRI\n11180..11181 ; XID_Continue # Mn [2] SHARADA SIGN CANDRABINDU..SHARADA SIGN ANUSVARA\n11182 ; XID_Continue # Mc SHARADA SIGN VISARGA\n11183..111B2 ; XID_Continue # Lo [48] SHARADA LETTER A..SHARADA LETTER HA\n111B3..111B5 ; XID_Continue # Mc [3] SHARADA VOWEL SIGN AA..SHARADA VOWEL SIGN II\n111B6..111BE ; XID_Continue # Mn [9] SHARADA VOWEL SIGN U..SHARADA VOWEL SIGN O\n111BF..111C0 ; XID_Continue # Mc [2] SHARADA VOWEL SIGN AU..SHARADA SIGN VIRAMA\n111C1..111C4 ; XID_Continue # Lo [4] SHARADA SIGN AVAGRAHA..SHARADA OM\n111CA..111CC ; XID_Continue # Mn [3] SHARADA SIGN NUKTA..SHARADA EXTRA SHORT VOWEL MARK\n111D0..111D9 ; XID_Continue # Nd [10] SHARADA DIGIT ZERO..SHARADA DIGIT NINE\n111DA ; XID_Continue # Lo SHARADA EKAM\n111DC ; XID_Continue # Lo SHARADA HEADSTROKE\n11200..11211 ; XID_Continue # Lo [18] KHOJKI LETTER A..KHOJKI LETTER JJA\n11213..1122B ; XID_Continue # Lo [25] KHOJKI LETTER NYA..KHOJKI LETTER LLA\n1122C..1122E ; XID_Continue # Mc [3] KHOJKI VOWEL SIGN AA..KHOJKI VOWEL SIGN II\n1122F..11231 ; XID_Continue # Mn [3] KHOJKI VOWEL SIGN U..KHOJKI VOWEL SIGN AI\n11232..11233 ; XID_Continue # Mc [2] KHOJKI VOWEL SIGN O..KHOJKI VOWEL SIGN AU\n11234 ; XID_Continue # Mn KHOJKI SIGN ANUSVARA\n11235 ; XID_Continue # Mc KHOJKI SIGN VIRAMA\n11236..11237 ; XID_Continue # Mn [2] KHOJKI SIGN NUKTA..KHOJKI SIGN SHADDA\n11280..11286 ; XID_Continue # Lo [7] MULTANI LETTER A..MULTANI LETTER GA\n11288 ; XID_Continue # Lo MULTANI LETTER GHA\n1128A..1128D ; XID_Continue # Lo [4] MULTANI LETTER CA..MULTANI LETTER JJA\n1128F..1129D ; XID_Continue # Lo [15] MULTANI LETTER NYA..MULTANI LETTER BA\n1129F..112A8 ; XID_Continue # Lo [10] MULTANI LETTER BHA..MULTANI LETTER RHA\n112B0..112DE ; XID_Continue # Lo [47] KHUDAWADI LETTER A..KHUDAWADI LETTER HA\n112DF ; XID_Continue # Mn KHUDAWADI SIGN ANUSVARA\n112E0..112E2 ; XID_Continue # Mc [3] KHUDAWADI VOWEL SIGN AA..KHUDAWADI VOWEL SIGN II\n112E3..112EA ; XID_Continue # Mn [8] KHUDAWADI VOWEL SIGN U..KHUDAWADI SIGN VIRAMA\n112F0..112F9 ; XID_Continue # Nd [10] KHUDAWADI DIGIT ZERO..KHUDAWADI DIGIT NINE\n11300..11301 ; XID_Continue # Mn [2] GRANTHA SIGN COMBINING ANUSVARA ABOVE..GRANTHA SIGN CANDRABINDU\n11302..11303 ; XID_Continue # Mc [2] GRANTHA SIGN ANUSVARA..GRANTHA SIGN VISARGA\n11305..1130C ; XID_Continue # Lo [8] GRANTHA LETTER A..GRANTHA LETTER VOCALIC L\n1130F..11310 ; XID_Continue # Lo [2] GRANTHA LETTER EE..GRANTHA LETTER AI\n11313..11328 ; XID_Continue # Lo [22] GRANTHA LETTER OO..GRANTHA LETTER NA\n1132A..11330 ; XID_Continue # Lo [7] GRANTHA LETTER PA..GRANTHA LETTER RA\n11332..11333 ; XID_Continue # Lo [2] GRANTHA LETTER LA..GRANTHA LETTER LLA\n11335..11339 ; XID_Continue # Lo [5] GRANTHA LETTER VA..GRANTHA LETTER HA\n1133C ; XID_Continue # Mn GRANTHA SIGN NUKTA\n1133D ; XID_Continue # Lo GRANTHA SIGN AVAGRAHA\n1133E..1133F ; XID_Continue # Mc [2] GRANTHA VOWEL SIGN AA..GRANTHA VOWEL SIGN I\n11340 ; XID_Continue # Mn GRANTHA VOWEL SIGN II\n11341..11344 ; XID_Continue # Mc [4] GRANTHA VOWEL SIGN U..GRANTHA VOWEL SIGN VOCALIC RR\n11347..11348 ; XID_Continue # Mc [2] GRANTHA VOWEL SIGN EE..GRANTHA VOWEL SIGN AI\n1134B..1134D ; XID_Continue # Mc [3] GRANTHA VOWEL SIGN OO..GRANTHA SIGN VIRAMA\n11350 ; XID_Continue # Lo GRANTHA OM\n11357 ; XID_Continue # Mc GRANTHA AU LENGTH MARK\n1135D..11361 ; XID_Continue # Lo [5] GRANTHA SIGN PLUTA..GRANTHA LETTER VOCALIC LL\n11362..11363 ; XID_Continue # Mc [2] GRANTHA VOWEL SIGN VOCALIC L..GRANTHA VOWEL SIGN VOCALIC LL\n11366..1136C ; XID_Continue # Mn [7] COMBINING GRANTHA DIGIT ZERO..COMBINING GRANTHA DIGIT SIX\n11370..11374 ; XID_Continue # Mn [5] COMBINING GRANTHA LETTER A..COMBINING GRANTHA LETTER PA\n11480..114AF ; XID_Continue # Lo [48] TIRHUTA ANJI..TIRHUTA LETTER HA\n114B0..114B2 ; XID_Continue # Mc [3] TIRHUTA VOWEL SIGN AA..TIRHUTA VOWEL SIGN II\n114B3..114B8 ; XID_Continue # Mn [6] TIRHUTA VOWEL SIGN U..TIRHUTA VOWEL SIGN VOCALIC LL\n114B9 ; XID_Continue # Mc TIRHUTA VOWEL SIGN E\n114BA ; XID_Continue # Mn TIRHUTA VOWEL SIGN SHORT E\n114BB..114BE ; XID_Continue # Mc [4] TIRHUTA VOWEL SIGN AI..TIRHUTA VOWEL SIGN AU\n114BF..114C0 ; XID_Continue # Mn [2] TIRHUTA SIGN CANDRABINDU..TIRHUTA SIGN ANUSVARA\n114C1 ; XID_Continue # Mc TIRHUTA SIGN VISARGA\n114C2..114C3 ; XID_Continue # Mn [2] TIRHUTA SIGN VIRAMA..TIRHUTA SIGN NUKTA\n114C4..114C5 ; XID_Continue # Lo [2] TIRHUTA SIGN AVAGRAHA..TIRHUTA GVANG\n114C7 ; XID_Continue # Lo TIRHUTA OM\n114D0..114D9 ; XID_Continue # Nd [10] TIRHUTA DIGIT ZERO..TIRHUTA DIGIT NINE\n11580..115AE ; XID_Continue # Lo [47] SIDDHAM LETTER A..SIDDHAM LETTER HA\n115AF..115B1 ; XID_Continue # Mc [3] SIDDHAM VOWEL SIGN AA..SIDDHAM VOWEL SIGN II\n115B2..115B5 ; XID_Continue # Mn [4] SIDDHAM VOWEL SIGN U..SIDDHAM VOWEL SIGN VOCALIC RR\n115B8..115BB ; XID_Continue # Mc [4] SIDDHAM VOWEL SIGN E..SIDDHAM VOWEL SIGN AU\n115BC..115BD ; XID_Continue # Mn [2] SIDDHAM SIGN CANDRABINDU..SIDDHAM SIGN ANUSVARA\n115BE ; XID_Continue # Mc SIDDHAM SIGN VISARGA\n115BF..115C0 ; XID_Continue # Mn [2] SIDDHAM SIGN VIRAMA..SIDDHAM SIGN NUKTA\n115D8..115DB ; XID_Continue # Lo [4] SIDDHAM LETTER THREE-CIRCLE ALTERNATE I..SIDDHAM LETTER ALTERNATE U\n115DC..115DD ; XID_Continue # Mn [2] SIDDHAM VOWEL SIGN ALTERNATE U..SIDDHAM VOWEL SIGN ALTERNATE UU\n11600..1162F ; XID_Continue # Lo [48] MODI LETTER A..MODI LETTER LLA\n11630..11632 ; XID_Continue # Mc [3] MODI VOWEL SIGN AA..MODI VOWEL SIGN II\n11633..1163A ; XID_Continue # Mn [8] MODI VOWEL SIGN U..MODI VOWEL SIGN AI\n1163B..1163C ; XID_Continue # Mc [2] MODI VOWEL SIGN O..MODI VOWEL SIGN AU\n1163D ; XID_Continue # Mn MODI SIGN ANUSVARA\n1163E ; XID_Continue # Mc MODI SIGN VISARGA\n1163F..11640 ; XID_Continue # Mn [2] MODI SIGN VIRAMA..MODI SIGN ARDHACANDRA\n11644 ; XID_Continue # Lo MODI SIGN HUVA\n11650..11659 ; XID_Continue # Nd [10] MODI DIGIT ZERO..MODI DIGIT NINE\n11680..116AA ; XID_Continue # Lo [43] TAKRI LETTER A..TAKRI LETTER RRA\n116AB ; XID_Continue # Mn TAKRI SIGN ANUSVARA\n116AC ; XID_Continue # Mc TAKRI SIGN VISARGA\n116AD ; XID_Continue # Mn TAKRI VOWEL SIGN AA\n116AE..116AF ; XID_Continue # Mc [2] TAKRI VOWEL SIGN I..TAKRI VOWEL SIGN II\n116B0..116B5 ; XID_Continue # Mn [6] TAKRI VOWEL SIGN U..TAKRI VOWEL SIGN AU\n116B6 ; XID_Continue # Mc TAKRI SIGN VIRAMA\n116B7 ; XID_Continue # Mn TAKRI SIGN NUKTA\n116C0..116C9 ; XID_Continue # Nd [10] TAKRI DIGIT ZERO..TAKRI DIGIT NINE\n11700..11719 ; XID_Continue # Lo [26] AHOM LETTER KA..AHOM LETTER JHA\n1171D..1171F ; XID_Continue # Mn [3] AHOM CONSONANT SIGN MEDIAL LA..AHOM CONSONANT SIGN MEDIAL LIGATING RA\n11720..11721 ; XID_Continue # Mc [2] AHOM VOWEL SIGN A..AHOM VOWEL SIGN AA\n11722..11725 ; XID_Continue # Mn [4] AHOM VOWEL SIGN I..AHOM VOWEL SIGN UU\n11726 ; XID_Continue # Mc AHOM VOWEL SIGN E\n11727..1172B ; XID_Continue # Mn [5] AHOM VOWEL SIGN AW..AHOM SIGN KILLER\n11730..11739 ; XID_Continue # Nd [10] AHOM DIGIT ZERO..AHOM DIGIT NINE\n118A0..118DF ; XID_Continue # L& [64] WARANG CITI CAPITAL LETTER NGAA..WARANG CITI SMALL LETTER VIYO\n118E0..118E9 ; XID_Continue # Nd [10] WARANG CITI DIGIT ZERO..WARANG CITI DIGIT NINE\n118FF ; XID_Continue # Lo WARANG CITI OM\n11AC0..11AF8 ; XID_Continue # Lo [57] PAU CIN HAU LETTER PA..PAU CIN HAU GLOTTAL STOP FINAL\n12000..12399 ; XID_Continue # Lo [922] CUNEIFORM SIGN A..CUNEIFORM SIGN U U\n12400..1246E ; XID_Continue # Nl [111] CUNEIFORM NUMERIC SIGN TWO ASH..CUNEIFORM NUMERIC SIGN NINE U VARIANT FORM\n12480..12543 ; XID_Continue # Lo [196] CUNEIFORM SIGN AB TIMES NUN TENU..CUNEIFORM SIGN ZU5 TIMES THREE DISH TENU\n13000..1342E ; XID_Continue # Lo [1071] EGYPTIAN HIEROGLYPH A001..EGYPTIAN HIEROGLYPH AA032\n14400..14646 ; XID_Continue # Lo [583] ANATOLIAN HIEROGLYPH A001..ANATOLIAN HIEROGLYPH A530\n16800..16A38 ; XID_Continue # Lo [569] BAMUM LETTER PHASE-A NGKUE MFON..BAMUM LETTER PHASE-F VUEQ\n16A40..16A5E ; XID_Continue # Lo [31] MRO LETTER TA..MRO LETTER TEK\n16A60..16A69 ; XID_Continue # Nd [10] MRO DIGIT ZERO..MRO DIGIT NINE\n16AD0..16AED ; XID_Continue # Lo [30] BASSA VAH LETTER ENNI..BASSA VAH LETTER I\n16AF0..16AF4 ; XID_Continue # Mn [5] BASSA VAH COMBINING HIGH TONE..BASSA VAH COMBINING HIGH-LOW TONE\n16B00..16B2F ; XID_Continue # Lo [48] PAHAWH HMONG VOWEL KEEB..PAHAWH HMONG CONSONANT CAU\n16B30..16B36 ; XID_Continue # Mn [7] PAHAWH HMONG MARK CIM TUB..PAHAWH HMONG MARK CIM TAUM\n16B40..16B43 ; XID_Continue # Lm [4] PAHAWH HMONG SIGN VOS SEEV..PAHAWH HMONG SIGN IB YAM\n16B50..16B59 ; XID_Continue # Nd [10] PAHAWH HMONG DIGIT ZERO..PAHAWH HMONG DIGIT NINE\n16B63..16B77 ; XID_Continue # Lo [21] PAHAWH HMONG SIGN VOS LUB..PAHAWH HMONG SIGN CIM NRES TOS\n16B7D..16B8F ; XID_Continue # Lo [19] PAHAWH HMONG CLAN SIGN TSHEEJ..PAHAWH HMONG CLAN SIGN VWJ\n16F00..16F44 ; XID_Continue # Lo [69] MIAO LETTER PA..MIAO LETTER HHA\n16F50 ; XID_Continue # Lo MIAO LETTER NASALIZATION\n16F51..16F7E ; XID_Continue # Mc [46] MIAO SIGN ASPIRATION..MIAO VOWEL SIGN NG\n16F8F..16F92 ; XID_Continue # Mn [4] MIAO TONE RIGHT..MIAO TONE BELOW\n16F93..16F9F ; XID_Continue # Lm [13] MIAO LETTER TONE-2..MIAO LETTER REFORMED TONE-8\n1B000..1B001 ; XID_Continue # Lo [2] KATAKANA LETTER ARCHAIC E..HIRAGANA LETTER ARCHAIC YE\n1BC00..1BC6A ; XID_Continue # Lo [107] DUPLOYAN LETTER H..DUPLOYAN LETTER VOCALIC M\n1BC70..1BC7C ; XID_Continue # Lo [13] DUPLOYAN AFFIX LEFT HORIZONTAL SECANT..DUPLOYAN AFFIX ATTACHED TANGENT HOOK\n1BC80..1BC88 ; XID_Continue # Lo [9] DUPLOYAN AFFIX HIGH ACUTE..DUPLOYAN AFFIX HIGH VERTICAL\n1BC90..1BC99 ; XID_Continue # Lo [10] DUPLOYAN AFFIX LOW ACUTE..DUPLOYAN AFFIX LOW ARROW\n1BC9D..1BC9E ; XID_Continue # Mn [2] DUPLOYAN THICK LETTER SELECTOR..DUPLOYAN DOUBLE MARK\n1D165..1D166 ; XID_Continue # Mc [2] MUSICAL SYMBOL COMBINING STEM..MUSICAL SYMBOL COMBINING SPRECHGESANG STEM\n1D167..1D169 ; XID_Continue # Mn [3] MUSICAL SYMBOL COMBINING TREMOLO-1..MUSICAL SYMBOL COMBINING TREMOLO-3\n1D16D..1D172 ; XID_Continue # Mc [6] MUSICAL SYMBOL COMBINING AUGMENTATION DOT..MUSICAL SYMBOL COMBINING FLAG-5\n1D17B..1D182 ; XID_Continue # Mn [8] MUSICAL SYMBOL COMBINING ACCENT..MUSICAL SYMBOL COMBINING LOURE\n1D185..1D18B ; XID_Continue # Mn [7] MUSICAL SYMBOL COMBINING DOIT..MUSICAL SYMBOL COMBINING TRIPLE TONGUE\n1D1AA..1D1AD ; XID_Continue # Mn [4] MUSICAL SYMBOL COMBINING DOWN BOW..MUSICAL SYMBOL COMBINING SNAP PIZZICATO\n1D242..1D244 ; XID_Continue # Mn [3] COMBINING GREEK MUSICAL TRISEME..COMBINING GREEK MUSICAL PENTASEME\n1D400..1D454 ; XID_Continue # L& [85] MATHEMATICAL BOLD CAPITAL A..MATHEMATICAL ITALIC SMALL G\n1D456..1D49C ; XID_Continue # L& [71] MATHEMATICAL ITALIC SMALL I..MATHEMATICAL SCRIPT CAPITAL A\n1D49E..1D49F ; XID_Continue # L& [2] MATHEMATICAL SCRIPT CAPITAL C..MATHEMATICAL SCRIPT CAPITAL D\n1D4A2 ; XID_Continue # L& MATHEMATICAL SCRIPT CAPITAL G\n1D4A5..1D4A6 ; XID_Continue # L& [2] MATHEMATICAL SCRIPT CAPITAL J..MATHEMATICAL SCRIPT CAPITAL K\n1D4A9..1D4AC ; XID_Continue # L& [4] MATHEMATICAL SCRIPT CAPITAL N..MATHEMATICAL SCRIPT CAPITAL Q\n1D4AE..1D4B9 ; XID_Continue # L& [12] MATHEMATICAL SCRIPT CAPITAL S..MATHEMATICAL SCRIPT SMALL D\n1D4BB ; XID_Continue # L& MATHEMATICAL SCRIPT SMALL F\n1D4BD..1D4C3 ; XID_Continue # L& [7] MATHEMATICAL SCRIPT SMALL H..MATHEMATICAL SCRIPT SMALL N\n1D4C5..1D505 ; XID_Continue # L& [65] MATHEMATICAL SCRIPT SMALL P..MATHEMATICAL FRAKTUR CAPITAL B\n1D507..1D50A ; XID_Continue # L& [4] MATHEMATICAL FRAKTUR CAPITAL D..MATHEMATICAL FRAKTUR CAPITAL G\n1D50D..1D514 ; XID_Continue # L& [8] MATHEMATICAL FRAKTUR CAPITAL J..MATHEMATICAL FRAKTUR CAPITAL Q\n1D516..1D51C ; XID_Continue # L& [7] MATHEMATICAL FRAKTUR CAPITAL S..MATHEMATICAL FRAKTUR CAPITAL Y\n1D51E..1D539 ; XID_Continue # L& [28] MATHEMATICAL FRAKTUR SMALL A..MATHEMATICAL DOUBLE-STRUCK CAPITAL B\n1D53B..1D53E ; XID_Continue # L& [4] MATHEMATICAL DOUBLE-STRUCK CAPITAL D..MATHEMATICAL DOUBLE-STRUCK CAPITAL G\n1D540..1D544 ; XID_Continue # L& [5] MATHEMATICAL DOUBLE-STRUCK CAPITAL I..MATHEMATICAL DOUBLE-STRUCK CAPITAL M\n1D546 ; XID_Continue # L& MATHEMATICAL DOUBLE-STRUCK CAPITAL O\n1D54A..1D550 ; XID_Continue # L& [7] MATHEMATICAL DOUBLE-STRUCK CAPITAL S..MATHEMATICAL DOUBLE-STRUCK CAPITAL Y\n1D552..1D6A5 ; XID_Continue # L& [340] MATHEMATICAL DOUBLE-STRUCK SMALL A..MATHEMATICAL ITALIC SMALL DOTLESS J\n1D6A8..1D6C0 ; XID_Continue # L& [25] MATHEMATICAL BOLD CAPITAL ALPHA..MATHEMATICAL BOLD CAPITAL OMEGA\n1D6C2..1D6DA ; XID_Continue # L& [25] MATHEMATICAL BOLD SMALL ALPHA..MATHEMATICAL BOLD SMALL OMEGA\n1D6DC..1D6FA ; XID_Continue # L& [31] MATHEMATICAL BOLD EPSILON SYMBOL..MATHEMATICAL ITALIC CAPITAL OMEGA\n1D6FC..1D714 ; XID_Continue # L& [25] MATHEMATICAL ITALIC SMALL ALPHA..MATHEMATICAL ITALIC SMALL OMEGA\n1D716..1D734 ; XID_Continue # L& [31] MATHEMATICAL ITALIC EPSILON SYMBOL..MATHEMATICAL BOLD ITALIC CAPITAL OMEGA\n1D736..1D74E ; XID_Continue # L& [25] MATHEMATICAL BOLD ITALIC SMALL ALPHA..MATHEMATICAL BOLD ITALIC SMALL OMEGA\n1D750..1D76E ; XID_Continue # L& [31] MATHEMATICAL BOLD ITALIC EPSILON SYMBOL..MATHEMATICAL SANS-SERIF BOLD CAPITAL OMEGA\n1D770..1D788 ; XID_Continue # L& [25] MATHEMATICAL SANS-SERIF BOLD SMALL ALPHA..MATHEMATICAL SANS-SERIF BOLD SMALL OMEGA\n1D78A..1D7A8 ; XID_Continue # L& [31] MATHEMATICAL SANS-SERIF BOLD EPSILON SYMBOL..MATHEMATICAL SANS-SERIF BOLD ITALIC CAPITAL OMEGA\n1D7AA..1D7C2 ; XID_Continue # L& [25] MATHEMATICAL SANS-SERIF BOLD ITALIC SMALL ALPHA..MATHEMATICAL SANS-SERIF BOLD ITALIC SMALL OMEGA\n1D7C4..1D7CB ; XID_Continue # L& [8] MATHEMATICAL SANS-SERIF BOLD ITALIC EPSILON SYMBOL..MATHEMATICAL BOLD SMALL DIGAMMA\n1D7CE..1D7FF ; XID_Continue # Nd [50] MATHEMATICAL BOLD DIGIT ZERO..MATHEMATICAL MONOSPACE DIGIT NINE\n1DA00..1DA36 ; XID_Continue # Mn [55] SIGNWRITING HEAD RIM..SIGNWRITING AIR SUCKING IN\n1DA3B..1DA6C ; XID_Continue # Mn [50] SIGNWRITING MOUTH CLOSED NEUTRAL..SIGNWRITING EXCITEMENT\n1DA75 ; XID_Continue # Mn SIGNWRITING UPPER BODY TILTING FROM HIP JOINTS\n1DA84 ; XID_Continue # Mn SIGNWRITING LOCATION HEAD NECK\n1DA9B..1DA9F ; XID_Continue # Mn [5] SIGNWRITING FILL MODIFIER-2..SIGNWRITING FILL MODIFIER-6\n1DAA1..1DAAF ; XID_Continue # Mn [15] SIGNWRITING ROTATION MODIFIER-2..SIGNWRITING ROTATION MODIFIER-16\n1E800..1E8C4 ; XID_Continue # Lo [197] MENDE KIKAKUI SYLLABLE M001 KI..MENDE KIKAKUI SYLLABLE M060 NYON\n1E8D0..1E8D6 ; XID_Continue # Mn [7] MENDE KIKAKUI COMBINING NUMBER TEENS..MENDE KIKAKUI COMBINING NUMBER MILLIONS\n1EE00..1EE03 ; XID_Continue # Lo [4] ARABIC MATHEMATICAL ALEF..ARABIC MATHEMATICAL DAL\n1EE05..1EE1F ; XID_Continue # Lo [27] ARABIC MATHEMATICAL WAW..ARABIC MATHEMATICAL DOTLESS QAF\n1EE21..1EE22 ; XID_Continue # Lo [2] ARABIC MATHEMATICAL INITIAL BEH..ARABIC MATHEMATICAL INITIAL JEEM\n1EE24 ; XID_Continue # Lo ARABIC MATHEMATICAL INITIAL HEH\n1EE27 ; XID_Continue # Lo ARABIC MATHEMATICAL INITIAL HAH\n1EE29..1EE32 ; XID_Continue # Lo [10] ARABIC MATHEMATICAL INITIAL YEH..ARABIC MATHEMATICAL INITIAL QAF\n1EE34..1EE37 ; XID_Continue # Lo [4] ARABIC MATHEMATICAL INITIAL SHEEN..ARABIC MATHEMATICAL INITIAL KHAH\n1EE39 ; XID_Continue # Lo ARABIC MATHEMATICAL INITIAL DAD\n1EE3B ; XID_Continue # Lo ARABIC MATHEMATICAL INITIAL GHAIN\n1EE42 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED JEEM\n1EE47 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED HAH\n1EE49 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED YEH\n1EE4B ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED LAM\n1EE4D..1EE4F ; XID_Continue # Lo [3] ARABIC MATHEMATICAL TAILED NOON..ARABIC MATHEMATICAL TAILED AIN\n1EE51..1EE52 ; XID_Continue # Lo [2] ARABIC MATHEMATICAL TAILED SAD..ARABIC MATHEMATICAL TAILED QAF\n1EE54 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED SHEEN\n1EE57 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED KHAH\n1EE59 ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED DAD\n1EE5B ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED GHAIN\n1EE5D ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED DOTLESS NOON\n1EE5F ; XID_Continue # Lo ARABIC MATHEMATICAL TAILED DOTLESS QAF\n1EE61..1EE62 ; XID_Continue # Lo [2] ARABIC MATHEMATICAL STRETCHED BEH..ARABIC MATHEMATICAL STRETCHED JEEM\n1EE64 ; XID_Continue # Lo ARABIC MATHEMATICAL STRETCHED HEH\n1EE67..1EE6A ; XID_Continue # Lo [4] ARABIC MATHEMATICAL STRETCHED HAH..ARABIC MATHEMATICAL STRETCHED KAF\n1EE6C..1EE72 ; XID_Continue # Lo [7] ARABIC MATHEMATICAL STRETCHED MEEM..ARABIC MATHEMATICAL STRETCHED QAF\n1EE74..1EE77 ; XID_Continue # Lo [4] ARABIC MATHEMATICAL STRETCHED SHEEN..ARABIC MATHEMATICAL STRETCHED KHAH\n1EE79..1EE7C ; XID_Continue # Lo [4] ARABIC MATHEMATICAL STRETCHED DAD..ARABIC MATHEMATICAL STRETCHED DOTLESS BEH\n1EE7E ; XID_Continue # Lo ARABIC MATHEMATICAL STRETCHED DOTLESS FEH\n1EE80..1EE89 ; XID_Continue # Lo [10] ARABIC MATHEMATICAL LOOPED ALEF..ARABIC MATHEMATICAL LOOPED YEH\n1EE8B..1EE9B ; XID_Continue # Lo [17] ARABIC MATHEMATICAL LOOPED LAM..ARABIC MATHEMATICAL LOOPED GHAIN\n1EEA1..1EEA3 ; XID_Continue # Lo [3] ARABIC MATHEMATICAL DOUBLE-STRUCK BEH..ARABIC MATHEMATICAL DOUBLE-STRUCK DAL\n1EEA5..1EEA9 ; XID_Continue # Lo [5] ARABIC MATHEMATICAL DOUBLE-STRUCK WAW..ARABIC MATHEMATICAL DOUBLE-STRUCK YEH\n1EEAB..1EEBB ; XID_Continue # Lo [17] ARABIC MATHEMATICAL DOUBLE-STRUCK LAM..ARABIC MATHEMATICAL DOUBLE-STRUCK GHAIN\n20000..2A6D6 ; XID_Continue # Lo [42711] CJK UNIFIED IDEOGRAPH-20000..CJK UNIFIED IDEOGRAPH-2A6D6\n2A700..2B734 ; XID_Continue # Lo [4149] CJK UNIFIED IDEOGRAPH-2A700..CJK UNIFIED IDEOGRAPH-2B734\n2B740..2B81D ; XID_Continue # Lo [222] CJK UNIFIED IDEOGRAPH-2B740..CJK UNIFIED IDEOGRAPH-2B81D\n2B820..2CEA1 ; XID_Continue # Lo [5762] CJK UNIFIED IDEOGRAPH-2B820..CJK UNIFIED IDEOGRAPH-2CEA1\n2F800..2FA1D ; XID_Continue # Lo [542] CJK COMPATIBILITY IDEOGRAPH-2F800..CJK COMPATIBILITY IDEOGRAPH-2FA1D\nE0100..E01EF ; XID_Continue # Mn [240] VARIATION SELECTOR-17..VARIATION SELECTOR-256\n\n# Total code points: 112333\n\"\n\nmodule Parser =\n open FParsec.Primitives\n open FParsec.CharParsers\n\n let hex2int c = (int c &&& 15) + (int c >>> 6)*9 // hex char to int\n\n let pCodePoint =\n manyMinMaxSatisfyL 4 6 isHex \"codepoint with 4-6 hex digits\"\n |>> fun s ->\n let mutable n = 0\n for i = 0 to s.Length - 1 do\n n <- n*16 + hex2int s[i]\n n\n\n let skipToBeginOfSection name =\n skipCharsTillString (\"Derived Property: \" + name) true System.Int32.MaxValue >>. skipRestOfLine true\n >>. skipMany (nextCharSatisfiesNot isHex >>. skipRestOfLine true)\n\n let str s = pstring s\n\n let range =\n pipe2 pCodePoint (str \"..\" >>. pCodePoint <|>% -1)\n (fun i1 i2 -> if i2 <> -1 then (i1, i2) else (i1, i1))\n .>> (spaces >>. str \";\" >>. skipRestOfLine true)\n\n let sectionEnd = newline >>. str \"# Total code points:\"\n\n let section name = skipToBeginOfSection name >>. many1 range .>> sectionEnd\n\n let xidRanges = section \"XID_Start\" .>>. section \"XID_Continue\"\n\n let parseXIdRanges() = run xidRanges xidProperties\n\nopen FParsec\n\nlet testCharPredicates() =\n let xidStartRanges, xidContinueRanges = match Parser.parseXIdRanges() with\n | CharParsers.Success(ranges, _, _ ) -> ranges\n | CharParsers.Failure(msg,_,_) -> failwith msg\n\n let checkPredicate fBmp fSmp ranges =\n let mutable lastLast = -1\n for (first, last) in ranges do\n for i = lastLast + 1 to first - 1 do\n let b = if i < 0x10000 then fBmp (char i) else fSmp (i - 0x10000)\n let isSurrogate = (i >= 0xd800 && i <= 0xdfff)\n b |> Equal isSurrogate\n for i = first to last do\n let b = if i < 0x10000 then fBmp (char i) else fSmp (i - 0x10000)\n b |> True\n lastLast <- last\n for i = lastLast + 1 to 0x10ffff do\n let b = if i < 0x10000 then fBmp (char i) else fSmp (i - 0x10000)\n let isSurrogate = (i >= 0xd800 && i <= 0xdfff)\n b |> Equal isSurrogate\n\n checkPredicate IdentifierValidator.IsXIdStartOrSurrogate FParsec.IdentifierValidator.IsXIdStartSmp xidStartRanges\n checkPredicate IdentifierValidator.IsXIdContinueOrSurrogate FParsec.IdentifierValidator.IsXIdContinueSmp xidContinueRanges\n\n let iv = new IdentifierValidator()\n let isIdStartOrSurrogateF = iv.IsIdStartOrSurrogateFunc\n let isIdContinueOrSurrogateF = iv.IsIdContinueOrSurrogateFunc\n let isIdContinueOrJoinControlOrSurrogateF = iv.IsIdContinueOrJoinControlOrSurrogateFunc\n\n for i = 0 to 0xffff do\n let c = char i\n\n IdentifierValidator.IsXIdContinueOrJoinControlOrSurrogate(c)\n |> Equal (IdentifierValidator.IsXIdContinueOrSurrogate(c) || (c >= '\\u200C' && c <= '\\u200D'))\n\n isIdStartOrSurrogateF(c) |> Equal (IdentifierValidator.IsXIdStartOrSurrogate(c))\n isIdContinueOrSurrogateF(c) |> Equal (IdentifierValidator.IsXIdContinueOrSurrogate(c))\n isIdContinueOrJoinControlOrSurrogateF(c) |> Equal (IdentifierValidator.IsXIdContinueOrJoinControlOrSurrogate(c))\n\n isIdStartOrSurrogateF '!' |> False\n isIdContinueOrSurrogateF '!' |> False\n isIdContinueOrJoinControlOrSurrogateF '!' |> False\n iv.SetIsAsciiIdNonStartChar('!')\n isIdStartOrSurrogateF '!' |> False\n isIdContinueOrSurrogateF '!' |> True\n isIdContinueOrJoinControlOrSurrogateF '!' |> True\n iv.SetIsAsciiIdStartChar('!')\n isIdStartOrSurrogateF '!' |> True\n isIdContinueOrSurrogateF '!' |> True\n isIdContinueOrJoinControlOrSurrogateF '!' |> True\n\n try IdentifierValidator.IsXIdStartSmp(-1) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdStartSmp(0x100000) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdStartSmp(System.Int32.MinValue) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdStartSmp(System.Int32.MaxValue) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n\n try IdentifierValidator.IsXIdContinueSmp(-1) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdContinueSmp(0x100000) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdContinueSmp(System.Int32.MinValue) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n try IdentifierValidator.IsXIdContinueSmp(System.Int32.MaxValue) |> ignore; Fail()\n with :? System.IndexOutOfRangeException -> ()\n\n\ntype IdFlags = IdentifierValidator.IdentifierCharFlags\n\nlet testIdentifierValidator() =\n let iv = IdentifierValidator()\n\n try iv.SetIsAsciiIdNonStartChar('\\u0000') with :? System.ArgumentOutOfRangeException -> ()\n try iv.SetIsAsciiIdNonStartChar('\\u0080') with :? System.ArgumentOutOfRangeException -> ()\n try iv.SetIsAsciiIdStartChar('\\u0000') with :? System.ArgumentOutOfRangeException -> ()\n try iv.SetIsAsciiIdStartChar('\\u0080') with :? System.ArgumentOutOfRangeException -> ()\n try iv.SetIsAsciiNoIdChar('\\u0000') with :? System.ArgumentOutOfRangeException -> ()\n try iv.SetIsAsciiNoIdChar('\\u0080') with :? System.ArgumentOutOfRangeException -> ()\n\n iv.IsIdStartOrSurrogateFunc('$') |> False\n iv.IsIdContinueOrSurrogateFunc('$') |> False\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('$') |> False\n iv.SetIsAsciiIdStartChar('$')\n iv.IsIdStartOrSurrogateFunc('$') |> True\n iv.IsIdContinueOrSurrogateFunc('$') |> True\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('$') |> True\n iv.SetIsAsciiIdNonStartChar('$')\n iv.IsIdStartOrSurrogateFunc('$') |> False\n iv.IsIdContinueOrSurrogateFunc('$') |> True\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('$') |> True\n iv.SetIsAsciiNoIdChar('$')\n iv.IsIdStartOrSurrogateFunc('$') |> False\n iv.IsIdContinueOrSurrogateFunc('$') |> False\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('$') |> False\n iv.SetIsAsciiIdStartChar('$')\n iv.IsIdStartOrSurrogateFunc('$') |> True\n iv.IsIdContinueOrSurrogateFunc('$') |> True\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('$') |> True\n\n iv.SetIsAsciiIdNonStartChar('?')\n iv.IsIdStartOrSurrogateFunc('?') |> False\n iv.IsIdContinueOrSurrogateFunc('?') |> True\n iv.IsIdContinueOrJoinControlOrSurrogateFunc('?') |> True\n\n let start = IdFlags.NonContinue ||| IdFlags.Continue\n\n let opts = Array.zeroCreate 128\n opts[int '$'] <- start\n opts[int '?'] <- IdFlags.Continue\n for i = 0 to 127 do\n if IdentifierValidator.IsXIdStartOrSurrogate(char i) then\n opts[i] <- start\n elif IdentifierValidator.IsXIdContinueOrSurrogate(char i) then\n opts[i] <- IdFlags.Continue\n\n let isSurrogate (str: string) (index: int) =\n System.Char.IsHighSurrogate(str[index])\n && index + 1 < str.Length\n && System.Char.IsLowSurrogate(str[index + 1])\n\n let isIdStart (opts: IdFlags[]) (str: string) (index: int) =\n let c = (str[index])\n if int c < opts.Length then int (opts[int c] &&& IdFlags.NonContinue) <> 0\n elif not (System.Char.IsSurrogate(c)) then\n IdentifierValidator.IsXIdStartOrSurrogate(c)\n else\n isSurrogate str index\n && IdentifierValidator.IsXIdStartSmp(System.Char.ConvertToUtf32(str, index) - 0x10000)\n\n let isIdContinue (opts: IdFlags[]) allowJoiner (str: string) (index: int) =\n let c = (str[index])\n if int c < opts.Length then int (opts[int c] &&& IdFlags.Continue) <> 0\n elif not (System.Char.IsSurrogate(c)) then\n IdentifierValidator.IsXIdContinueOrSurrogate(c)\n || (allowJoiner && c >= '\\u200c' && c <= '\\u200d')\n else\n isSurrogate str index\n && IdentifierValidator.IsXIdContinueSmp(System.Char.ConvertToUtf32(str, index) - 0x10000)\n\n let isIdentifier (opts: IdFlags[]) allowJoiner (str: string) =\n if str.Length > 0 && isIdStart opts str 0 then\n let rec loop iLast =\n let i = iLast + if isSurrogate str iLast then 2 else 1\n if i < str.Length then\n if isIdContinue opts allowJoiner str i then loop i\n else (false, i)\n else (true, -1)\n loop 0\n else (false, 0)\n\n isIdStart opts \"\\uFB1C\" 0 |> Equal false\n isIdContinue opts false \"\\uFB1C\" 0 |> Equal false\n isIdStart opts \"\\uFB1D\" 0 |> Equal true\n isIdContinue opts false \"\\uFB1D\" 0 |> Equal true\n isIdStart opts \"\\uFB1E\" 0 |> Equal false\n isIdContinue opts false \"\\uFB1E\" 0 |> Equal true\n\n let U = System.Char.ConvertFromUtf32 // the \\Uxxxxxxxx char escapes currently don't work in the F# compiler\n\n isIdStart opts (U 0x000101FD) 0 |> Equal false\n isIdContinue opts false (U 0x000101FD) 0 |> Equal true\n isIdStart opts (U 0x000101FF) 0 |> Equal false\n isIdContinue opts false (U 0x000101FF) 0 |> Equal false\n isIdStart opts (U 0x00010280) 0 |> Equal true\n isIdContinue opts false (U 0x00010280) 0 |> Equal true\n\n let chars = [|\"a\"; \" \"; \"1\"; \"$\"; \"?\"; \"\\u200C\"; \"\\u200D\"; \"\\uFB1C\"; \"\\uFB1D\"; \"\\uFB1E\";\n // We can't use simple string literals for latin-1 chars due to\n // normalization issue that is caused by the F# compiler not flagging\n // strings with chars in the 0x80-0xff range by a trailing 1 in the #US\n // metadata section of assemblies.\n // (The .NET CLR requires a trailing 1 for chars in 0x80-0xff, though\n // ECMA-335 does not, which is arguably an error in the ECMA spec.)\n U 0xa0; U 0xaa; U 0xb2;\n U 0x20a8; U 0x000101FD; U 0x000101FF; U 0x00010280;\n U 0x0001d400; U 0x0001d6c1; U 0x0001d7ce;|]\n\n\n let checkValidate (id: string) =\n let check allowJoiner =\n let isId, errorPos = isIdentifier opts allowJoiner id\n let idN = try id.Normalize(System.Text.NormalizationForm.FormKC)\n with :? System.ArgumentException -> null\n let isIdN, errorPosN = if idN <> null then isIdentifier opts allowJoiner idN\n else false, errorPos\n\n let mutable errorPos1 = 0\n let str1 = iv.ValidateAndNormalize(id, &errorPos1)\n\n if isId then\n str1 |> Equal id\n errorPos1 |> Equal -1\n else\n str1 |> Equal null\n errorPos1 |> Equal errorPos\n\n iv.NormalizationForm <- System.Text.NormalizationForm.FormKC\n let mutable errorPos2 = 0\n let str2 = iv.ValidateAndNormalize(id, &errorPos2)\n\n iv.NormalizeBeforeValidation <- true\n let mutable errorPos3 = 0\n let str3 = iv.ValidateAndNormalize(id, &errorPos3)\n\n iv.NormalizationForm <- enum 0\n iv.NormalizeBeforeValidation <- false\n\n if isId then\n str2 |> Equal idN\n errorPos2 |> Equal -1\n str3 |> Equal idN\n errorPos3 |> Equal -1\n else\n str2 |> Equal null\n errorPos2 |> Equal errorPos\n if isIdN then\n str3 |> Equal idN\n errorPos3 |> Equal -1\n else\n str3 |> Equal null\n errorPos3 |> Equal errorPosN\n\n check false\n iv.AllowJoinControlCharsAsIdContinueChars <- true\n check true\n iv.AllowJoinControlCharsAsIdContinueChars <- false\n\n let mutable errorPos = 0\n iv.NormalizationForm <- System.Text.NormalizationForm.FormC\n iv.ValidateAndNormalize(\"ϒ\\u0308\", &errorPos) |> Equal \"\\u03D4\"\n iv.NormalizationForm <- System.Text.NormalizationForm.FormKC\n iv.ValidateAndNormalize(\"ϒ\\u0308\", &errorPos) |> Equal \"\\u03AB\"\n iv.NormalizationForm <- enum 0\n\n let mutable i = 0\n checkValidate \"\"\n for c1 in chars do\n checkValidate c1\n for c2 in chars do\n checkValidate (System.String.Concat(c1, c2))\n for c3 in chars do\n i <- i + 1\n checkValidate (System.String.Concat(c1, c2, c3))\n\n // check illegal characters\n let d800 = string (char '\\uD800') // \"\\ud800\" doesn't work here\n let dc00 = string (char '\\uDC00')\n checkValidate d800\n checkValidate dc00\n checkValidate (d800 + \"a\")\n checkValidate (dc00 + \"a\")\n checkValidate \"\\uFFFF\"\n checkValidate (\"a\" + d800 + \"a\")\n checkValidate (\"a\" + dc00 + \"a\")\n checkValidate \"a\\uFFFF\"\n checkValidate (\"ab\" + d800)\n checkValidate (\"ab\" + dc00)\n checkValidate (d800 + d800)\n checkValidate (\"a\" + d800 + d800)\n checkValidate \"ab\\uFFFF\"\n\nlet run() =\n testCharPredicates()\n testIdentifierValidator()\n" }, { "path": "Test/OperatorPrecedenceParserTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2008-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.OperatorPrecedenceParserTests\n\nopen FParsec\nopen FParsec.Error\nopen FParsec.Primitives\nopen FParsec.CharParsers\n\nopen FParsec.Test.Test\n\n// the tests for this module are somewhat ad hoc and ugly...\n\ntype Expr = O1 of Expr*Expr\n | O2 of Expr*Expr\n | O3 of Expr*Expr\n | Pre1 of Expr\n | Pre2 of Expr\n | Po1 of Expr\n | Po2 of Expr\n | Val of int\n | T1 of Expr*Expr*Expr\n | T2 of Expr*Expr*Expr\n\ntype Expr2 = Op of (string*Position)*Expr2*Expr2\n | Pre of (string*Position)*Expr2\n | Post of (string*Position)*Expr2\n | Tern of (string*Position)*(string*Position)*Expr2*Expr2*Expr2\n | Value of Position*int\n\n\nlet ws = spaces\nlet ws1 = spaces1\n\nlet testRemove (opp: OperatorPrecedenceParser<_,_,_>) (op: Operator<_,_,_>) =\n try opp.AddOperator(op); Fail()\n with :? System.ArgumentException -> ()\n opp.RemoveOperator(op) |> True\n opp.RemoveOperator(op) |> False\n opp.AddOperator(op)\n match op.Type with\n | OperatorType.Prefix ->\n opp.RemovePrefixOperator(op.String) |> True\n opp.RemovePrefixOperator(op.String) |> False\n | OperatorType.Postfix ->\n opp.RemovePostfixOperator(op.String) |> True\n opp.RemovePostfixOperator(op.String) |> False\n | OperatorType.Infix when not op.IsTernary ->\n opp.RemoveInfixOperator(op.String) |> True\n opp.RemoveInfixOperator(op.String) |> False\n | OperatorType.Infix when op.IsTernary ->\n opp.RemoveTernaryOperator(op.String, op.TernaryRightString) |> True\n opp.RemoveTernaryOperator(op.String, op.TernaryRightString) |> False\n | _ -> Fail()\n\nlet testRemoveSeq rand opp ops =\n let ops = Seq.toArray ops\n shuffleArray rand ops\n for op in ops do\n testRemove opp op\n\nlet testOpParser() =\n let opp = new OperatorPrecedenceParser<_,_,_>()\n let expr = opp.ExpressionParser\n opp.TermParser <- preturn System.Int32.MinValue\n\n // check \"greedy\" op parsing, correct sorting and finding in internal op data structure\n opp.AddOperator(PrefixOperator(\"\\u0302\", ws, 1, true, fun _ -> 0))\n\n opp.AddOperator(PrefixOperator(\"\\u0303\", ws, 1, true, fun _ -> 1))\n opp.AddOperator(PrefixOperator(\"\\u0203\", ws, 1, true, fun _ -> 2))\n opp.AddOperator(PrefixOperator(\"\\u0403\", ws, 1, true, fun _ -> 3))\n opp.AddOperator(PrefixOperator(\"\\u0503\", ws, 1, true, fun _ -> 4))\n opp.AddOperator(PrefixOperator(\"\\u0103\", ws, 1, true, fun _ -> 5))\n\n opp.AddOperator(PrefixOperator(\"\\u0304\", ws, 1, true, fun _ -> -1))\n\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0303\", ws, 1, true, fun _ -> 6))\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0302\", ws, 1, true, fun _ -> 7))\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0304\", ws, 1, true, fun _ -> 8))\n\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0202\", ws, 1, true, fun _ -> 9))\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0203\", ws, 1, true, fun _ -> 10))\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0204\", ws, 1, true, fun _ -> 11))\n\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0404\", ws, 1, true, fun _ -> 12))\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0403\", ws, 1, true, fun _ -> 13))\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0402\", ws, 1, true, fun _ -> 14))\n\n opp.AddOperator(PrefixOperator(\"\\u0503\\u0403\", ws, 1, true, fun _ -> 15))\n opp.AddOperator(PrefixOperator(\"\\u0503\\u0402\", ws, 1, true, fun _ -> 16))\n opp.AddOperator(PrefixOperator(\"\\u0503\\u0404\", ws, 1, true, fun _ -> 17))\n\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0103\\u0103\\u0103\", ws, 1, true, fun _ -> 18))\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0103\\u0103\", ws, 1, true, fun _ -> 19))\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0102\\u0102\", ws, 1, true, fun _ -> 20))\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0102\", ws, 1, true, fun _ -> 21))\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0103\", ws, 1, true, fun _ -> 22))\n opp.AddOperator(PrefixOperator(\"\\u0103\\u0101\", ws, 1, true, fun _ -> 23))\n\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0303\\u0303\", ws, 1, true, fun _ -> 24))\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0303\\u0303\\u0303\", ws, 1, true, fun _ -> 25))\n opp.AddOperator(PrefixOperator(\"\\u0303\\u0302\\u0302\", ws, 1, true, fun _ -> 26))\n\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0202\\u0202\\u0202\", ws, 1, true, fun _ -> 27))\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0202\\u0202\", ws, 1, true, fun _ -> 28))\n opp.AddOperator(PrefixOperator(\"\\u0203\\u0203\\u0203\", ws, 1, true, fun _ -> 29))\n\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0403\\u0403\", ws, 1, true, fun _ -> 30))\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0402\\u0402\", ws, 1, true, fun _ -> 31))\n opp.AddOperator(PrefixOperator(\"\\u0403\\u0402\\u0402\\u402\", ws, 1, true, fun _ -> 32))\n\n opp.AddOperator(PrefixOperator(\"\\u0603\\u0602\", ws, 1, true, fun _ -> 33))\n opp.AddOperator(PrefixOperator(\"\\u0603\\u0603\", ws, 1, true, fun _ -> 34))\n\n let expectedPrefix = Errors.ExpectedPrefixOperator\n\n let ROk content result parser = ROkE content content.Length result expectedPrefix parser\n let ROkI content i result parser = ROkE content i result expectedPrefix parser\n\n expr |> ROkI \"\\u0301\" 0 System.Int32.MinValue\n\n expr |> ROk \"\\u0302\" 0\n\n expr |> ROk \"\\u0303\" 1\n expr |> ROk \"\\u0203\" 2\n expr |> ROk \"\\u0403\" 3\n expr |> ROk \"\\u0503\" 4\n expr |> ROk \"\\u0103\" 5\n\n expr |> ROkI \"\\u0003\" 0 System.Int32.MinValue\n expr |> ROkI \"\\u0703\" 0 System.Int32.MinValue\n\n expr |> ROk \"\\u0304\" -1\n\n expr |> ROk \"\\u0303\\u0303\" 6\n expr |> ROk \"\\u0303\\u0302\" 7\n expr |> ROk \"\\u0303\\u0304\" 8\n\n expr |> ROkI \"\\u0003\\u0303\" 0 System.Int32.MinValue\n expr |> ROkI \"\\u0703\\u0302\" 0 System.Int32.MinValue\n\n expr |> ROkI \"\\u0603\\u0601\" 0 System.Int32.MinValue\n expr |> ROk \"\\u0603\\u0602\" 33\n expr |> ROk \"\\u0603\\u0603\" 34\n expr |> ROkI \"\\u0603\\u0604\" 0 System.Int32.MinValue\n\n expr |> ROk \"\\u0203\\u0202\" 9\n expr |> ROk \"\\u0203\\u0203\" 10\n expr |> ROk \"\\u0203\\u0204\" 11\n\n expr |> ROk \"\\u0403\\u0404\" 12\n expr |> ROk \"\\u0403\\u0403\" 13\n expr |> ROk \"\\u0403\\u0402\" 14\n\n expr |> ROk \"\\u0503\\u0403\" 15\n expr |> ROk \"\\u0503\\u0402\" 16\n expr |> ROk \"\\u0503\\u0404\" 17\n\n expr |> ROk \"\\u0103\\u0103\\u0103\\u0103\" 18\n expr |> ROk \"\\u0103\\u0103\\u0103\" 19\n expr |> ROkI \"\\u0103\\u0103\\u0103\\u0102\" 3 19\n expr |> ROk \"\\u0103\\u0102\\u0102\" 20\n expr |> ROk \"\\u0103\\u0102\" 21\n expr |> ROk \"\\u0103\\u0103\" 22\n expr |> ROk \"\\u0103\\u0101\" 23\n expr |> ROkI \"\\u0103\\u0101\\u0102\" 2 23\n\n expr |> ROk \"\\u0303\\u0303\\u0303\" 24\n expr |> ROk \"\\u0303\\u0303\\u0303\\u0302\" 24\n expr |> ROk \"\\u0303\\u0303\\u0303\\u0303\" 25\n expr |> ROk \"\\u0303\\u0302\\u0302\" 26\n\n expr |> ROk \"\\u0203\\u0202\\u0202\\u0202\" 27\n expr |> ROk \"\\u0203\\u0202\\u0202\" 28\n expr |> ROkI \"\\u0203\\u0202\\u0202\\u0201\" 3 28\n expr |> ROk \"\\u0203\\u0203\\u0203\" 29\n\n expr |> ROk \"\\u0403\\u0403\\u0403\" 30\n expr |> ROk \"\\u0403\\u0402\\u0402\" 31\n expr |> ROk \"\\u0403\\u0402\\u0402\\u402\" 32\n expr |> ROkI \"\\u0403\\u0402\\u0402\\u0401\" 3 31\n\n // check whitespace parsing and parser state propagation\n\n let withMsg m p = p .>> (fail m <|>% ())\n\n\n let testPrefixOpParser wsParser termParser =\n opp.AddOperator(PrefixOperator(\"+\", wsParser, 1, true, fun x -> x + 1))\n opp.TermParser <- termParser\n let expr = opp.ExpressionParser\n let expr2 = pipe2 (many (pchar '+' >>? wsParser Strings.PrefixOperator)) termParser\n (fun ps i -> i + List.length ps)\n\n checkParserStr expr expr2 \"\"\n checkParserStr expr expr2 \"+\"\n checkParserStr expr expr2 \"1\"\n checkParserStr expr expr2 \"+\"\n checkParserStr expr expr2 \"+ \"\n checkParserStr expr expr2 \"+1\"\n checkParserStr expr expr2 \"+ 1\"\n checkParserStr expr expr2 \"++\"\n checkParserStr expr expr2 \"+ +\"\n checkParserStr expr expr2 \"+ + \"\n checkParserStr expr expr2 \"++1\"\n checkParserStr expr expr2 \"++ 1\"\n checkParserStr expr expr2 \"+ + 1\"\n\n opp.RemovePrefixOperator(\"+\") |> True\n\n\n testPrefixOpParser (ws |> withMsg \"e1\") (preturn 0 |> withMsg \"e2\")\n testPrefixOpParser (ws |> withMsg \"e1\") (fail \"e2\")\n testPrefixOpParser (ws |> withMsg \"e1\") (failFatally \"e2\")\n testPrefixOpParser (ws |> withMsg \"e1\") (preturn 0 |> withMsg \"e2\")\n testPrefixOpParser (fail \"e1\") (preturn 0 |> withMsg \"e2\" )\n testPrefixOpParser (failFatally \"e1\") (preturn 0 |> withMsg \"e2\" )\n testPrefixOpParser (ws |> withMsg \"e1\") (pint32 |> withMsg \"e2\")\n testPrefixOpParser (ws1 |> withMsg \"e1\") (preturn 0 |> withMsg \"e2\")\n testPrefixOpParser (ws1 >>. fail \"e1\") (preturn 0 |> withMsg \"e2\")\n testPrefixOpParser (ws1 |> withMsg \"e1\") (pint32 |> withMsg \"e2\")\n\n let testInfixOpParser wsParser termParser =\n opp.AddOperator(InfixOperator(\"+\", wsParser, 1, Associativity.Left, fun x y -> x + y))\n opp.TermParser <- termParser\n let expr = opp.ExpressionParser\n\n let expect label = preturn () label\n let term = expect Strings.PrefixOperator >>. termParser\n let infixOp = pstring \"+\" >>? wsParser Strings.InfixOperator\n let expr2 = pipe2 term ((infixOp >>. (term .>> (opt infixOp))) <|>% 0)\n (fun x y -> x + y)\n\n checkParserStr expr expr2 \"+\"\n checkParserStr expr expr2 \"+ \"\n checkParserStr expr expr2 \"1+\"\n checkParserStr expr expr2 \"1 +\"\n checkParserStr expr expr2 \"1 + \"\n checkParserStr expr expr2 \"1+2\"\n checkParserStr expr expr2 \"1 +2\"\n checkParserStr expr expr2 \"1 + 2\"\n checkParserStr expr expr2 \"1 + 2\"\n checkParserStr expr expr2 \"1+2 \"\n checkParserStr expr expr2 \"1 +2 \"\n checkParserStr expr expr2 \"1 + 2 \"\n checkParserStr expr expr2 \"1 + 2 \"\n\n opp.RemoveOperator(InfixOperator(\"+\", wsParser, 1, Associativity.Left, fun x y -> x + y)) |> False\n opp.RemoveInfixOperator(\"+\") |> True\n\n testInfixOpParser (ws |> withMsg \"e1\") (preturn 0 |> withMsg \"e2\")\n testInfixOpParser (fail \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n testInfixOpParser (failFatally \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n testInfixOpParser (ws |> withMsg \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n testInfixOpParser (ws1 |> withMsg \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n testInfixOpParser (ws1 >>. fail \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n testInfixOpParser (ws1 |> withMsg \"e1\") (pint32 .>> ws |> withMsg \"e2\")\n\n\n let testTernary2ndOpParser opWsParser =\n let wsm = (ws |> withMsg \"e1\")\n\n let term = (pint32 |> withMsg \"e2\") .>> wsm\n opp.TermParser <- term\n opp.AddOperator(TernaryOperator(\"?\", wsm, \":\", opWsParser, 1, Associativity.Left, fun x y z -> x + y + z))\n opp.MissingTernary2ndStringErrorFormatter <- fun (_, _, op, _) -> expected op.TernaryRightString\n\n let expr2 =\n let op str wsParser = skipString str >>? wsParser\n let expect label = preturn () label\n let term = expect \"prefix operator\" >>. term\n let op1 = op \"?\" wsm \"infix operator\"\n let op2 = expect Strings.InfixOperator >>. (op \":\" opWsParser \":\")\n pipe2 term (tuple2 (op1 >>. term) (op2 >>. term .>> expect Strings.InfixOperator) <|>% (0,0))\n (fun x (y,z) -> x + y + z)\n\n checkParserStr expr expr2 \"1 ?\"\n checkParserStr expr expr2 \"1 ?\"\n checkParserStr expr expr2 \"1 ?: 3\"\n checkParserStr expr expr2 \"1 ? : 3\"\n checkParserStr expr expr2 \"1 ? 2\"\n checkParserStr expr expr2 \"1 ? 2 \"\n checkParserStr expr expr2 \"1 ? 2: \"\n checkParserStr expr expr2 \"1 ? 2 :\"\n checkParserStr expr expr2 \"1 ? 2:3\"\n checkParserStr expr expr2 \"1 ? 2: 3\"\n checkParserStr expr expr2 \"1 ? 2 :3\"\n checkParserStr expr expr2 \"1 ? 2 : 3\"\n\n opp.RemoveTernaryOperator(\"?\", \":\") |> True\n\n testTernary2ndOpParser (ws |> withMsg \"e\")\n testTernary2ndOpParser (ws1 |> withMsg \"e\")\n testTernary2ndOpParser (fail \"e\")\n testTernary2ndOpParser (failFatally \"e\")\n testTernary2ndOpParser (ws1 >>. fail \"e\")\n\n let rand = new System.Random(1234)\n testRemoveSeq rand opp opp.Operators\n\n\nlet testConflictAfterStringParserHandling() =\n let opp = new OperatorPrecedenceParser<_,_,_>()\n let expr = opp.ExpressionParser\n opp.TermParser <- pint32\n\n let conflictError = messageError \"conflict\"\n opp.OperatorConflictErrorFormatter <- fun _ _ -> conflictError\n opp.AddOperator(PrefixOperator(\"+\", spaces, 1, false, fun x -> x + 1))\n opp.AddOperator(PrefixOperator(\"++\", spaces, 1, true, fun x -> x + 2))\n\n expr |> ROk \"+ ++1\" 5 4\n\n opp.RemovePrefixOperator(\"++\") |> True\n opp.AddOperator(PrefixOperator(\"++\", spaces, 1, false, fun x -> x + 2))\n\n expr |> RError \"+ ++1\" 2 conflictError\n\n opp.RemovePrefixOperator(\"++\") |> True\n opp.AddOperator(PrefixOperator(\"++\", spaces1, 1, false, fun x -> x + 2))\n\n expr |> RError \"+ ++1\" 2 (mergeErrors Errors.ExpectedPrefixOperator Errors.ExpectedInt32)\n\n opp.RemovePrefixOperator(\"++\") |> True\n opp.AddOperator(PrefixOperator(\"++\", failFatally \"e\", 1, false, fun x -> x + 2))\n\n expr |> RFatalError \"+ ++1\" 4 (messageError \"e\")\n\n opp.RemovePrefixOperator(\"++\") |> True\n\n opp.AddOperator(InfixOperator(\"+\", spaces, 1, Associativity.Left, fun x y -> x + y))\n opp.AddOperator(InfixOperator(\"-\", spaces, 1, Associativity.Right, fun x y -> x - y))\n\n expr |> RError \"1+2- 3\" 3 conflictError\n\n opp.RemoveInfixOperator(\"-\") |> True\n opp.AddOperator(InfixOperator(\"-\", spaces1, 1, Associativity.Right, fun x y -> x - y))\n\n expr |> ROkE \"1+2-3\" 3 3 Errors.ExpectedInfixOperator\n\n opp.RemoveInfixOperator(\"-\") |> True\n opp.AddOperator(InfixOperator(\"-\", failFatally \"e\", 1, Associativity.Right, fun x y -> x - y))\n expr |> RFatalError \"1+2- 3\" 4 (messageError \"e\")\n\nlet testAlternativeOpConstructors() =\n let opp = new OperatorPrecedenceParser<_,_,_>()\n let expr = opp.ExpressionParser\n let str = skipString\n\n let posWS = getPosition .>> ws\n\n let term = pipe2 getPosition (pint32 .>> ws) (fun pos x -> Value(pos, x))\n opp.TermParser <- term\n\n opp.AddOperator(PrefixOperator(\"-\", posWS, 1, true, (), fun pos x -> Pre((\"-\", pos), x)))\n opp.AddOperator(PostfixOperator(\"++\", posWS, 1, true, (), fun pos x -> Post((\"++\", pos), x)))\n opp.AddOperator(InfixOperator(\"*\", posWS, 1, Associativity.Left, (), fun pos x y -> Op((\"*\", pos), x, y)))\n opp.AddOperator(TernaryOperator(\"?\", posWS, \":\", posWS, 1, Associativity.Left, (), fun pos1 pos2 x y z -> Tern((\"?\", pos1), (\":\", pos2), x, y, z)))\n\n let op = many1SatisfyL (fun c -> match c with '*' | '+' | '-' | '?' | ':' -> true | _ -> false) \"operator\" .>>. posWS\n let expectInfixOrPostfix = fun stream -> Reply(Ok, Errors.ExpectedInfixOrPostfixOperator)\n let expr2 =\n pipe3 (tuple4 term op term op) (tuple2 op term) (tuple2 op (tuple2 op term))\n (fun (v12, multOp, v3, plusPlusOp) (qMarkOp, v4) (colonOp, (minusOp, v5)) ->\n Tern(qMarkOp, colonOp, Op(multOp, v12, Post(plusPlusOp, v3)), v4, Pre(minusOp, v5)))\n .>> expectInfixOrPostfix\n\n checkParserStr expr expr2 \"12 * 3++ ? 4 : -5\"\n\n let rand = new System.Random(1234)\n testRemoveSeq rand opp opp.Operators\n\n\nlet testPrecAndAssoc() =\n let opp = new OperatorPrecedenceParser<_,_,_>()\n let expr = opp.ExpressionParser\n\n opp.TermParser <- pint32 .>> ws |>> Val\n\n opp.AddOperator(InfixOperator(\"o1l\", ws, 1, Associativity.Left, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o1r\", ws, 1, Associativity.Right, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o1n\", ws, 1, Associativity.None, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o2l\", ws, 1, Associativity.Left, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o2r\", ws, 1, Associativity.Right, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o2n\", ws, 1, Associativity.None, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o3l\", ws, 1, Associativity.Left, fun x y -> O3(x,y)))\n opp.AddOperator(InfixOperator(\"o3r\", ws, 1, Associativity.Right, fun x y -> O3(x,y)))\n\n opp.AddOperator(InfixOperator(\"o1l*\", ws, 2, Associativity.Left, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o1r*\", ws, 2, Associativity.Right, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o1n*\", ws, 2, Associativity.None, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o2l*\", ws, 2, Associativity.Left, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o2r*\", ws, 2, Associativity.Right, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o2n*\", ws, 2, Associativity.None, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o3l*\", ws, 2, Associativity.Left, fun x y -> O3(x,y)))\n opp.AddOperator(InfixOperator(\"o3r*\", ws, 2, Associativity.Right, fun x y -> O3(x,y)))\n\n opp.AddOperator(InfixOperator(\"o1l**\", ws, 3, Associativity.Left, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o2l**\", ws, 3, Associativity.Left, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o3l**\", ws, 3, Associativity.Left, fun x y -> O3(x,y)))\n opp.AddOperator(InfixOperator(\"o1r**\", ws, 3, Associativity.Right, fun x y -> O1(x,y)))\n opp.AddOperator(InfixOperator(\"o2r**\", ws, 3, Associativity.Right, fun x y -> O2(x,y)))\n opp.AddOperator(InfixOperator(\"o3r**\", ws, 3, Associativity.Right, fun x y -> O3(x,y)))\n\n opp.AddOperator(TernaryOperator(\"t1l\", ws, \"tt1l\", ws, 1, Associativity.Left, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t1r\", ws, \"tt1r\", ws, 1, Associativity.Right, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t1n\", ws, \"tt1n\", ws, 1, Associativity.None, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2l\", ws, \"tt2l\", ws, 1, Associativity.Left, fun x y z -> T2(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2r\", ws, \"tt2r\", ws, 1, Associativity.Right, fun x y z -> T2(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2n\", ws, \"tt2n\", ws, 1, Associativity.None, fun x y z -> T2(x,y,z)))\n\n opp.AddOperator(TernaryOperator(\"t1l*\", ws, \"tt1l*\", ws, 2, Associativity.Left, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t1l**\", ws, \"tt1l**\", ws, 3, Associativity.Left, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t1r*\", ws, \"tt1r*\", ws, 2, Associativity.Right, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t1n*\", ws, \"tt1n*\", ws, 2, Associativity.None, fun x y z -> T1(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2l*\", ws, \"tt2l*\", ws, 2, Associativity.Left, fun x y z -> T2(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2r*\", ws, \"tt2r*\", ws, 2, Associativity.Right, fun x y z -> T2(x,y,z)))\n opp.AddOperator(TernaryOperator(\"t2n*\", ws, \"tt2n*\", ws, 2, Associativity.None, fun x y z -> T2(x,y,z)))\n\n let poOp1 = PostfixOperator(\"po1\", ws, 1, true, fun x -> Po1(x))\n opp.AddOperator(poOp1)\n opp.AddOperator(PostfixOperator(\"po1n\", ws, 1, false, fun x -> Po1(x)))\n opp.AddOperator(PostfixOperator(\"po2\", ws, 1, true, fun x -> Po2(x)))\n opp.AddOperator(PostfixOperator(\"po2n\", ws, 1, false, fun x -> Po2(x)))\n\n opp.AddOperator(PostfixOperator(\"po1*\", ws, 2, true, fun x -> Po1(x)))\n opp.AddOperator(PostfixOperator(\"po1n*\", ws, 2, false, fun x -> Po1(x)))\n opp.AddOperator(PostfixOperator(\"po2*\", ws, 2, true, fun x -> Po2(x)))\n opp.AddOperator(PostfixOperator(\"po2n*\", ws, 2, false, fun x -> Po2(x)))\n\n // do some tests without prefix operators defined (there's a separate code branch in OPP.ParseExpression)\n let expectedInfixOrPostfix = Errors.ExpectedInfixOrPostfixOperator\n let ROk content result parser = ROkE content content.Length result expectedInfixOrPostfix parser\n\n expr |> RError \"\" 0 Errors.ExpectedInt32\n expr |> ROk \"1 o1l 2 o2l 3\" (O2(O1(Val(1),Val(2)),Val(3)))\n expr |> ROk \"1 o1r* 2 o2r 3\" (O2(O1(Val(1),Val(2)),Val(3)))\n expr |> ROk \"1 o1r 2 o2r 3\" (O1(Val(1),O2(Val(2),Val(3))))\n expr |> ROk \"1 o1l 2 o2l* 3\" (O1(Val(1),O2(Val(2),Val(3))))\n expr |> ROk \"1 o1n 2 po1n\" (O1(Val(1), Po1(Val(2))))\n\n // add prefix operators\n\n opp.AddOperator(PrefixOperator(\"pre1\", ws, 1, true, fun x -> Pre1(x)))\n\n expr |> RError \"po1\" 0 (ErrorMessageList.Merge(Errors.ExpectedPrefixOperator, Errors.ExpectedInt32))\n\n opp.AddOperator(PrefixOperator(\"pre1n\", ws, 1, false, fun x -> Pre1(x)))\n opp.AddOperator(PrefixOperator(\"pre2\", ws, 1, true, fun x -> Pre2(x)))\n opp.AddOperator(PrefixOperator(\"pre2n\", ws, 1, false, fun x -> Pre2(x)))\n\n opp.AddOperator(PrefixOperator(\"pre1*\", ws, 2, true, fun x -> Pre1(x)))\n opp.AddOperator(PrefixOperator(\"pre1n*\", ws, 2, false, fun x -> Pre1(x)))\n opp.AddOperator(PrefixOperator(\"pre2*\", ws, 2, true, fun x -> Pre2(x)))\n opp.AddOperator(PrefixOperator(\"pre2n*\", ws, 2, false, fun x -> Pre2(x)))\n\n // add operators a second time with opposite fixity\n\n opp.AddOperator(PrefixOperator(\"o1l\", ws, 1, true, fun x -> failwith \"o1l\"))\n opp.AddOperator(PrefixOperator(\"o1r\", ws, 1, true, fun x -> failwith \"o1r\"))\n opp.AddOperator(PrefixOperator(\"o1n\", ws, 1, true, fun x -> failwith \"o1n\"))\n opp.AddOperator(PrefixOperator(\"o2l\", ws, 1, true, fun x -> failwith \"o2l\"))\n opp.AddOperator(PrefixOperator(\"o2r\", ws, 1, true, fun x -> failwith \"o2r\"))\n opp.AddOperator(PrefixOperator(\"o2n\", ws, 1, true, fun x -> failwith \"o2n\"))\n\n opp.AddOperator(PrefixOperator(\"o1l*\", ws, 2, true, fun x -> failwith \"o1l*\"))\n opp.AddOperator(PrefixOperator(\"o1l**\", ws, 3, true, fun x -> failwith \"o1l**\"))\n opp.AddOperator(PrefixOperator(\"o1r*\", ws, 2, true, fun x -> failwith \"o1r*\"))\n opp.AddOperator(PrefixOperator(\"o1n*\", ws, 2, true, fun x -> failwith \"o1n*\"))\n opp.AddOperator(PrefixOperator(\"o2l*\", ws, 2, true, fun x -> failwith \"o2l*\"))\n opp.AddOperator(PrefixOperator(\"o2r*\", ws, 2, true, fun x -> failwith \"o2r*\"))\n opp.AddOperator(PrefixOperator(\"o2n*\", ws, 2, true, fun x -> failwith \"o2n*\"))\n\n opp.AddOperator(PrefixOperator(\"t1l\", ws, 1, true, fun x -> failwith \"t1l\"))\n opp.AddOperator(PrefixOperator(\"t1r\", ws, 1, true, fun x -> failwith \"t1r\"))\n opp.AddOperator(PrefixOperator(\"t1n\", ws, 1, true, fun x -> failwith \"t1n\"))\n opp.AddOperator(PrefixOperator(\"t2l\", ws, 1, true, fun x -> failwith \"t2l\"))\n opp.AddOperator(PrefixOperator(\"t2r\", ws, 1, true, fun x -> failwith \"t2r\"))\n opp.AddOperator(PrefixOperator(\"t2n\", ws, 1, true, fun x -> failwith \"t2n\"))\n\n opp.AddOperator(PrefixOperator(\"t1l*\", ws, 2, true, fun x -> failwith \"t1l*\"))\n opp.AddOperator(PrefixOperator(\"t1l**\", ws, 3, true, fun x -> failwith \"t1l**\"))\n opp.AddOperator(PrefixOperator(\"t1r*\", ws, 2, true, fun x -> failwith \"t1r*\"))\n opp.AddOperator(PrefixOperator(\"t1n*\", ws, 2, true, fun x -> failwith \"t1n*\"))\n opp.AddOperator(PrefixOperator(\"t2l*\", ws, 2, true, fun x -> failwith \"t2l*\"))\n opp.AddOperator(PrefixOperator(\"t2r*\", ws, 2, true, fun x -> failwith \"t2r*\"))\n opp.AddOperator(PrefixOperator(\"t2n*\", ws, 2, true, fun x -> failwith \"t2n*\"))\n\n opp.AddOperator(PrefixOperator(\"po1\", ws, 1, true, fun x -> failwith \"po1\"))\n opp.AddOperator(PrefixOperator(\"po1n\", ws, 1, true, fun x -> failwith \"po1n\"))\n opp.AddOperator(PrefixOperator(\"po2\", ws, 1, true, fun x -> failwith \"po2\"))\n opp.AddOperator(PrefixOperator(\"po2n\", ws, 1, true, fun x -> failwith \"po2n\"))\n\n opp.AddOperator(PrefixOperator(\"po1*\", ws, 2, true, fun x -> failwith \"po1*\"))\n opp.AddOperator(PrefixOperator(\"po1n*\", ws, 2, true, fun x -> failwith \"po1n*\"))\n opp.AddOperator(PrefixOperator(\"po2*\", ws, 2, true, fun x -> failwith \"po2*\"))\n opp.AddOperator(PrefixOperator(\"po2n*\", ws, 2, true, fun x -> failwith \"po2n*\"))\n\n opp.AddOperator(InfixOperator(\"pre1\", ws, 1, Associativity.Left, fun x y -> failwith \"pre1\"))\n opp.AddOperator(InfixOperator(\"pre1n\", ws, 1, Associativity.Left, fun x y -> failwith \"pre1n\"))\n opp.AddOperator(PostfixOperator(\"pre2\", ws, 1, true, fun x -> failwith \"pre2\"))\n opp.AddOperator(PostfixOperator(\"pre2n\", ws, 1, true, fun x -> failwith \"pre2n\"))\n\n opp.AddOperator(InfixOperator(\"pre1*\", ws, 2, Associativity.Left, fun x y -> failwith \"pre1*\"))\n opp.AddOperator(InfixOperator(\"pre1n*\", ws, 2, Associativity.Left, fun x y -> failwith \"pre1n*\"))\n opp.AddOperator(PostfixOperator(\"pre2*\", ws, 2, true, fun x -> failwith \"pre2*\"))\n opp.AddOperator(PostfixOperator(\"pre2n*\", ws, 2, true, fun x -> failwith \"pre2n\"))\n\n expr |> ROk \"1 o1l 2 o2l 3\" (O2(O1(Val(1),Val(2)),Val(3)))\n expr |> ROk \"1 o1r* 2 o2r 3\" (O2(O1(Val(1),Val(2)),Val(3)))\n expr |> ROk \"1 o1r 2 o2r 3\" (O1(Val(1),O2(Val(2),Val(3))))\n expr |> ROk \"1 o1l 2 o2l* 3\" (O1(Val(1),O2(Val(2),Val(3))))\n\n expr |> ROk \"1 o1l 2 o2l* 3 o3l** 4\" (O1(Val(1),O2(Val(2),O3(Val(3),Val(4)))))\n expr |> ROk \"1 o1l 2 o2l* 3 o3l 4\" (O3((O1(Val(1),O2(Val(2),Val(3)))),Val(4)))\n expr |> ROk \"1 o1l 2 o2l** 3 o3l* 4\" (O1(Val(1),O3(O2(Val(2),Val(3)),Val(4))))\n\n expr |> ROk \"1 o1r 2 o2r* 3 o3r** 4\" (O1(Val(1),O2(Val(2),O3(Val(3),Val(4)))))\n expr |> ROk \"1 o1r 2 o2r* 3 o3r 4\" (O1(Val(1),O3(O2(Val(2),Val(3)),Val(4))))\n\n expr |> ROk \"1 t1l 2 tt1l 3 t2l 4 tt2l 5\" (T2(T1(Val(1),Val(2),Val(3)),Val(4),Val(5)))\n expr |> ROk \"1 t1r* 2 tt1r* 3 t2r 4 tt2r 5\" (T2(T1(Val(1),Val(2),Val(3)),Val(4),Val(5)))\n expr |> ROk \"1 t1r 2 tt1r 3 t2r 4 tt2r 5\" (T1(Val(1),Val(2),T2(Val(3),Val(4),Val(5))))\n expr |> ROk \"1 t1l 2 tt1l 3 t2l* 4 tt2l* 5\" (T1(Val(1),Val(2),T2(Val(3),Val(4),Val(5))))\n\n expr |> ROk \"1 t1l* 2 po1 tt1l* 3 t2l* 4 o1l 5 tt2l* 6\" (T2(T1(Val(1),Po1(Val(2)),Val(3)),O1(Val(4),Val(5)),Val(6)))\n expr |> ROk \"1 t1n 2 o1n 3 tt1n 4\" (T1(Val(1),O1(Val(2),Val(3)),Val(4)))\n\n expr |> ROk \"pre1 1 o1l 2\" (O1(Pre1(Val(1)), Val(2)))\n expr |> ROk \"pre1* 1 o1l 2\" (O1(Pre1(Val(1)), Val(2)))\n expr |> ROk \"pre1 1 o1l* 2\" (Pre1(O1(Val(1), Val(2))))\n\n expr |> ROk \"1 o1l pre1 2\" (O1(Val(1), Pre1(Val(2))))\n expr |> ROk \"1 o1l pre1* 2\" (O1(Val(1), Pre1(Val(2))))\n expr |> ROk \"1 o1l* pre1 2\" (O1(Val(1), Pre1(Val(2))))\n\n expr |> ROk \"1 o1l 2 po1\" (O1(Val(1), Po1(Val(2))))\n expr |> ROk \"1 o1l 2 po1*\" (O1(Val(1), Po1(Val(2))))\n expr |> ROk \"1 o1l* 2 po1\" (Po1(O1(Val(1), Val(2))))\n\n expr |> ROk \"1 o1l pre1 2 po1\" (O1(Val(1), Po1(Pre1(Val(2)))))\n expr |> ROk \"1 o1l pre1* 2 po1\" (O1(Val(1), Po1(Pre1(Val(2)))))\n expr |> ROk \"1 o1l pre1 2 po1*\" (O1(Val(1), Pre1(Po1(Val(2)))))\n expr |> ROk \"1 o1l* pre1 2 po1\" (Po1(O1(Val(1), Pre1(Val(2)))))\n expr |> ROk \"1 o1r* pre1 2 po1\" (Po1(O1(Val(1), Pre1(Val(2)))))\n expr |> ROk \"1 o1l* pre1 2 po1*\" (O1(Val(1), Pre1(Po1(Val(2)))))\n\n expr |> ROk \"1 o1l 2 po1*\" (O1(Val(1), Po1(Val(2))))\n expr |> ROk \"1 o1l* 2 po1\" (Po1(O1(Val(1), Val(2))))\n\n expr |> ROk \"1 o1l 2 o2l* 3 po1\" (O1(Val(1),Po1(O2(Val(2),Val(3)))))\n\n expr |> ROk \"1 o1l pre1 pre2 2\" (O1(Val(1), Pre1(Pre2(Val(2)))))\n expr |> ROk \"1 o1l pre1* pre2 2\" (O1(Val(1), Pre1(Pre2(Val(2)))))\n expr |> ROk \"1 o1l pre1 pre2* 2\" (O1(Val(1), Pre1(Pre2(Val(2)))))\n\n expr |> ROk \"1 o1l 2 po1 po2\" (O1(Val(1), Po2(Po1(Val(2)))))\n expr |> ROk \"1 o1l* 2 po1 po2\" (Po2(Po1(O1(Val(1), Val(2)))))\n expr |> ROk \"1 o1l* 2 po1 po2*\" (Po2(Po1(O1(Val(1), Val(2)))))\n expr |> ROk \"1 o1l* 2 po1* po2\" (Po2(O1(Val(1), Po1(Val(2)))))\n\n expr |> ROk \"1 o1l pre1 2 po1 po2\" (O1(Val(1), Po2(Po1(Pre1(Val(2))))))\n expr |> ROk \"1 o1l pre1 2 po1* po2\" (O1(Val(1), Po2(Pre1(Po1(Val(2))))))\n expr |> ROk \"1 o1l pre1 2 po1* po2*\" (O1(Val(1), Pre1(Po2(Po1(Val(2))))))\n expr |> ROk \"1 o1l pre1 2 po1 po2*\" (O1(Val(1), Po2(Po1(Pre1(Val(2))))))\n expr |> ROk \"1 o1l* pre1 2 po1 po2\" (Po2(Po1(O1(Val(1), Pre1(Val(2))))))\n expr |> ROk \"1 o1l* pre1 2 po1* po2\" (Po2(O1(Val(1), Pre1(Po1(Val(2))))))\n expr |> ROk \"1 o1l* pre1 2 po1* po2\" (Po2(O1(Val(1), Pre1(Po1(Val(2))))))\n\n expr |> ROk \"pre1 1 o1l 2 po1\" (O1(Pre1(Val(1)), Po1(Val(2))))\n expr |> ROk \"pre1 1 o1l* 2 po1\" (Po1(Pre1(O1(Val(1), Val(2)))))\n expr |> ROk \"pre1* 1 o1l* 2 po1\" (Po1(O1(Pre1(Val(1)), Val(2))))\n expr |> ROk \"pre1 1 o1l* 2 po1*\" (Pre1(O1(Val(1), Po1(Val(2)))))\n expr |> ROk \"pre1 1 o1l** 2 po1*\" (Pre1(Po1(O1(Val(1), Val(2)))))\n\n opp.OperatorConflictErrorFormatter <- fun _ _ -> messageError \"conflict\"\n\n let conflictE = messageError \"conflict\"\n\n expr |> ROk \"pre1n 1 o1n 2\" (O1(Pre1(Val(1)), Val(2)))\n expr |> ROk \"1 po1n o1n 2\" (O1(Po1(Val(1)), Val(2)))\n expr |> ROk \"1 o1n pre1n 2\" (O1(Val(1), Pre1(Val(2))))\n expr |> ROk \"1 o1n 2 po1n\" (O1(Val(1), Po1(Val(2))))\n\n\n expr |> ROk \"1 o1l* 2 o2r 3\" (O2(O1(Val(1),Val(2)),Val(3)))\n expr |> ROk \"1 o1l 2 o2r* 3\" (O1(Val(1),O2(Val(2),Val(3))))\n expr |> RError \"1 o1l 2 o2r 3\" 9 conflictE\n expr |> RError \"1 o1l 2 o2n 3\" 9 conflictE\n expr |> RError \"1 o1n 2 o2n 3\" 9 conflictE\n expr |> RError \"1 o1l 2 o2l* 3 o2r 4\" 16 conflictE\n expr |> RError \"1 o1l 2 o2l* 3 o2n 4\" 16 conflictE\n expr |> RError \"1 o1n 2 o2l* 3 o2n 4\" 16 conflictE\n\n expr |> ROk \"1 t1l* 2 tt1l* 3 t2r 4 tt2r 5\" (T2(T1(Val(1),Val(2),Val(3)),Val(4),Val(5)))\n expr |> ROk \"1 t1l 2 tt1l 3 t2r* 4 tt2r* 5\" (T1(Val(1),Val(2),T2(Val(3),Val(4),Val(5))))\n expr |> RError \"1 t1l 2 tt1l 3 t2r 4 tt2r 5\" 17 conflictE\n\n expr |> RError \"1 t1l 2 tt1l 3 o1r 4\" 17 conflictE\n expr |> RError \"1 o1r 2 t1l 3 tt1l 4\" 9 conflictE\n\n expr |> ROk \"pre1n 1 po1\" (Po1(Pre1(Val(1))))\n expr |> ROk \"pre1 1 po1n\" (Po1(Pre1(Val(1))))\n expr |> ROk \"pre1n* 1 po1n\" (Po1(Pre1(Val(1))))\n expr |> ROk \"pre1n 1 po1n*\" (Pre1(Po1(Val(1))))\n expr |> RError \"pre1n 1 po1n\" 9 conflictE\n\n expr |> ROk \"pre1 pre2n 1\" (Pre1(Pre2(Val(1))))\n expr |> ROk \"pre1n pre2 1\" (Pre1(Pre2(Val(1))))\n expr |> ROk \"pre1n* pre2n 1\" (Pre1(Pre2(Val(1))))\n expr |> ROk \"pre1n pre2n* 1\" (Pre1(Pre2(Val(1))))\n expr |> RError \"pre1n pre2n 1\" 7 conflictE\n\n expr |> ROk \"1 po1 po2n\" (Po2(Po1(Val(1))))\n expr |> ROk \"1 po1n po2\" (Po2(Po1(Val(1))))\n expr |> ROk \"1 po1n* po2n\" (Po2(Po1(Val(1))))\n expr |> ROk \"1 po1n po2n*\" (Po2(Po1(Val(1))))\n expr |> RError \"1 po1n po2n\" 9 conflictE\n\n let rand = new System.Random(1234)\n testRemoveSeq rand opp opp.Operators\n\nlet testExceptions() =\n let opp = new OperatorPrecedenceParser()\n opp.AddOperator(TernaryOperator(\"?\", spaces, \":\", spaces, 1, Associativity.Left, fun _ _ _ -> 0))\n try opp.AddOperator(TernaryOperator(\"??\", spaces, \":\", spaces, 1, Associativity.Left, fun _ _ _ -> 0))\n with :? System.ArgumentException -> ()\n try opp.AddOperator(PrefixOperator(\":\", spaces, 2, false, fun x -> 0))\n with :? System.ArgumentException -> ()\n\n opp.AddOperator(PrefixOperator(\"+\", spaces, 1, true, fun x -> 0))\n opp.AddOperator(InfixOperator(\"-\", spaces, 1, Associativity.Left, fun x y -> 0))\n try opp.AddOperator(PrefixOperator(\":\", spaces, 2, false, fun x -> 0))\n with :? System.ArgumentException -> ()\n try opp.AddOperator(TernaryOperator(\"x\", spaces, \"+\", spaces, 1, Associativity.Left, fun _ _ _ -> 0))\n with :? System.ArgumentException -> ()\n try opp.AddOperator(TernaryOperator(\"x\", spaces, \"-\", spaces, 1, Associativity.Left, fun _ _ _ -> 0))\n with :? System.ArgumentException -> ()\n\n try PrefixOperator(null, spaces, 1, true, fun x -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try PrefixOperator(\"\", spaces, 1, true, fun x -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try PrefixOperator(\"+\", Unchecked.defaultof<_>, 1, true, fun x -> x) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try PrefixOperator(\"+\", spaces, 0, true, fun x -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n (PrefixOperator(\"+\", spaces, 1, true, fun x -> 0)).IsAssociative |> True\n try InfixOperator(\"+\", spaces, 1, enum -1 , fun x y -> x + y) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try InfixOperator(\"+\", spaces, 1, enum 3 , fun x y -> x + y) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n try PrefixOperator(\"+\", spaces, 1, true, (), Unchecked.defaultof<_>) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n try InfixOperator(\"+\", spaces, 1, Associativity.Left, (), Unchecked.defaultof<_>) |> ignore; Fail()\n with :? System.ArgumentNullException -> ()\n\n try TernaryOperator(null, spaces, \"2\", spaces, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"\", spaces, \"2\", spaces, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, null, spaces, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"\", spaces, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", Unchecked.defaultof<_>, \"2\", spaces, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"2\", Unchecked.defaultof<_>, 1, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"2\", spaces, 0, Associativity.Left, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"2\", spaces, 1, enum -1, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"2\", spaces, 1, enum 3, fun x y z -> x) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try TernaryOperator(\"1\", spaces, \"2\", spaces, 1, Associativity.Left, (), Unchecked.defaultof<_>) |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\nlet run() =\n testOpParser()\n testConflictAfterStringParserHandling()\n testAlternativeOpConstructors()\n testPrecAndAssoc()\n testExceptions()\n" }, { "path": "Test/PrimitivesTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.PrimitivesTests\n\nopen FParsec\nopen FParsec.Error\n\nmodule Reference =\n\n []\n let Ok = FParsec.Primitives.Ok\n []\n let Error = FParsec.Primitives.Error\n []\n let FatalError = FParsec.Primitives.FatalError\n\n type Parser<'a,'u> = FParsec.Primitives.Parser<'a,'u>\n\n type FParsec.Reply<'a>\n with\n member t.WithError(error: ErrorMessageList) =\n Reply(t.Status, t.Result, error)\n\n let reconstructErrorReply (reply: Reply<_>) =\n Reply(reply.Status, reply.Error)\n\n let preturn x =\n fun stream -> Reply(x)\n\n let pzero : Parser<'a,'u> =\n fun stream -> Reply(Error, NoErrorMessages)\n\n let (>>=) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) =\n fun stream ->\n let reply1 = p stream\n if reply1.Status = Ok then\n let p2 = f reply1.Result\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n if stateTag1 <> stream.StateTag then reply2\n else reply2.WithError(mergeErrors reply1.Error reply2.Error)\n else reconstructErrorReply reply1\n\n let (>>%) p x = p >>= fun _ -> preturn x\n\n let (>>.) p1 p2 = p1 >>= fun _ -> p2\n let (.>>) p1 p2 = p1 >>= fun x -> p2 >>% x\n\n let (.>>.) p1 p2 =\n p1 >>= fun x1 ->\n p2 >>= fun x2 -> preturn (x1, x2)\n\n let between popen pclose p = popen >>. (p .>> pclose)\n\n let (|>>) p f =\n p >>= fun a -> preturn (f a)\n\n let pipe2 p1 p2 f =\n p1 >>= fun x1 ->\n p2 >>= fun x2 -> preturn (f x1 x2)\n\n let pipe3 p1 p2 p3 f =\n p1 >>= fun x1 ->\n p2 >>= fun x2 ->\n p3 >>= fun x3 -> preturn (f x1 x2 x3)\n\n let pipe4 p1 p2 p3 p4 f =\n p1 >>= fun x1 ->\n p2 >>= fun x2 ->\n p3 >>= fun x3 ->\n p4 >>= fun x4 -> preturn (f x1 x2 x3 x4)\n\n let pipe5 p1 p2 p3 p4 p5 f =\n p1 >>= fun x1 ->\n p2 >>= fun x2 ->\n p3 >>= fun x3 ->\n p4 >>= fun x4 ->\n p5 >>= fun x5 -> preturn (f x1 x2 x3 x4 x5)\n\n let () (p: Parser<'a,'u>) label : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let reply = p stream\n if stateTag <> stream.StateTag then reply\n else reply.WithError(expected label)\n\n let () (p: Parser<'a,'u>) label : Parser<'a,'u> =\n fun stream ->\n let mutable state = stream.State\n let reply = p stream\n if reply.Status = Ok then\n if state.Tag <> stream.StateTag then reply\n else reply.WithError(expected label)\n else\n if state.Tag <> stream.StateTag then\n let error = compoundError label stream reply.Error\n stream.BacktrackTo(&state)\n Reply(FatalError, error)\n else\n let error = match reply.Error with\n | ErrorMessageList(NestedError(pos, ustate, msgs), NoErrorMessages)\n -> ErrorMessageList(CompoundError(label, pos, ustate, msgs), NoErrorMessages)\n | _ -> expected label\n reply.WithError(error)\n\n let fail msg : Parser<'a,'u> =\n fun stream -> Reply(Error, messageError msg)\n\n let failFatally msg : Parser<'a,'u> =\n fun stream -> Reply(FatalError, messageError msg)\n\n let (<|>) (p1: Parser<'a,'u>) (p2: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let reply1 = p1 stream\n if reply1.Status = Error && stateTag = stream.StateTag then\n let reply2 = p2 stream\n if stateTag <> stream.StateTag then reply2\n else reply2.WithError(mergeErrors reply1.Error reply2.Error)\n else reply1\n\n let choice (ps: seq>) =\n List.fold (fun p pc -> p <|> pc) pzero (List.ofSeq ps)\n\n let (<|>%) p x = p <|> preturn x\n\n let opt p = (p |>> Some) <|>% None\n\n let optional p = (p >>% ()) <|>% ()\n\n let notEmpty (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let stateTag = stream.StateTag\n let reply = p stream\n if reply.Status <> Ok || stateTag <> stream.StateTag then reply\n else Reply(Error, reply.Error)\n\n let attempt (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let mutable state = stream.State\n let reply = p stream\n if reply.Status = Ok then reply\n elif state.Tag = stream.StateTag then\n Reply(Error, reply.Error)\n else\n let error = nestedError stream reply.Error\n stream.BacktrackTo(&state)\n Reply(Error, error)\n\n let (>>=?) (p: Parser<'a,'u>) (f: 'a -> Parser<'b,'u>) : Parser<'b,'u> =\n fun stream ->\n let mutable state = stream.State\n let reply1 = p stream\n if reply1.Status = Ok then\n let p2 = f reply1.Result\n let stateTag1 = stream.StateTag\n let reply2 = p2 stream\n if stateTag1 <> stream.StateTag then reply2\n else\n let error = mergeErrors reply1.Error reply2.Error\n if reply2.Status <> Error then reply2.WithError(error)\n elif state.Tag = stateTag1 then Reply(Error, error)\n else\n let error = nestedError stream error\n stream.BacktrackTo(&state)\n Reply(Error, error)\n else reconstructErrorReply reply1\n\n let (>>?) p1 p2 = p1 >>=? fun _ -> p2\n let (.>>?) p1 p2 = p1 >>=? fun x -> p2 >>% x\n\n let (.>>.?) p1 p2 =\n p1 >>=? fun x1 ->\n p2 >>= fun x2 -> preturn (x1, x2)\n\n let lookAhead (p: Parser<'a,'u>) : Parser<'a,'u> =\n fun stream ->\n let mutable state = stream.State\n let reply = p stream\n if reply.Status = Ok then\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state)\n Reply(reply.Result)\n else\n if state.Tag = stream.StateTag then\n Reply(Error, reply.Error)\n else\n let error = nestedError stream reply.Error\n stream.BacktrackTo(&state)\n Reply(Error, error)\n\n let followedByE (p: Parser<'a,'u>) error : Parser =\n fun stream ->\n let mutable state = stream.State\n let reply = p stream\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state)\n if reply.Status = Ok then Reply(())\n else Reply(Error, error)\n\n let followedBy p = followedByE p NoErrorMessages\n let followedByL p label = followedByE p (expected label)\n\n let notFollowedByE (p: Parser<'a,'u>) error : Parser =\n fun stream ->\n let mutable state = stream.State\n let reply = p stream\n if state.Tag <> stream.StateTag then\n stream.BacktrackTo(&state)\n if reply.Status <> Ok then Reply(())\n else Reply(Error, error)\n\n let notFollowedBy p = notFollowedByE p NoErrorMessages\n let notFollowedByL p label = notFollowedByE p (unexpected label)\n\n let tuple2 p1 p2 = pipe2 p1 p2 (fun a b -> (a, b))\n let tuple3 p1 p2 p3 = pipe3 p1 p2 p3 (fun a b c -> (a, b, c))\n let tuple4 p1 p2 p3 p4 = pipe4 p1 p2 p3 p4 (fun a b c d -> (a, b, c, d))\n let tuple5 p1 p2 p3 p4 p5 = pipe5 p1 p2 p3 p4 p5 (fun a b c d e -> (a, b, c, d, e))\n\n let parray n (p : Parser<_,_>) =\n let rec loop i =\n if i = n then preturn []\n else\n p >>= fun hd -> (loop (i + 1) |>> fun tl -> hd::tl)\n loop 0 |>> Array.ofList\n\n let skipArray n p = parray n p |>> ignore\n\n // Note that the actual implemention of `many` tries to guard against\n // an infinite loop/recursion by throwing an exception if the given parser\n // argument succeeds without changing the stream.\n\n let rec many p = many1 p <|>% []\n and many1 p = p >>= fun hd -> many p |>> (fun tl -> hd::tl)\n\n // a version of many (p1 .>>. p2) that does not succeed if `p1` succeeds\n /// without changing the parser state and `p2` fails without changing the state\n let rec manyPair p1 p2 =\n p1 |>> (fun x1 ->\n p2 >>= fun x2 ->\n manyPair p1 p2 |>> fun tl -> (x1, x2)::tl)\n <|>% (preturn [])\n >>= fun p -> p\n\n let rec sepBy p sep = sepBy1 p sep <|>% []\n and sepBy1 p sep =\n p >>= fun hd -> manyPair sep p |>> fun sepPs -> hd::(List.map snd sepPs)\n\n let rec sepEndBy p sep = sepEndBy1 p sep <|>% []\n and sepEndBy1 p sep =\n p >>= fun hd -> sep >>. sepEndBy p sep <|>% [] |>> fun tl -> hd::tl\n\n let manyTill (p: Parser<'a,'u>) (endp: Parser<'b,'u>) =\n let rec parse (stream: CharStream<'u>) acc error =\n let stateTag = stream.StateTag\n let replyE = endp stream\n if replyE.Status = Error && stateTag = stream.StateTag then\n let replyP = p stream\n if replyP.Status = Ok then\n if stateTag = stream.StateTag then\n failwith \"infinite loop\"\n parse stream (replyP.Result::acc) replyP.Error\n else\n let error = if stateTag <> stream.StateTag then replyP.Error\n else mergeErrors (mergeErrors error replyE.Error) replyP.Error\n Reply(replyP.Status, error)\n else\n let error = if stateTag <> stream.StateTag then replyE.Error\n else mergeErrors error replyE.Error\n if replyE.Status = Ok then\n Reply(Ok, List.rev acc, error)\n else\n Reply(replyE.Status, error)\n\n fun stream -> parse stream [] NoErrorMessages\n\n let many1Till p endp = pipe2 p (manyTill p endp) (fun hd tl -> hd::tl)\n\n let chainl1 p op =\n p >>= fun x -> manyPair op p |>> fun opPs ->\n List.fold (fun x (f, y) -> f x y) x opPs\n\n let chainl p op x = chainl1 p op <|>% x\n\n let chainr1 p op =\n let rec calc x rhs =\n match rhs with\n | (f, y)::tl -> f x (calc y tl)\n | [] -> x\n\n pipe2 p (manyPair op p) calc\n\n let chainr p op x = chainr1 p op <|>% x\n\n\nopen FParsec.Primitives\nopen FParsec.Test.Test\n\n\nlet testPrimitives() =\n let content = \"the content doesn't matter\"\n use stream = new FParsec.CharStream(content, 0, content.Length)\n\n\n let ps1 = Array.append (constantTestParsers 1 (expected \"1\")) [|(fun s -> s.UserState <- s.UserState + 1; Reply(1))|]\n let ps1b = constantTestParsers 11 (expected \"1b\")\n let ps1c = constantTestParsers 111 (expected \"1c\")\n let ps1d = constantTestParsers 1111 (expected \"1d\")\n\n let parserSeq4 =\n seq {for p1 in ps1 do\n for p2 in ps1b do\n for p3 in ps1c do\n for p4 in ps1d do\n yield [p1;p2;p3;p4]}\n\n let ps2 = constantTestParsers 2u (expected \"2\")\n let ps2b = constantTestParsers 22u (expected \"2b\")\n let ps2c = constantTestParsers 222u (expected \"2c\")\n let ps3 = constantTestParsers 3s (expected \"3\")\n let ps4 = constantTestParsers 4L (expected \"4\")\n let ps5 = constantTestParsers 5y (expected \"5\")\n\n let checkParser p1 p2 = checkParser p1 p2 stream\n\n let checkComb comb1 comb2 =\n for p in ps1 do\n checkParser (comb1 p) (comb2 p)\n\n let checkCombA comb1 comb2 arg =\n let adapt comb p = comb p arg\n checkComb (adapt comb1) (adapt comb2)\n\n let checkComb2 comb1 comb2 =\n for p1 in ps1 do\n for p2 in ps2 do\n checkParser (comb1 p1 p2) (comb2 p1 p2)\n\n let checkComb2A comb1 comb2 arg =\n let adapt comb p1 p2 = comb p1 p2 arg\n checkComb2 (adapt comb1) (adapt comb2)\n\n let checkBind bind1 bind2 =\n for p1 in ps1 do\n for p2 in ps2 do\n let f1 = fun r ->\n r |> Equal 1\n p2\n let f2 = fun r state ->\n r |> Equal 1\n p2 state\n checkParser (bind1 p1 f1) (bind2 p1 f1)\n checkParser (bind1 p1 f2) (bind2 p1 f2)\n\n let checkComb3 comb1 comb2 =\n for p1 in ps1 do\n for p2 in ps2 do\n for p3 in ps3 do\n checkParser (comb1 p1 p2 p3) (comb2 p1 p2 p3)\n\n let checkComb3A comb1 comb2 arg =\n let adapt comb p1 p2 p3 = comb p1 p2 p3 arg\n checkComb3 (adapt comb1) (adapt comb2)\n\n let checkComb4 comb1 comb2 =\n for p1 in ps1 do\n for p2 in ps2 do\n for p3 in ps3 do\n for p4 in ps4 do\n checkParser (comb1 p1 p2 p3 p4) (comb2 p1 p2 p3 p4)\n let checkComb4A comb1 comb2 arg =\n let adapt comb p1 p2 p3 p4 = comb p1 p2 p3 p4 arg\n checkComb4 (adapt comb1) (adapt comb2)\n\n let checkComb5 comb1 comb2 =\n for p1 in ps1 do\n for p2 in ps2 do\n for p3 in ps3 do\n for p4 in ps4 do\n for p5 in ps5 do\n checkParser (comb1 p1 p2 p3 p4 p5) (comb2 p1 p2 p3 p4 p5)\n let checkComb5A comb1 comb2 arg =\n let adapt comb p1 p2 p3 p4 p5 = comb p1 p2 p3 p4 p5 arg\n checkComb5 (adapt comb1) (adapt comb2)\n\n\n let testBasicPrimitives() =\n checkParser (preturn 42) (Reference.preturn 42)\n checkParser pzero Reference.pzero\n checkCombA () Reference.() \"test\"\n checkBind (>>=) Reference.(>>=)\n checkCombA (>>%) Reference.(>>%) \"test\"\n checkComb2 (>>.) Reference.(>>.)\n checkComb2 (.>>) Reference.(.>>)\n checkComb2 (.>>.) Reference.(.>>.)\n checkComb3 between Reference.between\n checkCombA (|>>) Reference.(|>>) (fun x -> x + 3)\n checkComb2A pipe2 Reference.pipe2 (fun a b -> (a, b))\n checkComb3A pipe3 Reference.pipe3 (fun a b c -> (a, b, c))\n checkComb4A pipe4 Reference.pipe4 (fun a b c d -> (a, b, c, d))\n checkComb5A pipe5 Reference.pipe5 (fun a b c d e -> (a, b, c, d, e))\n checkComb2 tuple2 Reference.tuple2\n checkComb3 tuple3 Reference.tuple3\n checkComb4 tuple4 Reference.tuple4\n checkComb5 tuple5 Reference.tuple5\n\n checkCombA () Reference.() \"test\"\n checkCombA () Reference.() \"test\"\n let btestp : Parser<_,_> =\n fun stream ->\n let mutable state0 = stream.State\n stream.Skip()\n stream.UserState <- stream.UserState + 1\n let error = nestedError stream (expected \"test\")\n stream.BacktrackTo(&state0)\n Reply(Error, error)\n\n checkParser (() btestp \"btest\") (Reference.() btestp \"btest\")\n checkParser (fail \"test\") (Reference.fail \"test\")\n checkParser (failFatally \"test\") (Reference.failFatally \"test\")\n\n for p1 in ps1 do\n for p2 in ps1b do\n checkParser ((<|>) p1 p2) (Reference.(<|>) p1 p2)\n\n for ps in Seq.append (Seq.singleton []) parserSeq4 do\n let refChoice = Reference.choice ps\n let refChoiceL = refChoice \"test\"\n\n // choice and choiceL use different implementations depending on whether\n // the type of the supplied sequence is a list, an array or a seq,\n // so we must test all 3 input types.\n let psa = Array.ofSeq ps\n let pss = match ps with\n | [] -> Seq.empty\n | [p1;p2;p3;p4] -> seq {yield p1\n yield p2\n yield p3\n yield p4}\n | _ -> failwith \"shouldn't happen\"\n\n checkParser (choice ps) refChoice\n checkParser (choiceL ps \"test\") refChoiceL\n checkParser (choice psa) refChoice\n checkParser (choiceL psa \"test\") refChoiceL\n checkParser (choice pss) refChoice\n checkParser (choiceL pss \"test\") refChoiceL\n\n checkCombA (<|>%) Reference.(<|>%) 99\n checkComb opt Reference.opt\n checkComb optional Reference.optional\n checkComb notEmpty Reference.notEmpty\n checkComb attempt Reference.attempt\n checkBind (>>=?) Reference.(>>=?)\n checkComb2 (>>?) Reference.(>>?)\n checkComb2 (.>>?) Reference.(.>>?)\n checkComb2 (.>>.?) Reference.(.>>.?)\n checkComb followedBy Reference.followedBy\n checkCombA followedByL Reference.followedByL \"test\"\n checkComb notFollowedBy Reference.notFollowedBy\n checkCombA notFollowedByL Reference.notFollowedByL \"test\"\n checkComb lookAhead Reference.lookAhead\n\n testBasicPrimitives()\n\n let testPArray() =\n // parray\n for ps in parserSeq4 do\n let p1, p2, pr = seqParserAndReset2 (List.ofSeq ps)\n checkParser (parray 0 p1) (Reference.parray 0 p2); pr()\n checkParser (parray 1 p1) (Reference.parray 1 p2); pr()\n checkParser (parray 2 p1) (Reference.parray 2 p2); pr()\n checkParser (parray 3 p1) (Reference.parray 3 p2); pr()\n checkParser (parray 4 p1) (Reference.parray 4 p2); pr()\n checkParser (skipArray 0 p1) (Reference.skipArray 0 p2); pr()\n checkParser (skipArray 1 p1) (Reference.skipArray 1 p2); pr()\n checkParser (skipArray 2 p1) (Reference.skipArray 2 p2); pr()\n checkParser (skipArray 3 p1) (Reference.skipArray 3 p2); pr()\n checkParser (skipArray 4 p1) (Reference.skipArray 4 p2); pr()\n\n testPArray()\n\n let foldTestF acc x = (acc + 1)*(acc + 1) + x\n let reduceOrDefault f d lst = match lst with\n | [] -> d\n | _ -> List.reduce f lst\n\n let testMany() =\n let manySeq3 = // parserSeq4 without parsers that return Ok without changing the state\n seq {for p1 in ps1[1..] do\n for p2 in ps1b[1..] do\n for p3 in ps1c[1..] do\n yield [p1;p2;p3]}\n\n let f = foldTestF\n\n for ps in manySeq3 do\n let p1, p2, pr = seqParserAndReset2 ps\n\n let rMany = Reference.many p2\n checkParser (many p1) rMany; pr()\n checkParser (skipMany p1) (rMany |>> ignore); pr()\n\n let rMany1 = Reference.many1 p2\n checkParser (many1 p1) rMany1; pr()\n checkParser (skipMany1 p1) (rMany1 |>> ignore); pr()\n\n try many (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n try many1 (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n\n testMany()\n\n let sepByTestParsers r1 e1 r2 e2 =\n let ps1 = constantTestParsers r1 e1\n let ps2 = constantTestParsers r2 e2\n // all parser combinations except \"ok without state change\", \"ok without state change\"\n seq {\n for p2 in ps2[1..] do\n yield ps1[0], p2\n for p1 in ps1[1..] do\n for p2 in ps2 do\n yield p1, p2\n }\n\n let testSeqEndBy() =\n let sepEndSeq3 =\n seq {for p1 in (constantTestParsers 1 (expected \"p1\"))[1..] do\n for sep1, p2 in sepByTestParsers 'a' (expected \"sep1\") 2 (expected \"p2\") do\n for sep2, p3 in sepByTestParsers 'b' (expected \"sep2\") 3 (expected \"p3\") do\n yield [p1; p2; p3;], [sep1; sep2;]}\n\n let f = foldTestF\n\n let mutable i = 0\n for ps, ss in sepEndSeq3 do\n i <- i + 1\n let p1, p2, pr = seqParserAndReset2 ps\n let s1, s2, sr = seqParserAndReset2 ss\n\n let rSepBy = Reference.sepBy p2 s2\n checkParser (sepBy p1 s1) rSepBy; pr(); sr()\n checkParser (skipSepBy p1 s1) (rSepBy |>> ignore); pr(); sr()\n\n let rSepBy1 = Reference.sepBy1 p2 s2\n checkParser (sepBy1 p1 s1) rSepBy1; pr(); sr()\n checkParser (skipSepBy1 p1 s1) (rSepBy1 |>> ignore); pr(); sr()\n\n let rSepEndBy = Reference.sepEndBy p2 s2\n checkParser (sepEndBy p1 s1) rSepEndBy; pr(); sr()\n checkParser (skipSepEndBy p1 s1) (rSepEndBy |>> ignore); pr(); sr()\n\n let rSepEndBy1 = Reference.sepEndBy1 p2 s2\n checkParser (sepEndBy1 p1 s1) rSepEndBy1; pr(); sr()\n checkParser (skipSepEndBy1 p1 s1) (rSepEndBy1 |>> ignore); pr(); sr()\n\n try sepBy (preturn 0) (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n try sepBy1 (preturn 0) (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n try sepEndBy (preturn 0) (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n try sepEndBy1 (preturn 0) (preturn 0) stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n\n testSeqEndBy()\n\n let testManyTill() =\n\n let manyTillSeq3 =\n seq {for endp1 in ps2 do\n for p1 in ps1[1..] do\n for endp2 in ps2b do\n for p2 in ps1b[1..] do\n for endp3 in ps2c do\n for p3 in ps1c[1..] do\n yield [p1; p2; p3], [endp1; endp2; endp3; ps2c[0]]}\n\n let f = foldTestF\n\n let mutable i = 0\n for ps, es in manyTillSeq3 do\n i <- i + 1\n let p1, p2, pr = seqParserAndReset2 ps\n let e1, e2, er = seqParserAndReset2 es\n\n let rManyTill = Reference.manyTill p2 e2\n checkParser (manyTill p1 e1) rManyTill; pr(); er()\n checkParser (skipManyTill p1 e1) (rManyTill |>> ignore); pr(); er()\n\n let rMany1Till = Reference.many1Till p2 e2\n checkParser (many1Till p1 e1) rMany1Till; pr(); er()\n checkParser (skipMany1Till p1 e1) (rMany1Till |>> ignore); pr(); er()\n\n\n try manyTill (preturn 0) (fail \"test\") stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n\n try many1Till (preturn 0) (fail \"test\") stream |> ignore; Fail ()\n with :? System.InvalidOperationException -> ()\n\n testManyTill()\n\n\n let testChain() =\n let chainSeq3 =\n seq {for p1 in ps1 do\n for op1, p2 in sepByTestParsers (+) (expected \"op1\") 2 (expected \"p2\") do\n for op2, p3 in sepByTestParsers (*) (expected \"op2\") 3 (expected \"p3\") do\n yield [p1; p2; p3;], [op1; op2;]}\n\n for ps, ops in chainSeq3 do\n let p1, p2, pr = seqParserAndReset2 ps\n let op1, op2, opr = seqParserAndReset2 ops\n\n checkParser (chainl p1 op1 -1) (Reference.chainl p2 op2 -1); pr(); opr()\n checkParser (chainl1 p1 op1) (Reference.chainl1 p2 op2); pr(); opr()\n checkParser (chainr p1 op1 -1) (Reference.chainr p2 op2 -1); pr(); opr()\n checkParser (chainr1 p1 op1) (Reference.chainr1 p2 op2); pr(); opr()\n\n testChain()\n\n\nlet run() =\n testPrimitives()\n\n" }, { "path": "Test/RangeTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.RangeTests\n\n#if !LOW_TRUST\n\nopen FParsec\nopen FParsec.Range\n\nopen FParsec.Test.Test\n\nlet int32Min = System.Int32.MinValue\nlet int32Max = System.Int32.MaxValue\n\nlet testLabel1, testLabel2 =\n let dm = new System.Reflection.Emit.DynamicMethod(\"__DummMethodForObtainingLabels\", null, null)\n let ilg = dm.GetILGenerator()\n let label1, label2 = ilg.DefineLabel(), ilg.DefineLabel()\n ilg.Emit(System.Reflection.Emit.OpCodes.Ret);\n dm.CreateDelegate(typeof) |> ignore\n label1, label2\n\nlet testCheckRanges() =\n let l, l2 = testLabel1, testLabel2\n\n#if DEBUG\n // in DEBUG builds one can't construct invalid ranges (due to an assert check)\n#else\n try checkRangesAreValidSortedAndUnconnected [|Range(1, -1)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(1, -1)|] [|l|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(2, 1)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(2, 1)|] [|l; l2|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n#endif\n\n try checkRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(0, 0)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(0, 0)|] [|l; l2|]|> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(1, 1)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected[|Range(0, 0); Range(1, 1)|] [|l; l|]|> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(2, 2); Range(3, 3);|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(0, 0); Range(2, 2); Range(3, 3);|] [|l; l; l|]|> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(1, 1); Range(0, 0)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(1, 1); Range(0, 0)|] [|l; l2|]|> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(int32Min, int32Max); Range(int32Max, int32Max)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(int32Min, int32Max); Range(int32Max, int32Max)|] [|l; l2|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkRangesAreValidSortedAndUnconnected [|Range(int32Min, int32Max); Range(int32Min, int32Min)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(int32Min, int32Max); Range(int32Min, int32Min)|] [|l; l2|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try checkLabelRangesAreValidSortedAndUnconnected [|Range(int32Min, int32Max)|] [|l; l2|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n\nlet testSortAndMergeRanges() =\n sortAndMergeRanges false [||] |> Equal [||]\n sortAndMergeRanges false [|Range(1, 1); Range(-1,-1)|] |> Equal [|Range(-1, -1); Range(1, 1)|]\n\n sortAndMergeRanges false [|Range(int32Min, int32Max)|] |> Equal [|Range(int32Min, int32Max)|]\n sortAndMergeRanges false [|Range(int32Min, 1); Range(2, int32Max)|] |> Equal [|Range(int32Min, int32Max)|]\n sortAndMergeRanges false [|Range(2, int32Max); Range(int32Min, 1)|] |> Equal [|Range(int32Min, int32Max)|]\n\n sortAndMergeRanges false [|Range(1, 2); Range(3, 4); Range(5, 6)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges false [|Range(1, 2); Range(4, 4); Range(5, 6)|] |> Equal [|Range(1, 2); Range(4, 6)|]\n sortAndMergeRanges false [|Range(1, 2); Range(4, 4); Range(6, 6)|] |> Equal [|Range(1, 2); Range(4, 4); Range(6, 6)|]\n\n sortAndMergeRanges true [|Range(int32Min, int32Max); Range(int32Min + 1, int32Max)|] |> Equal [|Range(int32Min, int32Max)|]\n sortAndMergeRanges true [|Range(int32Min, int32Max); Range(int32Min, int32Max - 1)|] |> Equal [|Range(int32Min, int32Max)|]\n\n sortAndMergeRanges true [|Range(0,0); Range(int32Min, int32Max); Range(int32Min + 1, int32Max)|] |> Equal [|Range(int32Min, int32Max)|]\n sortAndMergeRanges true [|Range(0,0); Range(int32Min, int32Max); Range(int32Min, int32Max - 1)|] |> Equal [|Range(int32Min, int32Max)|]\n sortAndMergeRanges true [|Range(int32Min + 1, int32Max); Range(int32Min, int32Max - 1)|] |> Equal [|Range(int32Min, int32Max)|]\n\n sortAndMergeRanges true [|Range(1, 3); Range(3, 4); Range(5, 6)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges true [|Range(1, 2); Range(3, 5); Range(5, 6)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges true [|Range(1, 2); Range(2, 5); Range(5, 6)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges true [|Range(1, 5); Range(3, 4); Range(5, 6)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges true [|Range(1, 5); Range(5, 6); Range(3, 5)|] |> Equal [|Range(1, 6)|]\n sortAndMergeRanges true [|Range(1, 5); Range(3, 5); Range(5, 6); Range(1, 7)|] |> Equal [|Range(1, 7)|]\n\n#if DEBUG\n // in DEBUG builds one can't construct invalid ranges (due to an assert check)\n#else\n try sortAndMergeRanges false [|Range(1, -1);|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeRanges false [|Range(0, 0); Range(2, 1)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n#endif\n\n try sortAndMergeRanges false [|Range(0, 0); Range(0, 0)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeRanges false [|Range(0, 0); Range(1, 2); Range(2, 2)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeRanges false [|Range(int32Min, int32Max); Range(0, 0)|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\nlet testSortAndMergeKeyValueRanges() =\n let cmp = System.Collections.Generic.EqualityComparer.Default\n sortAndMergeKeyValueRanges null [||] |> Equal ([||], [||])\n sortAndMergeKeyValueRanges null [|Range(1, 1), 0; Range(-1,-1), 1|] |> Equal ([|Range(-1, -1); Range(1, 1)|], [|1; 0|])\n sortAndMergeKeyValueRanges cmp [|Range(1, 1), 0; Range(-1,-1), 1|] |> Equal ([|Range(-1, -1); Range(1, 1)|], [|1; 0|])\n sortAndMergeKeyValueRanges cmp [|Range(1, 1), 0; Range(-1, 0), 0|] |> Equal ([|Range(-1, 1)|], [|0|])\n\n sortAndMergeKeyValueRanges null [|Range(2, int32Max), 0; Range(int32Min, 1), 0|] |> Equal ([|Range(int32Min, 1); Range(2, int32Max)|], [|0; 0|])\n sortAndMergeKeyValueRanges cmp [|Range(2, int32Max), 0; Range(int32Min, 1), 0|] |> Equal ([|Range(int32Min, int32Max)|], [|0|])\n\n sortAndMergeKeyValueRanges cmp [|Range(1, 2), 0; Range(3, 4), 0; Range(6, 6), 0|] |> Equal ([|Range(1, 4); Range(6, 6)|], [|0; 0|])\n sortAndMergeKeyValueRanges cmp [|Range(1, 1), 0; Range(3, 4), 0; Range(5, 6), 0|] |> Equal ([|Range(1, 1); Range(3, 6)|], [|0; 0|])\n\n#if DEBUG\n // in DEBUG builds one can't construct invalid ranges (due to an assert check)\n#else\n try sortAndMergeKeyValueRanges null [|Range(1, -1), 0;|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeKeyValueRanges null [|Range(0, 0), 0; Range(2, 1), 0|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n#endif\n\n try sortAndMergeKeyValueRanges null [|Range(0, 0), 0; Range(0, 0), 1|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeKeyValueRanges null [|Range(0, 0), 0; Range(1, 2), 1; Range(2, 2), 2|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try sortAndMergeKeyValueRanges null [|Range(int32Min, int32Max), 0; Range(0, 0), 1|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\nlet testMergeSortedKeyLabelRanges() =\n let l, l2 = testLabel1, testLabel2\n mergeSortedKeyLabelRanges [||] [||] |> Equal ([||], [||])\n mergeSortedKeyLabelRanges [|1|] [|l|] |> Equal ([|Range(1,1)|], [|l|])\n mergeSortedKeyLabelRanges [|1;2;3|] [|l;l;l|] |> Equal ([|Range(1,3)|], [|l|])\n mergeSortedKeyLabelRanges [|1;2;3|] [|l;l2;l|] |> Equal ([|Range(1,1);Range(2,2);Range(3,3)|], [|l;l2;l|])\n mergeSortedKeyLabelRanges [|1;2;3;5|] [|l;l;l;l|] |> Equal ([|Range(1,3); Range(5,5)|], [|l;l|])\n mergeSortedKeyLabelRanges [|1;2;3;5;6|] [|l;l;l;l;l|] |> Equal ([|Range(1,3); Range(5,6)|], [|l;l|])\n mergeSortedKeyLabelRanges [|1;2;3;5;6|] [|l;l2;l2;l;l|] |> Equal ([|Range(1,1); Range(2,3); Range(5,6)|], [|l;l2;l|])\n mergeSortedKeyLabelRanges [|1;3;5|] [|l;l;l|] |> Equal ([|Range(1, 1); Range(3, 3); Range(5,5)|], [|l;l;l|])\n mergeSortedKeyLabelRanges [|int32Max - 1; int32Max|] [|l;l|] |> Equal ([|Range(int32Max - 1, int32Max)|], [|l|])\n\n try mergeSortedKeyLabelRanges [||] [|l|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try mergeSortedKeyLabelRanges [|1; 1|] [|l; l|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\n try mergeSortedKeyLabelRanges [|1; 0|] [|l; l|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\nlet testCollectSortAndMergeRanges() =\n let cmp = System.Collections.Generic.EqualityComparer.Default\n let rand = new System.Random(1234)\n\n // check all possible subsets of {1,2,...,N}\n let N = 8\n let set = new ResizeArray<_>(N)\n for n = 0 to (1 <<< N) - 1 do\n set.Clear()\n\n let mutable b = n\n let mutable i = 1\n while b <> 0 do\n if (b &&& 1) <> 0 then set.Add(i)\n b <- b >>> 1\n i <- i + 1\n let keys = set.ToArray()\n let ranges = keys |> Array.map (fun k -> Range(k, k))\n\n let mergedRanges1 = collectSortAndMergeRanges keys\n let mergedRanges2 = sortAndMergeRanges false ranges\n mergedRanges1 |> Equal mergedRanges2\n\n let mergedRanges1b, rangeValues = collectSortAndMergeKeyValueRanges\n cmp\n (keys |> Array.map (fun k -> k,0))\n mergedRanges1b |> Equal mergedRanges2\n rangeValues.Length |> Equal mergedRanges2.Length\n rangeValues |> Array.forall ((=) 0) |> True\n\n if n <> 0 then\n set.Add(keys[rand.Next(keys.Length)])\n set.Add(keys[rand.Next(keys.Length)])\n let keys2 = set.ToArray()\n shuffleArray rand keys2\n let mergedRanges3 = collectSortAndMergeRanges keys2\n mergedRanges3 |> Equal mergedRanges2\n\n collectSortAndMergeRanges [|int32Max; int32Min|] |> Equal [|Range(int32Min, int32Min); Range(int32Max, int32Max)|]\n collectSortAndMergeRanges [|int32Max - 1; int32Max; int32Min; int32Min + 1|] |> Equal [|Range(int32Min, int32Min + 1); Range(int32Max - 1, int32Max)|]\n\n collectSortAndMergeKeyValueRanges null [|2,0; 1,0|] |> Equal ([|Range(1, 1); Range(2, 2)|], [|0; 0|])\n\n try collectSortAndMergeKeyValueRanges null [|1,1;2,2;1,3|] |> ignore\n Fail()\n with :? System.ArgumentException -> ()\n\nlet testSumsOfLengths() =\n sumOfLengths [|Range(1,1)|] 0 1 |> Equal 1.\n sumOfLengths [|Range(-1,1); Range(2,2)|] 0 2 |> Equal 4.\n sumOfLengths [|Range(int32Min, int32Max)|] 0 1 |> Equal (double int32Max - double int32Min + 1.)\n sumOfLengths [|Range(int32Min, int32Max); Range(int32Min, 0); Range(1, int32Max); Range(int32Min, int32Max);|] 1 3 |> Equal (double int32Max - double int32Min + 1.)\n\n sumOfCappedLengths 1 [|Range(1,1)|] 0 1 |> Equal 1.\n sumOfCappedLengths 3 [|Range(-1,1); Range(2,2)|] 0 2 |> Equal 4.\n sumOfCappedLengths 1 [|Range(-1,1); Range(2,2)|] 0 2 |> Equal 2.\n\n sumOfCappedLengths int32Max [|Range(int32Min, int32Max)|] 0 1 |> Equal (double int32Max)\n sumOfCappedLengths 1 [|Range(int32Min, int32Max)|] 0 1 |> Equal 1.\n sumOfCappedLengths int32Max [|Range(int32Min, int32Max); Range(int32Min, 0); Range(1, int32Max); Range(int32Min, int32Max);|] 1 3 |> Equal (double int32Max + double int32Max)\n sumOfCappedLengths int32Max [|Range(int32Min, -2); Range(-1,0); Range(1, int32Max)|] 0 3 |> Equal (double int32Max - double int32Min + 1.)\n\nlet testFindPivot() =\n findPivot [|Range(1,1)|] 0 1 |> Equal (0, false)\n findPivot [|Range(1,1); Range(3,3)|] 0 2 |> Equal (0, true)\n findPivot [|Range(1,1); Range(3,3)|] 1 2 |> Equal (1, false)\n findPivot [|Range(1,1); Range(3,4)|] 0 2|> Equal (1, false)\n findPivot [|Range(0,1); Range(2,3); Range(5,5)|] 0 3 |> Equal (1, true)\n findPivot [|Range(0,1); Range(2,3); Range(5,5); Range(8, 10)|] 0 4 |> Equal (2, true)\n findPivot [|Range(1, 1); Range(3,3); Range(5,5); Range(7,7)|] 0 4 |> Equal (1, true)\n findPivot [|Range(1, 1); Range(2,3); Range(5,6); Range(7,7)|] 0 4 |> Equal (1, true)\n findPivot [|Range(1, 1); Range(2,5); Range(6,8)|] 0 3 |> Equal (1, true)\n\nlet run() =\n testCheckRanges()\n testSortAndMergeRanges()\n testSortAndMergeKeyValueRanges()\n testMergeSortedKeyLabelRanges()\n testCollectSortAndMergeRanges()\n testSumsOfLengths()\n testFindPivot()\n\n#endif" }, { "path": "Test/StaticMappingTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2010-2011\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.StaticMappingTests\n\n#if !LOW_TRUST\n\nopen FParsec\nopen FParsec.Range\nopen FParsec.StaticMapping\n\nopen FParsec.Test.Test\n\ntype EqualityComparer<'a> = System.Collections.Generic.EqualityComparer<'a>\n\nlet testCreateIndicatorFunction() =\n let test (ranges: Range[]) value (minValue: int) (maxValue: int) indicator =\n let mutable i = minValue\n for r in ranges do\n while i < r.Min do\n indicator i |> Equal (not value)\n i <- i + 1\n while i <= r.Max do\n indicator i |> Equal value\n i <- i + 1\n while i <= maxValue do\n indicator i |> Equal (not value)\n i <- i + 1\n\n FParsec.Emit.noBitVectorTests <- true\n\n // check all possible subsets of {1,2,...,N}\n let N = 11\n let set = new ResizeArray<_>(N)\n for n = 0 to (1 <<< N) - 1 do\n let mutable b = n\n let mutable i = 1\n while b <> 0 do\n if (b &&& 1) <> 0 then set.Add(i)\n b <- b >>> 1\n i <- i + 1\n\n let ranges = collectSortAndMergeRanges set\n let indicator = createStaticIntIndicatorFunctionImpl 0 0. 0 (N + 1) false ranges\n test ranges true 0 (N + 1) indicator\n let indicator2 = createStaticIntIndicatorFunctionImpl 0 0. 0 (N + 1) true ranges\n test ranges false 0 (N + 1) indicator2\n\n set.Clear()\n\n FParsec.Emit.noBitVectorTests <- false\n\n let rand = new System.Random(1234)\n\n // check some random subsets of {1,2,...,N}\n let N = 16000\n for n = 1 to 1000 do\n let p = rand.NextDouble()\n for i = 0 to N do\n if rand.NextDouble() <= p then\n set.Add(i)\n let invert = rand.NextDouble() <= 0.5\n let ranges = collectSortAndMergeRanges set\n let indicator = createStaticIntIndicatorFunctionImpl\n 32 0.4 0 N invert ranges\n test ranges (not invert) 0 N indicator\n set.Clear()\n\n let () =\n let ranges = [|Range(1,1); Range(30,31)|]\n let indicator = createStaticIntIndicatorFunctionImpl 0 0. 0 31 false ranges\n test ranges true 0 31 indicator\n\n let () =\n let ranges = [|Range(1,1); Range(30,32)|]\n let indicator = createStaticIntIndicatorFunctionImpl 0 0. 0 32 false ranges\n test ranges true 0 32 indicator\n\n let () =\n let ranges = [|Range(1,1); Range(60,63)|]\n let indicator = createStaticIntIndicatorFunctionImpl 0 0. 0 63 false ranges\n test ranges true 0 63 indicator\n\n let () =\n let ranges = [|Range(1,1); Range(60,64)|]\n let indicator = createStaticIntIndicatorFunctionImpl 0 0. 0 64 false ranges\n test ranges true 0 64 indicator\n let () =\n let ranges = [|Range(1,1); Range(3,5); Range(6,7)|]\n let indicator = createStaticIntRangeIndicatorFunction false ranges\n test ranges true 0 9 indicator\n let indicator2 = createStaticIntIndicatorFunction false [1;3;4;5;6;7]\n test ranges true 0 9 indicator2\n\n let () =\n let indicator = createStaticCharIndicatorFunction false ['\\u0000';'\\ufffe';'\\uffff']\n indicator '\\u0000' |> Equal true\n indicator '\\u0001' |> Equal false\n indicator '\\ufffd' |> Equal false\n indicator '\\ufffe' |> Equal true\n indicator '\\uffff' |> Equal true\n let indicator2 = createStaticCharRangeIndicatorFunction false [Range(0,0); Range(0xfffe, 0xffff)]\n indicator2 '\\u0000' |> Equal true\n indicator2 '\\u0001' |> Equal false\n indicator2 '\\ufffd' |> Equal false\n indicator2 '\\ufffe' |> Equal true\n indicator2 '\\uffff' |> Equal true\n\n\n let () =\n try createStaticCharRangeIndicatorFunction false [Range(0xfffe, 0xffff); Range(-1,0);] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticCharRangeIndicatorFunction false [Range(0,0); Range(0xfffe, 0x10000)] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n ()\n\n\ntype TestStruct(value: int) = struct end\n\n[]\ntype TestStruct2 = struct\n val Field1: int64\n val Field2: int64\n new (value: int) = {Field1 = int64 value; Field2 = int64 value}\n override t.Equals(other: obj) =\n match other with\n | :? TestStruct2 as o -> t.Field1 = o.Field1 && t.Field2 = o.Field2\n | _ -> false\n override t.GetHashCode() = 0\nend\n\nlet testCreateStaticIntMapping() =\n let test (ranges: Range[]) (values: 't[]) defaultValue (minKey: int) (maxKey: int) mapping =\n let mutable i = minKey\n for r, value in Seq.zip ranges values do\n while i < r.Min do\n mapping i |> Equal defaultValue\n i <- i + 1\n while i <= r.Max do\n mapping i |> Equal value\n i <- i + 1\n while i <= maxKey do\n mapping i |> Equal defaultValue\n i <- i + 1\n\n let ranges = new ResizeArray()\n let rand = new System.Random(1234)\n let N = 16000\n for n = 0 to 2000 do\n ranges.Clear()\n\n let maxRangeLength = 1 + rand.Next(128)\n let mutable i = 0\n while i < N do\n let length = 1 + rand.Next(maxRangeLength)\n let i2 = min (i + length) N\n ranges.Add(Range(i, i2 - 1))\n i <- i2\n\n let ranges = ranges.ToArray()\n let values = Array.zeroCreate ranges.Length\n let mutable lastValue = 7\n for i = 0 to values.Length - 1 do\n // value in 0-7, but different from the last value\n lastValue <- (lastValue + 1 + rand.Next(7)) % 8\n values[i] <- byte lastValue\n\n let ranges, values = filterOutDefaultValueRanges EqualityComparer<_>.Default ranges values 0uy\n let mapping = createStaticIntMappingImpl\n //defaultMappingLengthCap defaultMappingDensityThreshold\n 16 0.90\n 0 (N - 1)\n 0uy ranges values\n test ranges values 0uy 0 (N - 1) mapping\n\n let test2 keyValues defaultValue =\n let mapping = createStaticIntMapping defaultValue keyValues\n for k,v in keyValues do\n mapping k |> Equal v\n\n mapping ((keyValues |> List.minBy (fun (k,v) -> k) |> fst) - 1) |> Equal defaultValue\n mapping ((keyValues |> List.maxBy (fun (k,v) -> k) |> fst) + 1) |> Equal defaultValue\n mapping System.Int32.MinValue |> Equal defaultValue\n\n test2 [0, 1; 1, 1; 2, 1;] 0\n\n test2 [1, true; 2, true; 3, true; 4, true] false\n test2 [1, '1'; 2, '2'; 3, '1'; 4, '0'] '0'\n test2 [1, 1y; 2, 2y; 3, 1y; 4, 0y] 0y\n test2 [1, 1uy; 2, 2uy; 3, 1uy; 4, 0uy] 0uy\n test2 [1, 1s; 2, 2s; 3, 1s; 4, 0s] 0s\n test2 [1, 1us; 2, 2us; 3, 1us; 4, 0us] 0us\n test2 [1, 1u; 2, 2u; 3, 1u; 4, 0u] 0u\n test2 [1, 1L; 2, 2L; 3, 1L; 4, 0L; 5, System.Int64.MaxValue] 0L\n test2 [1, 1UL; 2, 2UL; 3, 1UL; 4, 0UL; 5, System.UInt64.MaxValue] 0UL\n test2 [1, 1n; 2, 2n; 3, 1n; 4, 0n] 0n\n test2 [1, 1un; 2, 2un; 3, 1un; 4, 0un] 0un\n test2 [1, 1.f; 2, 2.f; 3, 1.f; 4, 0.f] 0.f\n test2 [1, 1.; 2, 2.; 3, 1.; 4, 0.] 0.\n test2 [1, \"1\"; 2, \"2\"; 3, \"1\"; 4, \"\"] \"\"\n test2 [1, \"1\"; 2, \"2\"; 3, \"1\"; 4, \"\"] null\n test2 [1, TestStruct(1); 2, TestStruct(2); 3, TestStruct(1); 4, TestStruct(0)] (TestStruct(0))\n test2 [1, TestStruct2(1); 2, TestStruct2(2); 3, TestStruct2(1); 4, TestStruct2(0)] (TestStruct2(0))\n test2 [1, FParsec.Associativity.Left; 2, FParsec.Associativity.Right; 3, FParsec.Associativity.Left; 4, FParsec.Associativity.None] FParsec.Associativity.None\n\n let () =\n let mapping = createStaticIntRangeMapping 0 [Range(1,1), 1; Range(3,3), 2; Range(4,5), 2; Range(6,6), 0]\n mapping 0 |> Equal 0\n mapping 1 |> Equal 1\n mapping 2 |> Equal 0\n mapping 3 |> Equal 2\n mapping 4 |> Equal 2\n mapping 5 |> Equal 2\n mapping 6 |> Equal 0\n mapping 7 |> Equal 0\n\n let () =\n try createStaticIntMapping 0 [1, 0; 1, 0] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticIntRangeMapping 0 [Range(0, 1), 0; Range(1, 2), 0] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n ()\n\nlet testCreateStaticStringMapping() =\n let testStringComparison() =\n for nn = 64 to 71 do\n let chars = [|for i = 1 to nn do yield char (32 + i)|]\n let str = new string(chars)\n for n = 0 to str.Length - 1 do\n let subStr = str.Substring(0, n)\n let mapping = createStaticStringMapping 0 [str, 1; subStr, 2]\n mapping str |> Equal 1\n mapping subStr |> Equal 2\n for i = 0 to chars.Length - 1 do\n let c = chars[i]\n chars[i] <- char (int c + 1)\n mapping (new string(chars)) |> Equal 0\n chars[i] <- char 0\n mapping (new string(chars)) |> Equal 0\n chars[i] <- c\n\n\n testStringComparison()\n\n let test defaultValue stringValues defaultTestStrings =\n let mapping = createStaticStringMapping defaultValue stringValues\n for str, value in stringValues do\n let v = mapping str\n v |> Equal value\n for str in defaultTestStrings do\n let v = mapping str\n v |> Equal defaultValue\n\n test 0 [] [\"\\u0000\"]\n test 0 [\"\", 1] [\"\\u0000\"]\n test 0 [\"\\u0000\", 1] [\"\"; \"a\"]\n test 0 [\"\", 1; \"\\u0000\", 2] [\"a\"]\n\n test 0 [\"\\u0000\", 1; \"\\u0001\", 2] [\"\"; \"\\u0002\"]\n test 0 [\"\\u0000\", 1; \"\\u0001\", 2; \"\\u0002\\u0003\", 3] [\"\"; \"\\u0002\"]\n test 0 [\"\", 1; \"\\u0000\", 2; \"\\u0001\", 3] [\"\\u0002\"]\n test 0 [\"\\u0001\", 3; \"\\u0000\", 2; \"\", 1] [\"\\u0002\"]\n test 0 [\"\\u0001\", 2; \"\\u0002\\u0003\", 3] [\"\"; \"\\u0000\"; \"\\u0002\"]\n test 0 [\"\\u0001\", 1; \"\\u0002\", 2] [\"\"; \"\\u0000\"]\n test 0 [\"\", 1; \"\\u0001\", 2; \"\\u0002\", 3] [\"\\u0000\"; \"a\"]\n test 0 [\"\\u0001\", 1; \"\\u0002\", 1] [\"\"; \"\\u0000\"; \"a\"]\n test 0 [\"\", 1; \"\\u0001\", 2; \"\\u0002\", 2] [\"\\u0000\"; \"a\"]\n test 0 [\"\", 1; \"\\u0000\", 2; \"\\u0001\", 2] [\"\\u0002\"; \"a\"]\n\n\n test 0 [\"prefix1\", 1; \"prefix2\", 2; \"prefix3\", 3] [\"\"; \"prefix\"; \"prefix\\u0000\"]\n test 0 [\"prefix1\", 2; \"prefix2\", 2; \"prefix3\", 2] [\"\"; \"prefix\"; \"prefix\\u0000\"]\n test 0 [\"prefix1postfix\", 2; \"prefix2postfix\", 2; \"prefix3postfix.\", 2] [\"\"; \"prefix\"; \"prefix\\u0000\"]\n test 0 [\"\", -1; \"prefix\", 1; \"prefix1\", 1; \"prefix2\", 2; \"prefix3\", 3] [\"prefix\\u0000\"]\n\n test 0 [\"\", -1; \"postfix1\", 1; \"postfix2\", 2;\n \"test/postfix1\", 1; \"test/postfix2\", 2;\n \"test/test/postfix1\", 1; \"test/test/postfix2\", 2;\n \"test/test/test/postfix1\", 1; \"test/test/test/postfix2\", 2;\n \"test/test/test|postfix1\", 1; \"test/test/test|postfix2\", 2] []\n\n test 0\n [|\"abstract\", 1; \"and\", 2; \"as\", 3; \"assert\", 4;\n \"base\", 5; \"begin\", 6; \"class\", 7; \"default\", 8;\n \"delegate\", 9; \"done\", 10; \"downcast\", 11; \"downto\", 12;\n \"elif\", 13; \"else\", 14; \"end\", 15; \"exception\", 16;\n \"extern\", 17; \"finally\", 18; \"for\", 19; \"fun\", 20;\n \"function\", 21; \"if\", 22; \"in\", 23; \"inherit\", 24;\n \"inline\", 25; \"interface\", 26; \"internal\", 27; \"lazy\", 28;\n \"match\", 29; \"member\", 30; \"module\", 31; \"mutable\", 32;\n \"namespace\", 33; \"new\", 34; \"of\", 35; \"open\", 36;\n \"or\", 37; \"override\", 38; \"private\", 39; \"public\", 40;\n \"rec\", 41; \"static\", 42; \"struct\", 43; \"then\", 44;\n \"to\", 45; \"try\", 46; \"type\", 47; \"upcast\", 48;\n \"use\", 49; \"val\", 50; \"void\", 51; \"when\", 52;\n \"with\", 53; \"false\", 54; \"true\", 55; \"let\", 56;\n \"do\", 57; \"while\", 58; \"yield\", 59; \"return\", 60;\n \"asr\", 61;\" land\", 61; \"lor\", 63; \"lsl\", 64;\n \"lsr\", 65; \"lxor\", 66; \"mod\", 67; \"sig\", 68;\n \"atomic\", 69; \"break\", 70; \"checked\", 71; \"component\", 72;\n \"const\", 73; \"constraint\", 74; \"constructor\", 75; \"continue\", 76;\n \"eager\", 77; \"event\", 78; \"external\", 79; \"fixed\", 80;\n \"functor\", 81; \"global\", 82; \"include\", 83; \"method\", 84;\n \"mixin\", 85; \"object\", 86; \"parallel\", 87; \"process\", 88;\n \"protected\", 89; \"pure\", 90; \"sealed\", 91; \"tailcall\", 92;\n \"trait\", 93; \"virtual\", 94; \"volatile\", 95|]\n []\n\n test \"\" [\"1\", \"1\"; \"2\", \"2\"; \"221\", \"221\"; \"222\", \"222\"; \"3\", \"1\"; \"4\", \"4\"] [\"\"; \"0\"]\n test null [\"1\", \"1\"; \"2\", \"2\"; \"221\", \"221\"; \"222\", \"222\"; \"3\", \"1\"; \"4\", \"\"] [\"\"; \"0\"]\n test (TestStruct(0)) [\"1\", TestStruct(1); \"2\", TestStruct(2); \"221\", TestStruct(221); \"222\", TestStruct(222); \"3\", TestStruct(1); \"4\", TestStruct(0)] [\"\"; \"0\"]\n test (TestStruct2(0)) [\"1\", TestStruct2(1); \"2\", TestStruct2(2); \"221\", TestStruct2(221); \"222\", TestStruct2(222); \"3\", TestStruct2(1); \"4\", TestStruct2(0)] [\"\"; \"0\"]\n test FParsec.Associativity.Left [\"1\", FParsec.Associativity.None; \"2\", FParsec.Associativity.Right; \"3\", FParsec.Associativity.None] [\"\"; \"4\"]\n\n try createStaticStringMapping 0 [null, 1] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticStringMapping 0 [\"1\", 1; null, 2] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticStringMapping 0 [\"\", 1; \"\", 2] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticStringMapping 0 [\"1\", 1; \"\", 2; \"3\", 3; \"\", 4] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticStringMapping 0 [\"1\", 1; \"1\", 2] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n try createStaticStringMapping 0 [\"0\", 1; \"1\", 2; \"3\", 3; \"1\", 4] |> ignore; Fail()\n with :? System.ArgumentException -> ()\n\n try createStaticStringMapping 0 [] null |> ignore; Fail()\n with :? System.NullReferenceException\n | :? System.ArgumentNullException -> ()\n\n try createStaticStringMapping 0 [\"1\", 1] null |> ignore; Fail()\n with :? System.NullReferenceException\n | :? System.ArgumentNullException -> ()\n\n try createStaticStringMapping 0 [\"1\", 1; \"2\", 2] null |> ignore; Fail()\n with :? System.NullReferenceException\n | :? System.ArgumentNullException -> ()\n\nlet run() =\n testCreateIndicatorFunction()\n testCreateStaticIntMapping()\n testCreateStaticStringMapping()\n\n#endif\n" }, { "path": "Test/StringBufferTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2009-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.StringBufferTests\n\n#if LOW_TRUST\n\nlet run() = ()\n\n#else\n\nopen FParsec.Test.Test\n\ntype StringBuffer = FParsec.StringBuffer\n\n// This test relies on the internal assert checks in StringBuffer,\n// hence this is only really a proper test in Debug builds or\n// if you compile FParsecCS with the DEBUG_STRINGBUFFER define.\n\nlet test() =\n let ty = typeof\n let getStaticField name = getStaticField ty name\n let minChunkSize = getStaticField \"MinChunkSize\" : int\n let firstSegmentSmallSize = getStaticField \"FirstSegmentSmallSize\" : int\n let firstSegmentLargeSize = getStaticField \"FirstSegmentLargeSize\" : int\n let maxSegmentSize = getStaticField \"MaxSegmentSize\" : int\n\n let testConstructor() =\n let buffer1 = StringBuffer.Create(0)\n buffer1.Dispose()\n let buffer1 = StringBuffer.Create(firstSegmentSmallSize)\n buffer1.Dispose()\n let buffer2 = StringBuffer.Create(maxSegmentSize + 1)\n buffer2.Dispose()\n try StringBuffer.Create(System.Int32.MaxValue) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n try StringBuffer.Create(-1) |> ignore; Fail()\n with :? System.ArgumentOutOfRangeException -> ()\n\n testConstructor()\n\n let rand = System.Random(1054754)\n let maxBufferSize = 196608\n let maxTotalSize = 1 <<< 22\n let buffers = new ResizeArray<_>()\n let mutable allocated = 0\n let mutable maxReached = false\n for i = 1 to 10000 do\n if (not maxReached && rand.Next(2) = 0) || buffers.Count = 0 then\n let maxSize = rand.Next(maxBufferSize + 1)\n let n = rand.Next(1, 11)\n for i = 1 to n do\n let size = rand.Next(maxSize + 1)\n if allocated + size < maxTotalSize then\n let buffer = StringBuffer.Create(size)\n allocated <- allocated + buffer.Length\n buffers.Add(buffer)\n else\n maxReached <- true\n else\n maxReached <- false\n let n = rand.Next(1, buffers.Count + 1)\n for i = 1 to n do\n let idx = rand.Next(buffers.Count)\n let buffer = buffers[idx]\n allocated <- allocated - buffer.Length\n buffer.Dispose()\n buffers.RemoveAt(idx)\n buffers.Reverse()\n for b in buffers do b.Dispose()\n\nlet run() =\n test()\n\n#endif\n" }, { "path": "Test/Test-LowTrust.fsproj", "content": "\n \n net6\n \n\n \n\n \n \n \n \n\n" }, { "path": "Test/Test.fs", "content": "// Copyright (c) Stephan Tolksdorf 2007-2009\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.Test\n\nopen System.Runtime.CompilerServices\n#if NETCORE\nopen System.Reflection\n#endif\n\nopen FParsec\nopen FParsec.Error\nopen FParsec.Primitives\n\nexception TestFailed of string\n\nlet Fail() = raise (TestFailed(\"Test failed.\"))\n\nlet True cond =\n if not cond then Fail ()\n\nlet False cond =\n if cond then Fail ()\n\nlet IsNull x =\n match box x with\n | null -> ()\n | _ -> Fail()\n\n[]\nlet EqualFail_ a b =\n Fail()\n\n// inline to allow the F# compiler to optimize the equality comparison\nlet inline Equal a b =\n if not (a = b) then EqualFail_ a b\n\nlet NotEqual a b =\n if a = b then Fail ()\n\nlet ReferenceEqual (a: 't) (b: 't) =\n if not (System.Object.ReferenceEquals(a, b)) then Fail ()\n\n\n\nlet private ROkE_ withNewline (content: string) nSkippedChars result error (parser: Parser<_,_>) =\n use stream = new CharStream(content, 0, content.Length)\n let mutable reply = parser stream\n if reply.Status <> Ok\n || reply.Result <> result\n || reply.Error <> error\n || stream.Index <> (int64 nSkippedChars)\n || stream.LineBegin = 0L <> (not withNewline)\n then\n System.Diagnostics.Debugger.Break()\n stream.Seek(0L)\n stream.SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(1L)\n stream.SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(0L)\n reply <- parser stream\n reply.Status |> Equal Ok\n reply.Result |> Equal result\n reply.Error |> Equal error\n stream.Index |> Equal (int64 nSkippedChars)\n stream.LineBegin = 0L |> Equal (not withNewline)\n\nlet ROk content nSkippedChars result parser = ROkE_ false content nSkippedChars result NoErrorMessages parser\nlet ROkE content nSkippedChars result error parser = ROkE_ false content nSkippedChars result error parser\nlet ROkNL content nSkippedChars result parser = ROkE_ true content nSkippedChars result NoErrorMessages parser\n\nlet private RError_ status (content: string) nSkippedChars error (parser: Parser<_,_>) =\n use stream = new CharStream(content, 0, content.Length)\n let mutable reply = parser stream\n if reply.Status <> status\n || reply.Error <> error\n || stream.Index <> (int64 nSkippedChars)\n then\n System.Diagnostics.Debugger.Break()\n stream.Seek(0L)\n stream.SetLine_WithoutCheckAndWithoutIncrementingTheStateTag(1L)\n stream.SetLineBegin_WithoutCheckAndWithoutIncrementingTheStateTag(0L)\n reply <- parser stream\n reply.Status |> Equal Error\n reply.Error |> Equal error\n stream.Index |> Equal (int64 nSkippedChars)\n\n\nlet RError content nSkippedChars error parser = RError_ Error content nSkippedChars error parser\nlet RFatalError content nSkippedChars error parser = RError_ FatalError content nSkippedChars error parser\n\n\n\n//let EqualParser (parser1: Parser<'a,'u>) state (parser2: Parser<'a,'u>) =\n// let reply1 = parser1 state\n// let reply2 = parser2 state\n// Equal reply1 reply2\n\n\n// we use the following flag to allow test parsers with mutable state\n// to repeat the last action in order to simplify debugging\nlet mutable checkParserRepeat = false\n\nlet checkParser (parser1: Parser<'a,'u>) (parser2: Parser<'a,'u>) (stream: CharStream<'u>) =\n let state0 = stream.State\n\n let mutable reply1 = parser1 stream\n let mutable state1 = stream.State\n let mutable index1 = stream.Index\n\n stream.BacktrackTo(state0)\n\n let mutable reply2 = parser2 stream\n let mutable state2 = stream.State\n let mutable index2 = stream.Index\n\n if reply1 <> reply2\n || index1 <> index2\n || state1.Line <> state2.Line\n || state1.LineBegin <> state2.LineBegin\n || state1.Name <> state2.Name\n || state1.UserState <> state2.UserState\n || (state1.Tag <> state0.Tag) <> (state2.Tag <> state0.Tag)\n then\n if System.Diagnostics.Debugger.IsAttached then\n System.Diagnostics.Debugger.Break()\n checkParserRepeat <- true\n // step into the following parser calls to see what goes wrong\n\n stream.BacktrackTo(state0)\n\n reply1 <- parser1 stream\n state1 <- stream.State\n index1 <- stream.Index\n\n stream.BacktrackTo(state0)\n\n reply2 <- parser2 stream\n state2 <- stream.State\n index2 <- stream.Index\n\n stream.BacktrackTo(state0)\n\n Equal reply1.Status reply2.Status\n Equal reply1.Error reply2.Error\n if reply1.Status = Ok then\n Equal reply1.Result reply2.Result\n Equal index1 stream.Index\n Equal state1.Line stream.Line\n Equal state1.LineBegin stream.LineBegin\n Equal state1.Name stream.Name\n Equal state1.UserState stream.UserState\n Equal (state1.Tag <> state0.Tag) (stream.StateTag <> state0.Tag)\n\nlet checkParserStr parser1 parser2 (str: string) =\n use stream = new CharStream(str, 0, str.Length)\n checkParser parser1 parser2 stream\n\nlet constantTestParsers r e : Parser<'a, int>[] = [| // we rely on the order of these parsers\n fun s -> Reply(Ok, r, e);\n fun s -> s.UserState <- s.UserState + 1; Reply(Ok, r, e)\n fun s -> Reply(Error, e);\n fun s -> s.UserState <- s.UserState + 1; Reply(Error, e);\n fun s -> Reply(FatalError, e);\n fun s -> s.UserState <- s.UserState + 1; Reply(FatalError, e);\n|]\n\n\n/// Builds a parser from a list of constant test parsers. The first parser\n/// will be used for the first invocation, the second for the next\n/// invocation, and so on. The reset function can be used to reset the aggregate parser.\nlet seqParserAndReset ps =\n let mutable psr = ps\n let mutable inRepeat = false\n (fun stream ->\n if checkParserRepeat && not inRepeat then\n inRepeat <- true\n psr <- ps\n match psr with\n | hd::tl -> psr <- tl; hd stream\n | [] -> Reply(Error, NoErrorMessages)),\n (fun () -> psr <- ps)\n\nlet seqParserAndReset2 ps =\n let p1, p1r = seqParserAndReset ps\n let p2, p2r = seqParserAndReset ps\n p1, p2, (fun () -> p1r(); p2r())\n\nlet setStaticField (t: System.Type) name v =\n t.GetField(name, BindingFlags.NonPublic ||| BindingFlags.Static).SetValue(null, v)\n\nlet getStaticField (t: System.Type) name =\n unbox (t.GetField(name, BindingFlags.NonPublic ||| BindingFlags.Static).GetValue())\n\n\nlet shuffleArray (rand: System.Random) (xs: 'a[]) =\n let n = xs.Length\n for i = 0 to n - 2 do\n let r = rand.Next(n - i - 1);\n let t = xs[i]\n xs[i] <- xs[i + r]\n xs[i + r] <- t\n\nlet inline _1< ^t when ^t : (static member One : ^t) > = LanguagePrimitives.GenericOne< ^t >\n" }, { "path": "Test/Test.fsproj", "content": "\n \n net6\n \n\n \n\n \n \n \n \n\n" }, { "path": "Test/Test.targets", "content": "\n \n Test\n FParsec.Test\n Exe\n false\n \n\n \n\n \n $(DefineConstants);USE_STATIC_MAPPING_FOR_IS_ANY_OF;DISABLE_STREAM_BACKTRACKING_TESTS\n \n\n \n $(DefineConstants);NETCORE\n \n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n\n" }, { "path": "Test/TextTests.fs", "content": "// Copyright (c) Stephan Tolksdorf 2009-2010\n// License: Simplified BSD License. See accompanying documentation.\n\nmodule FParsec.Test.TextTests\n\n#if NETCORE\nopen System.Reflection\n#endif\n\nopen FParsec.Test.Test\n\ntype Text = FParsec.Text\n\nlet testFoldCase() =\n Text.FoldCase(null) |> Equal null\n for s in [\"\"; \"a\"; \"aa\"; \"aaa\"] do\n Text.FoldCase(s) |> ReferenceEqual s\n\n Text.FoldCase(\"A\") |> Equal \"a\"\n Text.FoldCase(\"aA\") |> Equal \"aa\"\n Text.FoldCase(\"aaA\") |> Equal \"aaa\"\n Text.FoldCase(\"abcAOUÄÖÜdef\") |> Equal \"abcaouäöüdef\"\n\n\n let oneToOneMappings =\n let a = typeof.Assembly\n getStaticField (a.GetType(\"FParsec.CaseFoldTable\")) \"oneToOneMappings\" : string\n\n let mutable j = 0\n for i in 0..2..(oneToOneMappings.Length - 2) do\n let c = int oneToOneMappings[i]\n for k = j to c - 1 do\n Text.FoldCase((char k).ToString())[0] |> Equal (char k)\n Text.FoldCase((char c).ToString())[0] |> Equal oneToOneMappings[i + 1]\n j <- c + 1\n j |> Equal 0xff3b\n\nlet testNormalizeNewlines() =\n Text.NormalizeNewlines(null) |> Equal null\n Text.NormalizeNewlines(\"\") |> ReferenceEqual \"\"\n Text.NormalizeNewlines(\"ab\") |> ReferenceEqual \"ab\"\n\n let check (cs: char[]) n =\n let str = new string(cs, 0, n)\n let nstr = str.Replace(\"\\r\\n\", \"\\n\").Replace(\"\\r\", \"\\n\")\n let nstr2 = Text.NormalizeNewlines(str)\n Equal nstr nstr2\n\n let rec test (cs: char[]) n i =\n if i < n then\n cs[i] <- '\\r'\n test cs n (i + 1)\n cs[i] <- '\\n'\n test cs n (i + 1)\n cs[i] <- '_'\n test cs n (i + 1)\n else\n check cs n\n\n let N = 10\n let cs = Array.zeroCreate N\n\n for n = 1 to 8 do\n // test all possible character sequences of length n consisting of '\\r','\\n' or '_' chars\n test cs n 0\n\n // make sure there is no size-specific copying problem\n for n = 1 to 24 do\n let s = new string('_', n)\n Text.NormalizeNewlines(\"\\r\" + s) |> Equal (\"\\n\" + s)\n Text.NormalizeNewlines(\"\\r\\n\" + s) |> Equal (\"\\n\" + s)\n\n Text.NormalizeNewlines(\"_\\n_\\r\\n_\\r\\r_\\r\\n_\\r\\n\\n\\r_\") |> Equal \"_\\n_\\n_\\n\\n_\\n_\\n\\n\\n_\"\n\nlet testCountTextElements() =\n let countTextElementsRef s =\n let te = System.Globalization.StringInfo.GetTextElementEnumerator(s)\n let mutable count = 0\n while te.MoveNext() do count <- count + 1\n count\n\n let chars = [|\"\\u0020\"; \"\\u007e\"; \"\\U0001D41A\";\n \"\\u001F\";\" \\u007F\"; // control\n \"\\u00AD\"; \"\\U0001D173\"; // format\n string '\\ud800'; // surrogate (uses string '...' to work around an fsc parser issue)\n \"\\u0333\"; \"\\U000101FD\" // nonspacing mark\n \"\\u0BBE\"; \"\\U0001D166\" // spacing combining mark\n \"\\u20DD\" // enclosing mark\n |]\n\n for c in chars do\n Text.CountTextElements(c) |> Equal (countTextElementsRef c)\n\n for c1 in chars do\n for c2 in chars do\n let s = c1 + c2\n Text.CountTextElements(s) |> Equal (countTextElementsRef s)\n\n let rand = System.Random(1234)\n let strings = Array.zeroCreate 5\n for i = 0 to 100000 do\n for j = 0 to strings.Length - 1 do\n strings[j] <- chars[rand.Next()%chars.Length]\n let s = System.String.Concat(strings)\n Text.CountTextElements(s) |> Equal (countTextElementsRef s)\n\nlet testIsSurrogate() =\n for c = 0 to 0xffff do\n let c = char c\n Text.IsSurrogate(c) |> Equal (System.Char.IsSurrogate(c))\n Text.IsLowSurrogate(c) |> Equal (System.Char.IsLowSurrogate(c))\n Text.IsHighSurrogate(c) |> Equal (System.Char.IsHighSurrogate(c))\n\nlet testIsWhitespace() =\n for c = 0 to 0xffff do\n Text.IsWhitespace(char c) |> Equal (System.Char.IsWhiteSpace(char c))\n\n\nlet run() =\n testNormalizeNewlines()\n testFoldCase()\n testCountTextElements()\n testIsWhitespace()\n testIsSurrogate()" }, { "path": "global.json", "content": "{\n \"sdk\": {\n \"version\": \"7.0.100\",\n \"rollForward\": \"latestMajor\"\n }\n}\n" }, { "path": "pack.ps1", "content": "# This PowerShell script builds the FParsec NuGet packages. \n#\n# Run this script from the VS2019 Command Prompt, e.g. with \n# powershell -ExecutionPolicy ByPass -File pack.ps1 -versionSuffix \"\" > pack.out.txt\n# or on macOS e.g. with\n# pwsh -File pack.ps1 -versionSuffix \"\" > pack.out.txt\n\nParam(\n [string]$versionSuffix = \"dev\"\n)\n\n$ErrorActionPreference = 'Stop'\n\n$configSuffices = $('-LowTrust') # The non-LowTrust version currently doesn't pass the tests.\n\n$testTargetFrameworks = @{'' = $('net6')\n '-LowTrust' = $('net6')}\n\nfunction invoke([string] $cmd) {\n echo ''\n echo $cmd\n Invoke-Expression $cmd\n if ($LastExitCode -ne 0) {\n throw \"Non-zero exit code: $LastExitCode\"\n }\n}\n\nforeach ($folder in $(\"nupkgs\", \"FParsecCS\\obj\", \"FParsecCS\\bin\", \"FParsec\\obj\", \"FParsec\\bin\")) {\n try {\n Remove-Item $folder -recurse\n } catch {}\n}\n\nforeach ($configSuffix in $configSuffices) {\n $config = \"Release$configSuffix\"\n $props = \"-c $config -p:VersionSuffix=$versionSuffix -p:FParsecNuGet=true -p:Platform=AnyCPU\"\n invoke \"dotnet build FParsec/FParsec$configSuffix.fsproj $props -v n\"\n invoke \"dotnet pack FParsec/FParsec$configSuffix.fsproj $props -o \"\"$pwd\\nupkgs\"\"\"\n invoke \"dotnet build Test/Test$configSuffix.fsproj $props -v n\"\n foreach ($tf in $testTargetFrameworks[$configSuffix]) {\n invoke \"dotnet run --no-build --project Test/Test$configSuffix.fsproj $props\"\n }\n}\n" }, { "path": "readme.md", "content": "# FParsec\n\nFParsec is a [parser combinator](https://en.wikipedia.org/wiki/Parser_combinator) library for [F#](http://fsharp.org/).\n\nWith FParsec you can implement [recursive‐descent](https://en.wikipedia.org/wiki/Recursive_descent_parser) text parsers for [formal grammars](https://en.wikipedia.org/wiki/Formal_grammar).\n\nFParsec’s features include:\n- support for context‐sensitive, infinite look‐ahead grammars,\n- automatically generated, highly readable error messages,\n- Unicode support,\n- efficient support for very large files,\n- an embeddable, runtime‐configurable [operator‐precedence parser](https://en.wikipedia.org/wiki/Operator-precedence_parser) component,\n- a simple, efficient and easily extensible API,\n- an implementation thoroughly optimized for performance,\n- comprehensive documentation,\n- a permissive open source license.\n\n## Documentation\n\n- [FParsec vs alternatives](http://www.quanttec.com/fparsec/about/fparsec-vs-alternatives.html)\n- [NuGet packages and building FParsec from source](http://www.quanttec.com/fparsec/download-and-installation.html)\n- [Tutorial](http://www.quanttec.com/fparsec/tutorial.html)\n- [User's guide](http://www.quanttec.com/fparsec/users-guide/)\n- [Parser quick reference](http://www.quanttec.com/fparsec/reference/parser-overview.html)\n- [Reference](http://www.quanttec.com/fparsec/reference/)\n\n## License\n\n- *Code*: 2-clause BSD license (\"Simplified BSD License\")\n- *Data*: FParsec includes some data derived from the Unicode Character Database which is distributed under the [Unicode, Inc. License Agreement](http://www.unicode.org/copyright.html#Exhibit1).\n- *Documentation*: Creative Commons Attribution‐NonCommercial 3.0 Unported License\n\nSee the [www.quanttec.com/fparsec/license.html](http://www.quanttec.com/fparsec/license.html) for more details.\n\n" } ]