[ { "path": ".github/workflows/build.yml", "content": "# This workflow uses actions that are not certified by GitHub.\n# They are provided by a third-party and are governed by\n# separate terms of service, privacy policy, and support\n# documentation.\n\nname: Build\n\non:\n push:\n branches: [ \"master\" ]\n pull_request:\n branches: [ \"master\" ]\n\npermissions:\n contents: read\n\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - name: Set up JDK 8\n uses: actions/setup-java@v3\n with:\n java-version: '8'\n distribution: 'temurin'\n cache: 'sbt'\n - name: Run tests\n run: sbt test\n - name: Build one-jar\n run: sbt one-jar\n - name: Upload a Build Artifact\n uses: actions/upload-artifact@v3.1.2\n with:\n name: postgresql-to-sqlite\n path: target/scala-2.11/postgresql-to-sqlite*.jar\n" }, { "path": ".gitignore", "content": "*.class\n*.log\n\n# sbt specific\n.cache\n.history\n.lib/\ndist/*\ntarget/\nlib_managed/\nsrc_managed/\nproject/boot/\nproject/plugins/project/\n\n# Scala-IDE specific\n.scala_dependencies\n.worksheet\n/bin/\n" }, { "path": ".travis.yml", "content": "language: scala\nscala:\n - 2.11.12\nbranches:\n only:\n - master\njdk:\n - openjdk9\n\nscript:\n - sbt ++$TRAVIS_SCALA_VERSION test\n - sbt ++$TRAVIS_SCALA_VERSION one-jar\n" }, { "path": "Dockerfile", "content": "FROM hseeberger/scala-sbt:8u222_1.3.5_2.13.1\r\nENV psource=database.dump\r\nENV starget=sqllight.db\r\nRUN mkdir -p /p2s\r\nWORKDIR /p2s\r\nCOPY . ./\r\nRUN sbt one-jar\r\nRUN cp target/scala-2.11/postgresql-to-sqlite_2.11-*-one-jar.jar pg2sqlite.jar\r\nCMD exec java -jar pg2sqlite.jar -d \"$psource\" -o \"$starget\"\r\n" }, { "path": "LICENSE", "content": "The MIT License (MIT)\n\nCopyright (c) 2015 Aleksander Guryanov\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n" }, { "path": "README.md", "content": "# postgresql-to-sqlite (pg2sqlite)\n[![Build](https://github.com/caiiiycuk/postgresql-to-sqlite/actions/workflows/build.yml/badge.svg)](https://github.com/caiiiycuk/postgresql-to-sqlite/actions/workflows/build.yml)\n\nEasy to use solution to create sqlite database from postgresql dump.\n\n* default [`pg_dump`](http://www.postgresql.org/docs/9.4/static/app-pgdump.html) script format\n* as fast as possible\n* silently ignore unsupported postgresql features\n* gzip support\n\n## Installing\n\nIn [release section](https://github.com/caiiiycuk/postgresql-to-sqlite/releases/) you can download pre-built version of pg2sqlite.jar\n\n## How to use\n\n1. Install jre (java) on your PC\n\n2. Create dump from postgresql database\n```sh\npg_dump -h host -U user -f database.dump database\n```\n\n3. Make sqlite database from it\n```\njava -jar pg2sqlite-1.0.3.jar -d database.dump -o sqlite.db\n```\n\n## Command line arguments\n\n`pg2sqlite -d -o [-f ]`\n\n* **-d** `` - file that contains dump of postgresql database (made by pg_dump, accepts .gz)\n* **-o** `` - file name of newly created sqlite3 database\n* **-f** `` - default: false, force database re-creation if database file alredy exists\n* **-t** `` - default: integer, change sqlite3 date class (read below)\n\n## Timestamps\n\nSQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in [Date And Time Functions](https://www.sqlite.org/lang_datefunc.html) of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:\n\n* TEXT as ISO8601 strings (\"YYYY-MM-DD HH:MM:SS.SSS\").\n* REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.\n* INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.\n\nBy default pg2sqlite uses **INTEGER** to store dates, but you can change this with **-t** argument (`-t text` or `-t real`), use it like this:\n\n```sh\njava -jar pg2sqlite-1.0.3.jar -d database.dump -o sqlite.db -t text\n```\n\n## Tips\n\npg2sqlite does not support database schemas. If your dump file includes schema definition It will print errors like this:\n```\nCreate Table - Exception:\nunknown database \n[SQL] 'CREATE TABLE .table (...;'\n```\nYou can easily fix dump file with `sed`:\n```sh\n# sed 's/\\.//' -i database.dump\nsed 's/public\\.//' -i database.dump\npg2sqlite -d output.dump -o sqlite.db\n```\nWhere `public` is a schema name.\n\n## How to build\n```sh\ngit clone https://github.com/caiiiycuk/postgresql-to-sqlite.git\ncd postgresql-to-sqlite\nsbt one-jar\ncp target/scala-2.11/postgresql-to-sqlite_2.11-0.0.1-SNAPSHOT-one-jar.jar pg2sqlite.jar\n```\n\n## Docker\n\nClone the repository and run \n```\ndocker build -t postgresql-to-sqlite:latest .\n```\ninside the postgresql-to-sqlite folder.\n\nUse \n```\ndocker run -v /home/john/dbdata:/dbdata -e psource='/dbdata/pqdump.sql' -e starget='/dbdata/output.sqlite' -it postgresql-to-sqlite:latest\n```\nwhere\n- -v: is the volume where the pqdump file is located. (and later the output file)\n- -e: `psource` is the pqdump filename and folder & `starget` the sqlite filename and folder\n\np.s. the schema removal has to be done outside the container\n\n## Support\n\nIf you appreciate this project, please consider voting for it on Stack Overflow:\n\nhttps://stackoverflow.com/questions/6148421/how-to-convert-a-postgres-database-to-sqlite/69293251#69293251\n" }, { "path": "project/Build.scala", "content": "import com.github.retronym.SbtOneJar\nimport sbt._\nimport Keys._\n\nobject Build extends Build {\n\n lazy val project = Project(\"root\", file(\".\"), settings = Seq(\n name := \"postgresql-to-sqlite\",\n organization := \"com.github.caiiiycuk\",\n version := \"1.1.1\",\n scalaVersion := \"2.11.12\",\n\n libraryDependencies ++= Seq(\n \"com.github.scopt\" %% \"scopt\" % \"3.3.0\",\n \"ch.qos.logback\" % \"logback-classic\" % \"1.1.2\",\n \"org.xerial\" % \"sqlite-jdbc\" % \"3.42.0.0\",\n \"org.scalatest\" %% \"scalatest\" % \"2.2.4\" % \"test\"\n )\n ) ++ SbtOneJar.oneJarSettings)\n\n}\n" }, { "path": "project/build.properties", "content": "sbt.version=0.13.18\n" }, { "path": "project/plugins.sbt", "content": "addSbtPlugin(\"com.typesafe.sbteclipse\" % \"sbteclipse-plugin\" % \"4.0.0\")\n\naddSbtPlugin(\"org.scalastyle\" %% \"scalastyle-sbt-plugin\" % \"0.7.0\")\n\naddSbtPlugin(\"org.scala-sbt.plugins\" % \"sbt-onejar\" % \"0.8\")\n" }, { "path": "scalastyle-config.xml", "content": "\n Scalastyle standard configuration\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/Boot.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nimport com.github.caiiiycuk.pg2sqlite.command.CommandException\nimport com.github.caiiiycuk.pg2sqlite.iterator.LineIterator\nimport com.github.caiiiycuk.pg2sqlite.values.ValueParseException\n\nimport ch.qos.logback.classic.Level\n\nobject Boot extends App with Log {\n\n val root = org.slf4j.LoggerFactory.getLogger(org.slf4j.Logger.ROOT_LOGGER_NAME).asInstanceOf[ch.qos.logback.classic.Logger]\n root.setLevel(Level.INFO)\n\n val config = Config.parse(args)\n import config._\n\n val size = pgdump.length()\n val connection = Connection.sqlite(sqlite, config.dateClass)\n val iterator = LineIterator(pgdump)\n val loggedIterator = LoggedIterator(iterator, () => 100.0 * iterator.readed / size)\n val dumpInserter = new DumpInserter(connection)\n\n log.info(s\"'$pgdump' (${toMb(size)} Mb) -> '$sqlite'\")\n\n val success = try {\n dumpInserter.insert(loggedIterator)\n true\n } catch {\n case e: CommandException =>\n log.error(e.getMessage)\n false\n case e: ValueParseException =>\n log.error(e.getMessage)\n false\n case e: Throwable =>\n log.error(e.getMessage, e)\n false\n }\n\n iterator.close\n connection.close\n\n if (success) {\n log.info(\"Well done...\")\n } else {\n log.error(\"Task failed...\")\n }\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/Config.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nimport java.io.File\n\ncase class Config(pgdump: File = new File(\"dump\"), sqlite: File = new File(\"db\"),\n force: Boolean = false, dateClass: String = Connection.DEFAULT_DATE_CLASS)\n\nobject Config extends Log {\n private val parser = new scopt.OptionParser[Config](\"postgresql-to-sqlite\") {\n head(\"postgresql-to-sqlite\")\n\n opt[File]('d', \"dump\") required() valueName (\"\") action { (v, c) =>\n c.copy(pgdump = v)\n } text (\"postgresql dump generated by pg_dump\")\n\n opt[File]('o', \"out\") required() valueName (\"\") action { (v, c) =>\n c.copy(sqlite = v)\n } text (\"sqlite3 database to create\")\n\n opt[Boolean]('f', \"force\") optional() valueName (\"\") action { (v, c) =>\n c.copy(force = v)\n } text (\"recreate database if exists\")\n\n opt[String]('t', \"timestamps\") optional() valueName (\"\") action { (v, c) =>\n val dc = v.toUpperCase()\n if (dc.equals(Connection.TEXT_DATE_CLASS) || dc.equals(Connection.REAL_DATE_CLASS)) {\n c.copy(dateClass = dc)\n } else {\n c\n }\n } text (\"Change sqlite3 date class (default: INTEGER)\")\n\n checkConfig { c =>\n import c._\n\n if (!pgdump.exists()) {\n failure(s\"Dump '${pgdump}' does not exists\")\n } else if (sqlite.exists()) {\n if (force) {\n sqlite.delete()\n success\n } else {\n failure(s\"Database '${sqlite}' already exists\")\n }\n } else {\n success\n }\n }\n }\n\n def parse(args: Array[String]) = {\n parser.parse(args, Config()) match {\n case Some(config) =>\n Option(System.getenv(\"SQLITE_TMPDIR\")) match {\n case None =>\n log.warn(\"You should set SQLITE_TMPDIR environment variable to control where sqlite stores temp files\")\n case _ =>\n }\n\n config\n case _ =>\n System.exit(1)\n ???\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/Connection.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nimport org.sqlite.SQLiteConfig\n\nimport java.sql.DriverManager\nimport java.sql.Statement\nimport java.sql.PreparedStatement\nimport scala.collection.mutable.ListBuffer\nimport java.sql.ResultSet\nimport scala.annotation.tailrec\nimport java.io.File\nimport java.util.Properties\n\ntrait ConnectionHolder {\n def makeConnection: java.sql.Connection\n\n def db: String\n}\n\nobject Connection {\n final val DEFAULT_DATE_CLASS = \"INTEGER\"\n final val TEXT_DATE_CLASS = \"TEXT\"\n final val REAL_DATE_CLASS = \"REAL\"\n private final val DATE_CLASS_PRAGMA = \"date_class\"\n private final val FETCH_SIZE = 8192\n private final val MAX_VARIABLE_NUMBER = 999\n\n def sqlite(dbFile: File, dateClass: String = DEFAULT_DATE_CLASS): Connection = {\n val connectionHolder = new ConnectionHolder {\n override def makeConnection: java.sql.Connection = {\n val properties = new Properties()\n properties.setProperty(DATE_CLASS_PRAGMA, dateClass)\n implicit val connection = DriverManager.getConnection(s\"jdbc:sqlite:$dbFile\", properties)\n\n connection.setAutoCommit(true)\n sqlitePragmas()\n\n connection.setAutoCommit(false)\n connection\n }\n\n override def db = dbFile.toString\n }\n\n new Connection(connectionHolder)\n }\n\n private def sqlitePragmas()(implicit connection: java.sql.Connection) = {\n assert(SQLiteConfig.Pragma.DATE_CLASS.pragmaName.equals(DATE_CLASS_PRAGMA));\n val statement = connection.createStatement()\n statement.executeUpdate(s\"PRAGMA ${SQLiteConfig.Pragma.SYNCHRONOUS.pragmaName} = OFF\")\n statement.executeUpdate(s\"PRAGMA ${SQLiteConfig.Pragma.JOURNAL_MODE.pragmaName} = OFF\")\n statement.executeUpdate(s\"PRAGMA ${SQLiteConfig.Pragma.LIMIT_WORKER_THREADS.pragmaName} = 64\")\n statement.executeUpdate(s\"PRAGMA ${SQLiteConfig.Pragma.MAX_PAGE_COUNT.pragmaName} = 2147483646\")\n statement.executeUpdate(s\"PRAGMA ${SQLiteConfig.Pragma.CACHE_SIZE.pragmaName} = 65536\")\n statement.executeUpdate(\"PRAGMA cache_spill = true\")\n statement.close\n }\n}\n\nclass Connection(connectionHolder: ConnectionHolder) {\n\n import Connection._\n\n final val MAX_VARIABLE_NUMBER = Connection.MAX_VARIABLE_NUMBER\n\n lazy val connection = connectionHolder.makeConnection\n\n lazy val db = connectionHolder.db\n\n def withStatement[T](block: (Statement) => T): T = {\n val statement = connection.createStatement()\n val t = block(statement)\n statement.close\n t\n }\n\n def withPreparedStatement[T](sql: String, keepAlive: Boolean = false)(block: (PreparedStatement) => T): T = {\n val statement = connection.prepareStatement(sql)\n statement.setFetchSize(FETCH_SIZE)\n\n val t = block(statement)\n if (!keepAlive) statement.close\n t\n }\n\n def close = {\n connection.commit\n connection.close\n }\n\n def execute(sql: String) = {\n withStatement { statement =>\n statement.executeUpdate(sql)\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/DumpInserter.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nimport scala.annotation.tailrec\nimport com.github.caiiiycuk.pg2sqlite.command._\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.schema.Schema\n\nobject DumpInserter {\n val COMMANDS = List(CreateTable, Copy, CreateIndex)\n}\n\nclass DumpInserter(connection: Connection) {\n\n import DumpInserter._\n\n implicit val schema = new Schema()\n\n @tailrec\n final def insert(iterator: Iterator[Line]): Unit = {\n if (iterator.hasNext) {\n val head = iterator.next()\n val fullIterator = Iterator(head) ++ iterator\n\n COMMANDS.find(_.matchHead(head)).foreach { command =>\n command.apply(connection, fullIterator)\n }\n\n insert(iterator)\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/Log.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nimport org.slf4j.LoggerFactory\n\ntrait Log {\n\n protected lazy val log = LoggerFactory.getLogger(getClass)\n\n def toMb(length: Long) = {\n length / 1024 / 1024\n }\n\n def humanizeMsTime(time: Long) = {\n val ms = time % 1000\n val s = time / 1000 % 60\n val m = time / 1000 / 60\n\n s\"${m}m ${s}s ${ms}ms\"\n }\n\n def humanizeElapsedAndRemaning(startAt: Long, progress: Double): String = {\n val elapsed = System.currentTimeMillis - startAt\n val remaining = (elapsed / progress - elapsed).toInt\n\n s\"elapsed: ${humanizeMsTime(elapsed)} / remaining: ${humanizeMsTime(remaining)}\"\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/LoggedIterator.scala", "content": "package com.github.caiiiycuk.pg2sqlite\n\nobject LoggedIterator {\n final val DEFAULT_SENSIVITY = 10\n}\n\ncase class LoggedIterator[T](iterator: Iterator[T],\n progress: () => Double, sensivity: Int = LoggedIterator.DEFAULT_SENSIVITY)\n extends Iterator[T] with Log {\n\n val startAt = System.currentTimeMillis\n var currentProgress: Long = 0L\n\n override def hasNext = iterator.hasNext\n\n override def next(): T = {\n val value = iterator.next\n val newProgress = progress()\n val intProgress = (newProgress * sensivity).toLong\n\n if (intProgress > currentProgress) {\n val elapsedAndRemaining = humanizeElapsedAndRemaning(startAt, newProgress / 100)\n log.info(s\"Progress ${intProgress.toDouble / sensivity}%, ${elapsedAndRemaining}...\\t\")\n currentProgress = intProgress\n }\n\n value\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/command/Command.scala", "content": "package com.github.caiiiycuk.pg2sqlite.command\n\nimport scala.util.matching.Regex\nimport scala.annotation.tailrec\nimport com.github.caiiiycuk.pg2sqlite.Connection\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.schema.Schema\n\ntrait Command {\n\n def matchHead(head: Line): Boolean =\n matchHead(head.text)\n\n def matchHead(head: String): Boolean\n\n def apply(connection: Connection, iterator: Iterator[Line])(implicit schema: Schema)\n\n @tailrec\n final protected def takeUntil(iterator: Iterator[Line],\n when: (String) => Boolean,\n buffer: List[Line] = Nil): List[Line] = {\n if (!iterator.hasNext) {\n buffer.reverse\n } else {\n val line = iterator.next\n val newBuffer = line :: buffer\n\n if (when(line.text)) {\n newBuffer.reverse\n } else {\n takeUntil(iterator, when, newBuffer)\n }\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/command/CommandException.scala", "content": "package com.github.caiiiycuk.pg2sqlite.command\n\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\n\ncase class CommandException(command: String, cause: Throwable, context: List[String])\n extends Exception(s\"\"\"\n$command - Exception:\n\\t${cause.getMessage}\n\\t${context.mkString(\"\\n\\t\")},\n\"\"\", cause)\n\nobject CommandException {\n def apply(command: String, cause: Throwable, sql: String, rows: List[Line], context: List[String] = Nil): CommandException = {\n val default = List(s\"[SQL] '$sql'\", s\"[LINE #${rows.head.num}] ${rows.mkString(\" \")}\")\n CommandException(command, cause, default ++ context)\n }\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/command/Copy.scala", "content": "package com.github.caiiiycuk.pg2sqlite.command\n\nimport com.github.caiiiycuk.pg2sqlite.Connection\nimport com.github.caiiiycuk.pg2sqlite.Log\nimport com.github.caiiiycuk.pg2sqlite.values.LineToValues\nimport java.sql.SQLException\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.schema.Schema\nimport com.github.caiiiycuk.pg2sqlite.values.ValueParseException\nimport com.github.caiiiycuk.pg2sqlite.dsl.DSL._\n\nobject Copy extends Command with Log {\n\n import LineToValues._\n\n private val TABLE_NAME_POSITION = 1\n private val activator = \"^(?i)copy\".r\n\n override def matchHead(head: String): Boolean = {\n activator.findFirstIn(head).isDefined\n }\n\n override def apply(connection: Connection, iterator: Iterator[Line])(implicit schema: Schema) = {\n val rows = takeUntil(iterator, _.contains(\";\"))\n val rawSql = rows.mkString(\" \")\n\n val (tableName, sql, columnTypes) = try {\n val tableName = rawSql.tokens(TABLE_NAME_POSITION)\n val columns = rawSql.takeBraces.head.columns.map(_.name).toList\n\n val marks = (\"?,\" * columns.size).dropRight(1)\n val sql = s\"insert into $tableName(${columns.map(column => s\"[$column]\").mkString(\",\")}) values($marks)\"\n\n val columnTypes = schema.columnsToTypeConstants(tableName, columns)\n\n (tableName, sql, columnTypes)\n } catch {\n case t: Throwable =>\n throw CommandException(s\"COPY - Unable to find TABLE NAME or COLUMNS in '$rawSql'\",\n t, rawSql, rows)\n }\n\n if (schema.shouldExcludeTable(tableName)) {\n log.info(s\"Skipping '$sql'\")\n } else {\n log.info(s\"COPY table '$tableName'\")\n connection.withPreparedStatement(sql) { statement =>\n iterator.takeWhile(!_.startsWith(\"\\\\.\")).foreach { row =>\n val values = try {\n toValues(row.text)(columnTypes)\n } catch {\n case e: ValueParseException =>\n throw CommandException(\"COPY\", e, sql, rows,\n List(s\"[DATA #${row.num}] '$row'\",\n s\"[COLUMN,TYPE] ${schema.columns(tableName).map(_.toString).mkString(\" \")}\"))\n }\n\n try {\n values.foreach(_.apply(statement))\n statement.executeUpdate()\n } catch {\n case e: SQLException =>\n val vals = values.map(_.toString).mkString(\", \")\n throw CommandException(\"COPY\", e, sql, rows,\n List(s\"[DATA #${row.num}] '$row'\", s\"[VALUES] '$vals'\"))\n }\n\n }\n }\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/command/CreateIndex.scala", "content": "package com.github.caiiiycuk.pg2sqlite.command\n\nimport java.sql.SQLException\nimport scala.annotation.tailrec\nimport com.github.caiiiycuk.pg2sqlite.Connection\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.schema.Schema\nimport com.github.caiiiycuk.pg2sqlite.dsl.DSL._\nimport com.github.caiiiycuk.pg2sqlite.Log\n\nobject CreateIndex extends Command with Log {\n\n private val INDEX_NAME_POSITION = 2\n private val TABLE_NAME_POSITION = 0\n private val activator = \"\"\"^(?i)create\\s+index\"\"\".r\n\n override def matchHead(head: String): Boolean = {\n activator.findFirstIn(head).isDefined\n }\n\n override def apply(connection: Connection, iterator: Iterator[Line])(implicit schema: Schema) = {\n val rows = takeUntil(iterator, _.contains(\";\"))\n val rawSql = rows.mkString(\" \").toLowerCase\n\n val (tableName, sql, columns) = try {\n val createIndexParts = rawSql.split(\"\"\"\\s+on\\s+\"\"\")\n val indexName = createIndexParts(0).tokens(INDEX_NAME_POSITION)\n val tableName = createIndexParts(1).tokens(TABLE_NAME_POSITION)\n val columns = rawSql.takeBraces.head.columns.map(column => s\"[${column.name}]\").mkString(\",\")\n\n (tableName, s\"CREATE INDEX $indexName ON $tableName ($columns)\", columns)\n } catch {\n case t: Throwable =>\n throw CommandException(s\"CREATE INDEX - Unable to find INDEX_NAME or TABLE NAME or COLUMNS in '$rawSql'\",\n t, rawSql, rows)\n }\n\n if (schema.shouldExcludeTable(tableName) ||\n columns.isEmpty) {\n log.info(s\"Skipping '$sql'\")\n } else {\n try {\n connection.execute(sql)\n } catch {\n case e: SQLException =>\n throw CommandException(\"Create Index\", e, sql, rows)\n }\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/command/CreateTable.scala", "content": "package com.github.caiiiycuk.pg2sqlite.command\n\nimport java.sql.SQLException\nimport scala.annotation.tailrec\nimport com.github.caiiiycuk.pg2sqlite.Connection\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.schema.Schema\nimport com.github.caiiiycuk.pg2sqlite.Log\nimport com.github.caiiiycuk.pg2sqlite.dsl.DSL._\n\nobject CreateTable extends Command with Log {\n\n private final val TABLE_NAME_POSITON = 2\n private final val activator = \"\"\"^(?i)create\\s+table\"\"\".r\n\n override def matchHead(head: String): Boolean = {\n activator.findFirstIn(head).isDefined\n }\n\n override def apply(connection: Connection, iterator: Iterator[Line])(implicit schema: Schema) = {\n val rows = takeUntil(iterator, _.contains(\";\"))\n val rawSql = rows.mkString(\" \")\n\n val (tableName, sql) = try {\n val table = rawSql.tokens(TABLE_NAME_POSITON)\n val columns = rawSql.takeBraces.head.columns\n\n columns.foreach { column =>\n schema.addColumn(table, column)\n }\n\n (table, s\"CREATE TABLE [$table] (${columns.map(column => s\"[${column.name}]\").mkString(\", \")});\")\n } catch {\n case t: Throwable =>\n throw CommandException(s\"CREATE TABLE - Unable to find TABLE NAME or COLUMNS in '$rawSql'\",\n t, rawSql, rows)\n }\n\n if (schema.shouldExcludeTable(tableName)) {\n log.info(s\"Skipping '$sql'\")\n } else {\n try {\n connection.execute(sql)\n } catch {\n case e: SQLException =>\n throw CommandException(\"Create Table\", e, sql, rows)\n }\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/dsl/DSL.scala", "content": "package com.github.caiiiycuk.pg2sqlite.dsl\n\nimport scala.annotation.tailrec\nimport com.github.caiiiycuk.pg2sqlite.schema.Column\n\nclass DSL(line: String) {\n\n import DSL._\n\n def dropBraces: String =\n dropBraces(line.toIterator)\n\n def takeBraces: List[String] = {\n takeBraces(line.toIterator)\n }\n\n def commaSplitRespectBraces: List[String] = {\n commaSplitRespectBraces(line.toIterator)\n }\n\n def tokens: List[String] = {\n line.replaceAll(\"\\\"|'\",\"\").split(\"\"\"\\s|:|,|\\(|\\)\"\"\").map(_.trim).filterNot(_.isEmpty).toList\n }\n\n def columns: List[Column] = {\n val columns = commaSplitRespectBraces(line.toIterator).map(_.trim).filterNot(_.isEmpty)\n\n columns.map(_.replaceAll(\"\\\"|'\", \"\")).flatMap { columnDefenition =>\n val partials = columnDefenition.split(\"\"\"\\s\"\"\")\n .map(_.trim.toLowerCase).filterNot(_.isEmpty).toList\n\n partials match {\n case head :: _ if head.startsWith(\"constraint\") =>\n None\n case head :: _ if head.startsWith(\"to_tsvector(\") =>\n val name = columnDefenition.takeBraces.head.tokens.last\n Some(Column(name, None))\n case head :: _ if head.startsWith(\"lower(\") || head.startsWith(\"upper(\") =>\n val name = columnDefenition.takeBraces.head.tokens.head\n Some(Column(name, None))\n case head :: sqlType :: _ =>\n Some(Column(head, Some(sqlType)))\n case head :: Nil =>\n Some(Column(head, None))\n case _ =>\n None\n }\n }\n }\n\n @tailrec\n private def takeBraces(line: Iterator[Char], nesting: Int = 0,\n acc: String = \"\", buff: List[String] = Nil): List[String] =\n if (line.hasNext) {\n val head = line.next\n\n val newAcc = if (nesting > 1 || (nesting > 0 && head != ')')) {\n acc + head\n } else {\n acc\n }\n\n if (head == '(') {\n takeBraces(line, nesting + 1, newAcc, buff)\n } else if (head == ')' && nesting == 1) {\n takeBraces(line, nesting - 1, \"\", newAcc :: buff)\n } else if (head == ')') {\n takeBraces(line, nesting - 1, newAcc, buff)\n } else {\n takeBraces(line, nesting, newAcc, buff)\n }\n } else if (acc.nonEmpty) {\n (acc :: buff).reverse\n } else {\n buff.reverse\n }\n\n @tailrec\n private def dropBraces(line: Iterator[Char], nesting: Int = 0, buff: String = \"\"): String =\n if (line.hasNext) {\n val head = line.next\n\n if (head == '(') {\n dropBraces(line, nesting + 1, buff)\n } else if (head == ')') {\n dropBraces(line, nesting - 1, buff)\n } else if (nesting == 0) {\n dropBraces(line, nesting, buff + head)\n } else {\n dropBraces(line, nesting, buff)\n }\n } else {\n buff\n }\n\n @tailrec\n private def commaSplitRespectBraces(line: Iterator[Char], nesting: Int = 0,\n acc: String = \"\", buff: List[String] = Nil): List[String] =\n if (line.hasNext) {\n val head = line.next\n\n if (head == '(') {\n commaSplitRespectBraces(line, nesting + 1, acc + head, buff)\n } else if (head == ')') {\n commaSplitRespectBraces(line, nesting - 1, acc + head, buff)\n } else if (head == ',' && nesting == 0) {\n commaSplitRespectBraces(line, nesting, \"\", acc :: buff)\n } else {\n commaSplitRespectBraces(line, nesting, acc + head, buff)\n }\n } else if (acc.nonEmpty) {\n (acc :: buff).reverse\n } else {\n buff.reverse\n }\n\n}\n\nobject DSL {\n\n implicit def toDSLClass(line: String): DSL = {\n new DSL(line)\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/iterator/Line.scala", "content": "package com.github.caiiiycuk.pg2sqlite.iterator\n\ncase class Line(num: Int, text: String) {\n def startsWith(value: String) =\n text.startsWith(value)\n\n override def toString(): String = text\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/iterator/LineIterator.scala", "content": "package com.github.caiiiycuk.pg2sqlite.iterator\n\nimport java.io.FileReader\nimport java.io.BufferedReader\nimport java.io.Closeable\nimport java.io.File\nimport scala.collection.TraversableOnce.flattenTraversableOnce\n\ntrait LineIterator extends Iterator[Line] with Closeable {\n def readed: Long\n}\n\nclass FileOptionStringIterator(file: File) extends Iterator[Option[String]] with Closeable {\n\n var readed = 0L\n\n private val reader = new FileReader(file) {\n override def read(buf: Array[Char], off: Int, len: Int) = {\n val count = super.read(buf, off, len)\n readed += count\n count\n }\n }\n\n private val bufferedReader = new BufferedReader(reader)\n\n private var current = Option(bufferedReader.readLine())\n\n override def hasNext: Boolean = {\n current.nonEmpty\n }\n\n override def next(): Option[String] = {\n val value = current\n current = Option(bufferedReader.readLine())\n value\n }\n\n override def close(): Unit = {\n bufferedReader.close\n }\n\n}\n\nobject LineIterator {\n def apply(file: File) = {\n val iterator = new FileOptionStringIterator(file)\n val flatIterator = iterator.flatten.zipWithIndex.map {\n case (text, index) =>\n Line(index + 1, text)\n }\n\n new LineIterator {\n override def hasNext: Boolean = flatIterator.hasNext\n override def next(): Line = flatIterator.next()\n override def close = iterator.close\n override def readed: Long = iterator.readed\n }\n }\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/schema/Column.scala", "content": "package com.github.caiiiycuk.pg2sqlite.schema\n\nimport java.sql.Types\n\nobject Column {\n val TYPE_DETECTORS = List(\n (\"\"\"boolean\"\"\".r -> Types.BOOLEAN),\n (\"\"\"int\"\"\".r -> Types.BIGINT),\n (\"\"\"float\"\"\".r -> Types.DOUBLE),\n (\"\"\"numeric\"\"\".r -> Types.NUMERIC),\n (\"\"\"bytea\"\"\".r -> Types.BLOB),\n (\"\"\"geometry\"\"\".r -> Types.BLOB),\n (\"\"\"timestamp\"\"\".r -> Types.TIMESTAMP),\n (\"\"\"time\"\"\".r -> Types.TIME),\n (\"\"\"date\"\"\".r -> Types.DATE),\n (\"\"\"char\"\"\".r -> Types.VARCHAR),\n (\"\"\"text\"\"\".r -> Types.VARCHAR))\n}\n\ncase class Column(name: String, sqlType: Option[String]) {\n\n import Column._\n\n lazy val typeConstant = sqlType.map {\n sqlType =>\n val nativeType = TYPE_DETECTORS.find {\n case (regex, _) =>\n regex.findFirstIn(sqlType).isDefined\n }\n\n nativeType.map(_._2)\n }.flatten\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/schema/Schema.scala", "content": "package com.github.caiiiycuk.pg2sqlite.schema\n\nimport java.sql.Types\n\nimport scala.collection.mutable.Map\n\nclass Schema(excludeTables: Set[String] = Set(\"sqlite_stat\")) {\n\n protected case class Table(columns: Map[String, Column] = Map.empty)\n\n val tables: Map[String, Table] = Map.empty\n\n def addColumn(tableName: String, column: Column) = {\n val loweredTableName = tableName.toLowerCase\n val table = tables.get(loweredTableName).getOrElse {\n val table = Table()\n tables += ((loweredTableName, table))\n table\n }\n table.columns += ((column.name.toLowerCase, column))\n }\n\n def columns(tableName: String) = {\n tables.get(tableName.toLowerCase).map(_.columns).getOrElse(Map.empty)\n }\n\n def columnsToTypeConstants(tableName: String, columns: List[String]): scala.collection.immutable.Map[Int, Int] = {\n tables.get(tableName.toLowerCase).map { table =>\n columns.zipWithIndex.flatMap {\n case (column, index) =>\n table.columns.get(column).flatMap { column =>\n column.typeConstant.map((index + 1, _))\n }\n }.toMap\n }.getOrElse(scala.collection.immutable.Map.empty)\n }\n\n def shouldExcludeTable(table: String) = {\n excludeTables.contains(table.toLowerCase)\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/values/LineToValues.scala", "content": "package com.github.caiiiycuk.pg2sqlite.values\n\nimport java.sql.Types\nimport java.util.Formatter.DateTime\nimport java.text.SimpleDateFormat\nimport java.util.Date\n\nobject LineToValues {\n\n val DOUBLE = \"\"\"^\\d+\\.\\d+$\"\"\".r\n val INTEGER = \"\"\"^\\d+$\"\"\".r\n\n val SIMPLE_TIMESTAMP_FORMAT = new SimpleDateFormat(\"yyyy-MM-dd HH:mm:ss\")\n val SIMPLE_DATE_FORMAT = new SimpleDateFormat(\"yyyy-MM-dd\")\n val SIMPLE_TIME_FORMAT = new SimpleDateFormat(\"HH:mm:ss\")\n\n val FORMATTER = Map(Types.DATE -> SIMPLE_DATE_FORMAT,\n Types.TIME -> SIMPLE_TIME_FORMAT,\n Types.TIMESTAMP -> SIMPLE_TIMESTAMP_FORMAT)\n\n val NO_HEX_DIGITS = \"\"\"[^0-9A-Fa-f]\"\"\".r\n\n def toValues(line: String)(implicit indexToType: Map[Int, Int]): List[Value] = {\n val parts = line.split(\"\\t\").map(_.trim)\n parts.zipWithIndex.map {\n case (value, index) =>\n toValue(index + 1, value)\n }.toList\n }\n\n def toValue(index: Int, value: String)(implicit indexToType: Map[Int, Int]) = {\n if (value == \"\"\"\\N\"\"\") {\n NullValue(index, indexToType.get(index))\n } else {\n indexToType.get(index).map { sqlType =>\n toValueWithKnownType(index, value, sqlType)\n }.getOrElse {\n value match {\n case DOUBLE(_*) =>\n toDoubleWithStringFallback(index, value)\n case INTEGER(_*) =>\n toIntegerWithDoubleFallback(index, value)\n case _ =>\n StringValue(index, value)\n }\n }\n }\n }\n\n def toValueWithKnownType(index: Int, value: String, sqlType: Int) = {\n sqlType match {\n case Types.BIGINT =>\n toIntegerWithDoubleFallback(index, value)\n case Types.DOUBLE | Types.NUMERIC =>\n toDoubleWithStringFallback(index, value)\n case Types.VARCHAR =>\n StringValue(index, value)\n case Types.BOOLEAN =>\n BooleanValue(index, value.toLowerCase != \"f\")\n case Types.TIMESTAMP | Types.TIME | Types.DATE =>\n val date = toDate(value, sqlType).getOrElse {\n throw new ValueParseException(s\"[COLUMN#${index}] Doesn`t know how to convert string '$value', to timestamp\")\n }\n DateValue(index, date, sqlType)\n case Types.BLOB =>\n BlobValue(index, hex2bytes(value))\n case _ =>\n throw new ValueParseException(s\"[COLUMN#${index}] Doesn`t know how to convert string '$value', to sql type '$sqlType'\")\n }\n }\n\n private def toDate(value: String, sqlType: Int): Option[Date] = {\n val formatter = FORMATTER(sqlType)\n\n try {\n Some(formatter.parse(value.take(formatter.toPattern().length)))\n } catch {\n case t: Throwable =>\n None\n }\n }\n\n private def toIntegerWithDoubleFallback(index: Int, value: String) = {\n try {\n IntegerValue(index, value.toLong)\n } catch {\n case e: NumberFormatException =>\n toDoubleWithStringFallback(index, value)\n }\n }\n\n private def toDoubleWithStringFallback(index: Int, value: String) = {\n try {\n RealValue(index, value.toDouble)\n } catch {\n case e: NumberFormatException =>\n StringValue(index, value)\n }\n }\n\n private def hex2bytes(value: String): Array[Byte] = {\n if (value.length % 2 != 0 || NO_HEX_DIGITS.findFirstIn(value).isDefined) {\n value.getBytes\n } else {\n javax.xml.bind.DatatypeConverter.parseHexBinary(value)\n }\n }\n\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/values/Value.scala", "content": "package com.github.caiiiycuk.pg2sqlite.values\n\nimport java.sql.PreparedStatement\nimport java.sql.Types\nimport java.util.Date\n\nabstract class Value(index: Int) {\n def apply(statement: PreparedStatement)\n}\n\ncase class NullValue(index: Int, sqlType: Option[Int]) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setNull(index, sqlType.getOrElse(Types.BIGINT))\n }\n}\n\ncase class BooleanValue(index: Int, value: Boolean) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setBoolean(index, value)\n }\n}\n\ncase class RealValue(index: Int, value: Double) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setDouble(index, value)\n }\n}\n\ncase class IntegerValue(index: Int, value: Long) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setLong(index, value)\n }\n}\n\ncase class StringValue(index: Int, value: String) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setString(index, value)\n }\n}\n\ncase class BlobValue(index: Int, value: Array[Byte]) extends Value(index) {\n def apply(statement: PreparedStatement) {\n statement.setBytes(index, value)\n }\n}\n\ncase class DateValue(index: Int, value: Date, dateType: Int) extends Value(index) {\n def apply(statement: PreparedStatement) {\n dateType match {\n case Types.DATE =>\n statement.setDate(index, new java.sql.Date(value.getTime))\n case Types.TIME =>\n statement.setTime(index, new java.sql.Time(value.getTime))\n case _ =>\n statement.setTimestamp(index, new java.sql.Timestamp(value.getTime))\n }\n }\n}\n" }, { "path": "src/main/scala/com/github/caiiiycuk/pg2sqlite/values/ValueParseException.scala", "content": "package com.github.caiiiycuk.pg2sqlite.values\n\nclass ValueParseException(message: String) extends Exception(message)\n" }, { "path": "src/test/scala/com/github/caiiiycuk/pg2sqlite/dsl/DSLTest.scala", "content": "package com.github.caiiiycuk.pg2sqlite.dsl\n\nimport org.scalatest.FlatSpec\nimport org.scalatest.Matchers\nimport com.github.caiiiycuk.pg2sqlite.dsl.DSL._\nimport com.github.caiiiycuk.pg2sqlite.schema.Column\n\nclass DslTest extends FlatSpec with Matchers {\n\n \"DSL\" should \"drop braces from line\" in {\n val TEST_STRING = \"\"\"\nid bigint DEFAULT nextval('hibernate_sequence'::regclass) NOT NULL,\nurl text,\nident character varying(20) DEFAULT \"substring\"(upper(md5((((999999999)::double precision * random()))::text)), 1, 8) NOT NULL,\ncreated_at timestamp without time zone DEFAULT now()\n\"\"\"\n\n TEST_STRING.dropBraces should equal(\"\"\"\nid bigint DEFAULT nextval NOT NULL,\nurl text,\nident character varying DEFAULT \"substring\" NOT NULL,\ncreated_at timestamp without time zone DEFAULT now\n\"\"\")\n }\n\n \"DSL\" should \"take columns parts\" in {\n val TEST_STRING = \"\"\"\ninsert into some(a, b, c) values (\"a\", 2, true);\n\"\"\"\n\n TEST_STRING.takeBraces should equal(List(\n \"a, b, c\", \"\"\"\"a\", 2, true\"\"\"))\n }\n\n \"DSL\" should \"extract tokens\" in {\n val TEST_STRING = \"\"\"\n insert(strange text) into(some buffer) table\n \"\"\"\n\n TEST_STRING.tokens should equal(List(\"insert\", \"strange\", \"text\", \"into\", \"some\", \"buffer\", \"table\"))\n }\n\n \"DSL\" should \"extract columns with type\" in {\n val COLUMNS = \"\"\"\nid bigint DEFAULT nextval('hibernate_sequence'::regclass) NOT NULL,\nurl text,\nident character varying(20) DEFAULT \"substring\"(upper(md5((((999999999)::double precision * random()))::text)), 1, 8) NOT NULL,\ncreated_at timestamp without time zone DEFAULT now()\n\"\"\"\n\n COLUMNS.columns should equal(\n List(Column(\"id\", Some(\"bigint\")),\n Column(\"url\", Some(\"text\")),\n Column(\"ident\", Some(\"character\")),\n Column(\"created_at\", Some(\"timestamp\"))))\n }\n\n \"DSL\" should \"exclude keywords (CONSTRAINTS, etc.) from columns list\" in {\n val COLUMNS = \"\"\"\nid integer DEFAULT nextval('hibernate_sequence'::regclass) NOT NULL,\nlocation geometry,\nowner_geoobject_id bigint,\nCONSTRAINT enforce_dims_location CHECK ((st_ndims(location) = 2)),\nCONSTRAINT enforce_geotype_location CHECK (((geometrytype(location) = 'POLYGON'::text)\n OR (location IS NULL))),\nCONSTRAINT enforce_srid_location CHECK ((st_srid(location) = 3395))\n\"\"\"\n\n COLUMNS.columns should equal(\n List(Column(\"id\", Some(\"integer\")),\n Column(\"location\", Some(\"geometry\")),\n Column(\"owner_geoobject_id\", Some(\"bigint\"))))\n }\n\n \"DSL\" should \"get column name from to_tsvector function call\" in {\n val COLUMNS = \"to_tsvector('libstemmer_serb_lat_no_diacrit'::regconfig, content)\"\n\n COLUMNS.columns should equal(\n List(Column(\"content\", None)))\n }\n\n \"DSL\" should \"get column name from lower/upper function call\" in {\n val COLUMNS = \"lower((email)::text),upper((email_up)::text)\"\n\n COLUMNS.columns should equal(\n List(Column(\"email\", None), Column(\"email_up\", None)))\n }\n\n \"DSL\" should \"split by comma respect braces\" in {\n val TEST_STRING = \"\"\"\nid bigint DEFAULT nextval('hibernate_sequence'::regclass) NOT NULL,\nurl text,\nident character varying(20) DEFAULT \"substring\"(upper(md5((((999999999)::double precision * random()))::text)), 1, 8) NOT NULL,\ncreated_at timestamp without time zone DEFAULT now()\n\"\"\".replaceAll(\"\\n\", \"\")\n\n val parts = TEST_STRING.commaSplitRespectBraces\n parts.length should equal(4)\n parts(0) should equal(\"id bigint DEFAULT nextval('hibernate_sequence'::regclass) NOT NULL\")\n parts(1) should equal(\"url text\")\n parts(2) should equal(\"ident character varying(20) DEFAULT \\\"substring\\\"(upper(md5((((999999999)::double precision * random()))::text)), 1, 8) NOT NULL\")\n parts(3) should equal(\"created_at timestamp without time zone DEFAULT now()\")\n }\n}" }, { "path": "src/test/scala/com/github/caiiiycuk/pg2sqlite/dsl/DumperTest.scala", "content": "package com.github.caiiiycuk.pg2sqlite.dsl\n\nimport org.scalatest.FlatSpec\nimport org.scalatest.Matchers\nimport com.github.caiiiycuk.pg2sqlite.iterator.Line\nimport com.github.caiiiycuk.pg2sqlite.{Connection, DumpInserter}\nimport org.scalatest.{BeforeAndAfter, FlatSpec, Matchers}\n\nimport java.io.File\n\nclass DumperTest extends FlatSpec with Matchers with BeforeAndAfter {\n\n val dbFile = new File(\"test.db\")\n\n private final val DATE_DUMP =\n \"\"\"\n |CREATE TABLE test (\n | current timestamp without time zone NOT NULL\n |);\n |\n |COPY test (current) FROM stdin;\n |2024-05-06 15:14:12\n |\\.\n |\"\"\".stripMargin\n\n private def makeConnection(dateClass: String = Connection.DEFAULT_DATE_CLASS) = {\n if (dbFile.exists()) {\n dbFile.delete()\n }\n\n Connection.sqlite(dbFile, dateClass)\n }\n\n after {\n new File(\"test.db\").delete()\n }\n\n \"dumper\" should \"generate db from test-case of issue#11\" in {\n val connection = makeConnection()\n val inserter = new DumpInserter(connection)\n val dump =\n \"\"\"\n |CREATE TYPE product_type AS ENUM (\n | 'Material',\n | 'Digital'\n |);\n |\n |CREATE TABLE product (\n | client_id integer NOT NULL,\n | order_product integer,\n | upper_price integer NOT NULL,\n | lower_price integer NOT NULL,\n | type product_type NOT NULL,\n | product_id integer NOT NULL--,\n | CONSTRAINT product_check CHECK (((lower_price > upper_price) AND (upper_price <= 200))),\n | CONSTRAINT product_order_product_check CHECK ((order_product > 0)),\n | CONSTRAINT product_upper_price_check CHECK ((upper_price >= 0))\n |);\n |\"\"\".stripMargin\n .split(\"\\n\")\n .zipWithIndex\n .map {\n case (text, num) =>\n Line(num, text)\n }\n\n inserter.insert(dump.iterator)\n connection.close\n }\n\n \"dumper\" should \"should respect date class (Default)\" in {\n val connection = makeConnection()\n val inserter = new DumpInserter(connection)\n val dump = DATE_DUMP.split(\"\\n\")\n .zipWithIndex\n .map {\n case (text, num) =>\n Line(num, text)\n }\n\n inserter.insert(dump.iterator)\n connection.withStatement { statment =>\n val rs = statment.executeQuery(\"SELECT * FROM test\")\n rs.next() should equal(true)\n rs.getLong(1) > 0 should equal(true)\n rs.close()\n }\n connection.close\n }\n\n \"dumper\" should \"should respect date class (text)\" in {\n val connection = makeConnection(Connection.TEXT_DATE_CLASS)\n val inserter = new DumpInserter(connection)\n val dump = DATE_DUMP.split(\"\\n\")\n .zipWithIndex\n .map {\n case (text, num) =>\n Line(num, text)\n }\n\n inserter.insert(dump.iterator)\n connection.withStatement { statment =>\n val rs = statment.executeQuery(\"SELECT * FROM test\")\n rs.next() should equal(true)\n rs.getString(1) should equal(\"2024-05-06 15:14:12.000\")\n rs.close()\n }\n connection.close\n }\n\n \"dumper\" should \"should respect date class (real)\" in {\n val connection = makeConnection(Connection.REAL_DATE_CLASS)\n val inserter = new DumpInserter(connection)\n val dump = DATE_DUMP.split(\"\\n\")\n .zipWithIndex\n .map {\n case (text, num) =>\n Line(num, text)\n }\n\n inserter.insert(dump.iterator)\n connection.withStatement { statment =>\n val rs = statment.executeQuery(\"SELECT * FROM test\")\n rs.next() should equal(true)\n rs.getDouble(1) > 0 should equal(true)\n rs.close()\n }\n connection.close\n }\n\n}\n" } ]