Showing preview only (283K chars total). Download the full file or copy to clipboard to get everything.
Repository: brianmhess/cassandra-loader
Branch: master
Commit: 2ccfc0d25566
Files: 57
Total size: 264.2 KB
Directory structure:
gitextract_9khq7mcr/
├── .gitignore
├── ChangeLog.md
├── LICENSE.md
├── README.md
├── build.gradle
├── sample/
│ ├── cassandra-schema.cql
│ ├── load.sh
│ └── titanic.csv
└── src/
├── main/
│ ├── java/
│ │ └── com/
│ │ └── datastax/
│ │ └── loader/
│ │ ├── AbstractDynamicRateLimiter.java
│ │ ├── CqlDelimLoad.java
│ │ ├── CqlDelimLoadTask.java
│ │ ├── CqlDelimParser.java
│ │ ├── CqlDelimUnload.java
│ │ ├── EnhancedSession.java
│ │ ├── Latency999RateLimiter.java
│ │ ├── LoaderRetryPolicy.java
│ │ ├── RateLimitedSession.java
│ │ ├── RateLimiter.java
│ │ ├── futures/
│ │ │ ├── AbstractFutureManager.java
│ │ │ ├── ActionFutureList.java
│ │ │ ├── ActionFutureSet.java
│ │ │ ├── FutureAction.java
│ │ │ ├── FutureManager.java
│ │ │ ├── JsonPrintingFutureAction.java
│ │ │ ├── JsonPrintingFutureList.java
│ │ │ ├── JsonPrintingFutureSet.java
│ │ │ ├── NullFutureAction.java
│ │ │ ├── PrintingFutureAction.java
│ │ │ ├── PrintingFutureList.java
│ │ │ └── PrintingFutureSet.java
│ │ └── parser/
│ │ ├── AbstractParser.java
│ │ ├── BigDecimalParser.java
│ │ ├── BigIntegerParser.java
│ │ ├── BooleanParser.java
│ │ ├── ByteBufferParser.java
│ │ ├── ByteParser.java
│ │ ├── DateParser.java
│ │ ├── DelimParser.java
│ │ ├── DoubleParser.java
│ │ ├── FloatParser.java
│ │ ├── IndexedLine.java
│ │ ├── InetAddressParser.java
│ │ ├── IntegerParser.java
│ │ ├── ListParser.java
│ │ ├── LocalDateParser.java
│ │ ├── LongParser.java
│ │ ├── MapParser.java
│ │ ├── NumberParser.java
│ │ ├── Parser.java
│ │ ├── SetParser.java
│ │ ├── ShortParser.java
│ │ ├── StringParser.java
│ │ └── UUIDParser.java
│ └── resources/
│ └── logback.xml
└── make/
├── buildit.sh
├── cassandra-loader.sh
└── unloader.sh
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
build/
.gradle/
*~
*.jar
cassandra-loader
*.BADINSERT
*.BADPARSE
*.LOG
================================================
FILE: ChangeLog.md
================================================
## 0.0.27
- Added support for specifying a TTL (Issue 67)
## 0.0.26
- Fixed issue with long schemas (Issue 65)
## 0.0.25
- Added support for Fetch Size in cassandra-unloader
## 0.0.24
- Added support for DATE (and associated -localDateFormat option)
- Added support for SHORTINT and TINYINT
- Fixed an issue with special characters in column names (Issue 59)
## 0.0.23
- Fixed case of commas in Map keys
- Fixed keystore/truststore-pw typo (was pwd)
- Fixed quoting of non-collections
- Added support for supplying comment character
## 0.0.22
- Added support for gzipped input files
- Fixed issue with quoted values (and in collections)
## 0.0.21
- Added support for jsonarray (one JSON array per file)
- Added support for jsonline (one JSON per line)
- Changed JVM settings to 1GB heap
- Added -charsPerColumn (default to 4096) to enable Univocity optimizations
- Removed explicit queries to metadata tables and use driver API calls
## 0.0.20
- Fixed delimiter in MapParser
- Catch NULLs in Map/Set/List parsing and throw in BADPARSE
## 0.0.19
- Converted to parse with Univocity CSV parser
- Added -where
## 0.0.18
- Support for Cassandra 3.0
## 0.0.17
- Fixed null collection issue / NPE (Issue 8)
## 0.0.16
- Unloader will quote collections (which the loader expects)
- Fixed collection issue (Issue 14)
- BLOBs are now Base64 encoded on unload, and should be Base64 to load (Issue 15)
- Support for quoted keyspace, table, and column names
## 0.0.15
- Better error handling for case when C* inserts are failing
## 0.0.14
- Updated cassandra-unloader to add support for collections,
consistency level, ssl, etc
## 0.0.13
- Added configFile
- added ssl options (with truststore and keystore)
## 0.0.12
- Added a rateFile to output CSV rate statistics
- added -skipCols to skip input columns
## 0.0.11
- Added support for quoted Keyspaces, Tables, and Columns
## 0.0.10
- You want collections? You got 'em
- Added progress reporting - you can specify the rate at which
the rate is reported via the -progressRate option
- Refactored RateLimiting - added it to a new RateLimitingSession
- Laid groundwork for Dynamic rate limiting - to be worked out
once we find a way to collect the right statistic
## 0.0.9
- Added -successDir and -failureDir
- Added return codes for the loader and unloader
- Refactored BoolStyle
- Cleaned up the readme a bit
================================================
FILE: LICENSE.md
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016, DataStax
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# CASSANDRA-LOADER
## Introduction
cassandra-loader is a general-purpose, delimited-file, bulk loader for
Cassandra. It supports a number of configuration options to enable bulk
loading of various types of delimited files, including
* comma-separated values
* tab-separated values
* customer delimiter-separated values
* header row
* comma as decimal separator
* ...
## Getting it
### Downloading
This utility has already been built, and is available at
https://github.com/brianmhess/cassandra-loader/releases/download/v0.0.27/cassandra-loader
Get it with wget:
```
wget https://github.com/brianmhess/cassandra-loader/releases/download/v0.0.27/cassandra-loader
```
### Building
To build this repository, simply clone this repo and run:
```
gradle loader
```
All of the dependencies are included (namely, the Java driver - currently
version 3.0.0). The output will be the cassandra-loader executable
in the build directory. There will also be an jar with all of the
dependencies included in the build/libs/cassandra-loader-uber-<version>.jar
## Documentation
To extract this README document, simply run (on the cassandra-loader
executable - (e.g., on build/cassandra-loader):
```
jar xf cassandra-loader README.md
```
## Run
To run cassandra-loader, simply run the cassandra-loader executable
(e.g., located at build/cassandra-loader):
```
cassandra-loader
```
If you built this with gradle, you can also run:
```
gradle run
```
This will print the usage statement.
The following will load the `myFileToLoad.csv` file into the Cassandra
cluster at IP address 1.2.3.4 into the `test.ltest` column family where
the myFileToLoad file has the format of 4 columns - and it gets the
data type information from the database - and using the default options:
```
cassandra-loader -f myFileToLoad.csv -host 1.2.3.4 -schema "test.ltest(a, b, c, d)"
```
## Options:
Switch | Option | Default | Description
-----------------:|-------------------:|---------------------------:|:----------
`-configFile` | Filename | none | Filename of configuration options
`-f` | Filename | <REQUIRED> | Filename to load - required.
`-host` | IP Address | <REQUIRED> | Cassandra connection point - required.
`-format` | Input format | delim | Format of the data. Options are "delim" or "json".
`-schema` | CQL schema | | Schema of input data - required for delim In the format "keySpace.table(col1,col2,...)" and in the order that the data will be in the file.
`-keyspace` | Keyspace name | | Name of keyspace (case sensitive) to load in to - required for json
`-table` | Table name | | Name of table (case sensitive) to load in to - required for json
`-port` | Port Number | 9042 | Cassandra native protocol port number
`-user` | Username | none | Cassandra username
`-pw` | Password | none | Cassandra password
`-ssl-truststore-path` | Truststore Path | none | Path to SSL truststore
`-ssl-truststore-pw` | Truststore Password | none | Password to SSL truststore
`-ssl-keystore-path` | Keystore Path | none | Path to SSL keystore
`-ssl-keystore-path` | Keystore Password | none | Password to SSL keystore
`-consistencyLevel` | Consistency Level | ONE | CQL Consistency Level
`-numThreads` | Number of threads | Number of CPUs | Number of threads to use (one per file)
`-numFutures` | Number of Futures | 1000 | Number of Java driver futures in flight.
`-numRetries` | Number of retries | 1 | Number of times to retry the INSERT before declaring defeat.
`-queryTimeout` | Timeout in seconds | 2 | Amount of time to wait for a query to finish before timing out.
`-ttl` | Time To Live | none | TTL to use when inserting these rows
`-delim` | Delimiter | , | Delimiter to use
`-charsPerColumn`| Characters per column | 4096 | Maximum characters per column
`-nullString` | Null String | <empty string> | String to represent NULL data
`-boolStyle` | Boolean Style | TRUE_FALSE | String for boolean values. Options are "1_0", "Y_N", "T_F", "YES_NO", "TRUE_FALSE".
`-decimalDelim` | Decimal delimiter | . | Delimiter for decimal values. Options are "." or ","
`-dateFormat` | Date Format String | default for Locale.ENGLISH | Date format string as specified in the SimpleDateFormat Java class: http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
`-skipRows` | Rows to skip | 0 | Number of rows to skip at the beginning of the file
`-skipCols` | Columns to skip | <not set> | Comma-separated list of columns to skip loading (0-counted)
`-maxRows` | Max rows to read | -1 | Maximum rows to read (after optional skipping of rows). -1 signifies all rows.
`-maxErrors` | Max parse errors | 10 | Maximum number of rows that do not parse to allow before exiting.
`-maxInsertErrors`| Max insert errors | 10 | Maximum number of rows that do not insert to allow before exiting.
`-badDir` | Bad directory | current directory | Directory to write badly parsed and badly inserted rows - as well as the log file.
`-rate` | Ingest rate | unlimited | Maximum rate to insert data - in rows/sec.
`-progressRate` | Progress rate | 100000 | How often to report the ingest rate (number of rows)
`-rateFile` | Rate Stats File | <not set> | File to contain CSV rate statistics
`-successDir` | Success directory | <not set> | Location to move successfully loaded files
`-failureDir` | Failure directory | <not set> | Location to move files that failed to load
`-batchSize` | Batch size | 1 | Size of unlogged batches. If set to 1 then no batching.
`-comment` | Comment character | $lt;not set> | Comment character.
## Comments
### Using stdin
You can send data in on stdin by specifying the filename (via the -f switch) as "stdin" (case insensitive).
That way, you could pipe data in from other commands:
```
grep IMPORTANT data.csv | cassandra-loader -f stdin -h 1.2.3.4 -schema "test.itest(a, b)"
```
### Support for collections
Collections are supported. Their format is the CQL native one.
Sets are started with '{' and ended with '}' and enclose a comma-separated list
{1,2,3} or {"a","b","c"}
Lists are started with '[' and ended with ']' and enclose a comma-separated list
[1,2,3] or ["a","b","c"]
Maps are started with '{' and ended with '}' and enclose a comma-separated list
of pairs that are separated by ':'
{1:1,2:2,3:3} or {"a":1, "b":2, "c":3}
All collections must be enclosed in double-quotes.
### Username/Password
If you specify either the username or the password, then you must specify both.
### Boolean Style
boolStyle is a case-insensitive test of the True and False strings. For the
different styles, the True and False strings are as follows:
```
Style | True | False
------------|------|-------
0_1 | 1 | 0
Y_N | Y | N
T_F | T | F
YES_NO | YES | NO
TRUE_FALSE | TRUE | FALSE
```
### Configuration file
configFile is a file with configuration options that are formatted just like on
the command line. This allows you to not specify arguments on the command line.
For example, you can specify passwords in the configFile and avoid having them on
the command line. The format is one switch and option per line:
```
-pwd mypassword
-host 1.2.3.4
```
### Miscelaneous
numFutures is a way to control the level of parallelism, but at some point
too many will actually slow down the load. The default of 500 is a decent
place to start.
If you use quoted strings, you need to use double-quotes. To escape a double-quote inside a quoted string, use the backslash to escape it ("\""). To create a backslash inside a quoted string, use two backslashes in a row ("\\"). If you quote your string, it will not be trimmed, but if you do not quote your string it will be trimmed.
Loading into counter columns is not supported.
The default nullString is the empty string. If you want empty strings to be saved as empty strings, set the nullString to something else.
If you do not set the successDir then files that successfully loaded will remain in their input directory. The same is true for failed files if you do not set the failureDir. You cannot set either if the input file is "stdin".
When using `jsonline`, all JSON field names are case-sensitive. When using `jsonline` or `jsonarray`, the `-keyspace` and `-table` arguments are case-sensitive.
## Usage Statement:
```
version: 0.0.27
Usage: -f <filename> -host <ipaddress> [OPTIONS]
OPTIONS:
-schema <schema> Table schema (when using delim)
-table <tableName> Table name (when using json)
-keyspace <keyspaceName> Keyspace name (when using json)
-configFile <filename> File with configuration options
-delim <delimiter> Delimiter to use [,]
-charsPerColumn <chars> Max number of chars per column [4096]
-dateFormat <dateFormatString> Date format for TIMESTAMP [default for Locale.ENGLISH]
-localDateFormat <formatString> Date format for DATE [yyyy-MM-dd]
-nullString <nullString> String that signifies NULL [none]
-comment <commentString> Comment symbol to use [none]
-skipRows <skipRows> Number of rows to skip [0]
-skipCols <columnsToSkip> Comma-separated list of columsn to skip in the input file
-maxRows <maxRows> Maximum number of rows to read (-1 means all) [-1]
-maxErrors <maxErrors> Maximum parse errors to endure [10]
-badDir <badDirectory> Directory for where to place badly parsed rows. [none]
-port <portNumber> CQL Port Number [9042]
-user <username> Cassandra username [none]
-pw <password> Password for user [none]
-ssl-truststore-path <path> Path to SSL truststore [none]
-ssl-truststore-pw <pwd> Password for SSL truststore [none]
-ssl-keystore-path <path> Path to SSL keystore [none]
-ssl-keystore-pw <pwd> Password for SSL keystore [none]
-consistencyLevel <CL> Consistency level [LOCAL_ONE]
-numFutures <numFutures> Number of CQL futures to keep in flight [1000]
-batchSize <batchSize> Number of INSERTs to batch together [1]
-decimalDelim <decimalDelim> Decimal delimiter [.] Other option is ','
-boolStyle <boolStyleString> Style for booleans [TRUE_FALSE]
-numThreads <numThreads> Number of concurrent threads (files) to load [num cores]
-queryTimeout <# seconds> Query timeout (in seconds) [2]
-numRetries <numRetries> Number of times to retry the INSERT [1]
-maxInsertErrors <# errors> Maximum INSERT errors to endure [10]
-rate <rows-per-second> Maximum insert rate [50000]
-progressRate <num txns> How often to report the insert rate [100000]
-rateFile <filename> Where to print the rate statistics
-successDir <dir> Directory where to move successfully loaded files
-failureDir <dir> Directory where to move files that did not successfully load
-nullsUnset [false|true] Treat nulls as unset [faslse]
-format [delim|jsonline|jsonarray] Format of data: delimited or JSON [delim]
-table <tableName> Table name (when using JSON)
-keyspace <keyspaceName> Keyspace name (when using JSON)
-ttl <TTL> TTL for all rows in this invocation [unset]
Examples:
cassandra-loader -f /path/to/file.csv -host localhost -schema "test.test3(a, b, c)"
cassandra-loader -f /path/to/directory -host 1.2.3.4 -schema "test.test3(a, b, c)" -delim "\t" -numThreads 10
cassandra-loader -f stdin -host localhost -schema "test.test3(a, b, c)" -user myuser -pw mypassword
```
##Examples:
Load file /path/to/file.csv into the test3 table in the test keyspace using
the cluster at localhost. Use the default options:
```
cassandra-loader -f /path/to/file.csv -host localhost -schema "test.test3(a, b, c)"
```
Load all the files from /path/to/directory into the test3 table in the test
keyspace using the cluster at 1.2.3.4. Use 10 threads and use tab as the
delimiter:
```
cassandra-loader -f /path/to/directory -host 1.2.3.4 -schema "test.test3(a, b, c)" -delim "\t" -numThreads 10
```
Load the data from stdin into the test3 table in the test keyspace using the
cluster at localhost. Use "myuser" as the username and "mypassword" as the
password:
```
cassandra-loader -f stdin -host localhost -schema "test.test3(a, b, c)" -user myuser -pw mypassword
```
##Sample
Included here is a set of sample data. It is in the sample/ directory.
You can set up the table and keyspace by running:
```
cqlsh -f sample/cassandra-schema.cql
```
To load the data, run:
```
cd sample
./load.sh
```
To check that things have succeeded, you can run:
```
wc -l titanic.csv
```
And:
```
cqlsh -e "SELECT COUNT(*) FROM titanic.surviors"
```
Both should return 891.
## cassandra-unloader
cassandra-unloader is a utility to dump the contents
of a Cassandra table to delimited file format. It uses
the same sorts of options as cassandra-loader so that the
output of cassandra-unloader could be piped into
cassandra-loader:
```
cassandra-unloader -f stdout -host host1 -schema "ks.table(a,b,c)" | cassandra-loader -f stdin -host host2 -schema "ks2.table2(x,y,z)"
```
Get it with wget:
```
wget https://github.com/brianmhess/cassandra-loader/releases/download/v0.0.27/cassandra-unloader
```
To build, run:
```
gradle unloader
```
To run cassandra-unloader, simply run the cassandra-unloader executable
(e.g., located at build/cassandra-unloader):
```
cassandra-unloader
```
###Usage statement:
```
version: 0.0.27
Usage: -f <outputStem> -host <ipaddress> -schema <schema> [OPTIONS]
OPTIONS:
-configFile <filename> File with configuration options
-format [delim|jsonline|jsonarray] Format of data: delimited or JSON [delim]
-delim <delimiter> Delimiter to use [,]
-dateFormat <dateFormatString> Date format for TIMESTAMP [default for Locale.ENGLISH]
-localDateFormat <FormatString> Date format for DATE [yyyy-MM-dd]
-nullString <nullString> String that signifies NULL [none]
-port <portNumber> CQL Port Number [9042]
-user <username> Cassandra username [none]
-pw <password> Password for user [none]
-ssl-truststore-path <path> Path to SSL truststore [none]
-ssl-truststore-pw <pwd> Password for SSL truststore [none]
-ssl-keystore-path <path> Path to SSL keystore [none]
-ssl-keystore-pw <pwd> Password for SSL keystore [none]
-consistencyLevel <CL> Consistency level [LOCAL_ONE]
-decimalDelim <decimalDelim> Decimal delimiter [.] Other option is ','
-boolStyle <boolStyleString> Style for booleans [TRUE_FALSE]
-numThreads <numThreads> Number of concurrent threads to unload [5]
-beginToken <tokenString> Begin token [none]
-endToken <tokenString> End token [none]
-where <predicate> WHERE clause [none]
-fetchSize <fetchSize> Fetch size to use [0]
```
A few simple examples using the `-where` are as follows:
```
cassandra-unloader -host localhost -f stdout -schema "testks.testtable(pkey,ccol,x,y)" -where "pkey=5"
cassandra-unloader -host localhost -f stdout -schema "testks.testtable(pkey,ccol,x,y)" -where "x = 100 ALLOW FILTERING"
```
================================================
FILE: build.gradle
================================================
apply plugin: 'java'
apply plugin: 'application'
def versionNum = '0.0.27'
allprojects {
tasks.withType(JavaCompile) {
options.compilerArgs << "-Xlint:unchecked"
options.compilerArgs << "-Xlint:deprecation"
}
}
task loader(type: Exec) {
dependsOn << 'uberloader'
commandLine './src/make/buildit.sh'
}
task unloader(type: Exec) {
dependsOn << 'uberunloader'
commandLine './src/make/unloader.sh'
}
jar {
baseName = 'cassandra-loader'
version = versionNum
}
repositories {
mavenCentral()
}
dependencies {
compile 'com.datastax.cassandra:cassandra-driver-core:3.2.0'
compile 'org.xerial.snappy:snappy-java:1.0.5'
compile 'net.jpountz.lz4:lz4:1.2.0'
compile 'ch.qos.logback:logback-classic:1.1.3'
compile 'com.univocity:univocity-parsers:2.2.3'
compile 'org.apache.commons:commons-lang3:3.0'
compile group: 'com.googlecode.json-simple', name: 'json-simple', version: '1.1.1'
}
task uberloader(type: Jar) {
dependsOn << 'compileJava'
from "src/main/resources/logback.xml"
from files(sourceSets.main.output.classesDir)
from(configurations.runtime.asFileTree.files.collect { zipTree(it) }) {
exclude "META-INF/**"
}
manifest {
attributes "Main-Class" : 'com.datastax.loader.CqlDelimLoad'
}
baseName = 'cassandra-loader-uber'
version = versionNum
}
task uberunloader(type: Jar) {
dependsOn << 'compileJava'
from "src/main/resources/logback.xml"
from files(sourceSets.main.output.classesDir)
from(configurations.runtime.asFileTree.files.collect { zipTree(it) }) {
exclude "META-INF/**"
}
manifest {
attributes "Main-Class" : 'com.datastax.loader.CqlDelimUnload'
}
baseName = 'cassandra-unloader-uber'
version = versionNum
}
================================================
FILE: sample/cassandra-schema.cql
================================================
create keyspace titanic WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
use titanic;
CREATE TABLE survivors (
id varchar,
survived boolean,
passenger_class int,
name varchar,
sex varchar,
age int,
num_siblings_spouse int,
num_parents_children int,
ticket_id varchar,
fare varchar,
cabin varchar,
port_of_embarkation varchar,
primary key (id));
================================================
FILE: sample/load.sh
================================================
../build/cassandra-loader -f titanic.csv -host localhost -schema "titanic.survivors(id, survived, passenger_class, name, sex, age, num_siblings_spouse, num_parents_children, ticket_id, fare, cabin, port_of_embarkation)" -boolStyle 1_0
================================================
FILE: sample/titanic.csv
================================================
1,0,3,"Braund, Mr. Owen Harris",male,22,1,0,A/5 21171,7.25,,S
2,1,1,"Cumings, Mrs. John Bradley (Florence Briggs Thayer)",female,38,1,0,PC 17599,71.2833,C85,C
3,1,3,"Heikkinen, Miss. Laina",female,26,0,0,STON/O2. 3101282,7.925,,S
4,1,1,"Futrelle, Mrs. Jacques Heath (Lily May Peel)",female,35,1,0,113803,53.1,C123,S
5,0,3,"Allen, Mr. William Henry",male,35,0,0,373450,8.05,,S
6,0,3,"Moran, Mr. James",male,,0,0,330877,8.4583,,Q
7,0,1,"McCarthy, Mr. Timothy J",male,54,0,0,17463,51.8625,E46,S
8,0,3,"Palsson, Master. Gosta Leonard",male,2,3,1,349909,21.075,,S
9,1,3,"Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)",female,27,0,2,347742,11.1333,,S
10,1,2,"Nasser, Mrs. Nicholas (Adele Achem)",female,14,1,0,237736,30.0708,,C
11,1,3,"Sandstrom, Miss. Marguerite Rut",female,4,1,1,PP 9549,16.7,G6,S
12,1,1,"Bonnell, Miss. Elizabeth",female,58,0,0,113783,26.55,C103,S
13,0,3,"Saundercock, Mr. William Henry",male,20,0,0,A/5. 2151,8.05,,S
14,0,3,"Andersson, Mr. Anders Johan",male,39,1,5,347082,31.275,,S
15,0,3,"Vestrom, Miss. Hulda Amanda Adolfina",female,14,0,0,350406,7.8542,,S
16,1,2,"Hewlett, Mrs. (Mary D Kingcome) ",female,55,0,0,248706,16,,S
17,0,3,"Rice, Master. Eugene",male,2,4,1,382652,29.125,,Q
18,1,2,"Williams, Mr. Charles Eugene",male,,0,0,244373,13,,S
19,0,3,"Vander Planke, Mrs. Julius (Emelia Maria Vandemoortele)",female,31,1,0,345763,18,,S
20,1,3,"Masselmani, Mrs. Fatima",female,,0,0,2649,7.225,,C
21,0,2,"Fynney, Mr. Joseph J",male,35,0,0,239865,26,,S
22,1,2,"Beesley, Mr. Lawrence",male,34,0,0,248698,13,D56,S
23,1,3,"McGowan, Miss. Anna ""Annie""",female,15,0,0,330923,8.0292,,Q
24,1,1,"Sloper, Mr. William Thompson",male,28,0,0,113788,35.5,A6,S
25,0,3,"Palsson, Miss. Torborg Danira",female,8,3,1,349909,21.075,,S
26,1,3,"Asplund, Mrs. Carl Oscar (Selma Augusta Emilia Johansson)",female,38,1,5,347077,31.3875,,S
27,0,3,"Emir, Mr. Farred Chehab",male,,0,0,2631,7.225,,C
28,0,1,"Fortune, Mr. Charles Alexander",male,19,3,2,19950,263,C23 C25 C27,S
29,1,3,"O'Dwyer, Miss. Ellen ""Nellie""",female,,0,0,330959,7.8792,,Q
30,0,3,"Todoroff, Mr. Lalio",male,,0,0,349216,7.8958,,S
31,0,1,"Uruchurtu, Don. Manuel E",male,40,0,0,PC 17601,27.7208,,C
32,1,1,"Spencer, Mrs. William Augustus (Marie Eugenie)",female,,1,0,PC 17569,146.5208,B78,C
33,1,3,"Glynn, Miss. Mary Agatha",female,,0,0,335677,7.75,,Q
34,0,2,"Wheadon, Mr. Edward H",male,66,0,0,C.A. 24579,10.5,,S
35,0,1,"Meyer, Mr. Edgar Joseph",male,28,1,0,PC 17604,82.1708,,C
36,0,1,"Holverson, Mr. Alexander Oskar",male,42,1,0,113789,52,,S
37,1,3,"Mamee, Mr. Hanna",male,,0,0,2677,7.2292,,C
38,0,3,"Cann, Mr. Ernest Charles",male,21,0,0,A./5. 2152,8.05,,S
39,0,3,"Vander Planke, Miss. Augusta Maria",female,18,2,0,345764,18,,S
40,1,3,"Nicola-Yarred, Miss. Jamila",female,14,1,0,2651,11.2417,,C
41,0,3,"Ahlin, Mrs. Johan (Johanna Persdotter Larsson)",female,40,1,0,7546,9.475,,S
42,0,2,"Turpin, Mrs. William John Robert (Dorothy Ann Wonnacott)",female,27,1,0,11668,21,,S
43,0,3,"Kraeff, Mr. Theodor",male,,0,0,349253,7.8958,,C
44,1,2,"Laroche, Miss. Simonne Marie Anne Andree",female,3,1,2,SC/Paris 2123,41.5792,,C
45,1,3,"Devaney, Miss. Margaret Delia",female,19,0,0,330958,7.8792,,Q
46,0,3,"Rogers, Mr. William John",male,,0,0,S.C./A.4. 23567,8.05,,S
47,0,3,"Lennon, Mr. Denis",male,,1,0,370371,15.5,,Q
48,1,3,"O'Driscoll, Miss. Bridget",female,,0,0,14311,7.75,,Q
49,0,3,"Samaan, Mr. Youssef",male,,2,0,2662,21.6792,,C
50,0,3,"Arnold-Franchi, Mrs. Josef (Josefine Franchi)",female,18,1,0,349237,17.8,,S
51,0,3,"Panula, Master. Juha Niilo",male,7,4,1,3101295,39.6875,,S
52,0,3,"Nosworthy, Mr. Richard Cater",male,21,0,0,A/4. 39886,7.8,,S
53,1,1,"Harper, Mrs. Henry Sleeper (Myna Haxtun)",female,49,1,0,PC 17572,76.7292,D33,C
54,1,2,"Faunthorpe, Mrs. Lizzie (Elizabeth Anne Wilkinson)",female,29,1,0,2926,26,,S
55,0,1,"Ostby, Mr. Engelhart Cornelius",male,65,0,1,113509,61.9792,B30,C
56,1,1,"Woolner, Mr. Hugh",male,,0,0,19947,35.5,C52,S
57,1,2,"Rugg, Miss. Emily",female,21,0,0,C.A. 31026,10.5,,S
58,0,3,"Novel, Mr. Mansouer",male,28.5,0,0,2697,7.2292,,C
59,1,2,"West, Miss. Constance Mirium",female,5,1,2,C.A. 34651,27.75,,S
60,0,3,"Goodwin, Master. William Frederick",male,11,5,2,CA 2144,46.9,,S
61,0,3,"Sirayanian, Mr. Orsen",male,22,0,0,2669,7.2292,,C
62,1,1,"Icard, Miss. Amelie",female,38,0,0,113572,80,B28,
63,0,1,"Harris, Mr. Henry Birkhardt",male,45,1,0,36973,83.475,C83,S
64,0,3,"Skoog, Master. Harald",male,4,3,2,347088,27.9,,S
65,0,1,"Stewart, Mr. Albert A",male,,0,0,PC 17605,27.7208,,C
66,1,3,"Moubarek, Master. Gerios",male,,1,1,2661,15.2458,,C
67,1,2,"Nye, Mrs. (Elizabeth Ramell)",female,29,0,0,C.A. 29395,10.5,F33,S
68,0,3,"Crease, Mr. Ernest James",male,19,0,0,S.P. 3464,8.1583,,S
69,1,3,"Andersson, Miss. Erna Alexandra",female,17,4,2,3101281,7.925,,S
70,0,3,"Kink, Mr. Vincenz",male,26,2,0,315151,8.6625,,S
71,0,2,"Jenkin, Mr. Stephen Curnow",male,32,0,0,C.A. 33111,10.5,,S
72,0,3,"Goodwin, Miss. Lillian Amy",female,16,5,2,CA 2144,46.9,,S
73,0,2,"Hood, Mr. Ambrose Jr",male,21,0,0,S.O.C. 14879,73.5,,S
74,0,3,"Chronopoulos, Mr. Apostolos",male,26,1,0,2680,14.4542,,C
75,1,3,"Bing, Mr. Lee",male,32,0,0,1601,56.4958,,S
76,0,3,"Moen, Mr. Sigurd Hansen",male,25,0,0,348123,7.65,F G73,S
77,0,3,"Staneff, Mr. Ivan",male,,0,0,349208,7.8958,,S
78,0,3,"Moutal, Mr. Rahamin Haim",male,,0,0,374746,8.05,,S
79,1,2,"Caldwell, Master. Alden Gates",male,0.83,0,2,248738,29,,S
80,1,3,"Dowdell, Miss. Elizabeth",female,30,0,0,364516,12.475,,S
81,0,3,"Waelens, Mr. Achille",male,22,0,0,345767,9,,S
82,1,3,"Sheerlinck, Mr. Jan Baptist",male,29,0,0,345779,9.5,,S
83,1,3,"McDermott, Miss. Brigdet Delia",female,,0,0,330932,7.7875,,Q
84,0,1,"Carrau, Mr. Francisco M",male,28,0,0,113059,47.1,,S
85,1,2,"Ilett, Miss. Bertha",female,17,0,0,SO/C 14885,10.5,,S
86,1,3,"Backstrom, Mrs. Karl Alfred (Maria Mathilda Gustafsson)",female,33,3,0,3101278,15.85,,S
87,0,3,"Ford, Mr. William Neal",male,16,1,3,W./C. 6608,34.375,,S
88,0,3,"Slocovski, Mr. Selman Francis",male,,0,0,SOTON/OQ 392086,8.05,,S
89,1,1,"Fortune, Miss. Mabel Helen",female,23,3,2,19950,263,C23 C25 C27,S
90,0,3,"Celotti, Mr. Francesco",male,24,0,0,343275,8.05,,S
91,0,3,"Christmann, Mr. Emil",male,29,0,0,343276,8.05,,S
92,0,3,"Andreasson, Mr. Paul Edvin",male,20,0,0,347466,7.8542,,S
93,0,1,"Chaffee, Mr. Herbert Fuller",male,46,1,0,W.E.P. 5734,61.175,E31,S
94,0,3,"Dean, Mr. Bertram Frank",male,26,1,2,C.A. 2315,20.575,,S
95,0,3,"Coxon, Mr. Daniel",male,59,0,0,364500,7.25,,S
96,0,3,"Shorney, Mr. Charles Joseph",male,,0,0,374910,8.05,,S
97,0,1,"Goldschmidt, Mr. George B",male,71,0,0,PC 17754,34.6542,A5,C
98,1,1,"Greenfield, Mr. William Bertram",male,23,0,1,PC 17759,63.3583,D10 D12,C
99,1,2,"Doling, Mrs. John T (Ada Julia Bone)",female,34,0,1,231919,23,,S
100,0,2,"Kantor, Mr. Sinai",male,34,1,0,244367,26,,S
101,0,3,"Petranec, Miss. Matilda",female,28,0,0,349245,7.8958,,S
102,0,3,"Petroff, Mr. Pastcho (""Pentcho"")",male,,0,0,349215,7.8958,,S
103,0,1,"White, Mr. Richard Frasar",male,21,0,1,35281,77.2875,D26,S
104,0,3,"Johansson, Mr. Gustaf Joel",male,33,0,0,7540,8.6542,,S
105,0,3,"Gustafsson, Mr. Anders Vilhelm",male,37,2,0,3101276,7.925,,S
106,0,3,"Mionoff, Mr. Stoytcho",male,28,0,0,349207,7.8958,,S
107,1,3,"Salkjelsvik, Miss. Anna Kristine",female,21,0,0,343120,7.65,,S
108,1,3,"Moss, Mr. Albert Johan",male,,0,0,312991,7.775,,S
109,0,3,"Rekic, Mr. Tido",male,38,0,0,349249,7.8958,,S
110,1,3,"Moran, Miss. Bertha",female,,1,0,371110,24.15,,Q
111,0,1,"Porter, Mr. Walter Chamberlain",male,47,0,0,110465,52,C110,S
112,0,3,"Zabour, Miss. Hileni",female,14.5,1,0,2665,14.4542,,C
113,0,3,"Barton, Mr. David John",male,22,0,0,324669,8.05,,S
114,0,3,"Jussila, Miss. Katriina",female,20,1,0,4136,9.825,,S
115,0,3,"Attalah, Miss. Malake",female,17,0,0,2627,14.4583,,C
116,0,3,"Pekoniemi, Mr. Edvard",male,21,0,0,STON/O 2. 3101294,7.925,,S
117,0,3,"Connors, Mr. Patrick",male,70.5,0,0,370369,7.75,,Q
118,0,2,"Turpin, Mr. William John Robert",male,29,1,0,11668,21,,S
119,0,1,"Baxter, Mr. Quigg Edmond",male,24,0,1,PC 17558,247.5208,B58 B60,C
120,0,3,"Andersson, Miss. Ellis Anna Maria",female,2,4,2,347082,31.275,,S
121,0,2,"Hickman, Mr. Stanley George",male,21,2,0,S.O.C. 14879,73.5,,S
122,0,3,"Moore, Mr. Leonard Charles",male,,0,0,A4. 54510,8.05,,S
123,0,2,"Nasser, Mr. Nicholas",male,32.5,1,0,237736,30.0708,,C
124,1,2,"Webber, Miss. Susan",female,32.5,0,0,27267,13,E101,S
125,0,1,"White, Mr. Percival Wayland",male,54,0,1,35281,77.2875,D26,S
126,1,3,"Nicola-Yarred, Master. Elias",male,12,1,0,2651,11.2417,,C
127,0,3,"McMahon, Mr. Martin",male,,0,0,370372,7.75,,Q
128,1,3,"Madsen, Mr. Fridtjof Arne",male,24,0,0,C 17369,7.1417,,S
129,1,3,"Peter, Miss. Anna",female,,1,1,2668,22.3583,F E69,C
130,0,3,"Ekstrom, Mr. Johan",male,45,0,0,347061,6.975,,S
131,0,3,"Drazenoic, Mr. Jozef",male,33,0,0,349241,7.8958,,C
132,0,3,"Coelho, Mr. Domingos Fernandeo",male,20,0,0,SOTON/O.Q. 3101307,7.05,,S
133,0,3,"Robins, Mrs. Alexander A (Grace Charity Laury)",female,47,1,0,A/5. 3337,14.5,,S
134,1,2,"Weisz, Mrs. Leopold (Mathilde Francoise Pede)",female,29,1,0,228414,26,,S
135,0,2,"Sobey, Mr. Samuel James Hayden",male,25,0,0,C.A. 29178,13,,S
136,0,2,"Richard, Mr. Emile",male,23,0,0,SC/PARIS 2133,15.0458,,C
137,1,1,"Newsom, Miss. Helen Monypeny",female,19,0,2,11752,26.2833,D47,S
138,0,1,"Futrelle, Mr. Jacques Heath",male,37,1,0,113803,53.1,C123,S
139,0,3,"Osen, Mr. Olaf Elon",male,16,0,0,7534,9.2167,,S
140,0,1,"Giglio, Mr. Victor",male,24,0,0,PC 17593,79.2,B86,C
141,0,3,"Boulos, Mrs. Joseph (Sultana)",female,,0,2,2678,15.2458,,C
142,1,3,"Nysten, Miss. Anna Sofia",female,22,0,0,347081,7.75,,S
143,1,3,"Hakkarainen, Mrs. Pekka Pietari (Elin Matilda Dolck)",female,24,1,0,STON/O2. 3101279,15.85,,S
144,0,3,"Burke, Mr. Jeremiah",male,19,0,0,365222,6.75,,Q
145,0,2,"Andrew, Mr. Edgardo Samuel",male,18,0,0,231945,11.5,,S
146,0,2,"Nicholls, Mr. Joseph Charles",male,19,1,1,C.A. 33112,36.75,,S
147,1,3,"Andersson, Mr. August Edvard (""Wennerstrom"")",male,27,0,0,350043,7.7958,,S
148,0,3,"Ford, Miss. Robina Maggie ""Ruby""",female,9,2,2,W./C. 6608,34.375,,S
149,0,2,"Navratil, Mr. Michel (""Louis M Hoffman"")",male,36.5,0,2,230080,26,F2,S
150,0,2,"Byles, Rev. Thomas Roussel Davids",male,42,0,0,244310,13,,S
151,0,2,"Bateman, Rev. Robert James",male,51,0,0,S.O.P. 1166,12.525,,S
152,1,1,"Pears, Mrs. Thomas (Edith Wearne)",female,22,1,0,113776,66.6,C2,S
153,0,3,"Meo, Mr. Alfonzo",male,55.5,0,0,A.5. 11206,8.05,,S
154,0,3,"van Billiard, Mr. Austin Blyler",male,40.5,0,2,A/5. 851,14.5,,S
155,0,3,"Olsen, Mr. Ole Martin",male,,0,0,Fa 265302,7.3125,,S
156,0,1,"Williams, Mr. Charles Duane",male,51,0,1,PC 17597,61.3792,,C
157,1,3,"Gilnagh, Miss. Katherine ""Katie""",female,16,0,0,35851,7.7333,,Q
158,0,3,"Corn, Mr. Harry",male,30,0,0,SOTON/OQ 392090,8.05,,S
159,0,3,"Smiljanic, Mr. Mile",male,,0,0,315037,8.6625,,S
160,0,3,"Sage, Master. Thomas Henry",male,,8,2,CA. 2343,69.55,,S
161,0,3,"Cribb, Mr. John Hatfield",male,44,0,1,371362,16.1,,S
162,1,2,"Watt, Mrs. James (Elizabeth ""Bessie"" Inglis Milne)",female,40,0,0,C.A. 33595,15.75,,S
163,0,3,"Bengtsson, Mr. John Viktor",male,26,0,0,347068,7.775,,S
164,0,3,"Calic, Mr. Jovo",male,17,0,0,315093,8.6625,,S
165,0,3,"Panula, Master. Eino Viljami",male,1,4,1,3101295,39.6875,,S
166,1,3,"Goldsmith, Master. Frank John William ""Frankie""",male,9,0,2,363291,20.525,,S
167,1,1,"Chibnall, Mrs. (Edith Martha Bowerman)",female,,0,1,113505,55,E33,S
168,0,3,"Skoog, Mrs. William (Anna Bernhardina Karlsson)",female,45,1,4,347088,27.9,,S
169,0,1,"Baumann, Mr. John D",male,,0,0,PC 17318,25.925,,S
170,0,3,"Ling, Mr. Lee",male,28,0,0,1601,56.4958,,S
171,0,1,"Van der hoef, Mr. Wyckoff",male,61,0,0,111240,33.5,B19,S
172,0,3,"Rice, Master. Arthur",male,4,4,1,382652,29.125,,Q
173,1,3,"Johnson, Miss. Eleanor Ileen",female,1,1,1,347742,11.1333,,S
174,0,3,"Sivola, Mr. Antti Wilhelm",male,21,0,0,STON/O 2. 3101280,7.925,,S
175,0,1,"Smith, Mr. James Clinch",male,56,0,0,17764,30.6958,A7,C
176,0,3,"Klasen, Mr. Klas Albin",male,18,1,1,350404,7.8542,,S
177,0,3,"Lefebre, Master. Henry Forbes",male,,3,1,4133,25.4667,,S
178,0,1,"Isham, Miss. Ann Elizabeth",female,50,0,0,PC 17595,28.7125,C49,C
179,0,2,"Hale, Mr. Reginald",male,30,0,0,250653,13,,S
180,0,3,"Leonard, Mr. Lionel",male,36,0,0,LINE,0,,S
181,0,3,"Sage, Miss. Constance Gladys",female,,8,2,CA. 2343,69.55,,S
182,0,2,"Pernot, Mr. Rene",male,,0,0,SC/PARIS 2131,15.05,,C
183,0,3,"Asplund, Master. Clarence Gustaf Hugo",male,9,4,2,347077,31.3875,,S
184,1,2,"Becker, Master. Richard F",male,1,2,1,230136,39,F4,S
185,1,3,"Kink-Heilmann, Miss. Luise Gretchen",female,4,0,2,315153,22.025,,S
186,0,1,"Rood, Mr. Hugh Roscoe",male,,0,0,113767,50,A32,S
187,1,3,"O'Brien, Mrs. Thomas (Johanna ""Hannah"" Godfrey)",female,,1,0,370365,15.5,,Q
188,1,1,"Romaine, Mr. Charles Hallace (""Mr C Rolmane"")",male,45,0,0,111428,26.55,,S
189,0,3,"Bourke, Mr. John",male,40,1,1,364849,15.5,,Q
190,0,3,"Turcin, Mr. Stjepan",male,36,0,0,349247,7.8958,,S
191,1,2,"Pinsky, Mrs. (Rosa)",female,32,0,0,234604,13,,S
192,0,2,"Carbines, Mr. William",male,19,0,0,28424,13,,S
193,1,3,"Andersen-Jensen, Miss. Carla Christine Nielsine",female,19,1,0,350046,7.8542,,S
194,1,2,"Navratil, Master. Michel M",male,3,1,1,230080,26,F2,S
195,1,1,"Brown, Mrs. James Joseph (Margaret Tobin)",female,44,0,0,PC 17610,27.7208,B4,C
196,1,1,"Lurette, Miss. Elise",female,58,0,0,PC 17569,146.5208,B80,C
197,0,3,"Mernagh, Mr. Robert",male,,0,0,368703,7.75,,Q
198,0,3,"Olsen, Mr. Karl Siegwart Andreas",male,42,0,1,4579,8.4042,,S
199,1,3,"Madigan, Miss. Margaret ""Maggie""",female,,0,0,370370,7.75,,Q
200,0,2,"Yrois, Miss. Henriette (""Mrs Harbeck"")",female,24,0,0,248747,13,,S
201,0,3,"Vande Walle, Mr. Nestor Cyriel",male,28,0,0,345770,9.5,,S
202,0,3,"Sage, Mr. Frederick",male,,8,2,CA. 2343,69.55,,S
203,0,3,"Johanson, Mr. Jakob Alfred",male,34,0,0,3101264,6.4958,,S
204,0,3,"Youseff, Mr. Gerious",male,45.5,0,0,2628,7.225,,C
205,1,3,"Cohen, Mr. Gurshon ""Gus""",male,18,0,0,A/5 3540,8.05,,S
206,0,3,"Strom, Miss. Telma Matilda",female,2,0,1,347054,10.4625,G6,S
207,0,3,"Backstrom, Mr. Karl Alfred",male,32,1,0,3101278,15.85,,S
208,1,3,"Albimona, Mr. Nassef Cassem",male,26,0,0,2699,18.7875,,C
209,1,3,"Carr, Miss. Helen ""Ellen""",female,16,0,0,367231,7.75,,Q
210,1,1,"Blank, Mr. Henry",male,40,0,0,112277,31,A31,C
211,0,3,"Ali, Mr. Ahmed",male,24,0,0,SOTON/O.Q. 3101311,7.05,,S
212,1,2,"Cameron, Miss. Clear Annie",female,35,0,0,F.C.C. 13528,21,,S
213,0,3,"Perkin, Mr. John Henry",male,22,0,0,A/5 21174,7.25,,S
214,0,2,"Givard, Mr. Hans Kristensen",male,30,0,0,250646,13,,S
215,0,3,"Kiernan, Mr. Philip",male,,1,0,367229,7.75,,Q
216,1,1,"Newell, Miss. Madeleine",female,31,1,0,35273,113.275,D36,C
217,1,3,"Honkanen, Miss. Eliina",female,27,0,0,STON/O2. 3101283,7.925,,S
218,0,2,"Jacobsohn, Mr. Sidney Samuel",male,42,1,0,243847,27,,S
219,1,1,"Bazzani, Miss. Albina",female,32,0,0,11813,76.2917,D15,C
220,0,2,"Harris, Mr. Walter",male,30,0,0,W/C 14208,10.5,,S
221,1,3,"Sunderland, Mr. Victor Francis",male,16,0,0,SOTON/OQ 392089,8.05,,S
222,0,2,"Bracken, Mr. James H",male,27,0,0,220367,13,,S
223,0,3,"Green, Mr. George Henry",male,51,0,0,21440,8.05,,S
224,0,3,"Nenkoff, Mr. Christo",male,,0,0,349234,7.8958,,S
225,1,1,"Hoyt, Mr. Frederick Maxfield",male,38,1,0,19943,90,C93,S
226,0,3,"Berglund, Mr. Karl Ivar Sven",male,22,0,0,PP 4348,9.35,,S
227,1,2,"Mellors, Mr. William John",male,19,0,0,SW/PP 751,10.5,,S
228,0,3,"Lovell, Mr. John Hall (""Henry"")",male,20.5,0,0,A/5 21173,7.25,,S
229,0,2,"Fahlstrom, Mr. Arne Jonas",male,18,0,0,236171,13,,S
230,0,3,"Lefebre, Miss. Mathilde",female,,3,1,4133,25.4667,,S
231,1,1,"Harris, Mrs. Henry Birkhardt (Irene Wallach)",female,35,1,0,36973,83.475,C83,S
232,0,3,"Larsson, Mr. Bengt Edvin",male,29,0,0,347067,7.775,,S
233,0,2,"Sjostedt, Mr. Ernst Adolf",male,59,0,0,237442,13.5,,S
234,1,3,"Asplund, Miss. Lillian Gertrud",female,5,4,2,347077,31.3875,,S
235,0,2,"Leyson, Mr. Robert William Norman",male,24,0,0,C.A. 29566,10.5,,S
236,0,3,"Harknett, Miss. Alice Phoebe",female,,0,0,W./C. 6609,7.55,,S
237,0,2,"Hold, Mr. Stephen",male,44,1,0,26707,26,,S
238,1,2,"Collyer, Miss. Marjorie ""Lottie""",female,8,0,2,C.A. 31921,26.25,,S
239,0,2,"Pengelly, Mr. Frederick William",male,19,0,0,28665,10.5,,S
240,0,2,"Hunt, Mr. George Henry",male,33,0,0,SCO/W 1585,12.275,,S
241,0,3,"Zabour, Miss. Thamine",female,,1,0,2665,14.4542,,C
242,1,3,"Murphy, Miss. Katherine ""Kate""",female,,1,0,367230,15.5,,Q
243,0,2,"Coleridge, Mr. Reginald Charles",male,29,0,0,W./C. 14263,10.5,,S
244,0,3,"Maenpaa, Mr. Matti Alexanteri",male,22,0,0,STON/O 2. 3101275,7.125,,S
245,0,3,"Attalah, Mr. Sleiman",male,30,0,0,2694,7.225,,C
246,0,1,"Minahan, Dr. William Edward",male,44,2,0,19928,90,C78,Q
247,0,3,"Lindahl, Miss. Agda Thorilda Viktoria",female,25,0,0,347071,7.775,,S
248,1,2,"Hamalainen, Mrs. William (Anna)",female,24,0,2,250649,14.5,,S
249,1,1,"Beckwith, Mr. Richard Leonard",male,37,1,1,11751,52.5542,D35,S
250,0,2,"Carter, Rev. Ernest Courtenay",male,54,1,0,244252,26,,S
251,0,3,"Reed, Mr. James George",male,,0,0,362316,7.25,,S
252,0,3,"Strom, Mrs. Wilhelm (Elna Matilda Persson)",female,29,1,1,347054,10.4625,G6,S
253,0,1,"Stead, Mr. William Thomas",male,62,0,0,113514,26.55,C87,S
254,0,3,"Lobb, Mr. William Arthur",male,30,1,0,A/5. 3336,16.1,,S
255,0,3,"Rosblom, Mrs. Viktor (Helena Wilhelmina)",female,41,0,2,370129,20.2125,,S
256,1,3,"Touma, Mrs. Darwis (Hanne Youssef Razi)",female,29,0,2,2650,15.2458,,C
257,1,1,"Thorne, Mrs. Gertrude Maybelle",female,,0,0,PC 17585,79.2,,C
258,1,1,"Cherry, Miss. Gladys",female,30,0,0,110152,86.5,B77,S
259,1,1,"Ward, Miss. Anna",female,35,0,0,PC 17755,512.3292,,C
260,1,2,"Parrish, Mrs. (Lutie Davis)",female,50,0,1,230433,26,,S
261,0,3,"Smith, Mr. Thomas",male,,0,0,384461,7.75,,Q
262,1,3,"Asplund, Master. Edvin Rojj Felix",male,3,4,2,347077,31.3875,,S
263,0,1,"Taussig, Mr. Emil",male,52,1,1,110413,79.65,E67,S
264,0,1,"Harrison, Mr. William",male,40,0,0,112059,0,B94,S
265,0,3,"Henry, Miss. Delia",female,,0,0,382649,7.75,,Q
266,0,2,"Reeves, Mr. David",male,36,0,0,C.A. 17248,10.5,,S
267,0,3,"Panula, Mr. Ernesti Arvid",male,16,4,1,3101295,39.6875,,S
268,1,3,"Persson, Mr. Ernst Ulrik",male,25,1,0,347083,7.775,,S
269,1,1,"Graham, Mrs. William Thompson (Edith Junkins)",female,58,0,1,PC 17582,153.4625,C125,S
270,1,1,"Bissette, Miss. Amelia",female,35,0,0,PC 17760,135.6333,C99,S
271,0,1,"Cairns, Mr. Alexander",male,,0,0,113798,31,,S
272,1,3,"Tornquist, Mr. William Henry",male,25,0,0,LINE,0,,S
273,1,2,"Mellinger, Mrs. (Elizabeth Anne Maidment)",female,41,0,1,250644,19.5,,S
274,0,1,"Natsch, Mr. Charles H",male,37,0,1,PC 17596,29.7,C118,C
275,1,3,"Healy, Miss. Hanora ""Nora""",female,,0,0,370375,7.75,,Q
276,1,1,"Andrews, Miss. Kornelia Theodosia",female,63,1,0,13502,77.9583,D7,S
277,0,3,"Lindblom, Miss. Augusta Charlotta",female,45,0,0,347073,7.75,,S
278,0,2,"Parkes, Mr. Francis ""Frank""",male,,0,0,239853,0,,S
279,0,3,"Rice, Master. Eric",male,7,4,1,382652,29.125,,Q
280,1,3,"Abbott, Mrs. Stanton (Rosa Hunt)",female,35,1,1,C.A. 2673,20.25,,S
281,0,3,"Duane, Mr. Frank",male,65,0,0,336439,7.75,,Q
282,0,3,"Olsson, Mr. Nils Johan Goransson",male,28,0,0,347464,7.8542,,S
283,0,3,"de Pelsmaeker, Mr. Alfons",male,16,0,0,345778,9.5,,S
284,1,3,"Dorking, Mr. Edward Arthur",male,19,0,0,A/5. 10482,8.05,,S
285,0,1,"Smith, Mr. Richard William",male,,0,0,113056,26,A19,S
286,0,3,"Stankovic, Mr. Ivan",male,33,0,0,349239,8.6625,,C
287,1,3,"de Mulder, Mr. Theodore",male,30,0,0,345774,9.5,,S
288,0,3,"Naidenoff, Mr. Penko",male,22,0,0,349206,7.8958,,S
289,1,2,"Hosono, Mr. Masabumi",male,42,0,0,237798,13,,S
290,1,3,"Connolly, Miss. Kate",female,22,0,0,370373,7.75,,Q
291,1,1,"Barber, Miss. Ellen ""Nellie""",female,26,0,0,19877,78.85,,S
292,1,1,"Bishop, Mrs. Dickinson H (Helen Walton)",female,19,1,0,11967,91.0792,B49,C
293,0,2,"Levy, Mr. Rene Jacques",male,36,0,0,SC/Paris 2163,12.875,D,C
294,0,3,"Haas, Miss. Aloisia",female,24,0,0,349236,8.85,,S
295,0,3,"Mineff, Mr. Ivan",male,24,0,0,349233,7.8958,,S
296,0,1,"Lewy, Mr. Ervin G",male,,0,0,PC 17612,27.7208,,C
297,0,3,"Hanna, Mr. Mansour",male,23.5,0,0,2693,7.2292,,C
298,0,1,"Allison, Miss. Helen Loraine",female,2,1,2,113781,151.55,C22 C26,S
299,1,1,"Saalfeld, Mr. Adolphe",male,,0,0,19988,30.5,C106,S
300,1,1,"Baxter, Mrs. James (Helene DeLaudeniere Chaput)",female,50,0,1,PC 17558,247.5208,B58 B60,C
301,1,3,"Kelly, Miss. Anna Katherine ""Annie Kate""",female,,0,0,9234,7.75,,Q
302,1,3,"McCoy, Mr. Bernard",male,,2,0,367226,23.25,,Q
303,0,3,"Johnson, Mr. William Cahoone Jr",male,19,0,0,LINE,0,,S
304,1,2,"Keane, Miss. Nora A",female,,0,0,226593,12.35,E101,Q
305,0,3,"Williams, Mr. Howard Hugh ""Harry""",male,,0,0,A/5 2466,8.05,,S
306,1,1,"Allison, Master. Hudson Trevor",male,0.92,1,2,113781,151.55,C22 C26,S
307,1,1,"Fleming, Miss. Margaret",female,,0,0,17421,110.8833,,C
308,1,1,"Penasco y Castellana, Mrs. Victor de Satode (Maria Josefa Perez de Soto y Vallejo)",female,17,1,0,PC 17758,108.9,C65,C
309,0,2,"Abelson, Mr. Samuel",male,30,1,0,P/PP 3381,24,,C
310,1,1,"Francatelli, Miss. Laura Mabel",female,30,0,0,PC 17485,56.9292,E36,C
311,1,1,"Hays, Miss. Margaret Bechstein",female,24,0,0,11767,83.1583,C54,C
312,1,1,"Ryerson, Miss. Emily Borie",female,18,2,2,PC 17608,262.375,B57 B59 B63 B66,C
313,0,2,"Lahtinen, Mrs. William (Anna Sylfven)",female,26,1,1,250651,26,,S
314,0,3,"Hendekovic, Mr. Ignjac",male,28,0,0,349243,7.8958,,S
315,0,2,"Hart, Mr. Benjamin",male,43,1,1,F.C.C. 13529,26.25,,S
316,1,3,"Nilsson, Miss. Helmina Josefina",female,26,0,0,347470,7.8542,,S
317,1,2,"Kantor, Mrs. Sinai (Miriam Sternin)",female,24,1,0,244367,26,,S
318,0,2,"Moraweck, Dr. Ernest",male,54,0,0,29011,14,,S
319,1,1,"Wick, Miss. Mary Natalie",female,31,0,2,36928,164.8667,C7,S
320,1,1,"Spedden, Mrs. Frederic Oakley (Margaretta Corning Stone)",female,40,1,1,16966,134.5,E34,C
321,0,3,"Dennis, Mr. Samuel",male,22,0,0,A/5 21172,7.25,,S
322,0,3,"Danoff, Mr. Yoto",male,27,0,0,349219,7.8958,,S
323,1,2,"Slayter, Miss. Hilda Mary",female,30,0,0,234818,12.35,,Q
324,1,2,"Caldwell, Mrs. Albert Francis (Sylvia Mae Harbaugh)",female,22,1,1,248738,29,,S
325,0,3,"Sage, Mr. George John Jr",male,,8,2,CA. 2343,69.55,,S
326,1,1,"Young, Miss. Marie Grice",female,36,0,0,PC 17760,135.6333,C32,C
327,0,3,"Nysveen, Mr. Johan Hansen",male,61,0,0,345364,6.2375,,S
328,1,2,"Ball, Mrs. (Ada E Hall)",female,36,0,0,28551,13,D,S
329,1,3,"Goldsmith, Mrs. Frank John (Emily Alice Brown)",female,31,1,1,363291,20.525,,S
330,1,1,"Hippach, Miss. Jean Gertrude",female,16,0,1,111361,57.9792,B18,C
331,1,3,"McCoy, Miss. Agnes",female,,2,0,367226,23.25,,Q
332,0,1,"Partner, Mr. Austen",male,45.5,0,0,113043,28.5,C124,S
333,0,1,"Graham, Mr. George Edward",male,38,0,1,PC 17582,153.4625,C91,S
334,0,3,"Vander Planke, Mr. Leo Edmondus",male,16,2,0,345764,18,,S
335,1,1,"Frauenthal, Mrs. Henry William (Clara Heinsheimer)",female,,1,0,PC 17611,133.65,,S
336,0,3,"Denkoff, Mr. Mitto",male,,0,0,349225,7.8958,,S
337,0,1,"Pears, Mr. Thomas Clinton",male,29,1,0,113776,66.6,C2,S
338,1,1,"Burns, Miss. Elizabeth Margaret",female,41,0,0,16966,134.5,E40,C
339,1,3,"Dahl, Mr. Karl Edwart",male,45,0,0,7598,8.05,,S
340,0,1,"Blackwell, Mr. Stephen Weart",male,45,0,0,113784,35.5,T,S
341,1,2,"Navratil, Master. Edmond Roger",male,2,1,1,230080,26,F2,S
342,1,1,"Fortune, Miss. Alice Elizabeth",female,24,3,2,19950,263,C23 C25 C27,S
343,0,2,"Collander, Mr. Erik Gustaf",male,28,0,0,248740,13,,S
344,0,2,"Sedgwick, Mr. Charles Frederick Waddington",male,25,0,0,244361,13,,S
345,0,2,"Fox, Mr. Stanley Hubert",male,36,0,0,229236,13,,S
346,1,2,"Brown, Miss. Amelia ""Mildred""",female,24,0,0,248733,13,F33,S
347,1,2,"Smith, Miss. Marion Elsie",female,40,0,0,31418,13,,S
348,1,3,"Davison, Mrs. Thomas Henry (Mary E Finck)",female,,1,0,386525,16.1,,S
349,1,3,"Coutts, Master. William Loch ""William""",male,3,1,1,C.A. 37671,15.9,,S
350,0,3,"Dimic, Mr. Jovan",male,42,0,0,315088,8.6625,,S
351,0,3,"Odahl, Mr. Nils Martin",male,23,0,0,7267,9.225,,S
352,0,1,"Williams-Lambert, Mr. Fletcher Fellows",male,,0,0,113510,35,C128,S
353,0,3,"Elias, Mr. Tannous",male,15,1,1,2695,7.2292,,C
354,0,3,"Arnold-Franchi, Mr. Josef",male,25,1,0,349237,17.8,,S
355,0,3,"Yousif, Mr. Wazli",male,,0,0,2647,7.225,,C
356,0,3,"Vanden Steen, Mr. Leo Peter",male,28,0,0,345783,9.5,,S
357,1,1,"Bowerman, Miss. Elsie Edith",female,22,0,1,113505,55,E33,S
358,0,2,"Funk, Miss. Annie Clemmer",female,38,0,0,237671,13,,S
359,1,3,"McGovern, Miss. Mary",female,,0,0,330931,7.8792,,Q
360,1,3,"Mockler, Miss. Helen Mary ""Ellie""",female,,0,0,330980,7.8792,,Q
361,0,3,"Skoog, Mr. Wilhelm",male,40,1,4,347088,27.9,,S
362,0,2,"del Carlo, Mr. Sebastiano",male,29,1,0,SC/PARIS 2167,27.7208,,C
363,0,3,"Barbara, Mrs. (Catherine David)",female,45,0,1,2691,14.4542,,C
364,0,3,"Asim, Mr. Adola",male,35,0,0,SOTON/O.Q. 3101310,7.05,,S
365,0,3,"O'Brien, Mr. Thomas",male,,1,0,370365,15.5,,Q
366,0,3,"Adahl, Mr. Mauritz Nils Martin",male,30,0,0,C 7076,7.25,,S
367,1,1,"Warren, Mrs. Frank Manley (Anna Sophia Atkinson)",female,60,1,0,110813,75.25,D37,C
368,1,3,"Moussa, Mrs. (Mantoura Boulos)",female,,0,0,2626,7.2292,,C
369,1,3,"Jermyn, Miss. Annie",female,,0,0,14313,7.75,,Q
370,1,1,"Aubart, Mme. Leontine Pauline",female,24,0,0,PC 17477,69.3,B35,C
371,1,1,"Harder, Mr. George Achilles",male,25,1,0,11765,55.4417,E50,C
372,0,3,"Wiklund, Mr. Jakob Alfred",male,18,1,0,3101267,6.4958,,S
373,0,3,"Beavan, Mr. William Thomas",male,19,0,0,323951,8.05,,S
374,0,1,"Ringhini, Mr. Sante",male,22,0,0,PC 17760,135.6333,,C
375,0,3,"Palsson, Miss. Stina Viola",female,3,3,1,349909,21.075,,S
376,1,1,"Meyer, Mrs. Edgar Joseph (Leila Saks)",female,,1,0,PC 17604,82.1708,,C
377,1,3,"Landergren, Miss. Aurora Adelia",female,22,0,0,C 7077,7.25,,S
378,0,1,"Widener, Mr. Harry Elkins",male,27,0,2,113503,211.5,C82,C
379,0,3,"Betros, Mr. Tannous",male,20,0,0,2648,4.0125,,C
380,0,3,"Gustafsson, Mr. Karl Gideon",male,19,0,0,347069,7.775,,S
381,1,1,"Bidois, Miss. Rosalie",female,42,0,0,PC 17757,227.525,,C
382,1,3,"Nakid, Miss. Maria (""Mary"")",female,1,0,2,2653,15.7417,,C
383,0,3,"Tikkanen, Mr. Juho",male,32,0,0,STON/O 2. 3101293,7.925,,S
384,1,1,"Holverson, Mrs. Alexander Oskar (Mary Aline Towner)",female,35,1,0,113789,52,,S
385,0,3,"Plotcharsky, Mr. Vasil",male,,0,0,349227,7.8958,,S
386,0,2,"Davies, Mr. Charles Henry",male,18,0,0,S.O.C. 14879,73.5,,S
387,0,3,"Goodwin, Master. Sidney Leonard",male,1,5,2,CA 2144,46.9,,S
388,1,2,"Buss, Miss. Kate",female,36,0,0,27849,13,,S
389,0,3,"Sadlier, Mr. Matthew",male,,0,0,367655,7.7292,,Q
390,1,2,"Lehmann, Miss. Bertha",female,17,0,0,SC 1748,12,,C
391,1,1,"Carter, Mr. William Ernest",male,36,1,2,113760,120,B96 B98,S
392,1,3,"Jansson, Mr. Carl Olof",male,21,0,0,350034,7.7958,,S
393,0,3,"Gustafsson, Mr. Johan Birger",male,28,2,0,3101277,7.925,,S
394,1,1,"Newell, Miss. Marjorie",female,23,1,0,35273,113.275,D36,C
395,1,3,"Sandstrom, Mrs. Hjalmar (Agnes Charlotta Bengtsson)",female,24,0,2,PP 9549,16.7,G6,S
396,0,3,"Johansson, Mr. Erik",male,22,0,0,350052,7.7958,,S
397,0,3,"Olsson, Miss. Elina",female,31,0,0,350407,7.8542,,S
398,0,2,"McKane, Mr. Peter David",male,46,0,0,28403,26,,S
399,0,2,"Pain, Dr. Alfred",male,23,0,0,244278,10.5,,S
400,1,2,"Trout, Mrs. William H (Jessie L)",female,28,0,0,240929,12.65,,S
401,1,3,"Niskanen, Mr. Juha",male,39,0,0,STON/O 2. 3101289,7.925,,S
402,0,3,"Adams, Mr. John",male,26,0,0,341826,8.05,,S
403,0,3,"Jussila, Miss. Mari Aina",female,21,1,0,4137,9.825,,S
404,0,3,"Hakkarainen, Mr. Pekka Pietari",male,28,1,0,STON/O2. 3101279,15.85,,S
405,0,3,"Oreskovic, Miss. Marija",female,20,0,0,315096,8.6625,,S
406,0,2,"Gale, Mr. Shadrach",male,34,1,0,28664,21,,S
407,0,3,"Widegren, Mr. Carl/Charles Peter",male,51,0,0,347064,7.75,,S
408,1,2,"Richards, Master. William Rowe",male,3,1,1,29106,18.75,,S
409,0,3,"Birkeland, Mr. Hans Martin Monsen",male,21,0,0,312992,7.775,,S
410,0,3,"Lefebre, Miss. Ida",female,,3,1,4133,25.4667,,S
411,0,3,"Sdycoff, Mr. Todor",male,,0,0,349222,7.8958,,S
412,0,3,"Hart, Mr. Henry",male,,0,0,394140,6.8583,,Q
413,1,1,"Minahan, Miss. Daisy E",female,33,1,0,19928,90,C78,Q
414,0,2,"Cunningham, Mr. Alfred Fleming",male,,0,0,239853,0,,S
415,1,3,"Sundman, Mr. Johan Julian",male,44,0,0,STON/O 2. 3101269,7.925,,S
416,0,3,"Meek, Mrs. Thomas (Annie Louise Rowley)",female,,0,0,343095,8.05,,S
417,1,2,"Drew, Mrs. James Vivian (Lulu Thorne Christian)",female,34,1,1,28220,32.5,,S
418,1,2,"Silven, Miss. Lyyli Karoliina",female,18,0,2,250652,13,,S
419,0,2,"Matthews, Mr. William John",male,30,0,0,28228,13,,S
420,0,3,"Van Impe, Miss. Catharina",female,10,0,2,345773,24.15,,S
421,0,3,"Gheorgheff, Mr. Stanio",male,,0,0,349254,7.8958,,C
422,0,3,"Charters, Mr. David",male,21,0,0,A/5. 13032,7.7333,,Q
423,0,3,"Zimmerman, Mr. Leo",male,29,0,0,315082,7.875,,S
424,0,3,"Danbom, Mrs. Ernst Gilbert (Anna Sigrid Maria Brogren)",female,28,1,1,347080,14.4,,S
425,0,3,"Rosblom, Mr. Viktor Richard",male,18,1,1,370129,20.2125,,S
426,0,3,"Wiseman, Mr. Phillippe",male,,0,0,A/4. 34244,7.25,,S
427,1,2,"Clarke, Mrs. Charles V (Ada Maria Winfield)",female,28,1,0,2003,26,,S
428,1,2,"Phillips, Miss. Kate Florence (""Mrs Kate Louise Phillips Marshall"")",female,19,0,0,250655,26,,S
429,0,3,"Flynn, Mr. James",male,,0,0,364851,7.75,,Q
430,1,3,"Pickard, Mr. Berk (Berk Trembisky)",male,32,0,0,SOTON/O.Q. 392078,8.05,E10,S
431,1,1,"Bjornstrom-Steffansson, Mr. Mauritz Hakan",male,28,0,0,110564,26.55,C52,S
432,1,3,"Thorneycroft, Mrs. Percival (Florence Kate White)",female,,1,0,376564,16.1,,S
433,1,2,"Louch, Mrs. Charles Alexander (Alice Adelaide Slow)",female,42,1,0,SC/AH 3085,26,,S
434,0,3,"Kallio, Mr. Nikolai Erland",male,17,0,0,STON/O 2. 3101274,7.125,,S
435,0,1,"Silvey, Mr. William Baird",male,50,1,0,13507,55.9,E44,S
436,1,1,"Carter, Miss. Lucile Polk",female,14,1,2,113760,120,B96 B98,S
437,0,3,"Ford, Miss. Doolina Margaret ""Daisy""",female,21,2,2,W./C. 6608,34.375,,S
438,1,2,"Richards, Mrs. Sidney (Emily Hocking)",female,24,2,3,29106,18.75,,S
439,0,1,"Fortune, Mr. Mark",male,64,1,4,19950,263,C23 C25 C27,S
440,0,2,"Kvillner, Mr. Johan Henrik Johannesson",male,31,0,0,C.A. 18723,10.5,,S
441,1,2,"Hart, Mrs. Benjamin (Esther Ada Bloomfield)",female,45,1,1,F.C.C. 13529,26.25,,S
442,0,3,"Hampe, Mr. Leon",male,20,0,0,345769,9.5,,S
443,0,3,"Petterson, Mr. Johan Emil",male,25,1,0,347076,7.775,,S
444,1,2,"Reynaldo, Ms. Encarnacion",female,28,0,0,230434,13,,S
445,1,3,"Johannesen-Bratthammer, Mr. Bernt",male,,0,0,65306,8.1125,,S
446,1,1,"Dodge, Master. Washington",male,4,0,2,33638,81.8583,A34,S
447,1,2,"Mellinger, Miss. Madeleine Violet",female,13,0,1,250644,19.5,,S
448,1,1,"Seward, Mr. Frederic Kimber",male,34,0,0,113794,26.55,,S
449,1,3,"Baclini, Miss. Marie Catherine",female,5,2,1,2666,19.2583,,C
450,1,1,"Peuchen, Major. Arthur Godfrey",male,52,0,0,113786,30.5,C104,S
451,0,2,"West, Mr. Edwy Arthur",male,36,1,2,C.A. 34651,27.75,,S
452,0,3,"Hagland, Mr. Ingvald Olai Olsen",male,,1,0,65303,19.9667,,S
453,0,1,"Foreman, Mr. Benjamin Laventall",male,30,0,0,113051,27.75,C111,C
454,1,1,"Goldenberg, Mr. Samuel L",male,49,1,0,17453,89.1042,C92,C
455,0,3,"Peduzzi, Mr. Joseph",male,,0,0,A/5 2817,8.05,,S
456,1,3,"Jalsevac, Mr. Ivan",male,29,0,0,349240,7.8958,,C
457,0,1,"Millet, Mr. Francis Davis",male,65,0,0,13509,26.55,E38,S
458,1,1,"Kenyon, Mrs. Frederick R (Marion)",female,,1,0,17464,51.8625,D21,S
459,1,2,"Toomey, Miss. Ellen",female,50,0,0,F.C.C. 13531,10.5,,S
460,0,3,"O'Connor, Mr. Maurice",male,,0,0,371060,7.75,,Q
461,1,1,"Anderson, Mr. Harry",male,48,0,0,19952,26.55,E12,S
462,0,3,"Morley, Mr. William",male,34,0,0,364506,8.05,,S
463,0,1,"Gee, Mr. Arthur H",male,47,0,0,111320,38.5,E63,S
464,0,2,"Milling, Mr. Jacob Christian",male,48,0,0,234360,13,,S
465,0,3,"Maisner, Mr. Simon",male,,0,0,A/S 2816,8.05,,S
466,0,3,"Goncalves, Mr. Manuel Estanslas",male,38,0,0,SOTON/O.Q. 3101306,7.05,,S
467,0,2,"Campbell, Mr. William",male,,0,0,239853,0,,S
468,0,1,"Smart, Mr. John Montgomery",male,56,0,0,113792,26.55,,S
469,0,3,"Scanlan, Mr. James",male,,0,0,36209,7.725,,Q
470,1,3,"Baclini, Miss. Helene Barbara",female,0.75,2,1,2666,19.2583,,C
471,0,3,"Keefe, Mr. Arthur",male,,0,0,323592,7.25,,S
472,0,3,"Cacic, Mr. Luka",male,38,0,0,315089,8.6625,,S
473,1,2,"West, Mrs. Edwy Arthur (Ada Mary Worth)",female,33,1,2,C.A. 34651,27.75,,S
474,1,2,"Jerwan, Mrs. Amin S (Marie Marthe Thuillard)",female,23,0,0,SC/AH Basle 541,13.7917,D,C
475,0,3,"Strandberg, Miss. Ida Sofia",female,22,0,0,7553,9.8375,,S
476,0,1,"Clifford, Mr. George Quincy",male,,0,0,110465,52,A14,S
477,0,2,"Renouf, Mr. Peter Henry",male,34,1,0,31027,21,,S
478,0,3,"Braund, Mr. Lewis Richard",male,29,1,0,3460,7.0458,,S
479,0,3,"Karlsson, Mr. Nils August",male,22,0,0,350060,7.5208,,S
480,1,3,"Hirvonen, Miss. Hildur E",female,2,0,1,3101298,12.2875,,S
481,0,3,"Goodwin, Master. Harold Victor",male,9,5,2,CA 2144,46.9,,S
482,0,2,"Frost, Mr. Anthony Wood ""Archie""",male,,0,0,239854,0,,S
483,0,3,"Rouse, Mr. Richard Henry",male,50,0,0,A/5 3594,8.05,,S
484,1,3,"Turkula, Mrs. (Hedwig)",female,63,0,0,4134,9.5875,,S
485,1,1,"Bishop, Mr. Dickinson H",male,25,1,0,11967,91.0792,B49,C
486,0,3,"Lefebre, Miss. Jeannie",female,,3,1,4133,25.4667,,S
487,1,1,"Hoyt, Mrs. Frederick Maxfield (Jane Anne Forby)",female,35,1,0,19943,90,C93,S
488,0,1,"Kent, Mr. Edward Austin",male,58,0,0,11771,29.7,B37,C
489,0,3,"Somerton, Mr. Francis William",male,30,0,0,A.5. 18509,8.05,,S
490,1,3,"Coutts, Master. Eden Leslie ""Neville""",male,9,1,1,C.A. 37671,15.9,,S
491,0,3,"Hagland, Mr. Konrad Mathias Reiersen",male,,1,0,65304,19.9667,,S
492,0,3,"Windelov, Mr. Einar",male,21,0,0,SOTON/OQ 3101317,7.25,,S
493,0,1,"Molson, Mr. Harry Markland",male,55,0,0,113787,30.5,C30,S
494,0,1,"Artagaveytia, Mr. Ramon",male,71,0,0,PC 17609,49.5042,,C
495,0,3,"Stanley, Mr. Edward Roland",male,21,0,0,A/4 45380,8.05,,S
496,0,3,"Yousseff, Mr. Gerious",male,,0,0,2627,14.4583,,C
497,1,1,"Eustis, Miss. Elizabeth Mussey",female,54,1,0,36947,78.2667,D20,C
498,0,3,"Shellard, Mr. Frederick William",male,,0,0,C.A. 6212,15.1,,S
499,0,1,"Allison, Mrs. Hudson J C (Bessie Waldo Daniels)",female,25,1,2,113781,151.55,C22 C26,S
500,0,3,"Svensson, Mr. Olof",male,24,0,0,350035,7.7958,,S
501,0,3,"Calic, Mr. Petar",male,17,0,0,315086,8.6625,,S
502,0,3,"Canavan, Miss. Mary",female,21,0,0,364846,7.75,,Q
503,0,3,"O'Sullivan, Miss. Bridget Mary",female,,0,0,330909,7.6292,,Q
504,0,3,"Laitinen, Miss. Kristina Sofia",female,37,0,0,4135,9.5875,,S
505,1,1,"Maioni, Miss. Roberta",female,16,0,0,110152,86.5,B79,S
506,0,1,"Penasco y Castellana, Mr. Victor de Satode",male,18,1,0,PC 17758,108.9,C65,C
507,1,2,"Quick, Mrs. Frederick Charles (Jane Richards)",female,33,0,2,26360,26,,S
508,1,1,"Bradley, Mr. George (""George Arthur Brayton"")",male,,0,0,111427,26.55,,S
509,0,3,"Olsen, Mr. Henry Margido",male,28,0,0,C 4001,22.525,,S
510,1,3,"Lang, Mr. Fang",male,26,0,0,1601,56.4958,,S
511,1,3,"Daly, Mr. Eugene Patrick",male,29,0,0,382651,7.75,,Q
512,0,3,"Webber, Mr. James",male,,0,0,SOTON/OQ 3101316,8.05,,S
513,1,1,"McGough, Mr. James Robert",male,36,0,0,PC 17473,26.2875,E25,S
514,1,1,"Rothschild, Mrs. Martin (Elizabeth L. Barrett)",female,54,1,0,PC 17603,59.4,,C
515,0,3,"Coleff, Mr. Satio",male,24,0,0,349209,7.4958,,S
516,0,1,"Walker, Mr. William Anderson",male,47,0,0,36967,34.0208,D46,S
517,1,2,"Lemore, Mrs. (Amelia Milley)",female,34,0,0,C.A. 34260,10.5,F33,S
518,0,3,"Ryan, Mr. Patrick",male,,0,0,371110,24.15,,Q
519,1,2,"Angle, Mrs. William A (Florence ""Mary"" Agnes Hughes)",female,36,1,0,226875,26,,S
520,0,3,"Pavlovic, Mr. Stefo",male,32,0,0,349242,7.8958,,S
521,1,1,"Perreault, Miss. Anne",female,30,0,0,12749,93.5,B73,S
522,0,3,"Vovk, Mr. Janko",male,22,0,0,349252,7.8958,,S
523,0,3,"Lahoud, Mr. Sarkis",male,,0,0,2624,7.225,,C
524,1,1,"Hippach, Mrs. Louis Albert (Ida Sophia Fischer)",female,44,0,1,111361,57.9792,B18,C
525,0,3,"Kassem, Mr. Fared",male,,0,0,2700,7.2292,,C
526,0,3,"Farrell, Mr. James",male,40.5,0,0,367232,7.75,,Q
527,1,2,"Ridsdale, Miss. Lucy",female,50,0,0,W./C. 14258,10.5,,S
528,0,1,"Farthing, Mr. John",male,,0,0,PC 17483,221.7792,C95,S
529,0,3,"Salonen, Mr. Johan Werner",male,39,0,0,3101296,7.925,,S
530,0,2,"Hocking, Mr. Richard George",male,23,2,1,29104,11.5,,S
531,1,2,"Quick, Miss. Phyllis May",female,2,1,1,26360,26,,S
532,0,3,"Toufik, Mr. Nakli",male,,0,0,2641,7.2292,,C
533,0,3,"Elias, Mr. Joseph Jr",male,17,1,1,2690,7.2292,,C
534,1,3,"Peter, Mrs. Catherine (Catherine Rizk)",female,,0,2,2668,22.3583,,C
535,0,3,"Cacic, Miss. Marija",female,30,0,0,315084,8.6625,,S
536,1,2,"Hart, Miss. Eva Miriam",female,7,0,2,F.C.C. 13529,26.25,,S
537,0,1,"Butt, Major. Archibald Willingham",male,45,0,0,113050,26.55,B38,S
538,1,1,"LeRoy, Miss. Bertha",female,30,0,0,PC 17761,106.425,,C
539,0,3,"Risien, Mr. Samuel Beard",male,,0,0,364498,14.5,,S
540,1,1,"Frolicher, Miss. Hedwig Margaritha",female,22,0,2,13568,49.5,B39,C
541,1,1,"Crosby, Miss. Harriet R",female,36,0,2,WE/P 5735,71,B22,S
542,0,3,"Andersson, Miss. Ingeborg Constanzia",female,9,4,2,347082,31.275,,S
543,0,3,"Andersson, Miss. Sigrid Elisabeth",female,11,4,2,347082,31.275,,S
544,1,2,"Beane, Mr. Edward",male,32,1,0,2908,26,,S
545,0,1,"Douglas, Mr. Walter Donald",male,50,1,0,PC 17761,106.425,C86,C
546,0,1,"Nicholson, Mr. Arthur Ernest",male,64,0,0,693,26,,S
547,1,2,"Beane, Mrs. Edward (Ethel Clarke)",female,19,1,0,2908,26,,S
548,1,2,"Padro y Manent, Mr. Julian",male,,0,0,SC/PARIS 2146,13.8625,,C
549,0,3,"Goldsmith, Mr. Frank John",male,33,1,1,363291,20.525,,S
550,1,2,"Davies, Master. John Morgan Jr",male,8,1,1,C.A. 33112,36.75,,S
551,1,1,"Thayer, Mr. John Borland Jr",male,17,0,2,17421,110.8833,C70,C
552,0,2,"Sharp, Mr. Percival James R",male,27,0,0,244358,26,,S
553,0,3,"O'Brien, Mr. Timothy",male,,0,0,330979,7.8292,,Q
554,1,3,"Leeni, Mr. Fahim (""Philip Zenni"")",male,22,0,0,2620,7.225,,C
555,1,3,"Ohman, Miss. Velin",female,22,0,0,347085,7.775,,S
556,0,1,"Wright, Mr. George",male,62,0,0,113807,26.55,,S
557,1,1,"Duff Gordon, Lady. (Lucille Christiana Sutherland) (""Mrs Morgan"")",female,48,1,0,11755,39.6,A16,C
558,0,1,"Robbins, Mr. Victor",male,,0,0,PC 17757,227.525,,C
559,1,1,"Taussig, Mrs. Emil (Tillie Mandelbaum)",female,39,1,1,110413,79.65,E67,S
560,1,3,"de Messemaeker, Mrs. Guillaume Joseph (Emma)",female,36,1,0,345572,17.4,,S
561,0,3,"Morrow, Mr. Thomas Rowan",male,,0,0,372622,7.75,,Q
562,0,3,"Sivic, Mr. Husein",male,40,0,0,349251,7.8958,,S
563,0,2,"Norman, Mr. Robert Douglas",male,28,0,0,218629,13.5,,S
564,0,3,"Simmons, Mr. John",male,,0,0,SOTON/OQ 392082,8.05,,S
565,0,3,"Meanwell, Miss. (Marion Ogden)",female,,0,0,SOTON/O.Q. 392087,8.05,,S
566,0,3,"Davies, Mr. Alfred J",male,24,2,0,A/4 48871,24.15,,S
567,0,3,"Stoytcheff, Mr. Ilia",male,19,0,0,349205,7.8958,,S
568,0,3,"Palsson, Mrs. Nils (Alma Cornelia Berglund)",female,29,0,4,349909,21.075,,S
569,0,3,"Doharr, Mr. Tannous",male,,0,0,2686,7.2292,,C
570,1,3,"Jonsson, Mr. Carl",male,32,0,0,350417,7.8542,,S
571,1,2,"Harris, Mr. George",male,62,0,0,S.W./PP 752,10.5,,S
572,1,1,"Appleton, Mrs. Edward Dale (Charlotte Lamson)",female,53,2,0,11769,51.4792,C101,S
573,1,1,"Flynn, Mr. John Irwin (""Irving"")",male,36,0,0,PC 17474,26.3875,E25,S
574,1,3,"Kelly, Miss. Mary",female,,0,0,14312,7.75,,Q
575,0,3,"Rush, Mr. Alfred George John",male,16,0,0,A/4. 20589,8.05,,S
576,0,3,"Patchett, Mr. George",male,19,0,0,358585,14.5,,S
577,1,2,"Garside, Miss. Ethel",female,34,0,0,243880,13,,S
578,1,1,"Silvey, Mrs. William Baird (Alice Munger)",female,39,1,0,13507,55.9,E44,S
579,0,3,"Caram, Mrs. Joseph (Maria Elias)",female,,1,0,2689,14.4583,,C
580,1,3,"Jussila, Mr. Eiriik",male,32,0,0,STON/O 2. 3101286,7.925,,S
581,1,2,"Christy, Miss. Julie Rachel",female,25,1,1,237789,30,,S
582,1,1,"Thayer, Mrs. John Borland (Marian Longstreth Morris)",female,39,1,1,17421,110.8833,C68,C
583,0,2,"Downton, Mr. William James",male,54,0,0,28403,26,,S
584,0,1,"Ross, Mr. John Hugo",male,36,0,0,13049,40.125,A10,C
585,0,3,"Paulner, Mr. Uscher",male,,0,0,3411,8.7125,,C
586,1,1,"Taussig, Miss. Ruth",female,18,0,2,110413,79.65,E68,S
587,0,2,"Jarvis, Mr. John Denzil",male,47,0,0,237565,15,,S
588,1,1,"Frolicher-Stehli, Mr. Maxmillian",male,60,1,1,13567,79.2,B41,C
589,0,3,"Gilinski, Mr. Eliezer",male,22,0,0,14973,8.05,,S
590,0,3,"Murdlin, Mr. Joseph",male,,0,0,A./5. 3235,8.05,,S
591,0,3,"Rintamaki, Mr. Matti",male,35,0,0,STON/O 2. 3101273,7.125,,S
592,1,1,"Stephenson, Mrs. Walter Bertram (Martha Eustis)",female,52,1,0,36947,78.2667,D20,C
593,0,3,"Elsbury, Mr. William James",male,47,0,0,A/5 3902,7.25,,S
594,0,3,"Bourke, Miss. Mary",female,,0,2,364848,7.75,,Q
595,0,2,"Chapman, Mr. John Henry",male,37,1,0,SC/AH 29037,26,,S
596,0,3,"Van Impe, Mr. Jean Baptiste",male,36,1,1,345773,24.15,,S
597,1,2,"Leitch, Miss. Jessie Wills",female,,0,0,248727,33,,S
598,0,3,"Johnson, Mr. Alfred",male,49,0,0,LINE,0,,S
599,0,3,"Boulos, Mr. Hanna",male,,0,0,2664,7.225,,C
600,1,1,"Duff Gordon, Sir. Cosmo Edmund (""Mr Morgan"")",male,49,1,0,PC 17485,56.9292,A20,C
601,1,2,"Jacobsohn, Mrs. Sidney Samuel (Amy Frances Christy)",female,24,2,1,243847,27,,S
602,0,3,"Slabenoff, Mr. Petco",male,,0,0,349214,7.8958,,S
603,0,1,"Harrington, Mr. Charles H",male,,0,0,113796,42.4,,S
604,0,3,"Torber, Mr. Ernst William",male,44,0,0,364511,8.05,,S
605,1,1,"Homer, Mr. Harry (""Mr E Haven"")",male,35,0,0,111426,26.55,,C
606,0,3,"Lindell, Mr. Edvard Bengtsson",male,36,1,0,349910,15.55,,S
607,0,3,"Karaic, Mr. Milan",male,30,0,0,349246,7.8958,,S
608,1,1,"Daniel, Mr. Robert Williams",male,27,0,0,113804,30.5,,S
609,1,2,"Laroche, Mrs. Joseph (Juliette Marie Louise Lafargue)",female,22,1,2,SC/Paris 2123,41.5792,,C
610,1,1,"Shutes, Miss. Elizabeth W",female,40,0,0,PC 17582,153.4625,C125,S
611,0,3,"Andersson, Mrs. Anders Johan (Alfrida Konstantia Brogren)",female,39,1,5,347082,31.275,,S
612,0,3,"Jardin, Mr. Jose Neto",male,,0,0,SOTON/O.Q. 3101305,7.05,,S
613,1,3,"Murphy, Miss. Margaret Jane",female,,1,0,367230,15.5,,Q
614,0,3,"Horgan, Mr. John",male,,0,0,370377,7.75,,Q
615,0,3,"Brocklebank, Mr. William Alfred",male,35,0,0,364512,8.05,,S
616,1,2,"Herman, Miss. Alice",female,24,1,2,220845,65,,S
617,0,3,"Danbom, Mr. Ernst Gilbert",male,34,1,1,347080,14.4,,S
618,0,3,"Lobb, Mrs. William Arthur (Cordelia K Stanlick)",female,26,1,0,A/5. 3336,16.1,,S
619,1,2,"Becker, Miss. Marion Louise",female,4,2,1,230136,39,F4,S
620,0,2,"Gavey, Mr. Lawrence",male,26,0,0,31028,10.5,,S
621,0,3,"Yasbeck, Mr. Antoni",male,27,1,0,2659,14.4542,,C
622,1,1,"Kimball, Mr. Edwin Nelson Jr",male,42,1,0,11753,52.5542,D19,S
623,1,3,"Nakid, Mr. Sahid",male,20,1,1,2653,15.7417,,C
624,0,3,"Hansen, Mr. Henry Damsgaard",male,21,0,0,350029,7.8542,,S
625,0,3,"Bowen, Mr. David John ""Dai""",male,21,0,0,54636,16.1,,S
626,0,1,"Sutton, Mr. Frederick",male,61,0,0,36963,32.3208,D50,S
627,0,2,"Kirkland, Rev. Charles Leonard",male,57,0,0,219533,12.35,,Q
628,1,1,"Longley, Miss. Gretchen Fiske",female,21,0,0,13502,77.9583,D9,S
629,0,3,"Bostandyeff, Mr. Guentcho",male,26,0,0,349224,7.8958,,S
630,0,3,"O'Connell, Mr. Patrick D",male,,0,0,334912,7.7333,,Q
631,1,1,"Barkworth, Mr. Algernon Henry Wilson",male,80,0,0,27042,30,A23,S
632,0,3,"Lundahl, Mr. Johan Svensson",male,51,0,0,347743,7.0542,,S
633,1,1,"Stahelin-Maeglin, Dr. Max",male,32,0,0,13214,30.5,B50,C
634,0,1,"Parr, Mr. William Henry Marsh",male,,0,0,112052,0,,S
635,0,3,"Skoog, Miss. Mabel",female,9,3,2,347088,27.9,,S
636,1,2,"Davis, Miss. Mary",female,28,0,0,237668,13,,S
637,0,3,"Leinonen, Mr. Antti Gustaf",male,32,0,0,STON/O 2. 3101292,7.925,,S
638,0,2,"Collyer, Mr. Harvey",male,31,1,1,C.A. 31921,26.25,,S
639,0,3,"Panula, Mrs. Juha (Maria Emilia Ojala)",female,41,0,5,3101295,39.6875,,S
640,0,3,"Thorneycroft, Mr. Percival",male,,1,0,376564,16.1,,S
641,0,3,"Jensen, Mr. Hans Peder",male,20,0,0,350050,7.8542,,S
642,1,1,"Sagesser, Mlle. Emma",female,24,0,0,PC 17477,69.3,B35,C
643,0,3,"Skoog, Miss. Margit Elizabeth",female,2,3,2,347088,27.9,,S
644,1,3,"Foo, Mr. Choong",male,,0,0,1601,56.4958,,S
645,1,3,"Baclini, Miss. Eugenie",female,0.75,2,1,2666,19.2583,,C
646,1,1,"Harper, Mr. Henry Sleeper",male,48,1,0,PC 17572,76.7292,D33,C
647,0,3,"Cor, Mr. Liudevit",male,19,0,0,349231,7.8958,,S
648,1,1,"Simonius-Blumer, Col. Oberst Alfons",male,56,0,0,13213,35.5,A26,C
649,0,3,"Willey, Mr. Edward",male,,0,0,S.O./P.P. 751,7.55,,S
650,1,3,"Stanley, Miss. Amy Zillah Elsie",female,23,0,0,CA. 2314,7.55,,S
651,0,3,"Mitkoff, Mr. Mito",male,,0,0,349221,7.8958,,S
652,1,2,"Doling, Miss. Elsie",female,18,0,1,231919,23,,S
653,0,3,"Kalvik, Mr. Johannes Halvorsen",male,21,0,0,8475,8.4333,,S
654,1,3,"O'Leary, Miss. Hanora ""Norah""",female,,0,0,330919,7.8292,,Q
655,0,3,"Hegarty, Miss. Hanora ""Nora""",female,18,0,0,365226,6.75,,Q
656,0,2,"Hickman, Mr. Leonard Mark",male,24,2,0,S.O.C. 14879,73.5,,S
657,0,3,"Radeff, Mr. Alexander",male,,0,0,349223,7.8958,,S
658,0,3,"Bourke, Mrs. John (Catherine)",female,32,1,1,364849,15.5,,Q
659,0,2,"Eitemiller, Mr. George Floyd",male,23,0,0,29751,13,,S
660,0,1,"Newell, Mr. Arthur Webster",male,58,0,2,35273,113.275,D48,C
661,1,1,"Frauenthal, Dr. Henry William",male,50,2,0,PC 17611,133.65,,S
662,0,3,"Badt, Mr. Mohamed",male,40,0,0,2623,7.225,,C
663,0,1,"Colley, Mr. Edward Pomeroy",male,47,0,0,5727,25.5875,E58,S
664,0,3,"Coleff, Mr. Peju",male,36,0,0,349210,7.4958,,S
665,1,3,"Lindqvist, Mr. Eino William",male,20,1,0,STON/O 2. 3101285,7.925,,S
666,0,2,"Hickman, Mr. Lewis",male,32,2,0,S.O.C. 14879,73.5,,S
667,0,2,"Butler, Mr. Reginald Fenton",male,25,0,0,234686,13,,S
668,0,3,"Rommetvedt, Mr. Knud Paust",male,,0,0,312993,7.775,,S
669,0,3,"Cook, Mr. Jacob",male,43,0,0,A/5 3536,8.05,,S
670,1,1,"Taylor, Mrs. Elmer Zebley (Juliet Cummins Wright)",female,,1,0,19996,52,C126,S
671,1,2,"Brown, Mrs. Thomas William Solomon (Elizabeth Catherine Ford)",female,40,1,1,29750,39,,S
672,0,1,"Davidson, Mr. Thornton",male,31,1,0,F.C. 12750,52,B71,S
673,0,2,"Mitchell, Mr. Henry Michael",male,70,0,0,C.A. 24580,10.5,,S
674,1,2,"Wilhelms, Mr. Charles",male,31,0,0,244270,13,,S
675,0,2,"Watson, Mr. Ennis Hastings",male,,0,0,239856,0,,S
676,0,3,"Edvardsson, Mr. Gustaf Hjalmar",male,18,0,0,349912,7.775,,S
677,0,3,"Sawyer, Mr. Frederick Charles",male,24.5,0,0,342826,8.05,,S
678,1,3,"Turja, Miss. Anna Sofia",female,18,0,0,4138,9.8417,,S
679,0,3,"Goodwin, Mrs. Frederick (Augusta Tyler)",female,43,1,6,CA 2144,46.9,,S
680,1,1,"Cardeza, Mr. Thomas Drake Martinez",male,36,0,1,PC 17755,512.3292,B51 B53 B55,C
681,0,3,"Peters, Miss. Katie",female,,0,0,330935,8.1375,,Q
682,1,1,"Hassab, Mr. Hammad",male,27,0,0,PC 17572,76.7292,D49,C
683,0,3,"Olsvigen, Mr. Thor Anderson",male,20,0,0,6563,9.225,,S
684,0,3,"Goodwin, Mr. Charles Edward",male,14,5,2,CA 2144,46.9,,S
685,0,2,"Brown, Mr. Thomas William Solomon",male,60,1,1,29750,39,,S
686,0,2,"Laroche, Mr. Joseph Philippe Lemercier",male,25,1,2,SC/Paris 2123,41.5792,,C
687,0,3,"Panula, Mr. Jaako Arnold",male,14,4,1,3101295,39.6875,,S
688,0,3,"Dakic, Mr. Branko",male,19,0,0,349228,10.1708,,S
689,0,3,"Fischer, Mr. Eberhard Thelander",male,18,0,0,350036,7.7958,,S
690,1,1,"Madill, Miss. Georgette Alexandra",female,15,0,1,24160,211.3375,B5,S
691,1,1,"Dick, Mr. Albert Adrian",male,31,1,0,17474,57,B20,S
692,1,3,"Karun, Miss. Manca",female,4,0,1,349256,13.4167,,C
693,1,3,"Lam, Mr. Ali",male,,0,0,1601,56.4958,,S
694,0,3,"Saad, Mr. Khalil",male,25,0,0,2672,7.225,,C
695,0,1,"Weir, Col. John",male,60,0,0,113800,26.55,,S
696,0,2,"Chapman, Mr. Charles Henry",male,52,0,0,248731,13.5,,S
697,0,3,"Kelly, Mr. James",male,44,0,0,363592,8.05,,S
698,1,3,"Mullens, Miss. Katherine ""Katie""",female,,0,0,35852,7.7333,,Q
699,0,1,"Thayer, Mr. John Borland",male,49,1,1,17421,110.8833,C68,C
700,0,3,"Humblen, Mr. Adolf Mathias Nicolai Olsen",male,42,0,0,348121,7.65,F G63,S
701,1,1,"Astor, Mrs. John Jacob (Madeleine Talmadge Force)",female,18,1,0,PC 17757,227.525,C62 C64,C
702,1,1,"Silverthorne, Mr. Spencer Victor",male,35,0,0,PC 17475,26.2875,E24,S
703,0,3,"Barbara, Miss. Saiide",female,18,0,1,2691,14.4542,,C
704,0,3,"Gallagher, Mr. Martin",male,25,0,0,36864,7.7417,,Q
705,0,3,"Hansen, Mr. Henrik Juul",male,26,1,0,350025,7.8542,,S
706,0,2,"Morley, Mr. Henry Samuel (""Mr Henry Marshall"")",male,39,0,0,250655,26,,S
707,1,2,"Kelly, Mrs. Florence ""Fannie""",female,45,0,0,223596,13.5,,S
708,1,1,"Calderhead, Mr. Edward Pennington",male,42,0,0,PC 17476,26.2875,E24,S
709,1,1,"Cleaver, Miss. Alice",female,22,0,0,113781,151.55,,S
710,1,3,"Moubarek, Master. Halim Gonios (""William George"")",male,,1,1,2661,15.2458,,C
711,1,1,"Mayne, Mlle. Berthe Antonine (""Mrs de Villiers"")",female,24,0,0,PC 17482,49.5042,C90,C
712,0,1,"Klaber, Mr. Herman",male,,0,0,113028,26.55,C124,S
713,1,1,"Taylor, Mr. Elmer Zebley",male,48,1,0,19996,52,C126,S
714,0,3,"Larsson, Mr. August Viktor",male,29,0,0,7545,9.4833,,S
715,0,2,"Greenberg, Mr. Samuel",male,52,0,0,250647,13,,S
716,0,3,"Soholt, Mr. Peter Andreas Lauritz Andersen",male,19,0,0,348124,7.65,F G73,S
717,1,1,"Endres, Miss. Caroline Louise",female,38,0,0,PC 17757,227.525,C45,C
718,1,2,"Troutt, Miss. Edwina Celia ""Winnie""",female,27,0,0,34218,10.5,E101,S
719,0,3,"McEvoy, Mr. Michael",male,,0,0,36568,15.5,,Q
720,0,3,"Johnson, Mr. Malkolm Joackim",male,33,0,0,347062,7.775,,S
721,1,2,"Harper, Miss. Annie Jessie ""Nina""",female,6,0,1,248727,33,,S
722,0,3,"Jensen, Mr. Svend Lauritz",male,17,1,0,350048,7.0542,,S
723,0,2,"Gillespie, Mr. William Henry",male,34,0,0,12233,13,,S
724,0,2,"Hodges, Mr. Henry Price",male,50,0,0,250643,13,,S
725,1,1,"Chambers, Mr. Norman Campbell",male,27,1,0,113806,53.1,E8,S
726,0,3,"Oreskovic, Mr. Luka",male,20,0,0,315094,8.6625,,S
727,1,2,"Renouf, Mrs. Peter Henry (Lillian Jefferys)",female,30,3,0,31027,21,,S
728,1,3,"Mannion, Miss. Margareth",female,,0,0,36866,7.7375,,Q
729,0,2,"Bryhl, Mr. Kurt Arnold Gottfrid",male,25,1,0,236853,26,,S
730,0,3,"Ilmakangas, Miss. Pieta Sofia",female,25,1,0,STON/O2. 3101271,7.925,,S
731,1,1,"Allen, Miss. Elisabeth Walton",female,29,0,0,24160,211.3375,B5,S
732,0,3,"Hassan, Mr. Houssein G N",male,11,0,0,2699,18.7875,,C
733,0,2,"Knight, Mr. Robert J",male,,0,0,239855,0,,S
734,0,2,"Berriman, Mr. William John",male,23,0,0,28425,13,,S
735,0,2,"Troupiansky, Mr. Moses Aaron",male,23,0,0,233639,13,,S
736,0,3,"Williams, Mr. Leslie",male,28.5,0,0,54636,16.1,,S
737,0,3,"Ford, Mrs. Edward (Margaret Ann Watson)",female,48,1,3,W./C. 6608,34.375,,S
738,1,1,"Lesurer, Mr. Gustave J",male,35,0,0,PC 17755,512.3292,B101,C
739,0,3,"Ivanoff, Mr. Kanio",male,,0,0,349201,7.8958,,S
740,0,3,"Nankoff, Mr. Minko",male,,0,0,349218,7.8958,,S
741,1,1,"Hawksford, Mr. Walter James",male,,0,0,16988,30,D45,S
742,0,1,"Cavendish, Mr. Tyrell William",male,36,1,0,19877,78.85,C46,S
743,1,1,"Ryerson, Miss. Susan Parker ""Suzette""",female,21,2,2,PC 17608,262.375,B57 B59 B63 B66,C
744,0,3,"McNamee, Mr. Neal",male,24,1,0,376566,16.1,,S
745,1,3,"Stranden, Mr. Juho",male,31,0,0,STON/O 2. 3101288,7.925,,S
746,0,1,"Crosby, Capt. Edward Gifford",male,70,1,1,WE/P 5735,71,B22,S
747,0,3,"Abbott, Mr. Rossmore Edward",male,16,1,1,C.A. 2673,20.25,,S
748,1,2,"Sinkkonen, Miss. Anna",female,30,0,0,250648,13,,S
749,0,1,"Marvin, Mr. Daniel Warner",male,19,1,0,113773,53.1,D30,S
750,0,3,"Connaghton, Mr. Michael",male,31,0,0,335097,7.75,,Q
751,1,2,"Wells, Miss. Joan",female,4,1,1,29103,23,,S
752,1,3,"Moor, Master. Meier",male,6,0,1,392096,12.475,E121,S
753,0,3,"Vande Velde, Mr. Johannes Joseph",male,33,0,0,345780,9.5,,S
754,0,3,"Jonkoff, Mr. Lalio",male,23,0,0,349204,7.8958,,S
755,1,2,"Herman, Mrs. Samuel (Jane Laver)",female,48,1,2,220845,65,,S
756,1,2,"Hamalainen, Master. Viljo",male,0.67,1,1,250649,14.5,,S
757,0,3,"Carlsson, Mr. August Sigfrid",male,28,0,0,350042,7.7958,,S
758,0,2,"Bailey, Mr. Percy Andrew",male,18,0,0,29108,11.5,,S
759,0,3,"Theobald, Mr. Thomas Leonard",male,34,0,0,363294,8.05,,S
760,1,1,"Rothes, the Countess. of (Lucy Noel Martha Dyer-Edwards)",female,33,0,0,110152,86.5,B77,S
761,0,3,"Garfirth, Mr. John",male,,0,0,358585,14.5,,S
762,0,3,"Nirva, Mr. Iisakki Antino Aijo",male,41,0,0,SOTON/O2 3101272,7.125,,S
763,1,3,"Barah, Mr. Hanna Assi",male,20,0,0,2663,7.2292,,C
764,1,1,"Carter, Mrs. William Ernest (Lucile Polk)",female,36,1,2,113760,120,B96 B98,S
765,0,3,"Eklund, Mr. Hans Linus",male,16,0,0,347074,7.775,,S
766,1,1,"Hogeboom, Mrs. John C (Anna Andrews)",female,51,1,0,13502,77.9583,D11,S
767,0,1,"Brewe, Dr. Arthur Jackson",male,,0,0,112379,39.6,,C
768,0,3,"Mangan, Miss. Mary",female,30.5,0,0,364850,7.75,,Q
769,0,3,"Moran, Mr. Daniel J",male,,1,0,371110,24.15,,Q
770,0,3,"Gronnestad, Mr. Daniel Danielsen",male,32,0,0,8471,8.3625,,S
771,0,3,"Lievens, Mr. Rene Aime",male,24,0,0,345781,9.5,,S
772,0,3,"Jensen, Mr. Niels Peder",male,48,0,0,350047,7.8542,,S
773,0,2,"Mack, Mrs. (Mary)",female,57,0,0,S.O./P.P. 3,10.5,E77,S
774,0,3,"Elias, Mr. Dibo",male,,0,0,2674,7.225,,C
775,1,2,"Hocking, Mrs. Elizabeth (Eliza Needs)",female,54,1,3,29105,23,,S
776,0,3,"Myhrman, Mr. Pehr Fabian Oliver Malkolm",male,18,0,0,347078,7.75,,S
777,0,3,"Tobin, Mr. Roger",male,,0,0,383121,7.75,F38,Q
778,1,3,"Emanuel, Miss. Virginia Ethel",female,5,0,0,364516,12.475,,S
779,0,3,"Kilgannon, Mr. Thomas J",male,,0,0,36865,7.7375,,Q
780,1,1,"Robert, Mrs. Edward Scott (Elisabeth Walton McMillan)",female,43,0,1,24160,211.3375,B3,S
781,1,3,"Ayoub, Miss. Banoura",female,13,0,0,2687,7.2292,,C
782,1,1,"Dick, Mrs. Albert Adrian (Vera Gillespie)",female,17,1,0,17474,57,B20,S
783,0,1,"Long, Mr. Milton Clyde",male,29,0,0,113501,30,D6,S
784,0,3,"Johnston, Mr. Andrew G",male,,1,2,W./C. 6607,23.45,,S
785,0,3,"Ali, Mr. William",male,25,0,0,SOTON/O.Q. 3101312,7.05,,S
786,0,3,"Harmer, Mr. Abraham (David Lishin)",male,25,0,0,374887,7.25,,S
787,1,3,"Sjoblom, Miss. Anna Sofia",female,18,0,0,3101265,7.4958,,S
788,0,3,"Rice, Master. George Hugh",male,8,4,1,382652,29.125,,Q
789,1,3,"Dean, Master. Bertram Vere",male,1,1,2,C.A. 2315,20.575,,S
790,0,1,"Guggenheim, Mr. Benjamin",male,46,0,0,PC 17593,79.2,B82 B84,C
791,0,3,"Keane, Mr. Andrew ""Andy""",male,,0,0,12460,7.75,,Q
792,0,2,"Gaskell, Mr. Alfred",male,16,0,0,239865,26,,S
793,0,3,"Sage, Miss. Stella Anna",female,,8,2,CA. 2343,69.55,,S
794,0,1,"Hoyt, Mr. William Fisher",male,,0,0,PC 17600,30.6958,,C
795,0,3,"Dantcheff, Mr. Ristiu",male,25,0,0,349203,7.8958,,S
796,0,2,"Otter, Mr. Richard",male,39,0,0,28213,13,,S
797,1,1,"Leader, Dr. Alice (Farnham)",female,49,0,0,17465,25.9292,D17,S
798,1,3,"Osman, Mrs. Mara",female,31,0,0,349244,8.6833,,S
799,0,3,"Ibrahim Shawah, Mr. Yousseff",male,30,0,0,2685,7.2292,,C
800,0,3,"Van Impe, Mrs. Jean Baptiste (Rosalie Paula Govaert)",female,30,1,1,345773,24.15,,S
801,0,2,"Ponesell, Mr. Martin",male,34,0,0,250647,13,,S
802,1,2,"Collyer, Mrs. Harvey (Charlotte Annie Tate)",female,31,1,1,C.A. 31921,26.25,,S
803,1,1,"Carter, Master. William Thornton II",male,11,1,2,113760,120,B96 B98,S
804,1,3,"Thomas, Master. Assad Alexander",male,0.42,0,1,2625,8.5167,,C
805,1,3,"Hedman, Mr. Oskar Arvid",male,27,0,0,347089,6.975,,S
806,0,3,"Johansson, Mr. Karl Johan",male,31,0,0,347063,7.775,,S
807,0,1,"Andrews, Mr. Thomas Jr",male,39,0,0,112050,0,A36,S
808,0,3,"Pettersson, Miss. Ellen Natalia",female,18,0,0,347087,7.775,,S
809,0,2,"Meyer, Mr. August",male,39,0,0,248723,13,,S
810,1,1,"Chambers, Mrs. Norman Campbell (Bertha Griggs)",female,33,1,0,113806,53.1,E8,S
811,0,3,"Alexander, Mr. William",male,26,0,0,3474,7.8875,,S
812,0,3,"Lester, Mr. James",male,39,0,0,A/4 48871,24.15,,S
813,0,2,"Slemen, Mr. Richard James",male,35,0,0,28206,10.5,,S
814,0,3,"Andersson, Miss. Ebba Iris Alfrida",female,6,4,2,347082,31.275,,S
815,0,3,"Tomlin, Mr. Ernest Portage",male,30.5,0,0,364499,8.05,,S
816,0,1,"Fry, Mr. Richard",male,,0,0,112058,0,B102,S
817,0,3,"Heininen, Miss. Wendla Maria",female,23,0,0,STON/O2. 3101290,7.925,,S
818,0,2,"Mallet, Mr. Albert",male,31,1,1,S.C./PARIS 2079,37.0042,,C
819,0,3,"Holm, Mr. John Fredrik Alexander",male,43,0,0,C 7075,6.45,,S
820,0,3,"Skoog, Master. Karl Thorsten",male,10,3,2,347088,27.9,,S
821,1,1,"Hays, Mrs. Charles Melville (Clara Jennings Gregg)",female,52,1,1,12749,93.5,B69,S
822,1,3,"Lulic, Mr. Nikola",male,27,0,0,315098,8.6625,,S
823,0,1,"Reuchlin, Jonkheer. John George",male,38,0,0,19972,0,,S
824,1,3,"Moor, Mrs. (Beila)",female,27,0,1,392096,12.475,E121,S
825,0,3,"Panula, Master. Urho Abraham",male,2,4,1,3101295,39.6875,,S
826,0,3,"Flynn, Mr. John",male,,0,0,368323,6.95,,Q
827,0,3,"Lam, Mr. Len",male,,0,0,1601,56.4958,,S
828,1,2,"Mallet, Master. Andre",male,1,0,2,S.C./PARIS 2079,37.0042,,C
829,1,3,"McCormack, Mr. Thomas Joseph",male,,0,0,367228,7.75,,Q
830,1,1,"Stone, Mrs. George Nelson (Martha Evelyn)",female,62,0,0,113572,80,B28,
831,1,3,"Yasbeck, Mrs. Antoni (Selini Alexander)",female,15,1,0,2659,14.4542,,C
832,1,2,"Richards, Master. George Sibley",male,0.83,1,1,29106,18.75,,S
833,0,3,"Saad, Mr. Amin",male,,0,0,2671,7.2292,,C
834,0,3,"Augustsson, Mr. Albert",male,23,0,0,347468,7.8542,,S
835,0,3,"Allum, Mr. Owen George",male,18,0,0,2223,8.3,,S
836,1,1,"Compton, Miss. Sara Rebecca",female,39,1,1,PC 17756,83.1583,E49,C
837,0,3,"Pasic, Mr. Jakob",male,21,0,0,315097,8.6625,,S
838,0,3,"Sirota, Mr. Maurice",male,,0,0,392092,8.05,,S
839,1,3,"Chip, Mr. Chang",male,32,0,0,1601,56.4958,,S
840,1,1,"Marechal, Mr. Pierre",male,,0,0,11774,29.7,C47,C
841,0,3,"Alhomaki, Mr. Ilmari Rudolf",male,20,0,0,SOTON/O2 3101287,7.925,,S
842,0,2,"Mudd, Mr. Thomas Charles",male,16,0,0,S.O./P.P. 3,10.5,,S
843,1,1,"Serepeca, Miss. Augusta",female,30,0,0,113798,31,,C
844,0,3,"Lemberopolous, Mr. Peter L",male,34.5,0,0,2683,6.4375,,C
845,0,3,"Culumovic, Mr. Jeso",male,17,0,0,315090,8.6625,,S
846,0,3,"Abbing, Mr. Anthony",male,42,0,0,C.A. 5547,7.55,,S
847,0,3,"Sage, Mr. Douglas Bullen",male,,8,2,CA. 2343,69.55,,S
848,0,3,"Markoff, Mr. Marin",male,35,0,0,349213,7.8958,,C
849,0,2,"Harper, Rev. John",male,28,0,1,248727,33,,S
850,1,1,"Goldenberg, Mrs. Samuel L (Edwiga Grabowska)",female,,1,0,17453,89.1042,C92,C
851,0,3,"Andersson, Master. Sigvard Harald Elias",male,4,4,2,347082,31.275,,S
852,0,3,"Svensson, Mr. Johan",male,74,0,0,347060,7.775,,S
853,0,3,"Boulos, Miss. Nourelain",female,9,1,1,2678,15.2458,,C
854,1,1,"Lines, Miss. Mary Conover",female,16,0,1,PC 17592,39.4,D28,S
855,0,2,"Carter, Mrs. Ernest Courtenay (Lilian Hughes)",female,44,1,0,244252,26,,S
856,1,3,"Aks, Mrs. Sam (Leah Rosen)",female,18,0,1,392091,9.35,,S
857,1,1,"Wick, Mrs. George Dennick (Mary Hitchcock)",female,45,1,1,36928,164.8667,,S
858,1,1,"Daly, Mr. Peter Denis ",male,51,0,0,113055,26.55,E17,S
859,1,3,"Baclini, Mrs. Solomon (Latifa Qurban)",female,24,0,3,2666,19.2583,,C
860,0,3,"Razi, Mr. Raihed",male,,0,0,2629,7.2292,,C
861,0,3,"Hansen, Mr. Claus Peter",male,41,2,0,350026,14.1083,,S
862,0,2,"Giles, Mr. Frederick Edward",male,21,1,0,28134,11.5,,S
863,1,1,"Swift, Mrs. Frederick Joel (Margaret Welles Barron)",female,48,0,0,17466,25.9292,D17,S
864,0,3,"Sage, Miss. Dorothy Edith ""Dolly""",female,,8,2,CA. 2343,69.55,,S
865,0,2,"Gill, Mr. John William",male,24,0,0,233866,13,,S
866,1,2,"Bystrom, Mrs. (Karolina)",female,42,0,0,236852,13,,S
867,1,2,"Duran y More, Miss. Asuncion",female,27,1,0,SC/PARIS 2149,13.8583,,C
868,0,1,"Roebling, Mr. Washington Augustus II",male,31,0,0,PC 17590,50.4958,A24,S
869,0,3,"van Melkebeke, Mr. Philemon",male,,0,0,345777,9.5,,S
870,1,3,"Johnson, Master. Harold Theodor",male,4,1,1,347742,11.1333,,S
871,0,3,"Balkic, Mr. Cerin",male,26,0,0,349248,7.8958,,S
872,1,1,"Beckwith, Mrs. Richard Leonard (Sallie Monypeny)",female,47,1,1,11751,52.5542,D35,S
873,0,1,"Carlsson, Mr. Frans Olof",male,33,0,0,695,5,B51 B53 B55,S
874,0,3,"Vander Cruyssen, Mr. Victor",male,47,0,0,345765,9,,S
875,1,2,"Abelson, Mrs. Samuel (Hannah Wizosky)",female,28,1,0,P/PP 3381,24,,C
876,1,3,"Najib, Miss. Adele Kiamie ""Jane""",female,15,0,0,2667,7.225,,C
877,0,3,"Gustafsson, Mr. Alfred Ossian",male,20,0,0,7534,9.8458,,S
878,0,3,"Petroff, Mr. Nedelio",male,19,0,0,349212,7.8958,,S
879,0,3,"Laleff, Mr. Kristo",male,,0,0,349217,7.8958,,S
880,1,1,"Potter, Mrs. Thomas Jr (Lily Alexenia Wilson)",female,56,0,1,11767,83.1583,C50,C
881,1,2,"Shelley, Mrs. William (Imanita Parrish Hall)",female,25,0,1,230433,26,,S
882,0,3,"Markun, Mr. Johann",male,33,0,0,349257,7.8958,,S
883,0,3,"Dahlberg, Miss. Gerda Ulrika",female,22,0,0,7552,10.5167,,S
884,0,2,"Banfield, Mr. Frederick James",male,28,0,0,C.A./SOTON 34068,10.5,,S
885,0,3,"Sutehall, Mr. Henry Jr",male,25,0,0,SOTON/OQ 392076,7.05,,S
886,0,3,"Rice, Mrs. William (Margaret Norton)",female,39,0,5,382652,29.125,,Q
887,0,2,"Montvila, Rev. Juozas",male,27,0,0,211536,13,,S
888,1,1,"Graham, Miss. Margaret Edith",female,19,0,0,112053,30,B42,S
889,0,3,"Johnston, Miss. Catherine Helen ""Carrie""",female,,1,2,W./C. 6607,23.45,,S
890,1,1,"Behr, Mr. Karl Howell",male,26,0,0,111369,30,C148,C
891,0,3,"Dooley, Mr. Patrick",male,32,0,0,370376,7.75,,Q
================================================
FILE: src/main/java/com/datastax/loader/AbstractDynamicRateLimiter.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
public abstract class AbstractDynamicRateLimiter extends RateLimiter {
protected long lastCheck = 0;
protected long howOften;
protected double lastStat;
protected double maxStat;
protected double minStat;
protected double downFraction;
protected double upFraction;
protected boolean invertLogic;
public AbstractDynamicRateLimiter(double inRate, long inHowOften,
double inMaxStat, double inMinStat,
double inDownFraction,
double inUpFraction,
boolean inInvertLogic) {
this(inRate, Long.MAX_VALUE, inHowOften, inMaxStat, inMinStat,
inDownFraction, inUpFraction, inInvertLogic);
}
public AbstractDynamicRateLimiter(double inRate, long inUpdateRate,
long inHowOften, double inMaxStat,
double inMinStat, double inDownFraction,
double inUpFraction,
boolean inInvertLogic) {
super(inRate);
howOften = inHowOften;
maxStat = inMaxStat;
minStat = inMinStat;
downFraction = inDownFraction;
upFraction = inUpFraction;
invertLogic = inInvertLogic;
}
public void acquire() {
this.acquire(1);
}
public synchronized void acquire(int permits) {
long currTime = System.currentTimeMillis();
if (currTime - lastCheck > howOften) {
adjustRate();
lastCheck = currTime;
}
super.acquire(permits);
}
protected synchronized void adjustRate() {
double currStat = getCurrStat();
if (statTooHigh(currStat)) {
if (invertLogic)
adjustRateUp();
else
adjustRateDown();
System.err.println("Adjusting rate down : " + currStat + " > " + maxStat + " " + super.getRate());
}
else if (statTooLow(currStat)) {
if (invertLogic)
adjustRateDown();
else
adjustRateUp();
System.err.println("Adjusting rate up : " + currStat + " > " + maxStat + " " + super.getRate());
}
}
protected synchronized boolean statTooHigh(double currStat) {
return currStat > maxStat;
}
protected synchronized boolean statTooLow(double currStat) {
return currStat < minStat;
}
protected synchronized void adjustRateDown() {
double currRate = super.getRate();
super.setRate(currRate - (currRate * downFraction));
}
protected synchronized void adjustRateUp() {
double currRate = super.getRate();
super.setRate(currRate + (currRate * upFraction));
}
protected abstract double getCurrStat();
}
================================================
FILE: src/main/java/com/datastax/loader/CqlDelimLoad.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import com.datastax.loader.parser.BooleanParser;
import com.datastax.loader.futures.FutureManager;
import com.datastax.loader.futures.PrintingFutureSet;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.util.HashMap;
import java.util.Set;
import java.util.HashSet;
import java.util.Deque;
import java.util.ArrayDeque;
import java.util.Comparator;
import java.util.Arrays;
import java.util.Locale;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.BufferedOutputStream;
import java.io.PrintStream;
import java.io.FileNotFoundException;
import java.text.ParseException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.KeyStoreException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Metrics;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.ProtocolVersion;
import com.datastax.driver.core.PoolingOptions;
import com.datastax.driver.core.ProtocolOptions;
import com.datastax.driver.core.HostDistance;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.ResultSetFuture;
import com.datastax.driver.core.SSLOptions;
import com.datastax.driver.core.RemoteEndpointAwareJdkSSLOptions;
import com.datastax.driver.core.policies.TokenAwarePolicy;
import com.datastax.driver.core.policies.DCAwareRoundRobinPolicy;
import com.codahale.metrics.Timer;
import org.apache.commons.lang3.StringEscapeUtils;
public class CqlDelimLoad {
private String version = "0.0.27";
private String host = null;
private int port = 9042;
private String username = null;
private String password = null;
private String truststorePath = null;
private String truststorePwd = null;
private String keystorePath = null;
private String keystorePwd = null;
private Cluster cluster = null;
private Session session = null;
private ConsistencyLevel consistencyLevel = ConsistencyLevel.LOCAL_ONE;
private int numFutures = 1000;
private int inNumFutures = -1;
private int queryTimeout = 2;
private long maxInsertErrors = 10;
private int numRetries = 1;
private double rate = 50000.0;
private long progressRate = 100000;
private RateLimiter rateLimiter = null;
private String rateFile = null;
private PrintStream rateStream = null;
private Integer inTtl = null;
private int ttl = -1;
private String cqlSchema = null;
private String table = null;
private String keyspace = null;
private String format = "delim";
private long maxErrors = 10;
private long skipRows = 0;
private String skipCols = null;
private long maxRows = -1;
private String badDir = ".";
private String filename = null;
public static String STDIN = "stdin";
public static String STDERR = "stderr";
private String successDir = null;
private String failureDir = null;
private Locale locale = null;
private BooleanParser.BoolStyle boolStyle = null;
private String dateFormatString = null;
private String localDateFormatString = "yyyy-MM-dd";
private String nullString = null;
private String commentString = null;
private String delimiter = null;
private int charsPerColumn = 4096;
private int numThreads = Runtime.getRuntime().availableProcessors();
private int batchSize = 1;
private boolean nullsUnset = false;
private String usage() {
StringBuilder usage = new StringBuilder("version: ").append(version).append("\n");
usage.append("Usage: -f <filename> -host <ipaddress> [OPTIONS]\n");
usage.append("OPTIONS:\n");
usage.append(" -schema <schema> Table schema (when using delim)\n");
usage.append(" -table <tableName> Table name (when using json)\n");
usage.append(" -keyspace <keyspaceName> Keyspace name (when using json)\n");
usage.append(" -configFile <filename> File with configuration options\n");
usage.append(" -delim <delimiter> Delimiter to use [,]\n");
usage.append(" -charsPerColumn <chars> Max number of chars per column [4096]\n");
usage.append(" -dateFormat <dateFormatString> Date format for TIMESTAMP [default for Locale.ENGLISH]\n");
usage.append(" -localDateFormat <formatString> Date format for DATE [yyyy-MM-dd]\n");
usage.append(" -nullString <nullString> String that signifies NULL [none]\n");
usage.append(" -comment <commentString> Comment symbol to use [none]\n");
usage.append(" -skipRows <skipRows> Number of rows to skip [0]\n");
usage.append(" -skipCols <columnsToSkip> Comma-separated list of columsn to skip in the input file\n");
usage.append(" -maxRows <maxRows> Maximum number of rows to read (-1 means all) [-1]\n");
usage.append(" -maxErrors <maxErrors> Maximum parse errors to endure [10]\n");
usage.append(" -badDir <badDirectory> Directory for where to place badly parsed rows. [none]\n");
usage.append(" -port <portNumber> CQL Port Number [9042]\n");
usage.append(" -user <username> Cassandra username [none]\n");
usage.append(" -pw <password> Password for user [none]\n");
usage.append(" -ssl-truststore-path <path> Path to SSL truststore [none]\n");
usage.append(" -ssl-truststore-pw <pwd> Password for SSL truststore [none]\n");
usage.append(" -ssl-keystore-path <path> Path to SSL keystore [none]\n");
usage.append(" -ssl-keystore-pw <pwd> Password for SSL keystore [none]\n");
usage.append(" -consistencyLevel <CL> Consistency level [LOCAL_ONE]\n");
usage.append(" -numFutures <numFutures> Number of CQL futures to keep in flight [1000]\n");
usage.append(" -batchSize <batchSize> Number of INSERTs to batch together [1]\n");
usage.append(" -decimalDelim <decimalDelim> Decimal delimiter [.] Other option is ','\n");
usage.append(" -boolStyle <boolStyleString> Style for booleans [TRUE_FALSE]\n");
usage.append(" -numThreads <numThreads> Number of concurrent threads (files) to load [num cores]\n");
usage.append(" -queryTimeout <# seconds> Query timeout (in seconds) [2]\n");
usage.append(" -numRetries <numRetries> Number of times to retry the INSERT [1]\n");
usage.append(" -maxInsertErrors <# errors> Maximum INSERT errors to endure [10]\n");
usage.append(" -rate <rows-per-second> Maximum insert rate [50000]\n");
usage.append(" -progressRate <num txns> How often to report the insert rate [100000]\n");
usage.append(" -rateFile <filename> Where to print the rate statistics\n");
usage.append(" -successDir <dir> Directory where to move successfully loaded files\n");
usage.append(" -failureDir <dir> Directory where to move files that did not successfully load\n");
usage.append(" -nullsUnset [false|true] Treat nulls as unset [faslse]\n");
usage.append(" -format [delim|jsonline|jsonarray] Format of data: delimited or JSON [delim]\n");
usage.append(" -table <tableName> Table name (when using JSON)\n");
usage.append(" -keyspace <keyspaceName> Keyspace name (when using JSON)\n");
usage.append(" -ttl <TTL> TTL for all rows in this invocation [unset]\n");
usage.append("\n\nExamples:\n");
usage.append("cassandra-loader -f /path/to/file.csv -host localhost -schema \"test.test3(a, b, c)\"\n");
usage.append("cassandra-loader -f /path/to/directory -host 1.2.3.4 -schema \"test.test3(a, b, c)\" -delim \"\\t\" -numThreads 10\n");
usage.append("cassandra-loader -f stdin -host localhost -schema \"test.test3(a, b, c)\" -user myuser -pw mypassword\n");
return usage.toString();
}
private boolean validateArgs() {
if (format.equalsIgnoreCase("delim")) {
if (null == cqlSchema) {
System.err.println("If you specify format " + format + " you must provide a schema");
return false;
}
if (null != keyspace)
System.err.println("Format is " + format + ", ignoring keyspace");
if (null != table)
System.err.println("Format is " + format + ", ignoring table");
}
else if (format.equalsIgnoreCase("jsonline")
|| format.equalsIgnoreCase("jsonarray")) {
if (null == keyspace) {
System.err.println("If you specify format " + format + " you must provide a keyspace");
return false;
}
if (null == table) {
System.err.println("If you specify format " + format + " you must provide a table");
return false;
}
if (null != cqlSchema)
System.err.println("Format is " + format + ", ignoring schema");
}
else {
System.err.println("Unknown format option");
return false;
}
if (0 >= numFutures) {
System.err.println("Number of futures must be positive (" + numFutures + ")");
return false;
}
if (0 >= batchSize) {
System.err.println("Batch size must be positive (" + batchSize + ")");
return false;
}
if (0 >= queryTimeout) {
System.err.println("Query timeout must be positive");
return false;
}
if (0 > maxInsertErrors) {
System.err.println("Maximum number of insert errors must be non-negative");
return false;
}
if (0 > numRetries) {
System.err.println("Number of retries must be non-negative");
return false;
}
if (0 > skipRows) {
System.err.println("Number of rows to skip must be non-negative");
return false;
}
if (0 >= maxRows) {
System.err.println("Maximum number of rows to load must be positive");
return false;
}
if (0 > maxErrors) {
System.err.println("Maximum number of parse errors must be non-negative");
return false;
}
if (0 > progressRate) {
System.err.println("Progress rate must be non-negative");
return false;
}
if (1 > numThreads) {
System.err.println("Number of threads must be non-negative");
return false;
}
if (!STDIN.equalsIgnoreCase(filename)) {
File infile = new File(filename);
if ((!infile.isFile()) && (!infile.isDirectory())) {
System.err.println("The -f argument needs to be a file or a directory");
return false;
}
if (infile.isDirectory()) {
File[] infileList = infile.listFiles();
if (infileList.length < 1) {
System.err.println("The directory supplied is empty");
return false;
}
}
}
if (null != successDir) {
if (STDIN.equalsIgnoreCase(filename)) {
System.err.println("Cannot specify -successDir with stdin");
return false;
}
File sdir = new File(successDir);
if (!sdir.isDirectory()) {
System.err.println("-successDir must be a directory");
return false;
}
}
if (null != failureDir) {
if (STDIN.equalsIgnoreCase(filename)) {
System.err.println("Cannot specify -failureDir with stdin");
return false;
}
File sdir = new File(failureDir);
if (!sdir.isDirectory()) {
System.err.println("-failureDir must be a directory");
return false;
}
}
if ((null == username) && (null != password)) {
System.err.println("If you supply the password, you must supply the username");
return false;
}
if ((null != username) && (null == password)) {
System.err.println("If you supply the username, you must supply the password");
return false;
}
if ((null == truststorePath) && (null != truststorePwd)) {
System.err.println("If you supply the ssl-truststore-pw, you must supply the ssl-truststore-path");
return false;
}
if ((null != truststorePath) && (null == truststorePwd)) {
System.err.println("If you supply the ssl-truststore-path, you must supply the ssl-truststore-pw");
return false;
}
if ((null == keystorePath) && (null != keystorePwd)) {
System.err.println("If you supply the ssl-keystore-pw, you must supply the ssl-keystore-path");
return false;
}
if ((null != keystorePath) && (null == keystorePwd)) {
System.err.println("If you supply the ssl-keystore-path, you must supply the ssl-keystore-pw");
return false;
}
File tfile = null;
if (null != truststorePath) {
tfile = new File(truststorePath);
if (!tfile.isFile()) {
System.err.println("truststore file must be a file");
return false;
}
}
if (null != keystorePath) {
tfile = new File(keystorePath);
if (!tfile.isFile()) {
System.err.println("keystore file must be a file");
return false;
}
}
if (0 > rate) {
System.err.println("Rate must be positive");
return false;
}
if (0 > charsPerColumn) {
System.err.println("charsPerColumn must be positive");
return false;
}
return true;
}
private boolean processConfigFile(String fname, Map<String, String> amap)
throws IOException, FileNotFoundException {
File cFile = new File(fname);
if (!cFile.isFile()) {
System.err.println("Configuration File must be a file");
return false;
}
BufferedReader cReader = new BufferedReader(new FileReader(cFile));
String line;
while ((line = cReader.readLine()) != null) {
String[] fields = line.trim().split("\\s+");
if (2 != fields.length) {
System.err.println("Bad line in config file: " + line);
return false;
}
if (null == amap.get(fields[0])) {
amap.put(fields[0], fields[1]);
}
}
return true;
}
private boolean parseArgs(String[] args) throws IOException, FileNotFoundException {
String tkey;
if (args.length == 0) {
System.err.println("No arguments specified");
return false;
}
if (0 != args.length % 2) {
System.err.println("Not an even number of parameters");
return false;
}
Map<String, String> amap = new HashMap<String,String>();
for (int i = 0; i < args.length; i+=2)
amap.put(args[i], args[i+1]);
if (null != (tkey = amap.remove("-configFile")))
if (!processConfigFile(tkey, amap))
return false;
host = amap.remove("-host");
if (null == host) { // host is required
System.err.println("Must provide a host");
return false;
}
filename = amap.remove("-f");
if (null == filename) { // filename is required
System.err.println("Must provide a filename/directory");
return false;
}
if (null != (tkey = amap.remove("-format"))) format = tkey;
if (null != (tkey = amap.remove("-schema"))) cqlSchema = tkey;
if (null != (tkey = amap.remove("-table"))) table = tkey;
if (null != (tkey = amap.remove("-keyspace"))) keyspace = tkey;
if (null != (tkey = amap.remove("-port"))) port = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-user"))) username = tkey;
if (null != (tkey = amap.remove("-pw"))) password = tkey;
if (null != (tkey = amap.remove("-ssl-truststore-path"))) truststorePath = tkey;
if (null != (tkey = amap.remove("-ssl-truststore-pw"))) truststorePwd = tkey;
if (null != (tkey = amap.remove("-ssl-keystore-path"))) keystorePath = tkey;
if (null != (tkey = amap.remove("-ssl-keystore-pw"))) keystorePwd = tkey;
if (null != (tkey = amap.remove("-consistencyLevel"))) consistencyLevel = ConsistencyLevel.valueOf(tkey);
if (null != (tkey = amap.remove("-numFutures"))) inNumFutures = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-batchSize"))) batchSize = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-queryTimeout"))) queryTimeout = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-maxInsertErrors"))) maxInsertErrors = Long.parseLong(tkey);
if (null != (tkey = amap.remove("-numRetries"))) numRetries = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-maxErrors"))) maxErrors = Long.parseLong(tkey);
if (null != (tkey = amap.remove("-skipRows"))) skipRows = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-skipCols"))) skipCols = tkey;
if (null != (tkey = amap.remove("-maxRows"))) maxRows = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-badDir"))) badDir = tkey;
if (null != (tkey = amap.remove("-dateFormat"))) dateFormatString = tkey;
if (null != (tkey = amap.remove("-localDateFormat"))) localDateFormatString = tkey;
if (null != (tkey = amap.remove("-nullString"))) nullString = tkey;
if (null != (tkey = amap.remove("-comment"))) commentString = tkey;
if (null != (tkey = amap.remove("-delim"))) delimiter = tkey;
if (null != (tkey = amap.remove("-numThreads"))) numThreads = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-rate"))) rate = Double.parseDouble(tkey);
if (null != (tkey = amap.remove("-progressRate"))) progressRate = Long.parseLong(tkey);
if (null != (tkey = amap.remove("-ttl"))) inTtl = new Integer(tkey);
if (null != (tkey = amap.remove("-rateFile"))) rateFile = tkey;
if (null != (tkey = amap.remove("-successDir"))) successDir = tkey;
if (null != (tkey = amap.remove("-failureDir"))) failureDir = tkey;
if (null != (tkey = amap.remove("-decimalDelim"))) {
if (tkey.equals(","))
locale = Locale.FRANCE;
}
if (null != (tkey = amap.remove("-boolStyle"))) {
boolStyle = BooleanParser.getBoolStyle(tkey);
if (null == boolStyle) {
System.err.println("Bad boolean style. Options are: " + BooleanParser.getOptions());
return false;
}
}
if (null != (tkey = amap.remove("-nullsUnset"))) nullsUnset = Boolean.parseBoolean(tkey);
if (null != (tkey = amap.remove("-charsPerColumn"))) charsPerColumn = Integer.parseInt(tkey);
if (-1 == maxRows)
maxRows = Long.MAX_VALUE;
if (-1 == maxErrors)
maxErrors = Long.MAX_VALUE;
if (-1 == maxInsertErrors)
maxInsertErrors = Long.MAX_VALUE;
if (!amap.isEmpty()) {
for (String k : amap.keySet())
System.err.println("Unrecognized option: " + k);
return false;
}
if (0 < inNumFutures)
numFutures = inNumFutures / numThreads;
if (null != inTtl) {
if (1 > inTtl.intValue()) {
System.err.println("TTL must be greater than 1");
return false;
}
else
ttl = inTtl.intValue();
}
if (null != keyspace)
keyspace = quote(keyspace);
if (null != table)
table = quote(table);
return validateArgs();
}
private String quote(String instr) {
String ret = instr;
if ((ret.startsWith("\"")) && (ret.endsWith("\"")))
ret = ret.substring(1, ret.length() - 1);
return "\"" + ret + "\"";
}
private SSLOptions createSSLOptions()
throws KeyStoreException, FileNotFoundException, IOException, NoSuchAlgorithmException,
KeyManagementException, CertificateException, UnrecoverableKeyException {
TrustManagerFactory tmf = null;
KeyStore tks = KeyStore.getInstance("JKS");
tks.load((InputStream) new FileInputStream(new File(truststorePath)),
truststorePwd.toCharArray());
tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(tks);
KeyManagerFactory kmf = null;
if (null != keystorePath) {
KeyStore kks = KeyStore.getInstance("JKS");
kks.load((InputStream) new FileInputStream(new File(keystorePath)),
keystorePwd.toCharArray());
kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(kks, keystorePwd.toCharArray());
}
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf != null? kmf.getKeyManagers() : null,
tmf != null ? tmf.getTrustManagers() : null,
new SecureRandom());
return RemoteEndpointAwareJdkSSLOptions.builder().withSSLContext(sslContext).build();
}
private boolean setup()
throws IOException, KeyStoreException, NoSuchAlgorithmException, KeyManagementException,
CertificateException, UnrecoverableKeyException {
// Connect to Cassandra
Session tsession = null;
try {
PoolingOptions pOpts = new PoolingOptions();
pOpts.setMaxConnectionsPerHost(HostDistance.LOCAL, 8);
pOpts.setCoreConnectionsPerHost(HostDistance.LOCAL, 8);
Cluster.Builder clusterBuilder = Cluster.builder()
.addContactPoint(host)
.withPort(port)
//.withCompression(ProtocolOptions.Compression.LZ4)
.withPoolingOptions(pOpts)
.withLoadBalancingPolicy(new TokenAwarePolicy( DCAwareRoundRobinPolicy.builder().build()))
;
if (null != username)
clusterBuilder = clusterBuilder.withCredentials(username, password);
if (null != truststorePath)
clusterBuilder = clusterBuilder.withSSL(createSSLOptions());
cluster = clusterBuilder.build();
if (null == cluster) {
throw new IOException("Could not create cluster");
}
tsession = cluster.connect();
}
catch (IllegalArgumentException e){
System.err.println("Could not connect to the cluster, check your hosts");
//e.printStackTrace();
return false;
}
catch (Exception e){
System.err.println(e.getStackTrace());
e.printStackTrace();
return false;
}
if ((0 > cluster.getConfiguration().getProtocolOptions()
.getProtocolVersion().compareTo(ProtocolVersion.V4))
&& nullsUnset) {
System.err.println("Cannot use nullsUnset with ProtocolVersion less than V4 (prior to Cassandra 3.0");
cleanup();
return false;
}
if (null != rateFile) {
if (STDERR.equalsIgnoreCase(rateFile)) {
rateStream = System.err;
}
else {
rateStream = new PrintStream(new BufferedOutputStream(new FileOutputStream(rateFile)), true);
}
}
Metrics metrics = cluster.getMetrics();
com.codahale.metrics.Timer timer = metrics.getRequestsTimer();
rateLimiter = new RateLimiter(rate, progressRate, timer, rateStream);
//rateLimiter = new Latency999RateLimiter(rate, progressRate, 3000, 200, 10, 0.5, 0.1, cluster, false);
session = new RateLimitedSession(tsession, rateLimiter);
return true;
}
private void cleanup() {
rateLimiter.report(null, null);
if (null != rateStream)
rateStream.close();
if (null != session)
session.close();
if (null != cluster)
cluster.close();
}
public boolean run(String[] args)
throws IOException, ParseException, InterruptedException, ExecutionException, KeyStoreException,
NoSuchAlgorithmException, KeyManagementException, CertificateException,
UnrecoverableKeyException {
if (false == parseArgs(args)) {
System.err.println("Bad arguments");
System.err.println(usage());
return false;
}
// Setup
if (false == setup())
return false;
// open file
Deque<File> fileList = new ArrayDeque<File>();
File infile = null;
File[] inFileList = null;
boolean onefile = true;
if (STDIN.equalsIgnoreCase(filename)) {
infile = null;
}
else {
infile = new File(filename);
if (infile.isFile()) {
}
else {
inFileList = infile.listFiles();
if (inFileList.length < 1)
throw new IOException("directory is empty");
onefile = false;
Arrays.sort(inFileList,
new Comparator<File>() {
public int compare(File f1, File f2) {
return f1.getName().compareTo(f2.getName());
}
});
for (int i = 0; i < inFileList.length; i++)
fileList.push(inFileList[i]);
}
}
// Launch Threads
ExecutorService executor;
long total = 0;
if (onefile) {
// One file/stdin to process
executor = Executors.newSingleThreadExecutor();
Callable<Long> worker = new CqlDelimLoadTask(cqlSchema, delimiter,
charsPerColumn, nullString,
commentString,
dateFormatString,
localDateFormatString,
boolStyle, locale,
maxErrors, skipRows,
skipCols,
maxRows, badDir, infile,
session, consistencyLevel,
numFutures, batchSize,
numRetries, queryTimeout,
maxInsertErrors,
successDir, failureDir,
nullsUnset, format,
keyspace, table, ttl);
Future<Long> res = executor.submit(worker);
total = res.get();
executor.shutdown();
}
else {
executor = Executors.newFixedThreadPool(numThreads);
Set<Future<Long>> results = new HashSet<Future<Long>>();
while (!fileList.isEmpty()) {
File tFile = fileList.pop();
Callable<Long> worker = new CqlDelimLoadTask(cqlSchema, delimiter,
charsPerColumn, nullString,
commentString,
dateFormatString,
localDateFormatString,
boolStyle, locale,
maxErrors, skipRows,
skipCols,
maxRows, badDir, tFile,
session,
consistencyLevel,
numFutures, batchSize,
numRetries,
queryTimeout,
maxInsertErrors,
successDir, failureDir,
nullsUnset, format,
keyspace, table, ttl);
results.add(executor.submit(worker));
}
executor.shutdown();
for (Future<Long> res : results)
total += res.get();
}
// Cleanup
cleanup();
//System.err.println("Total rows inserted: " + total);
return true;
}
public static void main(String[] args)
throws IOException, ParseException, InterruptedException, ExecutionException,
KeyStoreException, NoSuchAlgorithmException, UnrecoverableKeyException,
CertificateException, KeyManagementException {
CqlDelimLoad cdl = new CqlDelimLoad();
boolean success = cdl.run(args);
if (success) {
System.exit(0);
} else {
System.exit(-1);
}
}
}
================================================
FILE: src/main/java/com/datastax/loader/CqlDelimLoadTask.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import com.datastax.driver.core.BatchStatement;
import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.ResultSetFuture;
import com.datastax.driver.core.Session;
import com.datastax.loader.futures.FutureManager;
import com.datastax.loader.futures.PrintingFutureSet;
import com.datastax.loader.futures.JsonPrintingFutureSet;
import com.datastax.loader.parser.BooleanParser;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.text.ParseException;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.Callable;
import java.util.concurrent.TimeUnit;
import java.util.zip.GZIPInputStream;
import java.util.zip.ZipException;
class CqlDelimLoadTask implements Callable<Long> {
private String BADPARSE = ".BADPARSE";
private String BADINSERT = ".BADINSERT";
private String LOG = ".LOG";
private Session session;
private String insert;
private PreparedStatement statement;
private BatchStatement batch;
private StringBuilder batchString;
private FutureManager fm;
private ConsistencyLevel consistencyLevel;
private CqlDelimParser cdp;
private long maxErrors;
private long skipRows;
private String skipCols = null;
private long maxRows;
private String badDir;
private String successDir;
private String failureDir;
private String readerName;
private PrintStream badParsePrinter = null;
private PrintStream badInsertPrinter = null;
private PrintStream logPrinter = null;
private String logFname = "";
private BufferedReader reader;
private File infile;
private int numFutures;
private int batchSize;
private long numInserted;
private int ttl = -1;
private String cqlSchema;
private Locale locale = null;
private BooleanParser.BoolStyle boolStyle = null;
private String dateFormatString = null;
private String localDateFormatString = null;
private String nullString = null;
private String commentString = null;
private String delimiter = null;
private int charsPerColumn = 4096;
private TimeUnit unit = TimeUnit.SECONDS;
private long queryTimeout = 2;
private int numRetries = 1;
private long maxInsertErrors = 10;
private long insertErrors = 0;
private boolean nullsUnset;
private String format = "delim";
private String keyspace = null;
private String table = null;
private JSONArray jsonArray;
public CqlDelimLoadTask(String inCqlSchema, String inDelimiter,
int inCharsPerColumn,
String inNullString, String inCommentString,
String inDateFormatString, String inLocalDateFormatString,
BooleanParser.BoolStyle inBoolStyle,
Locale inLocale,
long inMaxErrors, long inSkipRows,
String inSkipCols, long inMaxRows,
String inBadDir, File inFile,
Session inSession, ConsistencyLevel inCl,
int inNumFutures, int inBatchSize, int inNumRetries,
int inQueryTimeout, long inMaxInsertErrors,
String inSuccessDir, String inFailureDir,
boolean inNullsUnset, String inFormat,
String inKeyspace, String inTable, int inTtl) {
super();
cqlSchema = inCqlSchema;
delimiter = inDelimiter;
charsPerColumn = inCharsPerColumn;
nullString = inNullString;
commentString = inCommentString;
dateFormatString = inDateFormatString;
localDateFormatString = inLocalDateFormatString;
boolStyle = inBoolStyle;
locale = inLocale;
maxErrors = inMaxErrors;
skipRows = inSkipRows;
skipCols = inSkipCols;
maxRows = inMaxRows;
badDir = inBadDir;
infile = inFile;
session = inSession;
consistencyLevel = inCl;
numFutures = inNumFutures;
batchSize = inBatchSize;
numRetries = inNumRetries;
queryTimeout = inQueryTimeout;
maxInsertErrors = inMaxInsertErrors;
successDir = inSuccessDir;
failureDir = inFailureDir;
nullsUnset = inNullsUnset;
format = inFormat;
keyspace = inKeyspace;
table = inTable;
ttl = inTtl;
}
public Long call() throws IOException, ParseException, org.json.simple.parser.ParseException {
setup();
numInserted = execute();
return numInserted;
}
private void setup() throws IOException, ParseException, org.json.simple.parser.ParseException {
if (null == infile) {
reader = new BufferedReader(new InputStreamReader(System.in));
readerName = "stdin";
}
else {
InputStream is = null;
try {
is = new GZIPInputStream(new FileInputStream(infile));
}
catch (ZipException e) {
is = new FileInputStream(infile);
}
reader = new BufferedReader(new InputStreamReader(is));
readerName = infile.getName();
}
//setup json reader
if(format.equalsIgnoreCase("jsonarray")){
JSONParser parser = new JSONParser();
jsonArray= (JSONArray) parser.parse(reader);
}
// Prepare Badfile
if (null != badDir) {
badParsePrinter = new PrintStream(new BufferedOutputStream(new FileOutputStream(badDir + "/" + readerName + BADPARSE)));
badInsertPrinter = new PrintStream(new BufferedOutputStream(new FileOutputStream(badDir + "/" + readerName + BADINSERT)));
logFname = badDir + "/" + readerName + LOG;
logPrinter = new PrintStream(new BufferedOutputStream(new FileOutputStream(logFname)));
}
if (format.equalsIgnoreCase("delim")) {
cdp = new CqlDelimParser(cqlSchema, delimiter, charsPerColumn,
nullString, commentString,
dateFormatString, localDateFormatString,
boolStyle, locale,
skipCols, session, true, ttl);
}
else if (format.equalsIgnoreCase("jsonline")
|| format.equalsIgnoreCase("jsonarray")) {
cdp = new CqlDelimParser(keyspace, table, delimiter, charsPerColumn,
nullString, commentString,
dateFormatString, localDateFormatString,
boolStyle, locale,
skipCols, session, true, ttl);
}
insert = cdp.generateInsert();
statement = session.prepare(insert);
statement.setRetryPolicy(new LoaderRetryPolicy(numRetries));
statement.setConsistencyLevel(consistencyLevel);
batch = new BatchStatement(BatchStatement.Type.UNLOGGED);
batchString = new StringBuilder();
if (format.equalsIgnoreCase("delim")) {
fm = new PrintingFutureSet(numFutures, queryTimeout,
maxInsertErrors, logPrinter,
badInsertPrinter);
}
else if (format.equalsIgnoreCase("jsonline")
|| format.equalsIgnoreCase("jsonarray")) {
fm = new JsonPrintingFutureSet(numFutures, queryTimeout,
maxInsertErrors, logPrinter,
badInsertPrinter);
}
}
private void cleanup(boolean success) throws IOException {
if (null != badParsePrinter) {
if (format.equalsIgnoreCase("jsonarray"))
badParsePrinter.println("]");
badParsePrinter.close();
}
if (null != badInsertPrinter) {
if (format.equalsIgnoreCase("jsonarray"))
badInsertPrinter.println("]");
badInsertPrinter.close();
}
if (null != logPrinter)
logPrinter.close();
if (success) {
if (null != successDir) {
Path src = infile.toPath();
Path dst = Paths.get(successDir);
Files.move(src, dst.resolve(src.getFileName()),
StandardCopyOption.REPLACE_EXISTING);
}
}
else {
if (null != failureDir) {
Path src = infile.toPath();
Path dst = Paths.get(failureDir);
Files.move(src, dst.resolve(src.getFileName()),
StandardCopyOption.REPLACE_EXISTING);
}
}
}
private int sendInsert(List<Object> elements, String line) {
BoundStatement bind = statement.bind(elements.toArray());
ResultSetFuture resultSetFuture;
int retval = 0;
if (nullsUnset) {
for (int i = 0; i < elements.size(); i++)
if (null == elements.get(i))
bind.unset(i);
}
if (1 == batchSize) {
resultSetFuture = session.executeAsync(bind);
if (!fm.add(resultSetFuture, line)) {
System.err.println("There was an error. Please check the log file for more information (" + logFname + ")");
//cleanup(false);
return -2;
}
//numInserted += 1;
retval = 1;
}
else {
batch.add(bind);
batchString.append("\n").append(line);
if (batchSize == batch.size()) {
resultSetFuture = session.executeAsync(batch);
if (!fm.add(resultSetFuture, batchString.toString())) {
System.err.println("There was an error. Please check the log file for more information (" + logFname + ")");
//cleanup(false);
return -2;
}
int numInserted = batch.size();
batch.clear();
batchString.setLength(0);
retval = numInserted;
}
}
return retval;
}
private long execute() throws IOException {
String line = null;
int lineNumber = 0;
long numInserted = 0;
int numErrors = 0;
int curBatch = 0;
BoundStatement bind = null;
List<Object> elements = null;
System.err.println("*** Processing " + readerName);
if (format.equalsIgnoreCase("delim")
|| format.equalsIgnoreCase("jsonline")) {
while ((line = reader.readLine()) != null) {
lineNumber++;
if (skipRows > 0) {
skipRows--;
continue;
}
if (maxRows-- < 0)
break;
if (0 == line.trim().length())
continue;
elements = null;
if (format.equalsIgnoreCase("delim"))
elements = cdp.parse(line);
else if (format.equalsIgnoreCase("jsonline"))
elements = cdp.parseJson(line);
if (null != elements) {
int ret = sendInsert(elements, line);
if (-2 == ret) {
cleanup(false);
return -2;
}
numInserted += ret;
}
else {
if (null != logPrinter) {
logPrinter.println(String.format("Error parsing line %d in %s: %s", lineNumber, readerName, line));
}
System.err.println(String.format("Error parsing line %d in %s: %s", lineNumber, readerName, line));
if (null != badParsePrinter) {
badParsePrinter.println(line);
}
numErrors++;
if (maxErrors <= numErrors) {
if (null != logPrinter) {
logPrinter.println(String.format("Maximum number of errors exceeded (%d) for %s", numErrors, readerName));
}
System.err.println(String.format("Maximum number of errors exceeded (%d) for %s", numErrors, readerName));
cleanup(false);
return -1;
}
}
}
} // if (format.equalsIgnoreCase("delim"))
else if (format.equalsIgnoreCase("jsonarray")) {
boolean firstBadJson = true;
String badJsonDelim = "[\n";
List<String> columnBackbone = cdp.getColumnNames();
int columnCount = columnBackbone.size();
for (Object o : jsonArray) {
JSONObject jsonRow = (JSONObject) o;
String[] jsonElements = new String[columnCount];
jsonElements[0] = jsonRow.get(columnBackbone.get(0)).toString();
for (int i = 1; i < columnCount; i++) {
if (null != jsonRow.get(columnBackbone.get(i))) {
jsonElements[i] = jsonRow.get(columnBackbone.get(i)).toString();
} else {
jsonElements[i] = null;
}
}
if (null != (elements = cdp.parse(jsonElements))) {
int ret = sendInsert(elements, line);
if (-2 == ret) {
cleanup(false);
return -2;
}
numInserted += ret;
} else {
String badString = jsonRow.toJSONString();
if (null != logPrinter) {
logPrinter.println(String.format("Error parsing JSON item %d in %s: %s", lineNumber, readerName, badString));
}
System.err.println(String.format("Error parsing JSON item %d in %s: %s", lineNumber, readerName, badString));
if (null != badParsePrinter) {
badParsePrinter.println(badJsonDelim + badString);
if (firstBadJson) {
firstBadJson = false;
badJsonDelim = ",\n";
}
}
numErrors++;
if (maxErrors <= numErrors) {
if (null != logPrinter) {
logPrinter.println(String.format("Maximum number of errors exceeded (%d) for %s", numErrors, readerName));
}
System.err.println(String.format("Maximum number of errors exceeded (%d) for %s", numErrors, readerName));
cleanup(false);
return -1;
}
}
}
}// if (format.equalsIgnoreCase("json"))
// Send last partially filled batch
if ((batchSize > 1) && (batch.size() > 0)) {
ResultSetFuture resultSetFuture = session.executeAsync(batch);
if (!fm.add(resultSetFuture, line)) {
cleanup(false);
return -2;
}
numInserted += batch.size();
}
if (!fm.cleanup()) {
cleanup(false);
return -1;
}
if (null != logPrinter) {
logPrinter.println("*** DONE: " + readerName + " number of lines processed: " + lineNumber + " (" + numInserted + " inserted)");
}
System.err.println("*** DONE: " + readerName + " number of lines processed: " + lineNumber + " (" + numInserted + " inserted)");
cleanup(true);
return fm.getNumInserted();
}
}
================================================
FILE: src/main/java/com/datastax/loader/CqlDelimParser.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import com.datastax.driver.core.ColumnMetadata;
import com.datastax.driver.core.DataType;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.TableMetadata;
import com.datastax.driver.core.KeyspaceMetadata;
import com.datastax.driver.core.exceptions.InvalidTypeException;
import com.datastax.loader.parser.BigDecimalParser;
import com.datastax.loader.parser.BigIntegerParser;
import com.datastax.loader.parser.BooleanParser;
import com.datastax.loader.parser.ByteBufferParser;
import com.datastax.loader.parser.ByteParser;
import com.datastax.loader.parser.DateParser;
import com.datastax.loader.parser.DelimParser;
import com.datastax.loader.parser.DoubleParser;
import com.datastax.loader.parser.FloatParser;
import com.datastax.loader.parser.InetAddressParser;
import com.datastax.loader.parser.IntegerParser;
import com.datastax.loader.parser.ListParser;
import com.datastax.loader.parser.LocalDateParser;
import com.datastax.loader.parser.LongParser;
import com.datastax.loader.parser.MapParser;
import com.datastax.loader.parser.Parser;
import com.datastax.loader.parser.SetParser;
import com.datastax.loader.parser.ShortParser;
import com.datastax.loader.parser.StringParser;
import com.datastax.loader.parser.UUIDParser;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import com.univocity.parsers.csv.CsvParser;
import com.univocity.parsers.csv.CsvParserSettings;
public class CqlDelimParser {
private Map<DataType.Name, Parser> pmap;
private List<SchemaBits> sbl;
private List<String> columnNames;
private String keyspace;
private String tablename;
private DelimParser delimParser;
private JSONParser jsonParser;
private int ttl = -1;
public CqlDelimParser(String inCqlSchema, String inDelimiter, int inCharsPerColumn,
String inNullString, String inCommentString,
String inDateFormatString, String inLocalDateFormatString,
BooleanParser.BoolStyle inBoolStyle, Locale inLocale,
String skipList, Session session, boolean bLoader, int inTtl)
throws ParseException {
// Optionally provide things for the line parser - date format, boolean format, locale
ttl = inTtl;
initPmap(inDateFormatString, inLocalDateFormatString, inBoolStyle,
inLocale, bLoader);
processCqlSchema(inCqlSchema, session);
createDelimParser(inDelimiter, inCharsPerColumn, inNullString, inCommentString, skipList);
}
public CqlDelimParser(String inKeyspace, String inTable, String inDelimiter,
int inCharsPerColumn,
String inNullString, String inCommentString,
String inDateFormatString, String inLocalDateFormatString,
BooleanParser.BoolStyle inBoolStyle, Locale inLocale,
String skipList, Session session, boolean bLoader, int inTtl)
throws ParseException {
// Optionally provide things for the line parser - date format, boolean format, locale
ttl = inTtl;
keyspace = inKeyspace;
tablename = inTable;
initPmap(inDateFormatString, inLocalDateFormatString, inBoolStyle,
inLocale, bLoader);
processCqlSchema(session);
createDelimParser(inDelimiter, inCharsPerColumn, inNullString, inCommentString, skipList);
}
public List<String> getColumnNames() {
return columnNames;
}
private void setColumnNames(List<String> columnNames) {
this.columnNames = columnNames;
}
// used internally to store schema information
private class SchemaBits {
public String name;
public DataType.Name datatype;
public Parser parser;
}
// intialize the Parsers and the parser map
private void initPmap(String dateFormatString, String localDateFormatString,
BooleanParser.BoolStyle inBoolStyle,
Locale inLocale, boolean bLoader) {
pmap = new HashMap<DataType.Name, Parser>();
Parser byteParser = new ByteParser(inLocale, bLoader);
Parser shortParser = new ShortParser(inLocale, bLoader);
Parser integerParser = new IntegerParser(inLocale, bLoader);
Parser longParser = new LongParser(inLocale, bLoader);
Parser floatParser = new FloatParser(inLocale, bLoader);
Parser doubleParser = new DoubleParser(inLocale, bLoader);
Parser stringParser = new StringParser();
Parser booleanParser = new BooleanParser(inBoolStyle);
Parser uuidParser = new UUIDParser();
Parser bigDecimalParser = new BigDecimalParser();
Parser bigIntegerParser = new BigIntegerParser();
Parser byteBufferParser = new ByteBufferParser();
Parser inetAddressParser = new InetAddressParser();
Parser dateParser = new DateParser(dateFormatString);
Parser localDateParser = new LocalDateParser(localDateFormatString);
pmap.put(DataType.Name.ASCII, stringParser);
pmap.put(DataType.Name.BIGINT, longParser);
pmap.put(DataType.Name.BLOB, byteBufferParser);
pmap.put(DataType.Name.BOOLEAN, booleanParser);
pmap.put(DataType.Name.COUNTER, longParser);
pmap.put(DataType.Name.DATE , localDateParser);
pmap.put(DataType.Name.DECIMAL, bigDecimalParser);
pmap.put(DataType.Name.DOUBLE, doubleParser);
pmap.put(DataType.Name.FLOAT, floatParser);
pmap.put(DataType.Name.INET, inetAddressParser);
pmap.put(DataType.Name.INT, integerParser);
pmap.put(DataType.Name.SMALLINT , shortParser);
pmap.put(DataType.Name.TEXT, stringParser);
pmap.put(DataType.Name.TIME , longParser);
pmap.put(DataType.Name.TIMESTAMP, dateParser);
pmap.put(DataType.Name.TIMEUUID, uuidParser);
pmap.put(DataType.Name.TINYINT , byteParser);
pmap.put(DataType.Name.UUID, uuidParser);
pmap.put(DataType.Name.VARCHAR, stringParser);
pmap.put(DataType.Name.VARINT, bigIntegerParser);
}
// Validate the CQL schema, extract the keyspace and tablename, and process the rest of the schema
private void processCqlSchema(String cqlSchema, Session session) throws ParseException {
CsvParserSettings ks_settings = new CsvParserSettings();
ks_settings.getFormat().setLineSeparator("\n");
ks_settings.getFormat().setDelimiter('.');
ks_settings.getFormat().setQuote('\"');
ks_settings.getFormat().setQuoteEscape('\\');
ks_settings.getFormat().setCharToEscapeQuoteEscaping('\\');
ks_settings.setKeepQuotes(true);
ks_settings.setKeepEscapeSequences(true);
ks_settings.setMaxCharsPerColumn(-1);
CsvParser ks_parser = new CsvParser(ks_settings);
String[] ks_elements = ks_parser.parseLine(cqlSchema);
keyspace = ks_elements[0];
String table_string = cqlSchema.substring(keyspace.length() + 1);
CsvParserSettings table_settings = new CsvParserSettings();
table_settings.getFormat().setLineSeparator("\n");
table_settings.getFormat().setDelimiter('(');
table_settings.getFormat().setQuote('\"');
table_settings.getFormat().setQuoteEscape('\\');
table_settings.getFormat().setCharToEscapeQuoteEscaping('\\');
table_settings.setKeepQuotes(true);
table_settings.setKeepEscapeSequences(true);
table_settings.setMaxCharsPerColumn(-1);
CsvParser table_parser = new CsvParser(table_settings);
String[] table_elements = table_parser.parseLine(table_string);
tablename = table_elements[0];
String schemaString = table_string.substring(tablename.length() + 1,
table_string.length() - 1);
sbl = schemaBits(schemaString, session);
}
private void processCqlSchema(Session session) throws ParseException {
sbl = schemaBits(null, session);
}
private List<SchemaBits> schemaBits(String in, Session session) throws ParseException {
KeyspaceMetadata km = session.getCluster().getMetadata().getKeyspace(keyspace);
if (null == km) {
System.err.println("Keyspace " + keyspace + " not found.");
System.exit(-1);
}
TableMetadata tm = km.getTable(tablename);
if (null == tm) {
System.err.println("Table " + tablename + " not found.");
System.exit(-1);
}
List<String> inList = new ArrayList<String>();
if (null != in) {
CsvParserSettings settings = new CsvParserSettings();
settings.getFormat().setLineSeparator("\n");
settings.getFormat().setDelimiter(',');
settings.getFormat().setQuote('\"');
settings.getFormat().setQuoteEscape('\\');
settings.getFormat().setCharToEscapeQuoteEscaping('\\');
settings.setKeepQuotes(true);
settings.setKeepEscapeSequences(true);
settings.setMaxCharsPerColumn(-1);
CsvParser parser = new CsvParser(settings);
String[] tlist = parser.parseLine(in);
for (int i = 0; i < tlist.length; i++)
inList.add(tlist[i].trim());
}
else {
for (ColumnMetadata cm : tm.getColumns())
inList.add("\""+cm.getName()+"\"");
}
//keep the list of columns from metadata to use as column backbone for JSON
setColumnNames(inList);
List<SchemaBits> sbl = new ArrayList<SchemaBits>();
for (int i = 0; i < inList.size(); i++) {
String col = inList.get(i);
SchemaBits sb = new SchemaBits();
ColumnMetadata cm = tm.getColumn(col);
if (null == cm) {
System.err.println("Column " + col + " of table " + keyspace + "." + tablename + " not found");
System.exit(-1);
}
DataType dt = cm.getType();
sb.name = col;
sb.datatype = dt.getName();
if (dt.isCollection()) {
if (sb.datatype == DataType.Name.LIST) {
DataType.Name listType = dt.getTypeArguments().get(0).getName();
Parser listParser = pmap.get(listType);
if (null == listParser) {
throw new ParseException("List data type not recognized ("
+ listType + ")", i);
}
sb.parser = new ListParser(listParser, ',', '[', ']');
}
else if (sb.datatype == DataType.Name.SET) {
DataType.Name setType = dt.getTypeArguments().get(0).getName();
Parser setParser = pmap.get(setType);
if (null == setParser) {
throw new ParseException("Set data type not recognized ("
+ setType + ")", i);
}
sb.parser = new SetParser(setParser, ',', '{', '}');
}
else if (sb.datatype == DataType.Name.MAP) {
DataType.Name keyType = dt.getTypeArguments().get(0).getName();
Parser keyParser = pmap.get(keyType);
if (null == keyParser) {
throw new ParseException("Map key data type not recognized ("
+ keyType + ")", i);
}
DataType.Name valueType = dt.getTypeArguments().get(1).getName();
Parser valueParser = pmap.get(valueType);
if (null == valueParser) {
throw new ParseException("Map value data type not recognized ("
+ valueType + ")", i);
}
sb.parser = new MapParser(keyParser, valueParser, ',', '{', '}', ':');
}
else {
throw new ParseException("Collection data type not recognized ("
+ sb.datatype + ")", i);
}
}
else {
sb.parser = pmap.get(sb.datatype);
if (null == sb.parser) {
throw new ParseException("Column data type not recognized (" + sb.datatype + ")", i);
}
}
sbl.add(sb);
}
return sbl;
}
// Creates the DelimParser that will parse the line
private void createDelimParser(String delimiter, int charsPerColumn,
String nullString, String commentString,
String skipList) throws NumberFormatException {
delimParser = new DelimParser(delimiter, charsPerColumn, nullString, commentString);
for (int i = 0; i < sbl.size(); i++)
delimParser.add(sbl.get(i).parser);
if (null != skipList) {
for (String s : skipList.split(",")) {
delimParser.addSkip(Integer.parseInt(s.trim()));
}
}
jsonParser = new JSONParser();
}
// Convenience method to return the INSERT statement for a PreparedStatement.
public String generateInsert() {
String insert = "INSERT INTO " + keyspace + "." + tablename + "(" + sbl.get(0).name;
String qmarks = "?";
for (int i = 1; i < sbl.size(); i++) {
insert = insert + ", " + sbl.get(i).name;
qmarks = qmarks + ", ?";
}
insert = insert + ") VALUES (" + qmarks + ")";
if (0 < ttl)
insert = insert + " USING TTL " + ttl;
return insert;
}
public String generateSelect() {
String select = "SELECT " + sbl.get(0).name;
for (int i = 1; i < sbl.size(); i++) {
select = select + ", " + sbl.get(i).name;
}
select += " FROM " + keyspace + "." + tablename;
return select;
}
public String getKeyspace() {
return keyspace;
}
public String getTable() {
return tablename;
}
// Pass through to parse the line - the DelimParser we created will be used.
public List<Object> parse(String line) {
return delimParser.parse(line);
}
public List<Object> parse(String[] row) {
return delimParser.parse(row);
}
@SuppressWarnings("unchecked")
public List<Object> parseJson(String line) {
JSONObject jsonObject = null;
try {
jsonObject = (JSONObject)jsonParser.parse(line);
} catch (org.json.simple.parser.ParseException e) {
System.err.println(String.format("Invalid format in input %d: %s",line, e.getMessage()));
return null;
}
String[] row = new String[columnNames.size()];
Set<String> fields = (Set<String>)jsonObject.keySet();
for (int i = 0; i < columnNames.size(); i++) {
String s = columnNames.get(i);
Object o = jsonObject.get(s);
if (null != o)
row[i] = o.toString();
else row[i] = null;
fields.remove(s);
}
if (0 != fields.size()) {
for (String f : fields) {
System.err.println("Unknown JSON field " + f);
}
return null;
}
return parse(row);
}
public String format(Row row) throws IndexOutOfBoundsException, InvalidTypeException {
return delimParser.format(row);
}
public String formatJson(Row row) throws IndexOutOfBoundsException, InvalidTypeException {
String[] stringVals = delimParser.stringVals(row);
Map<String,String> pairs = new HashMap<String,String>();
for (int i = 0; i < sbl.size(); i++)
pairs.put(sbl.get(i).name, stringVals[i]);
return JSONObject.toJSONString(pairs);
}
}
================================================
FILE: src/main/java/com/datastax/loader/CqlDelimUnload.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import com.datastax.loader.parser.BooleanParser;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.util.HashMap;
import java.util.Set;
import java.util.HashSet;
import java.util.Deque;
import java.util.ArrayDeque;
import java.util.Locale;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.math.BigInteger;
import java.io.FileOutputStream;
import java.io.BufferedOutputStream;
import java.io.PrintStream;
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.InputStreamReader;
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.FileNotFoundException;
import java.text.ParseException;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.KeyStoreException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.ColumnMetadata;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.PoolingOptions;
import com.datastax.driver.core.HostDistance;
import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.SSLOptions;
import com.datastax.driver.core.RemoteEndpointAwareJdkSSLOptions;
import com.datastax.driver.core.policies.TokenAwarePolicy;
import com.datastax.driver.core.policies.DCAwareRoundRobinPolicy;
import com.datastax.driver.core.exceptions.QueryValidationException;
public class CqlDelimUnload {
private String version = "0.0.27";
private String host = null;
private int port = 9042;
private String username = null;
private String password = null;
private String truststorePath = null;
private String truststorePwd = null;
private String keystorePath = null;
private String keystorePwd = null;
private ConsistencyLevel consistencyLevel = ConsistencyLevel.LOCAL_ONE;
private Cluster cluster = null;
private Session session = null;
private String beginToken = "-9223372036854775808";
private String endToken = "9223372036854775807";
private String where = null;
private String cqlSchema = null;
private String filename = null;
private String format = "delim";
private Locale locale = null;
private BooleanParser.BoolStyle boolStyle = null;
private String dateFormatString = null;
private String localDateFormatString = "yyyy-MM-dd";
private String nullString = null;
private String delimiter = null;
private int numThreads = 5;
private int fetchSize = 0;
private String usage() {
StringBuilder usage = new StringBuilder("version: ").append(version).append("\n");
usage.append("Usage: -f <outputStem> -host <ipaddress> -schema <schema> [OPTIONS]\n");
usage.append("OPTIONS:\n");
usage.append(" -configFile <filename> File with configuration options\n");
usage.append(" -format [delim|jsonline|jsonarray] Format of data: delimited or JSON [delim]\n");
usage.append(" -delim <delimiter> Delimiter to use [,]\n");
usage.append(" -dateFormat <dateFormatString> Date format for TIMESTAMP [default for Locale.ENGLISH]\n");
usage.append(" -localDateFormat <FormatString> Date format for DATE [yyyy-MM-dd]\n");
usage.append(" -nullString <nullString> String that signifies NULL [none]\n");
usage.append(" -port <portNumber> CQL Port Number [9042]\n");
usage.append(" -user <username> Cassandra username [none]\n");
usage.append(" -pw <password> Password for user [none]\n");
usage.append(" -ssl-truststore-path <path> Path to SSL truststore [none]\n");
usage.append(" -ssl-truststore-pw <pwd> Password for SSL truststore [none]\n");
usage.append(" -ssl-keystore-path <path> Path to SSL keystore [none]\n");
usage.append(" -ssl-keystore-pw <pwd> Password for SSL keystore [none]\n");
usage.append(" -consistencyLevel <CL> Consistency level [LOCAL_ONE]\n");
usage.append(" -decimalDelim <decimalDelim> Decimal delimiter [.] Other option is ','\n");
usage.append(" -boolStyle <boolStyleString> Style for booleans [TRUE_FALSE]\n");
usage.append(" -numThreads <numThreads> Number of concurrent threads to unload [5]\n");
usage.append(" -beginToken <tokenString> Begin token [none]\n");
usage.append(" -endToken <tokenString> End token [none]\n");
usage.append(" -where <predicate> WHERE clause [none]\n");
usage.append(" -fetchSize <fetchSize> Fetch size to use [0]\n");
return usage.toString();
}
private boolean validateArgs() {
if (!format.equalsIgnoreCase("delim")
&& !format.equalsIgnoreCase("jsonline")
&& !format.equalsIgnoreCase("jsonarray")) {
System.err.println("Invalid format (" + format + ")");
return false;
}
if (numThreads < 1) {
System.err.println("Number of threads must be positive");
return false;
}
if (fetchSize < 0) {
System.err.println("Fetch size must be non-negative");
return false;
}
if ((null == username) && (null != password)) {
System.err.println("If you supply the password, you must supply the username");
return false;
}
if ((null != username) && (null == password)) {
System.err.println("If you supply the username, you must supply the password");
return false;
}
if (filename.equalsIgnoreCase("stdout")) {
numThreads = 1;
}
if ((null == truststorePath) && (null != truststorePwd)) {
System.err.println("If you supply the ssl-truststore-pw, you must supply the ssl-truststore-path");
return false;
}
if ((null != truststorePath) && (null == truststorePwd)) {
System.err.println("If you supply the ssl-truststore-path, you must supply the ssl-truststore-pw");
return false;
}
if ((null == keystorePath) && (null != keystorePwd)) {
System.err.println("If you supply the ssl-keystore-pw, you must supply the ssl-keystore-path");
return false;
}
if ((null != keystorePath) && (null == keystorePwd)) {
System.err.println("If you supply the ssl-keystore-path, you must supply the ssl-keystore-pw");
return false;
}
File tfile = null;
if (null != truststorePath) {
tfile = new File(truststorePath);
if (!tfile.isFile()) {
System.err.println("truststore file must be a file");
return false;
}
}
if (null != keystorePath) {
tfile = new File(keystorePath);
if (!tfile.isFile()) {
System.err.println("keystore file must be a file");
return false;
}
}
if ((null != beginToken) && (null == endToken)) {
System.err.println("If you supply the beginToken then you need to specify the endToken");
return false;
}
if ((null == beginToken) && (null != endToken)) {
System.err.println("If you supply the endToken then you need to specify the beginToken");
return false;
}
return true;
}
private boolean processConfigFile(String fname, Map<String, String> amap)
throws IOException, FileNotFoundException {
File cFile = new File(fname);
if (!cFile.isFile()) {
System.err.println("Configuration File must be a file");
return false;
}
BufferedReader cReader = new BufferedReader(new FileReader(cFile));
String line;
while ((line = cReader.readLine()) != null) {
String[] fields = line.trim().split("\\s+");
if (2 != fields.length) {
System.err.println("Bad line in config file: " + line);
return false;
}
if (null == amap.get(fields[0])) {
amap.put(fields[0], fields[1]);
}
}
return true;
}
private boolean parseArgs(String[] args)
throws IOException, FileNotFoundException {
String tkey;
if (args.length == 0) {
System.err.println("No arguments specified");
return false;
}
if (0 != args.length % 2)
return false;
Map<String, String> amap = new HashMap<String,String>();
for (int i = 0; i < args.length; i+=2) {
amap.put(args[i], args[i+1]);
}
if (null != (tkey = amap.remove("-configFile")))
if (!processConfigFile(tkey, amap))
return false;
host = amap.remove("-host");
if (null == host) { // host is required
System.err.println("Must provide a host");
return false;
}
filename = amap.remove("-f");
if (null == filename) { // filename is required
System.err.println("Must provide an output filename stem");
return false;
}
cqlSchema = amap.remove("-schema");
if (null == cqlSchema) { // schema is required
System.err.println("Must provide a schema");
return false;
}
if (null != (tkey = amap.remove("-port"))) port = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-user"))) username = tkey;
if (null != (tkey = amap.remove("-pw"))) password = tkey;
if (null != (tkey = amap.remove("-ssl-truststore-path"))) truststorePath = tkey;
if (null != (tkey = amap.remove("-ssl-truststore-pw"))) truststorePwd = tkey;
if (null != (tkey = amap.remove("-ssl-keystore-path"))) keystorePath = tkey;
if (null != (tkey = amap.remove("-ssl-keystore-pw"))) keystorePwd = tkey;
if (null != (tkey = amap.remove("-consistencyLevel"))) consistencyLevel = ConsistencyLevel.valueOf(tkey);
if (null != (tkey = amap.remove("-dateFormat"))) dateFormatString = tkey;
if (null != (tkey = amap.remove("-localDateFormat"))) localDateFormatString = tkey;
if (null != (tkey = amap.remove("-nullString"))) nullString = tkey;
if (null != (tkey = amap.remove("-delim"))) delimiter = tkey;
if (null != (tkey = amap.remove("-decimalDelim"))) {
if (tkey.equals(","))
locale = Locale.FRANCE;
}
if (null != (tkey = amap.remove("-boolStyle"))) {
boolStyle = BooleanParser.getBoolStyle(tkey);
if (null == boolStyle) {
System.err.println("Bad boolean style. Options are: " + BooleanParser.getOptions());
return false;
}
}
if (null != (tkey = amap.remove("-numThreads"))) numThreads = Integer.parseInt(tkey);
if (null != (tkey = amap.remove("-beginToken"))) beginToken = tkey;
if (null != (tkey = amap.remove("-endToken"))) endToken = tkey;
if (null != (tkey = amap.remove("-where"))) where = tkey;
if (null != (tkey = amap.remove("-format"))) format = tkey;
if (null != (tkey = amap.remove("-fetchSize"))) fetchSize = Integer.parseInt(tkey);
if (!amap.isEmpty()) {
for (String k : amap.keySet())
System.err.println("Unrecognized option: " + k);
return false;
}
return validateArgs();
}
private SSLOptions createSSLOptions()
throws KeyStoreException, FileNotFoundException, IOException, NoSuchAlgorithmException,
KeyManagementException, CertificateException, UnrecoverableKeyException {
TrustManagerFactory tmf = null;
KeyStore tks = KeyStore.getInstance("JKS");
tks.load((InputStream) new FileInputStream(new File(truststorePath)),
truststorePwd.toCharArray());
tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(tks);
KeyManagerFactory kmf = null;
if (null != keystorePath) {
KeyStore kks = KeyStore.getInstance("JKS");
kks.load((InputStream) new FileInputStream(new File(keystorePath)),
keystorePwd.toCharArray());
kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(kks, keystorePwd.toCharArray());
}
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf != null? kmf.getKeyManagers() : null,
tmf != null ? tmf.getTrustManagers() : null,
new SecureRandom());
return RemoteEndpointAwareJdkSSLOptions.builder().withSSLContext(sslContext).build();
}
private void setup()
throws IOException, KeyStoreException, NoSuchAlgorithmException, KeyManagementException,
CertificateException, UnrecoverableKeyException {
// Connect to Cassandra
PoolingOptions pOpts = new PoolingOptions();
pOpts.setCoreConnectionsPerHost(HostDistance.LOCAL, 4);
pOpts.setMaxConnectionsPerHost(HostDistance.LOCAL, 4);
Cluster.Builder clusterBuilder = Cluster.builder()
.addContactPoint(host)
.withPort(port)
.withPoolingOptions(pOpts)
.withLoadBalancingPolicy(new TokenAwarePolicy( DCAwareRoundRobinPolicy.builder().build()));
if (null != username)
clusterBuilder = clusterBuilder.withCredentials(username, password);
if (null != truststorePath)
clusterBuilder = clusterBuilder.withSSL(createSSLOptions());
cluster = clusterBuilder.build();
if (null == cluster) {
throw new IOException("Could not create cluster");
}
session = cluster.connect();
}
private void cleanup() {
if (null != session)
session.close();
if (null != cluster)
cluster.close();
}
public boolean run(String[] args)
throws IOException, ParseException, InterruptedException, ExecutionException,
KeyStoreException, NoSuchAlgorithmException, KeyManagementException,
CertificateException, UnrecoverableKeyException {
if (false == parseArgs(args)) {
System.err.println("Bad arguments");
System.err.println(usage());
return false;
}
// Setup
setup();
PrintStream pstream = null;
if (1 == numThreads) {
if (filename.equalsIgnoreCase("stdout")) {
pstream = System.out;
}
else {
pstream = new PrintStream(new BufferedOutputStream(new FileOutputStream(filename + ".0")));
}
beginToken = null;
endToken = null;
}
// Launch Threads
ExecutorService executor;
long total = 0;
if (null != pstream) {
// One file/stdin to process
executor = Executors.newSingleThreadExecutor();
Callable<Long> worker = new ThreadExecute(cqlSchema, delimiter,
nullString,
dateFormatString,
localDateFormatString,
boolStyle, locale,
pstream,
beginToken,
endToken, session,
consistencyLevel, where,
format, fetchSize);
Future<Long> res = executor.submit(worker);
total = res.get();
executor.shutdown();
}
else {
BigInteger begin = null;
BigInteger end = null;
BigInteger delta = null;
List<String> beginList = new ArrayList<String>();
List<String> endList = new ArrayList<String>();
if (null != beginToken) {
begin = new BigInteger(beginToken);
end = new BigInteger(endToken);
delta = end.subtract(begin).divide(new BigInteger(String.valueOf(numThreads)));
for (int mype = 0; mype < numThreads; mype++) {
if (mype < numThreads - 1) {
beginList.add(begin.add(delta.multiply(new BigInteger(String.valueOf(mype)))).toString());
endList.add(begin.add(delta.multiply(new BigInteger(String.valueOf(mype+1)))).toString());
}
else {
beginList.add(begin.add(delta.multiply(new BigInteger(String.valueOf(numThreads-1)))).toString());
endList.add(end.toString());
}
}
}
else {
// What's the right thing here?
// (1) Split into canonical token ranges - numThreads=numRanges
// (2) Split into subranges of canonical token ranges
// - if numThreads < numRanges, then reset numThreads=numRanges
// - let K=CEIL(numThreads/numRanges) and M=MOD(numThreads/numRanges), for the first M token ranges split into K subranges, and for the remaining ones split into K-1 subranges
// (?) Should there be an option for numThreads-per-range?
// (?) Should there be an option for numThreads=numRanges
}
executor = Executors.newFixedThreadPool(numThreads);
Set<Future<Long>> results = new HashSet<Future<Long>>();
for (int mype = 0; mype < numThreads; mype++) {
String tBeginString = beginList.get(mype);
String tEndString = endList.get(mype);
pstream = new PrintStream(new BufferedOutputStream(new FileOutputStream(filename + "." + mype)));
Callable<Long> worker = new ThreadExecute(cqlSchema, delimiter,
nullString,
dateFormatString,
localDateFormatString,
boolStyle, locale,
pstream,
tBeginString,
tEndString, session,
consistencyLevel,
where, format, fetchSize);
results.add(executor.submit(worker));
}
executor.shutdown();
for (Future<Long> res : results)
total += res.get();
}
System.err.println("Total rows retrieved: " + total);
// Cleanup
cleanup();
return true;
}
public static void main(String[] args)
throws IOException, ParseException, InterruptedException, ExecutionException,
KeyStoreException, NoSuchAlgorithmException, UnrecoverableKeyException,
CertificateException, KeyManagementException {
CqlDelimUnload cdu = new CqlDelimUnload();
boolean success = cdu.run(args);
if (success) {
System.exit(0);
} else {
System.exit(-1);
}
}
class ThreadExecute implements Callable<Long> {
private Session session;
private ConsistencyLevel consistencyLevel;
private PreparedStatement statement;
private CqlDelimParser cdp;
private String cqlSchema;
private String format = "delim";
private Locale locale = null;
private BooleanParser.BoolStyle boolStyle = null;
private String nullString = null;
private String delimiter = null;
private PrintStream writer = null;
private String beginToken = null;
private String endToken = null;
private String partitionKey = null;
private long numRead = 0;
private String where = null;
private String dateFormatString = null;
private String localDateFormatString = null;
private int fetchSize = 0;
public ThreadExecute(String inCqlSchema, String inDelimiter,
String inNullString,
String inDateFormatString,
String inLocalDateFormatString,
BooleanParser.BoolStyle inBoolStyle,
Locale inLocale,
PrintStream inWriter,
String inBeginToken, String inEndToken,
Session inSession, ConsistencyLevel inConsistencyLevel,
String inWhere, String inFormat, int inFetchSize) {
super();
cqlSchema = inCqlSchema;
delimiter = inDelimiter;
nullString = inNullString;
dateFormatString = inDateFormatString;
localDateFormatString = inLocalDateFormatString;
boolStyle = inBoolStyle;
locale = inLocale;
beginToken = inBeginToken;
endToken = inEndToken;
session = inSession;
writer = inWriter;
consistencyLevel = inConsistencyLevel;
where = inWhere;
format = inFormat;
fetchSize = inFetchSize;
}
public Long call() throws IOException, ParseException {
if (false == setup()) {
return 0L;
}
numRead = execute();
cleanup();
return numRead;
}
private String getPartitionKey(CqlDelimParser cdp, Session session) {
String keyspace = cdp.getKeyspace();
String table = cdp.getTable();
if (keyspace.startsWith("\"") && keyspace.endsWith("\""))
keyspace = keyspace.replaceAll("\"", "");
else
keyspace = keyspace.toLowerCase();
if (table.startsWith("\"") && table.endsWith("\""))
table = table.replaceAll("\"", "");
else
table = table.toLowerCase();
List<ColumnMetadata> lcm = session.getCluster().getMetadata()
.getKeyspace(keyspace).getTable(table).getPartitionKey();
String partitionKey = lcm.get(0).getName();
for (int i = 1; i < lcm.size(); i++) {
partitionKey = partitionKey + "," + lcm.get(i).getName();
}
return partitionKey;
}
private boolean setup() throws IOException, ParseException {
cdp = new CqlDelimParser(cqlSchema, delimiter, 4096, nullString,
null, dateFormatString, localDateFormatString,
boolStyle, locale, null, session, false, -1);
String select = cdp.generateSelect();
String partitionKey = getPartitionKey(cdp, session);
if (null != beginToken) {
select = select + " WHERE Token(" + partitionKey + ") > "
+ beginToken + " AND Token(" + partitionKey + ") <= "
+ endToken;
if (null != where)
select = select + " AND " + where;
}
else {
if (null != where)
select = select + " WHERE " + where;
}
try {
statement = session.prepare(select);
}
catch (QueryValidationException iqe) {
System.err.println("Error creating statement: " + iqe.getMessage());
System.err.println("CQL Query: " + select);
if (null != where)
System.err.println("Check your syntax for -where: " + where);
return false;
}
statement.setConsistencyLevel(consistencyLevel);
return true;
}
private void cleanup() throws IOException {
writer.flush();
writer.close();
}
private long execute() throws IOException {
BoundStatement bound = statement.bind();
bound.setFetchSize(fetchSize);
ResultSet rs = session.execute(bound);
numRead = 0;
String s = null;
String jsonArrayChar = "[\n";
for (Row row : rs) {
if (format.equalsIgnoreCase("jsonarray")) {
writer.print(jsonArrayChar);
jsonArrayChar = ",";
}
if (format.equalsIgnoreCase("delim")) {
s = cdp.format(row);
}
else if (format.equalsIgnoreCase("jsonline")
|| format.equalsIgnoreCase("jsonarray")) {
s = cdp.formatJson(row);
}
writer.println(s);
numRead++;
}
if (format.equalsIgnoreCase("jsonarray"))
writer.println("]");
return numRead;
}
}
}
================================================
FILE: src/main/java/com/datastax/loader/EnhancedSession.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import java.util.Map;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.CloseFuture;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.ResultSetFuture;
import com.datastax.driver.core.Statement;
import com.datastax.driver.core.RegularStatement;
import com.datastax.driver.core.PreparedStatement;
import com.google.common.util.concurrent.ListenableFuture;
public class EnhancedSession implements Session {
private Session session;
public EnhancedSession(Session inSession) {
session = inSession;
}
public void close() {
session.close();
}
public CloseFuture closeAsync() {
return session.closeAsync();
}
public ResultSet execute(Statement statement) {
return session.execute(statement);
}
public ResultSet execute(String query) {
return session.execute(query);
}
public ResultSet execute(String query, Object... values) {
return session.execute(query, values);
}
public ResultSet execute(String query, Map<String,Object> values) {
return session.execute(query, values);
}
public ResultSetFuture executeAsync(Statement statement) {
return session.executeAsync(statement);
}
public ResultSetFuture executeAsync(String query) {
return session.executeAsync(query);
}
public ResultSetFuture executeAsync(String query, Object... values) {
return session.executeAsync(query, values);
}
public ResultSetFuture executeAsync(String query, Map<String,Object> values) {
return session.executeAsync(query, values);
}
public Cluster getCluster() {
return session.getCluster();
}
public String getLoggedKeyspace() {
return session.getLoggedKeyspace();
}
public Session.State getState() {
return session.getState();
}
public EnhancedSession init() {
session.init();
return this;
}
public com.google.common.util.concurrent.ListenableFuture<Session> initAsync() {
return session.initAsync();
}
public boolean isClosed() {
return session.isClosed();
}
public PreparedStatement prepare(RegularStatement statement) {
return session.prepare(statement);
}
public PreparedStatement prepare(String query) {
return session.prepare(query);
}
public com.google.common.util.concurrent.ListenableFuture<PreparedStatement>
prepareAsync(RegularStatement statement) {
return session.prepareAsync(statement);
}
public com.google.common.util.concurrent.ListenableFuture<PreparedStatement>
prepareAsync(String query) {
return session.prepareAsync(query);
}
}
================================================
FILE: src/main/java/com/datastax/loader/Latency999RateLimiter.java
================================================
/*
* Copyright 2015 Brian Hess
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.datastax.loader;
import com.datastax.driver.core.Cluster;
public class Latency999RateLimiter extends AbstractDynamicRateLimiter {
private Cluster cluster;
public Latency999RateLimiter(double inRate, long inHowOften,
double inMaxStat, double inMinStat,
double inDownFraction,
double inUpFraction, Cluster inCluster,
boolean inInvertLogic) {
this(inRate, Long.MAX_VALUE, inHowOften, inMaxStat, inMinStat,
inDownFraction, inUpFraction, inCluster, inInvertLogic);
}
public Latency999RateLimiter(double inRate, long inUpdateRate,
long inHowOften, double inMaxStat,
double inMinStat, double inDownFraction,
double inUpFraction, Cluster inCluster,
boolean inInvertLogic) {
super(inRate, inUpdateRate, inHowOften, inMaxStat, inMinStat,
inDownFraction, inUpFraction, inInvertLogic);
cluster = inCluster;
}
protected synchronized double getCurrStat() {
return cluster.getMetrics().getRequestsTimer().getSnapshot().get999thPercentile();
}
}
================================================
FILE: src/main/java/com/datastax/loader/LoaderRetryPolicy.java
================================================
package com.datastax.loader;
import java.io.File;
import java.io.BufferedWriter;
import java.io.FileWriter;
import com.datastax.driver.core.policies.RetryPolicy;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.Statement;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.WriteType;
import com.datastax.driver.core.exceptions.DriverException;
class LoaderRetryPolicy implements RetryPolicy {
private int numRetries;
public LoaderRetryPolicy(int inNumRetries) {
numRetries = inNumRetries;
}
// Taken from DefaultRetryPolicy
public RetryDecision onReadTimeout(Statement statement, ConsistencyLevel cl,
int requiredResponses,
int receivedResponses,
boolean dataRetrieved, int nbRetry) {
if (nbRetry != 0)
return RetryDecision.rethrow();
return receivedResponses >= requiredResponses && !dataRetrieved
? RetryDecision.retry(cl)
: RetryDecision.rethrow();
}
// Taken from DefaultRetryPolicy
public RetryDecision onUnavailable(Statement statement, ConsistencyLevel cl,
int requiredReplica, int aliveReplica,
int nbRetry) {
return RetryDecision.rethrow();
}
public RetryDecision onWriteTimeout(Statement statement,
ConsistencyLevel cl,
WriteType writeType, int requiredAcks,
int receivedAcks, int nbRetry) {
if (nbRetry >= numRetries)
return RetryDecision.rethrow();
return RetryDecision.retry(cl);
}
public RetryPolicy.RetryDecision onRequestError(Statement statement,
ConsistencyLevel cl,
DriverException e,
gitextract_9khq7mcr/
├── .gitignore
├── ChangeLog.md
├── LICENSE.md
├── README.md
├── build.gradle
├── sample/
│ ├── cassandra-schema.cql
│ ├── load.sh
│ └── titanic.csv
└── src/
├── main/
│ ├── java/
│ │ └── com/
│ │ └── datastax/
│ │ └── loader/
│ │ ├── AbstractDynamicRateLimiter.java
│ │ ├── CqlDelimLoad.java
│ │ ├── CqlDelimLoadTask.java
│ │ ├── CqlDelimParser.java
│ │ ├── CqlDelimUnload.java
│ │ ├── EnhancedSession.java
│ │ ├── Latency999RateLimiter.java
│ │ ├── LoaderRetryPolicy.java
│ │ ├── RateLimitedSession.java
│ │ ├── RateLimiter.java
│ │ ├── futures/
│ │ │ ├── AbstractFutureManager.java
│ │ │ ├── ActionFutureList.java
│ │ │ ├── ActionFutureSet.java
│ │ │ ├── FutureAction.java
│ │ │ ├── FutureManager.java
│ │ │ ├── JsonPrintingFutureAction.java
│ │ │ ├── JsonPrintingFutureList.java
│ │ │ ├── JsonPrintingFutureSet.java
│ │ │ ├── NullFutureAction.java
│ │ │ ├── PrintingFutureAction.java
│ │ │ ├── PrintingFutureList.java
│ │ │ └── PrintingFutureSet.java
│ │ └── parser/
│ │ ├── AbstractParser.java
│ │ ├── BigDecimalParser.java
│ │ ├── BigIntegerParser.java
│ │ ├── BooleanParser.java
│ │ ├── ByteBufferParser.java
│ │ ├── ByteParser.java
│ │ ├── DateParser.java
│ │ ├── DelimParser.java
│ │ ├── DoubleParser.java
│ │ ├── FloatParser.java
│ │ ├── IndexedLine.java
│ │ ├── InetAddressParser.java
│ │ ├── IntegerParser.java
│ │ ├── ListParser.java
│ │ ├── LocalDateParser.java
│ │ ├── LongParser.java
│ │ ├── MapParser.java
│ │ ├── NumberParser.java
│ │ ├── Parser.java
│ │ ├── SetParser.java
│ │ ├── ShortParser.java
│ │ ├── StringParser.java
│ │ └── UUIDParser.java
│ └── resources/
│ └── logback.xml
└── make/
├── buildit.sh
├── cassandra-loader.sh
└── unloader.sh
SYMBOL INDEX (312 symbols across 45 files)
FILE: src/main/java/com/datastax/loader/AbstractDynamicRateLimiter.java
class AbstractDynamicRateLimiter (line 18) | public abstract class AbstractDynamicRateLimiter extends RateLimiter {
method AbstractDynamicRateLimiter (line 28) | public AbstractDynamicRateLimiter(double inRate, long inHowOften,
method AbstractDynamicRateLimiter (line 37) | public AbstractDynamicRateLimiter(double inRate, long inUpdateRate,
method acquire (line 51) | public void acquire() {
method acquire (line 55) | public synchronized void acquire(int permits) {
method adjustRate (line 64) | protected synchronized void adjustRate() {
method statTooHigh (line 82) | protected synchronized boolean statTooHigh(double currStat) {
method statTooLow (line 86) | protected synchronized boolean statTooLow(double currStat) {
method adjustRateDown (line 90) | protected synchronized void adjustRateDown() {
method adjustRateUp (line 95) | protected synchronized void adjustRateUp() {
method getCurrStat (line 100) | protected abstract double getCurrStat();
FILE: src/main/java/com/datastax/loader/CqlDelimLoad.java
class CqlDelimLoad (line 85) | public class CqlDelimLoad {
method usage (line 141) | private String usage() {
method validateArgs (line 194) | private boolean validateArgs() {
method processConfigFile (line 351) | private boolean processConfigFile(String fname, Map<String, String> amap)
method parseArgs (line 374) | private boolean parseArgs(String[] args) throws IOException, FileNotFo...
method quote (line 486) | private String quote(String instr) {
method createSSLOptions (line 493) | private SSLOptions createSSLOptions()
method setup (line 520) | private boolean setup()
method cleanup (line 584) | private void cleanup() {
method run (line 594) | public boolean run(String[] args)
method main (line 699) | public static void main(String[] args)
FILE: src/main/java/com/datastax/loader/CqlDelimLoadTask.java
class CqlDelimLoadTask (line 53) | class CqlDelimLoadTask implements Callable<Long> {
method CqlDelimLoadTask (line 104) | public CqlDelimLoadTask(String inCqlSchema, String inDelimiter,
method call (line 151) | public Long call() throws IOException, ParseException, org.json.simple...
method setup (line 157) | private void setup() throws IOException, ParseException, org.json.simp...
method cleanup (line 223) | private void cleanup(boolean success) throws IOException {
method sendInsert (line 254) | private int sendInsert(List<Object> elements, String line) {
method execute (line 292) | private long execute() throws IOException {
FILE: src/main/java/com/datastax/loader/CqlDelimParser.java
class CqlDelimParser (line 62) | public class CqlDelimParser {
method CqlDelimParser (line 72) | public CqlDelimParser(String inCqlSchema, String inDelimiter, int inCh...
method CqlDelimParser (line 86) | public CqlDelimParser(String inKeyspace, String inTable, String inDeli...
method getColumnNames (line 103) | public List<String> getColumnNames() {
method setColumnNames (line 107) | private void setColumnNames(List<String> columnNames) {
class SchemaBits (line 112) | private class SchemaBits {
method initPmap (line 119) | private void initPmap(String dateFormatString, String localDateFormatS...
method processCqlSchema (line 163) | private void processCqlSchema(String cqlSchema, Session session) throw...
method processCqlSchema (line 197) | private void processCqlSchema(Session session) throws ParseException {
method schemaBits (line 202) | private List<SchemaBits> schemaBits(String in, Session session) throws...
method createDelimParser (line 298) | private void createDelimParser(String delimiter, int charsPerColumn,
method generateInsert (line 313) | public String generateInsert() {
method generateSelect (line 326) | public String generateSelect() {
method getKeyspace (line 335) | public String getKeyspace() {
method getTable (line 339) | public String getTable() {
method parse (line 344) | public List<Object> parse(String line) {
method parse (line 348) | public List<Object> parse(String[] row) {
method parseJson (line 352) | @SuppressWarnings("unchecked")
method format (line 380) | public String format(Row row) throws IndexOutOfBoundsException, Invali...
method formatJson (line 384) | public String formatJson(Row row) throws IndexOutOfBoundsException, In...
FILE: src/main/java/com/datastax/loader/CqlDelimUnload.java
class CqlDelimUnload (line 76) | public class CqlDelimUnload {
method usage (line 107) | private String usage() {
method validateArgs (line 135) | private boolean validateArgs() {
method processConfigFile (line 204) | private boolean processConfigFile(String fname, Map<String, String> amap)
method parseArgs (line 227) | private boolean parseArgs(String[] args)
method createSSLOptions (line 302) | private SSLOptions createSSLOptions()
method setup (line 329) | private void setup()
method cleanup (line 353) | private void cleanup() {
method run (line 360) | public boolean run(String[] args)
method main (line 466) | public static void main(String[] args)
class ThreadExecute (line 479) | class ThreadExecute implements Callable<Long> {
method ThreadExecute (line 502) | public ThreadExecute(String inCqlSchema, String inDelimiter,
method call (line 530) | public Long call() throws IOException, ParseException {
method getPartitionKey (line 539) | private String getPartitionKey(CqlDelimParser cdp, Session session) {
method setup (line 560) | private boolean setup() throws IOException, ParseException {
method cleanup (line 591) | private void cleanup() throws IOException {
method execute (line 596) | private long execute() throws IOException {
FILE: src/main/java/com/datastax/loader/EnhancedSession.java
class EnhancedSession (line 31) | public class EnhancedSession implements Session {
method EnhancedSession (line 33) | public EnhancedSession(Session inSession) {
method close (line 37) | public void close() {
method closeAsync (line 41) | public CloseFuture closeAsync() {
method execute (line 45) | public ResultSet execute(Statement statement) {
method execute (line 49) | public ResultSet execute(String query) {
method execute (line 53) | public ResultSet execute(String query, Object... values) {
method execute (line 57) | public ResultSet execute(String query, Map<String,Object> values) {
method executeAsync (line 61) | public ResultSetFuture executeAsync(Statement statement) {
method executeAsync (line 65) | public ResultSetFuture executeAsync(String query) {
method executeAsync (line 69) | public ResultSetFuture executeAsync(String query, Object... values) {
method executeAsync (line 73) | public ResultSetFuture executeAsync(String query, Map<String,Object> v...
method getCluster (line 77) | public Cluster getCluster() {
method getLoggedKeyspace (line 81) | public String getLoggedKeyspace() {
method getState (line 85) | public Session.State getState() {
method init (line 89) | public EnhancedSession init() {
method initAsync (line 94) | public com.google.common.util.concurrent.ListenableFuture<Session> ini...
method isClosed (line 98) | public boolean isClosed() {
method prepare (line 102) | public PreparedStatement prepare(RegularStatement statement) {
method prepare (line 106) | public PreparedStatement prepare(String query) {
method prepareAsync (line 110) | public com.google.common.util.concurrent.ListenableFuture<PreparedStat...
method prepareAsync (line 115) | public com.google.common.util.concurrent.ListenableFuture<PreparedStat...
FILE: src/main/java/com/datastax/loader/Latency999RateLimiter.java
class Latency999RateLimiter (line 20) | public class Latency999RateLimiter extends AbstractDynamicRateLimiter {
method Latency999RateLimiter (line 23) | public Latency999RateLimiter(double inRate, long inHowOften,
method Latency999RateLimiter (line 32) | public Latency999RateLimiter(double inRate, long inUpdateRate,
method getCurrStat (line 42) | protected synchronized double getCurrStat() {
FILE: src/main/java/com/datastax/loader/LoaderRetryPolicy.java
class LoaderRetryPolicy (line 14) | class LoaderRetryPolicy implements RetryPolicy {
method LoaderRetryPolicy (line 17) | public LoaderRetryPolicy(int inNumRetries) {
method onReadTimeout (line 22) | public RetryDecision onReadTimeout(Statement statement, ConsistencyLev...
method onUnavailable (line 35) | public RetryDecision onUnavailable(Statement statement, ConsistencyLev...
method onWriteTimeout (line 41) | public RetryDecision onWriteTimeout(Statement statement,
method onRequestError (line 51) | public RetryPolicy.RetryDecision onRequestError(Statement statement,
method close (line 58) | public void close() {
method init (line 61) | public void init(Cluster cluster) {
FILE: src/main/java/com/datastax/loader/RateLimitedSession.java
class RateLimitedSession (line 24) | public class RateLimitedSession extends EnhancedSession {
method RateLimitedSession (line 27) | public RateLimitedSession(Session inSession, RateLimiter inRateLimiter) {
method numAcquires (line 34) | public long numAcquires() {
method getRate (line 38) | public double getRate() {
method execute (line 42) | public ResultSet execute(Statement statement) {
method execute (line 47) | public ResultSet execute(String query) {
method execute (line 52) | public ResultSet execute(String query, Object... values) {
method executeAsync (line 57) | public ResultSetFuture executeAsync(Statement statement) {
method executeAsync (line 65) | public ResultSetFuture executeAsync(String query) {
method executeAsync (line 70) | public ResultSetFuture executeAsync(String query, Object... values) {
FILE: src/main/java/com/datastax/loader/RateLimiter.java
class RateLimiter (line 24) | public class RateLimiter {
method RateLimiter (line 34) | public RateLimiter(double inRate) {
method RateLimiter (line 38) | public RateLimiter(double inRate, long inUpdateRate) {
method RateLimiter (line 42) | public RateLimiter(double inRate, long inUpdateRate,
method printHeader (line 57) | protected void printHeader() {
method printStats (line 61) | protected void printStats() {
method report (line 82) | public void report(Long currentVal, Long currentTime) {
method incrementAndReport (line 109) | protected synchronized void incrementAndReport(int permits) {
method acquire (line 119) | public void acquire() {
method acquire (line 124) | public void acquire(int permits) {
method getRate (line 129) | public double getRate() {
method setRate (line 133) | public synchronized void setRate(double permitsPerSecond) {
method toString (line 137) | public String toString() {
method tryAcquire (line 141) | public boolean tryAcquire() {
method tryAcquire (line 149) | public boolean tryAcquire(int permits) {
method tryAcquire (line 157) | public boolean tryAcquire(int permits, long timeout, TimeUnit unit) {
method tryAcquire (line 165) | public boolean tryAcquire(long timeout, TimeUnit unit) {
method numAcquires (line 173) | public long numAcquires() {
FILE: src/main/java/com/datastax/loader/futures/AbstractFutureManager.java
class AbstractFutureManager (line 6) | public abstract class AbstractFutureManager implements FutureManager {
method AbstractFutureManager (line 12) | public AbstractFutureManager(int inSize, long inQueryTimeout, long inM...
method add (line 18) | public abstract boolean add(ResultSetFuture future, String line);
method cleanup (line 20) | public abstract boolean cleanup();
method getNumInserted (line 22) | public abstract long getNumInserted();
FILE: src/main/java/com/datastax/loader/futures/ActionFutureList.java
class ActionFutureList (line 9) | public class ActionFutureList extends AbstractFutureManager {
method ActionFutureList (line 16) | public ActionFutureList(int inSize, long inQueryTimeout, long inMaxIns...
method add (line 25) | public boolean add(ResultSetFuture future, String line) {
method purgeFutures (line 36) | protected boolean purgeFutures() {
method cleanup (line 64) | public boolean cleanup() {
method getNumInserted (line 68) | public long getNumInserted() {
FILE: src/main/java/com/datastax/loader/futures/ActionFutureSet.java
class ActionFutureSet (line 15) | public class ActionFutureSet extends AbstractFutureManager {
method ActionFutureSet (line 21) | public ActionFutureSet(int inSize, long inQueryTimeout,
method add (line 31) | public boolean add(ResultSetFuture future, final String line) {
method cleanup (line 60) | public boolean cleanup() {
method getNumInserted (line 69) | public long getNumInserted() {
FILE: src/main/java/com/datastax/loader/futures/FutureAction.java
type FutureAction (line 5) | public interface FutureAction {
method onSuccess (line 6) | public void onSuccess(ResultSet rs, String line);
method onFailure (line 7) | public void onFailure(Throwable t, String line);
method onTooManyFailures (line 8) | public void onTooManyFailures();
FILE: src/main/java/com/datastax/loader/futures/FutureManager.java
type FutureManager (line 5) | public interface FutureManager {
method add (line 6) | public boolean add(ResultSetFuture future, String line);
method cleanup (line 8) | public boolean cleanup();
method getNumInserted (line 10) | public long getNumInserted();
FILE: src/main/java/com/datastax/loader/futures/JsonPrintingFutureAction.java
class JsonPrintingFutureAction (line 7) | public class JsonPrintingFutureAction extends PrintingFutureAction {
method JsonPrintingFutureAction (line 10) | public JsonPrintingFutureAction(PrintStream inLogPrinter,
method onFailure (line 15) | public void onFailure(Throwable t, String line) {
FILE: src/main/java/com/datastax/loader/futures/JsonPrintingFutureList.java
class JsonPrintingFutureList (line 7) | public class JsonPrintingFutureList extends ActionFutureList {
method JsonPrintingFutureList (line 8) | public JsonPrintingFutureList() {
method JsonPrintingFutureList (line 12) | public JsonPrintingFutureList(int inSize, long inQueryTimeout,
method JsonPrintingFutureList (line 17) | public JsonPrintingFutureList(int inSize, long inQueryTimeout,
FILE: src/main/java/com/datastax/loader/futures/JsonPrintingFutureSet.java
class JsonPrintingFutureSet (line 5) | public class JsonPrintingFutureSet extends ActionFutureSet {
method JsonPrintingFutureSet (line 6) | public JsonPrintingFutureSet() {
method JsonPrintingFutureSet (line 10) | public JsonPrintingFutureSet(int inSize, long inQueryTimeout,
method JsonPrintingFutureSet (line 15) | public JsonPrintingFutureSet(int inSize, long inQueryTimeout,
FILE: src/main/java/com/datastax/loader/futures/NullFutureAction.java
class NullFutureAction (line 5) | public class NullFutureAction implements FutureAction {
method onSuccess (line 6) | public void onSuccess(ResultSet rs, String line) { }
method onFailure (line 7) | public void onFailure(Throwable t, String line) { }
method onTooManyFailures (line 8) | public void onTooManyFailures() { }
FILE: src/main/java/com/datastax/loader/futures/PrintingFutureAction.java
class PrintingFutureAction (line 7) | public class PrintingFutureAction implements FutureAction {
method PrintingFutureAction (line 13) | public PrintingFutureAction(PrintStream inLogPrinter,
method onSuccess (line 20) | public void onSuccess(ResultSet rs, String line) {
method onFailure (line 29) | public void onFailure(Throwable t, String line) {
method onTooManyFailures (line 39) | public void onTooManyFailures() {
FILE: src/main/java/com/datastax/loader/futures/PrintingFutureList.java
class PrintingFutureList (line 7) | public class PrintingFutureList extends ActionFutureList {
method PrintingFutureList (line 8) | public PrintingFutureList() {
method PrintingFutureList (line 12) | public PrintingFutureList(int inSize, long inQueryTimeout, long inMaxI...
method PrintingFutureList (line 16) | public PrintingFutureList(int inSize, long inQueryTimeout, long inMaxI...
FILE: src/main/java/com/datastax/loader/futures/PrintingFutureSet.java
class PrintingFutureSet (line 5) | public class PrintingFutureSet extends ActionFutureSet {
method PrintingFutureSet (line 6) | public PrintingFutureSet() {
method PrintingFutureSet (line 10) | public PrintingFutureSet(int inSize, long inQueryTimeout,
method PrintingFutureSet (line 15) | public PrintingFutureSet(int inSize, long inQueryTimeout,
FILE: src/main/java/com/datastax/loader/parser/AbstractParser.java
class AbstractParser (line 26) | public abstract class AbstractParser implements Parser {
method parseIt (line 27) | public abstract Object parseIt(String toparse) throws ParseException;
method format (line 28) | public String format(Row row, int index) throws IndexOutOfBoundsExcept...
method format (line 33) | public abstract String format(Object o);
method parse (line 35) | public Object parse(String toparse) throws ParseException {
method parse (line 40) | public Object parse(IndexedLine il, String nullString, Character delim,
method prepareToParse (line 49) | public String prepareToParse(String retstring, String nullString, Char...
method getQuotedOrUnquoted (line 63) | public String getQuotedOrUnquoted(IndexedLine il, String nullString,
method extractUntil (line 88) | public String extractUntil(IndexedLine il, Character delim,
method quote (line 118) | public static String quote(String instr) {
method unquote (line 124) | public static String unquote(String instr) {
method unstripQuote (line 132) | public static String unstripQuote(String instr) {
method stripQuote (line 138) | public static String stripQuote(String instr) {
method escape (line 146) | public static String escape(String instr) {
method unescape (line 152) | public static String unescape(String instr) {
FILE: src/main/java/com/datastax/loader/parser/BigDecimalParser.java
class BigDecimalParser (line 23) | public class BigDecimalParser extends AbstractParser {
method parseIt (line 24) | public BigDecimal parseIt(String toparse) throws NumberFormatException {
method format (line 30) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/BigIntegerParser.java
class BigIntegerParser (line 23) | public class BigIntegerParser extends AbstractParser {
method parseIt (line 24) | public BigInteger parseIt(String toparse) throws NumberFormatException {
method format (line 30) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/BooleanParser.java
class BooleanParser (line 23) | public class BooleanParser extends AbstractParser {
type BoolStyle (line 24) | public static enum BoolStyle {
method BoolStyle (line 35) | BoolStyle(String inStyleStr, String inTrueStr, String inFalseStr) {
method getStyle (line 41) | public String getStyle() {
method getTrueStr (line 45) | public String getTrueStr() {
method getFalseStr (line 49) | public String getFalseStr() {
method BooleanParser (line 62) | public BooleanParser() {
method BooleanParser (line 66) | public BooleanParser(BoolStyle inBoolStyle) {
method BooleanParser (line 73) | public BooleanParser(String inBoolTrue, String inBoolFalse) {
method getBoolStyle (line 78) | public static BoolStyle getBoolStyle(String instr) {
method getOptions (line 87) | public static String getOptions() {
method parseIt (line 96) | public Boolean parseIt(String toparse) throws ParseException {
method format (line 106) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/ByteBufferParser.java
class ByteBufferParser (line 24) | public class ByteBufferParser extends AbstractParser {
method parseIt (line 25) | public ByteBuffer parseIt(String toparse) {
method format (line 32) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/ByteParser.java
class ByteParser (line 25) | public class ByteParser extends NumberParser {
method ByteParser (line 26) | public ByteParser() {
method ByteParser (line 30) | public ByteParser(Locale inLocale) {
method ByteParser (line 34) | public ByteParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Byte parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/DateParser.java
class DateParser (line 28) | public class DateParser extends AbstractParser {
method DateParser (line 30) | public DateParser(String inFormatString) {
method parseIt (line 37) | public Date parseIt(String toparse) throws ParseException {
method format (line 43) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/DelimParser.java
class DelimParser (line 28) | public class DelimParser {
method DelimParser (line 48) | public DelimParser() {
method DelimParser (line 52) | public DelimParser(String inDelimiter) {
method DelimParser (line 56) | public DelimParser(String inDelimiter, int inCharsPerColumn,
method add (line 94) | public void add(Parser p) {
method addSkip (line 100) | public void addSkip(int idx) {
method prepareToParse (line 107) | private String prepareToParse(String toparse) {
method parse (line 116) | public List<Object> parse(String line) {
method parseWithUnivocity (line 121) | public List<Object> parseWithUnivocity(String line) {
method parse (line 126) | public List<Object> parse(String[] row) {
method parseComplex (line 159) | public List<Object> parseComplex(String line) {
method getElements (line 187) | public Object[] getElements() {
method format (line 191) | public String format(Row row) throws IndexOutOfBoundsException, Invali...
method stringVals (line 201) | public String[] stringVals(Row row) throws IndexOutOfBoundsException, ...
FILE: src/main/java/com/datastax/loader/parser/DoubleParser.java
class DoubleParser (line 25) | public class DoubleParser extends NumberParser {
method DoubleParser (line 26) | public DoubleParser() {
method DoubleParser (line 30) | public DoubleParser(Locale inLocale) {
method DoubleParser (line 34) | public DoubleParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Double parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/FloatParser.java
class FloatParser (line 25) | public class FloatParser extends NumberParser {
method FloatParser (line 26) | public FloatParser() {
method FloatParser (line 30) | public FloatParser(Locale inLocale) {
method FloatParser (line 34) | public FloatParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Float parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/IndexedLine.java
class IndexedLine (line 20) | public class IndexedLine {
method IndexedLine (line 24) | public IndexedLine(char[] inBuffer) {
method IndexedLine (line 29) | public IndexedLine(String instr) {
method getNext (line 33) | public char getNext() throws ParseException {
method hasNext (line 41) | public boolean hasNext() {
method setIndex (line 45) | public boolean setIndex(int idx) throws ParseException {
method get (line 52) | public char get(int idx) throws ParseException {
method backup (line 58) | public boolean backup() {
method length (line 65) | public int length() {
method numRemaining (line 69) | public int numRemaining() {
method remaining (line 73) | public String remaining() {
FILE: src/main/java/com/datastax/loader/parser/InetAddressParser.java
class InetAddressParser (line 26) | public class InetAddressParser extends AbstractParser {
method parseIt (line 27) | public InetAddress parseIt(String toparse) throws ParseException {
method format (line 40) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/IntegerParser.java
class IntegerParser (line 25) | public class IntegerParser extends NumberParser {
method IntegerParser (line 26) | public IntegerParser() {
method IntegerParser (line 30) | public IntegerParser(Locale inLocale) {
method IntegerParser (line 34) | public IntegerParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Integer parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/ListParser.java
class ListParser (line 30) | public class ListParser extends AbstractParser {
method ListParser (line 42) | public ListParser(Parser inParser, char inCollectionDelim,
method parseIt (line 62) | public Object parseIt(String toparse) throws ParseException {
method format (line 89) | @SuppressWarnings("unchecked")
FILE: src/main/java/com/datastax/loader/parser/LocalDateParser.java
class LocalDateParser (line 23) | public class LocalDateParser extends AbstractParser {
method LocalDateParser (line 26) | public LocalDateParser(String inFormatString) {
method parseIt (line 30) | public LocalDate parseIt(String toparse) throws ParseException {
method format (line 38) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/LongParser.java
class LongParser (line 25) | public class LongParser extends NumberParser {
method LongParser (line 26) | public LongParser() {
method LongParser (line 30) | public LongParser(Locale inLocale) {
method LongParser (line 34) | public LongParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Long parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/MapParser.java
class MapParser (line 33) | public class MapParser extends AbstractParser {
method MapParser (line 47) | public MapParser(Parser inKeyParser, Parser inValueParser,
method parseIt (line 70) | public Object parseIt(String toparse) throws ParseException {
method format (line 99) | @SuppressWarnings("unchecked")
FILE: src/main/java/com/datastax/loader/parser/NumberParser.java
class NumberParser (line 29) | public class NumberParser extends AbstractParser {
method NumberParser (line 31) | public NumberParser() {
method NumberParser (line 35) | public NumberParser(Locale locale) {
method NumberParser (line 39) | public NumberParser(Locale locale, Boolean grouping) {
method parseIt (line 49) | public Number parseIt(String toparse) throws ParseException {
method format (line 55) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/Parser.java
type Parser (line 26) | public interface Parser {
method parse (line 27) | public Object parse(String toparse) throws ParseException;
method parse (line 28) | public Object parse(IndexedLine il, String nullString, Character delim,
method format (line 31) | public String format(Row row, int index) throws IndexOutOfBoundsExcept...
method format (line 32) | public String format(Object o);
FILE: src/main/java/com/datastax/loader/parser/SetParser.java
class SetParser (line 31) | public class SetParser extends AbstractParser {
method SetParser (line 43) | public SetParser(Parser inParser, char inCollectionDelim,
method parseIt (line 62) | public Object parseIt(String toparse) throws ParseException {
method format (line 90) | @SuppressWarnings("unchecked")
FILE: src/main/java/com/datastax/loader/parser/ShortParser.java
class ShortParser (line 25) | public class ShortParser extends NumberParser {
method ShortParser (line 26) | public ShortParser() {
method ShortParser (line 30) | public ShortParser(Locale inLocale) {
method ShortParser (line 34) | public ShortParser(Locale inLocale, Boolean grouping) {
method parseIt (line 38) | public Short parseIt(String toparse) throws ParseException {
FILE: src/main/java/com/datastax/loader/parser/StringParser.java
class StringParser (line 25) | public class StringParser extends AbstractParser {
method parseIt (line 26) | public String parseIt(String toparse) {
method format (line 30) | public String format(Object o) {
FILE: src/main/java/com/datastax/loader/parser/UUIDParser.java
class UUIDParser (line 24) | public class UUIDParser extends AbstractParser {
method parseIt (line 25) | public UUID parseIt(String toparse) throws IllegalArgumentException {
method format (line 30) | public String format(Object o) {
Condensed preview — 57 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (285K chars).
[
{
"path": ".gitignore",
"chars": 71,
"preview": "build/\n.gradle/\n*~\n*.jar\ncassandra-loader\n*.BADINSERT\n*.BADPARSE\n*.LOG\n"
},
{
"path": "ChangeLog.md",
"chars": 2370,
"preview": "## 0.0.27\n- Added support for specifying a TTL (Issue 67)\n\n## 0.0.26\n- Fixed issue with long schemas (Issue 65)\n\n## 0.0."
},
{
"path": "LICENSE.md",
"chars": 11339,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 16688,
"preview": "\n# CASSANDRA-LOADER\n\n## Introduction\n\ncassandra-loader is a general-purpose, delimited-file, bulk loader for \nCassandra."
},
{
"path": "build.gradle",
"chars": 1809,
"preview": "apply plugin: 'java'\napply plugin: 'application'\n\ndef versionNum = '0.0.27'\n\nallprojects {\n tasks.withType(JavaCompil"
},
{
"path": "sample/cassandra-schema.cql",
"chars": 383,
"preview": "create keyspace titanic WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};\n\nuse titanic;\n\nCREATE T"
},
{
"path": "sample/load.sh",
"chars": 235,
"preview": "../build/cassandra-loader -f titanic.csv -host localhost -schema \"titanic.survivors(id, survived, passenger_class, name,"
},
{
"path": "sample/titanic.csv",
"chars": 60222,
"preview": "1,0,3,\"Braund, Mr. Owen Harris\",male,22,1,0,A/5 21171,7.25,,S\n2,1,1,\"Cumings, Mrs. John Bradley (Florence Briggs Thayer)"
},
{
"path": "src/main/java/com/datastax/loader/AbstractDynamicRateLimiter.java",
"chars": 3541,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/CqlDelimLoad.java",
"chars": 32339,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/CqlDelimLoadTask.java",
"chars": 17177,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/CqlDelimParser.java",
"chars": 17007,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/CqlDelimUnload.java",
"chars": 27187,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/EnhancedSession.java",
"chars": 3422,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/Latency999RateLimiter.java",
"chars": 1896,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/LoaderRetryPolicy.java",
"chars": 2241,
"preview": "package com.datastax.loader;\n\nimport java.io.File;\nimport java.io.BufferedWriter;\nimport java.io.FileWriter;\n\nimport com"
},
{
"path": "src/main/java/com/datastax/loader/RateLimitedSession.java",
"chars": 2343,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/RateLimiter.java",
"chars": 6086,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/futures/AbstractFutureManager.java",
"chars": 701,
"preview": "package com.datastax.loader.futures;\n\nimport java.util.concurrent.TimeUnit;\nimport com.datastax.driver.core.ResultSetFut"
},
{
"path": "src/main/java/com/datastax/loader/futures/ActionFutureList.java",
"chars": 2264,
"preview": "package com.datastax.loader.futures;\n\nimport java.util.List;\nimport java.util.ArrayList;\n\nimport com.datastax.driver.cor"
},
{
"path": "src/main/java/com/datastax/loader/futures/ActionFutureSet.java",
"chars": 2346,
"preview": "package com.datastax.loader.futures;\n\nimport java.util.List;\nimport java.util.ArrayList;\nimport java.util.concurrent.Tim"
},
{
"path": "src/main/java/com/datastax/loader/futures/FutureAction.java",
"chars": 260,
"preview": "package com.datastax.loader.futures;\n\nimport com.datastax.driver.core.ResultSet;\n\npublic interface FutureAction {\n pu"
},
{
"path": "src/main/java/com/datastax/loader/futures/FutureManager.java",
"chars": 250,
"preview": "package com.datastax.loader.futures;\n\nimport com.datastax.driver.core.ResultSetFuture;\n\npublic interface FutureManager {"
},
{
"path": "src/main/java/com/datastax/loader/futures/JsonPrintingFutureAction.java",
"chars": 927,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\nimport java.util.concurrent.atomic.AtomicLong;\nimport "
},
{
"path": "src/main/java/com/datastax/loader/futures/JsonPrintingFutureList.java",
"chars": 872,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\n\nimport com.datastax.driver.core.ResultSetFuture;\n\npub"
},
{
"path": "src/main/java/com/datastax/loader/futures/JsonPrintingFutureSet.java",
"chars": 813,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\n\npublic class JsonPrintingFutureSet extends ActionFutu"
},
{
"path": "src/main/java/com/datastax/loader/futures/NullFutureAction.java",
"chars": 293,
"preview": "package com.datastax.loader.futures;\n\nimport com.datastax.driver.core.ResultSet;\n\npublic class NullFutureAction implemen"
},
{
"path": "src/main/java/com/datastax/loader/futures/PrintingFutureAction.java",
"chars": 1400,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\nimport java.util.concurrent.atomic.AtomicLong;\nimport "
},
{
"path": "src/main/java/com/datastax/loader/futures/PrintingFutureList.java",
"chars": 720,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\n\nimport com.datastax.driver.core.ResultSetFuture;\n\npub"
},
{
"path": "src/main/java/com/datastax/loader/futures/PrintingFutureSet.java",
"chars": 777,
"preview": "package com.datastax.loader.futures;\n\nimport java.io.PrintStream;\n\npublic class PrintingFutureSet extends ActionFutureSe"
},
{
"path": "src/main/java/com/datastax/loader/parser/AbstractParser.java",
"chars": 5361,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/BigDecimalParser.java",
"chars": 1129,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/BigIntegerParser.java",
"chars": 1129,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/BooleanParser.java",
"chars": 3622,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/ByteBufferParser.java",
"chars": 1242,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/ByteParser.java",
"chars": 1287,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/DateParser.java",
"chars": 1533,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/DelimParser.java",
"chars": 7468,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/DoubleParser.java",
"chars": 1297,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/FloatParser.java",
"chars": 1290,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/IndexedLine.java",
"chars": 2018,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/InetAddressParser.java",
"chars": 1421,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/IntegerParser.java",
"chars": 1304,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/ListParser.java",
"chars": 3863,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/LocalDateParser.java",
"chars": 1417,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/LongParser.java",
"chars": 1287,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/MapParser.java",
"chars": 4589,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/NumberParser.java",
"chars": 1805,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/Parser.java",
"chars": 1295,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/SetParser.java",
"chars": 3890,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/ShortParser.java",
"chars": 1294,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/StringParser.java",
"chars": 1046,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/java/com/datastax/loader/parser/UUIDParser.java",
"chars": 1097,
"preview": "/*\n * Copyright 2015 Brian Hess\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use"
},
{
"path": "src/main/resources/logback.xml",
"chars": 292,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n <appender name=\"console\" class=\"ch.qos.logback.core.ConsoleAppe"
},
{
"path": "src/make/buildit.sh",
"chars": 143,
"preview": "#!/bin/sh\n\ncat src/make/cassandra-loader.sh build/libs/cassandra-loader-uber*.jar > build/cassandra-loader && chmod 755 "
},
{
"path": "src/make/cassandra-loader.sh",
"chars": 263,
"preview": "#!/bin/sh\nMYSELF=`which \"$0\" 2>/dev/null`\n[ $? -gt 0 -a -f \"$0\" ] && MYSELF=\"./$0\"\njava=java\nif test -n \"$JAVA_HOME\"; th"
},
{
"path": "src/make/unloader.sh",
"chars": 149,
"preview": "#!/bin/sh\n\ncat src/make/cassandra-loader.sh build/libs/cassandra-unloader-uber*.jar > build/cassandra-unloader && chmod "
}
]
About this extraction
This page contains the full source code of the brianmhess/cassandra-loader GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 57 files (264.2 KB), approximately 76.4k tokens, and a symbol index with 312 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.