Repository: EnterpriseDB/pg_nosql_benchmark
Branch: master
Commit: b16b5fbcea57
Files: 7
Total size: 77.8 KB
Directory structure:
gitextract_yk5gzwpp/
├── LICENSE
├── README.md
├── lib/
│ ├── common_func_lib.sh
│ ├── mongo_func_lib.sh
│ └── pg_func_lib.sh
├── output/
│ └── sample_expected.out
└── pg_nosql_benchmark
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
pg_nosql_benchmark
Copyright (c) 2013-2014, EnterpriseDB Corporation
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: README.md
================================================
pg_nosql_benchmark
==================
This is tool for benchmarking Postgres (JSONB) and MongoDB (BSON)
Introduction
-------------
This is a benchmarking tool developed by EnterpriseDB to benchmark MongoDB 2.6 (BSON) and Postgres 9.4 (JSONB) database using JSON data. The current version focuses on data ingestion and simple select operations in single-instance environments - later versions will include a complete range of workloads (including deleting, updating, appending, and complex select operations) and they will also evaluate multi-server configurations.
This tool performs the following tasks to compare of MongoDB and PostgreSQL:
* The tool generates a large set of JSON documents (the number of documents is defined by the value of the variable json_rows in pg_nosql_benchmark)
* The data set is loaded into MongoDB and PostgreSQL using mongoimport and PostgreSQL's COPY command.
* The same data is loaded into MongoDB and PostgreSQL using the INSERT command.
* The tool executes 4 SELECT Queries in MongoDB & PostgreSQL.
Requirements
------------
* pg_nosql_benchmark uses CentOS 6.4 or later, and is designed for PostgreSQL 9.4 beta server and MongoDB 2.6.
* The configuration requires three servers
* Load generating server
* MongoDB server
* PostgreSQL server
* The MongoDB server and the PostgreSQL server should be configured identically
* The script is designed to run from the central load-generating server, which must have access to the MongoDB and PostgreSQL servers.
* The following environment variables in pg_nosql_benchmark control the execution:
PostgreSQL Variables:
```
PGHOME=/usr/pgsql-9.4 # Installation location of PostgreSQL binaries.
PGHOST="172.17.0.2" # Hostname/IP address of PostgreSQL
PGPORT="5432" # Port number on which PostgreSQL is running.
PGUSER="postgres" # PostgreSQL database username.
PGPASSWORD="postgres" # PostgreSQL database users password.
PGBIN=/usr/pgsql-9.4/bin # PostgreSQL binary location.
```
MongoDB Variables:
```
MONGO="/usr/bin/mongo" # Complete path of mongo Command binary
MONGOIMPORT="/usr/bin/mongoimport" # complete path of mongoimport binary
MONGOHOST="172.17.0.3" # Hostname/IP address of MongoDB
MONGOPORT="27017" # Port number on which MongoDB is running.
MONGOUSER="mongo" # Mongo database username
MONGOPASSWORD="mongo" # MongoDB database username's password
MONGODBNAME="benchmark" # mongoDB database name.
```
* To create the admin user in MongoDB use the following command on the MongoDB server:
```
> db.createUser({ user: "mongo",
pwd: "mongo",
roles:[{ role: "userAdmin",
db: "benchmark"
}]
})
```
* To create the super user in PostgreSQL use the following command:
```
CREATE USER postgres PASSWORD '<password>' WITH SUPERUSER;
```
For more information on CREATE USER command in PostgreSQL, please check:
http://www.postgresql.org/docs/9.4/static/sql-createuser.html
Recommended modules
--------------------
The following packages are needed to run the benchmark tool:
1. mongodb-org-2.6.3-1.x86_64
2. postgresql94-9.4beta1-1PGDG.rhel6.x86_64
3. bc-1.06.95-1.el6.x86_64
4. git-1.7.1-3.el6_4.1.x86_64
Installation
------------
To install this tool on the load generating server, use the following command:
1. git clone https://github.com/EnterpriseDB/pg_nosql_benchmark.git
2. cd pg_mongo_benchmark
3. chmod +x pg_nosql_benchmark
================================================
FILE: lib/common_func_lib.sh
================================================
#!/bin/bash
################################################################################
# Copyright EnterpriseDB Cooperation
# All rights reserved.
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of PostgreSQL nor the names of its contributors
# may be used to endorse or promote products derived from this
# software without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Author: Vibhor Kumar
# E-mail ID: vibhor.aim@gmail.com
################################################################################
# quit on any error
set -e
# verify any undefined shell variables
set -u
################################################################################
# function: print messages with process id
################################################################################
function process_log()
{
echo "PID: $$ [RUNTIME: $(date +'%m-%d-%y %H:%M:%S')] ${BASENAME}: $*" >&2
}
################################################################################
# function: exit_on_error
################################################################################
function exit_on_error()
{
process_log "ERROR: $*"
exit 1
}
################################################################################
# function: get_timestamp_in_nanoseconds
################################################################################
function get_timestamp_nano ()
{
echo $(date +"%F %T.%N")
}
################################################################################
# function: get_timestamp_diff_nanoseconds
################################################################################
function get_timestamp_diff_nano ()
{
typeset -r F_TIMESTAMP1="$1"
typeset -r F_TIMESTAMP2="$2"
local SECONDS_DIFF
local NANOSECONDS_DIFF
local SECONDS_NANO
SECONDS_DIFF=$(echo $(date -d "${F_TIMESTAMP1}" +%s) \
- $(date -d "${F_TIMESTAMP2}" +%s)|bc)
NANOSECONDS_DIFF=$(echo $(date -d "${F_TIMESTAMP1}" +%N) \
- $(date -d "${F_TIMESTAMP2}" +%N)|bc)
SECONDS_NANO=$(echo ${SECONDS_DIFF} \* 1000000000|bc)
printf "%d\n" $(((${SECONDS_NANO} + ${NANOSECONDS_DIFF})))
}
################################################################################
# function: json_seed_data
################################################################################
function json_seed_data ()
{
local INDX="$1"
local SEED_DATA
if [[ ${INDX} -eq 0 ]]
then
INDX=1
fi
SEED_DATA="{ \"name\" : \"AC3$((${RANDOM}/$INDX + $INDX )) Phone\", \"brand\" : \"ACME$((${RANDOM}/$INDX + $INDX ))\", \"type\" : \"phone\", \"price\" : 200, \"warranty_years\" : 1, \"available\" : true, \"description\": \"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"AC7$((${RANDOM}/$INDX + $INDX )) Phone\", \"brand\" : \"ACME\", \"type\" : \"phone\", \"price\" : 320, \"warranty_years\" : 1, \"available\" : false, \"description\":\"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"AC3$((${RANDOM}/$INDX + $INDX )) Series Charger\", \"type\" : [ \"accessory\", \"charger\" ], \"price\" : 19, \"warranty_years\" : 0.25, \"for\" : [ \"ac3\", \"ac7\", \"ac9\" ], \"description\": \"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"AC3$((${RANDOM}/$INDX + $INDX )) Case Green\", \"type\" : [ \"accessory\", \"case\" ], \"color\" : \"green\", \"price\" : 12, \"warranty_years\" : 0, \"description\": \"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"Phone Extended Warranty\", \"type\" : \"warranty\", \"price\" : 38, \"warranty_years\" : 2, \"for\" : [ \"ac3\", \"ac7\", \"ac9\", \"qp7\", \"qp8\", \"qp9\" ], \"description\": \"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"AC3$((${RANDOM}/$INDX + $INDX )) Case Black\", \"type\" : [ \"accessory\", \"case\" ], \"color\" : \"black\", \"price\" : 12.5, \"warranty_years\" : 0.25, \"available\" : false, \"for\" : \"ac3\", \"description\":\"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }
{ \"name\" : \"AC3 Case Red\", \"type\" : [ \"accessory\", \"case\" ], \"color\" : \"red\", \"price\" : 12, \"warranty_years\" : 0.25, \"available\" : true, \"for\" : \"ac3\" }
{ \"name\" : \"Phone Service Basic Plan$((${RANDOM}/$INDX + $INDX ))\", \"type\" : \"service$((${RANDOM}/$INDX + $INDX ))\", \"monthly_price\" : 40, \"limits\" : { \"voice\" : { \"units\" : \"minutes\", \"n\" : 400, \"over_rate\" : 0.05 }, \"data\" : { \"units\" : \"gigabytes\", \"n\" : 20, \"over_rate\" : 1 }, \"sms\" : { \"units\" : \"texts sent\", \"n\" : 100, \"over_rate\" : 0.001 } }, \"term_years\" : 2 }
{ \"name\" : \"Phone Service Core Plan\", \"type\" : \"service$((${RANDOM}/$INDX + $INDX ))\", \"monthly_price\" : 60, \"limits\" : { \"voice\" : { \"units\" : \"minutes\", \"n\" : 1000, \"over_rate\" : 0.05 }, \"data\" : { \"n\" : \"unlimited\", \"over_rate\" : 0 }, \"sms\" : { \"n\" : \"unlimited\", \"over_rate\" : 0 } }, \"term_years\" : 1, \"description\": \"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\"}
{ \"name\" : \"Phone Service Family Plan\", \"type\" : \"service\", \"monthly_price\" : 90, \"limits\" : { \"voice\" : { \"units\" : \"minutes\", \"n\" : 1200, \"over_rate\" : 0.05 }, \"data\" : { \"n\" : \"unlimited\", \"over_rate\" : 0 }, \"sms\" : { \"n\" : \"unlimited\", \"over_rate\" : 0 } }, \"sales_tax\" : true, \"term_years\" : 2 }
{ \"name\" : \"Cable TV Basic Service Package\", \"type\" : \"tv\", \"monthly_price\" : 50, \"term_years\" : 2, \"cancel_penalty\" : 25, \"sales_tax\" : true, \"additional_tarriffs\" : [ { \"kind\" : \"federal tarriff\", \"amount\" : { \"percent_of_service\" : 0.06 } }, { \"kind\" : \"misc tarriff\", \"amount\" : 2.25 } ], \"description\":\"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin eget elit ut nulla tempor viverra vel eu nulla. Sed luctus porttitor urna, ac dapibus velit fringilla et. Donec iaculis, dolor a vehicula dictum, augue neque suscipit augue, nec mollis massa neque in libero. Donec sed dapibus magna. Pellentesque at condimentum dolor. In nunc nibh, dignissim in risus a, blandit tincidunt velit. Vestibulum rutrum tempus sem eget tempus. Mauris sollicitudin purus auctor dolor vestibulum, vitae pulvinar neque suscipit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Phasellus lacus turpis, vulputate at adipiscing viverra, ultricies at lectus. Pellentesque ut porta leo, vel eleifend neque. Nunc sagittis metus at ante pellentesque, ut condimentum libero semper. In hac habitasse platea dictumst. In dapibus posuere posuere. Fusce vulputate augue eget tellus molestie, vitae egestas ante malesuada. Phasellus nunc mi, faucibus at elementum pharetra, aliquet a enim. In purus est, vulputate in nibh quis, faucibus dapibus magna. In accumsan libero velit, eu accumsan sem commodo id. In fringilla tempor augue, et feugiat erat convallis et. Sed aliquet eget ipsum eu vestibulum.Curabitur blandit leo nec condimentum semper. Mauris lectus sapien, rutrum a tincidunt id, euismod ac elit. Mauris suscipit et arcu et auctor. Quisque mollis magna vel mi viverra rutrum. Nulla non pretium magna. Cras sed tortor non tellus rutrum gravida eu at odio. Aliquam cursus fermentum erat, nec ullamcorper sem gravida sit amet. Donec viverra, erat vel ornare pulvinar, est ipsum accumsan massa, eu tristique lorem ante nec tortor. Sed suscipit iaculis faucibus. Maecenas a suscipit ligula, vitae faucibus turpis.Cras sed tellus auctor, tempor leo eu, molestie leo. Suspendisse ipsum tellus, egestas et ultricies eu, tempus a arcu. Cras laoreet, est dapibus consequat varius, nisi nisi placerat leo, et dictum ante tortor vitae est. Duis eu urna ac felis ullamcorper rutrum. Quisque iaculis, enim eget sodales vehicula, magna orci dignissim eros, nec volutpat massa urna in elit. In interdum pellentesque risus, feugiat pulvinar odio eleifend sit amet. Quisque congue libero quis dolor faucibus, a mollis nisl tempus.\" }"
echo "${SEED_DATA}"
}
################################################################################
# function: generate_json_data
################################################################################
function generate_json_rows ()
{
typeset -r NO_OF_ROWS="$1"
typeset -r FILENAME="$2"
rm -rf ${FILENAME}
process_log "creating json data."
NO_OF_LOOPS=$((${NO_OF_ROWS}/11 + 1 ))
for ((i=0;i<${NO_OF_LOOPS};i++))
do
json_seed_data $i >>${FILENAME}
done
}
################################################################################
# print integer arrays
################################################################################
function print_result()
{
typeset -r F_TAG="$1"
shift
typeset -r F_LOCALARRAY=(${@})
typeset F_ARRAYLENGTH
F_ARRAYLENGTH=${#F_LOCALARRAY[@]}
printf "%20s " "${F_TAG}"
for (( i=0 ; i < ${F_ARRAYLENGTH} ; i++ ))
do
printf "%14d " ${F_LOCALARRAY[${i}]}
done
printf "\n"
}
================================================
FILE: lib/mongo_func_lib.sh
================================================
#!/bin/bash
################################################################################
# Copyright EnterpriseDB Cooperation
# All rights reserved.
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of PostgreSQL nor the names of its contributors
# may be used to endorse or promote products derived from this
# software without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Author: Vibhor Kumar
# E-mail ID: vibhor.aim@gmail.com
################################################################################
# source common lib
################################################################################
DIRECTORY=$(dirname $0)
source ${DIRECTORY}/lib/common_func_lib.sh
################################################################################
# mongo_run_command: send command to mongodb
################################################################################
function run_mongo_command ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_MONGOCOMMAND="$6"
${MONGO} ${F_MONGOHOST}:${F_MONGOPORT}/${F_MONGODBNAME} \
--username ${F_MONGOUSER} \
--password ${F_MONGOPASSWORD} --quiet <<- EOF
DBQuery.shellBatchSize = 10000000000;
${F_MONGOCOMMAND}
EOF
}
################################################################################
# run_mong_file: execute list of commands on mongo
################################################################################
function run_mongo_file ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COMMANDFILE="$6"
${MONGO} ${F_MONGOHOST}:${F_MONGOPORT}/${F_MONGODBNAME} \
--username ${F_MONGOUSER} \
--password ${F_MONGOPASSWORD} --quiet < ${F_COMMANDFILE}
}
################################################################################
# function: drop_mongocollection drop specific collection in mongo
################################################################################
function drop_mongocollection ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_MONGOCOLLECTION="$6"
typeset -r F_MONGOCOMMAND="printjson(db.${F_MONGOCOLLECTION}.drop())"
process_log "dropping mongo collection ${F_MONGOCOLLECTION}"
run_mongo_command "${F_MONGOHOST}" "${F_MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" "${F_MONGOPASSWORD}" "${F_MONGOCOMMAND}" >/dev/null
}
################################################################################
# function: mongo_insert_maker
################################################################################
function mongo_json_insert_maker ()
{
typeset -r COLLECTION_NAME="$1"
typeset -r NO_OF_ROWS="$2"
typeset -r JSON_FILENAME="$3"
rm -rf ${JSON_FILENAME}
process_log "preparing mongo insert commands."
NO_OF_LOOPS=$((${NO_OF_ROWS}/11 + 1 ))
for ((i=0;i<${NO_OF_LOOPS};i++))
do
json_seed_data $i | sed "s/^/db.${COLLECTION_NAME}.insert( /" | \
sed "s/$/ )/" >>${JSON_FILENAME}
done
}
################################################################################
# function: benchmark mongo-import
################################################################################
function mongodb_import_benchmark ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_FILENAME="$7"
process_log "testing mongoimport."
start_time=$(get_timestamp_nano)
${MONGOIMPORT} --host ${F_MONGOHOST} --db ${F_MONGODBNAME} \
--username ${F_MONGOUSER} --password ${F_MONGOPASSWORD} \
--type json --port ${F_MONGOPORT} \
--collection ${F_COLLECTION} < ${F_FILENAME} >/dev/null \
2>>/dev/null
end_time=$(get_timestamp_nano)
total_time="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
echo "${total_time}"
}
################################################################################
# function: benchmark mongo inserts
################################################################################
function mongodb_inserts_benchmark ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_JSONINSERTS="$7"
process_log "testing inserts in mongo"
start_time=$(get_timestamp_nano)
run_mongo_file "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" "${F_MONGOPASSWORD}" \
"${F_JSONINSERTS}" >/dev/null
end_time=$(get_timestamp_nano)
total_time="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
echo "${total_time}"
}
################################################################################
# function: benchmark mongo-select
################################################################################
function mongodb_select_benchmark ()
{
F_MONGOHOST="$1"
F_MONGOPORT="$2"
F_MONGODBNAME="$3"
F_MONGOUSER="$4"
F_MONGOPASSWORD="$5"
F_COLLECTION="$6"
F_MONGOSELECT1="db.${F_COLLECTION}.find({ brand: 'ACME'})"
F_MONGOSELECT2="db.${F_COLLECTION}.find({ name: 'Phone Service Basic Plan'})"
F_MONGOSELECT3="db.${F_COLLECTION}.find({ name: 'AC3 Case Red'})"
F_MONGOSELECT4="db.${F_COLLECTION}.find({ type: 'service'})"
process_log "testing mongo FIRST SELECT."
start_time=$(get_timestamp_nano)
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGOSELECT1}" >/dev/null
end_time=$(get_timestamp_nano)
total_time1="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing mongo SECOND SELECT."
start_time=$(get_timestamp_nano)
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGOSELECT2}" >/dev/null
end_time=$(get_timestamp_nano)
total_time2="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing mongo THIRD SELECT."
start_time=$(get_timestamp_nano)
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGOSELECT3}" >/dev/null
end_time=$(get_timestamp_nano)
total_time3="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing mongo FOURTH SELECT."
start_time=$(get_timestamp_nano)
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGOSELECT4}" >/dev/null
end_time=$(get_timestamp_nano)
total_time4="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
AVG="$(( ($total_time1 + $total_time2 + $total_time3 + $total_time4)/4 ))"
echo "${AVG}"
}
################################################################################
# function: mongdb collection size
################################################################################
function mongo_collection_size ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_COMMAND="printjson(db.${F_COLLECTION}.stats())"
process_log "calculating the size of mongo collection."
output="$(run_mongo_command "${F_MONGOHOST}" "${F_MONGOPORT}" \
"${F_MONGODBNAME}" "${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_COMMAND}")"
collectionsize="$(echo ${output}|awk -F"," '{print $5}'|cut -d":" -f2)"
echo "${collectionsize}"
}
################################################################################
# function: mongdb version
################################################################################
function mongo_version ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COMMAND="db.version()"
output="$(run_mongo_command "${F_MONGOHOST}" "${F_MONGOPORT}" \
"${F_MONGODBNAME}" "${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_COMMAND}" )"
version=$(echo $output|awk '{print $2}')
echo "${version}"
}
################################################################################
# function: mongodb create_index
################################################################################
function mongodb_create_index ()
{
typeset -r F_MONGOHOST="$1"
typeset -r F_MONGOPORT="$2"
typeset -r F_MONGODBNAME="$3"
typeset -r F_MONGOUSER="$4"
typeset -r F_MONGOPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_MONGODBIDX1="db.${F_COLLECTION}.ensureIndex( { \"name\": 1});"
typeset -r F_MONGODBIDX2="db.${F_COLLECTION}.ensureIndex( { \"type\": 1});"
typeset -r F_MONGODBIDX3="db.${F_COLLECTION}.ensureIndex( { \"brand\": 1});"
process_log "creating index in mongodb."
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGODBIDX1}" >/dev/null
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGODBIDX2}" >/dev/null
run_mongo_command "${F_MONGOHOST}" "${MONGOPORT}" "${F_MONGODBNAME}" \
"${F_MONGOUSER}" \
"${F_MONGOPASSWORD}" "${F_MONGODBIDX3}" >/dev/null
}
================================================
FILE: lib/pg_func_lib.sh
================================================
#!/bin/bash
################################################################################
# Copyright EnterpriseDB Cooperation
# All rights reserved.
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of PostgreSQL nor the names of its contributors
# may be used to endorse or promote products derived from this
# software without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Author: Vibhor Kumar
# E-mail ID: vibhor.aim@gmail.com
################################################################################
# source common lib
################################################################################
source $DIRECTORY/lib/common_func_lib.sh
################################################################################
# function: pg_json_insert_maker
################################################################################
function pg_json_insert_maker ()
{
typeset -r COLLECTION_NAME="$1"
typeset -r NO_OF_ROWS="$2"
typeset -r JSON_FILENAME="$3"
process_log "preparing postgresql INSERTs."
rm -rf ${JSON_FILENAME}
NO_OF_LOOPS=$((${NO_OF_ROWS}/11 + 1 ))
for ((i=0;i<${NO_OF_LOOPS};i++))
do
json_seed_data $i | \
sed "s/^/INSERT INTO ${COLLECTION_NAME} VALUES(\$JSON\$/"| \
sed "s/$/\$JSON\$);/" >>${JSON_FILENAME}
done
}
################################################################################
# run_sql_file: send SQL from a file to database
################################################################################
function run_sql_file ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQLFILE="$6"
export PGPASSWORD="${F_PGPASSWORD}"
${PGHOME}/bin/psql -qAt -h ${F_PGHOST} -p ${F_PGPORT} -U ${F_PGUSER} \
--single-transaction -d ${F_DBNAME} -f "${F_SQLFILE}"
}
################################################################################
# run_sql: send SQL to database
################################################################################
function run_sql ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQL="$6"
export PGPASSWORD="${F_PGPASSWORD}"
${PGHOME}/bin/psql -qAt -h ${F_PGHOST} -p ${F_PGPORT} -U ${F_PGUSER} \
-d ${F_DBNAME} -c "${F_SQL}"
}
################################################################################
# function: remove_pgdb (remove postgresql database)
################################################################################
function remove_pg_db ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQL="DROP DATABASE IF EXISTS ${F_DBNAME};"
process_log "droping database ${F_DBNAME} if exists."
run_sql "${F_PGHOST}" "${F_PGPORT}" "postgres" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}" 2>/dev/null >/dev/null
}
################################################################################
# function: create_pgdb (create postgresql database)
################################################################################
function create_pg_db ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQL="CREATE DATABASE ${F_DBNAME};"
process_log "creating database ${F_DBNAME}."
run_sql "${F_PGHOST}" "${F_PGPORT}" "postgres" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}"
}
################################################################################
# function: relation_size (calculate postgresql relation size)
################################################################################
function pg_relation_size ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_RELATION="$6"
typeset -r F_SQL="SELECT pg_catalog.pg_relation_size('${F_RELATION}');"
process_log "calculating PostgreSQL collection size."
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}"
}
################################################################################
# function: check if database exists
################################################################################
function if_dbexists ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQL="SELECT COUNT(1)
FROM pg_catalog.pg_database
WHERE datname='${F_DBNAME}';"
output=$(run_sql "${F_PGHOST}" "${F_PGPORT}" "postgres" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"${F_SQL}")
echo ${output}
}
################################################################################
# function: mk_pgjson_collection create json table in PG
################################################################################
function mk_pg_json_collection ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_TABLE="$6"
typeset -r F_SQL1="DROP TABLE IF EXISTS ${F_TABLE} CASCADE;"
typeset -r F_SQL2="CREATE TABLE ${F_TABLE} (data JSONB);"
process_log "creating ${F_TABLE} collection in postgreSQL."
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL1}" \
>/dev/null 2>/dev/null
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL2}" \
>/dev/null 2>/dev/null
}
################################################################################
# function: pg_create_index create json table in PG
################################################################################
function pg_create_index_collection ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_TABLE="$6"
typeset -r F_SQL="CREATE INDEX ${F_TABLE}_idx ON ${F_TABLE} USING gin(data);"
process_log "creating index on postgreSQL collections."
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}" \
>/dev/null
}
################################################################################
# function: delete_json_data delete json data in PG
################################################################################
function delete_json_data ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_COLLECTION="$6"
process_log "droping json object in postgresql."
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"TRUNCATE TABLE ${F_COLLECTION};" >/dev/null
}
################################################################################
# function: pg_copy_benchmark
################################################################################
function pg_copy_benchmark ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_JSONFILE="$7"
typeset -r F_COPY="COPY ${F_COLLECTION} FROM STDIN;"
DBEXISTS=$(if_dbexists "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" \
"${F_PGUSER}" "${F_PGPASSWORD}")
process_log "loading data in postgresql using ${F_JSONFILE}."
start_time=$(get_timestamp_nano)
cat ${F_JSONFILE}|run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" \
"${F_PGUSER}" "${F_PGPASSWORD}" "${F_COPY}"
end_time=$(get_timestamp_nano)
total_time="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
echo "${total_time}"
}
################################################################################
# function: benchmark postgresql inserts
################################################################################
function pg_inserts_benchmark ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_INSERTS="$7"
process_log "inserting data in postgresql using ${F_INSERTS}."
start_time=$(get_timestamp_nano)
run_sql_file "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_INSERTS}"
end_time=$(get_timestamp_nano)
total_time="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
echo "${total_time}"
}
################################################################################
# function: benchmark postgresql select
################################################################################
function pg_select_benchmark ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_COLLECTION="$6"
typeset -r F_SELECT1="SELECT data
FROM ${F_COLLECTION}
WHERE (data->>'brand') = 'ACME';"
typeset -r F_SELECT2="SELECT data
FROM ${F_COLLECTION}
WHERE (data->>'name') = 'Phone Service Basic Plan';"
typeset -r F_SELECT3="SELECT data
FROM ${F_COLLECTION}
WHERE (data->>'name') = 'AC3 Case Red';"
typeset -r F_SELECT4="SELECT data
FROM ${F_COLLECTION}
WHERE (data->>'type') = 'service';"
local START end_time
process_log "testing FIRST SELECT in postgresql."
start_time=$(get_timestamp_nano)
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"${F_SELECT1}" >/dev/null || exit_on_error "failed to execute SELECT 1."
end_time=$(get_timestamp_nano)
total_time1="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing SECOND SELECT in postgresql."
start_time=$(get_timestamp_nano)
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"${F_SELECT2}" >/dev/null || exit_on_error "failed to execute SELECT 2."
end_time=$(get_timestamp_nano)
total_time2="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing THIRD SELECT in postgresql."
start_time=$(get_timestamp_nano)
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"${F_SELECT3}" >/dev/null || exit_on_error "failed to execute SELECT 3."
end_time=$(get_timestamp_nano)
total_time3="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
process_log "testing FOURTH SELECT in postgresql."
start_time=$(get_timestamp_nano)
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" \
"${F_SELECT4}" >/dev/null || exit_on_error "failed to execute SELECT 4."
end_time=$(get_timestamp_nano)
total_time4="$(get_timestamp_diff_nano "${end_time}" "${start_time}")"
AVG=$(( ($total_time1 + $total_time2 + $total_time3 + $total_time4 )/4 ))
echo "${AVG}"
}
################################################################################
# function: mk_pgjson_collection create json table in PG
################################################################################
function analyze_collections ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_TABLE="$6"
typeset -r F_SQL="VACUUM FREEZE ANALYZE ${F_TABLE};"
process_log "performing analyze in postgreSQL."
run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}" \
>/dev/null 2>/dev/null
}
################################################################################
# function: mk_pgjson_collection create json table in PG
################################################################################
function pg_version ()
{
typeset -r F_PGHOST="$1"
typeset -r F_PGPORT="$2"
typeset -r F_DBNAME="$3"
typeset -r F_PGUSER="$4"
typeset -r F_PGPASSWORD="$5"
typeset -r F_SQL="select split_part(version(),' ',2);"
version=$(run_sql "${F_PGHOST}" "${F_PGPORT}" "${F_DBNAME}" "${F_PGUSER}" \
"${F_PGPASSWORD}" "${F_SQL}")
echo "${version}"
}
================================================
FILE: output/sample_expected.out
================================================
PID: 1251 [RUNTIME: 07-21-14 23:21:24] pg_nosql_benchmark: MongoDB Version 2.6.3
PID: 1251 [RUNTIME: 07-21-14 23:21:24] pg_nosql_benchmark: PostgreSQL Version 9.4beta1
PID: 1251 [RUNTIME: 07-21-14 23:21:24] pg_nosql_benchmark: creating json data.
PID: 1251 [RUNTIME: 07-21-14 23:21:25] pg_nosql_benchmark: preparing postgresql INSERTs.
PID: 1251 [RUNTIME: 07-21-14 23:21:26] pg_nosql_benchmark: preparing mongo insert commands.
PID: 1251 [RUNTIME: 07-21-14 23:21:27] pg_nosql_benchmark: droping database benchmark if exists.
PID: 1251 [RUNTIME: 07-21-14 23:21:27] pg_nosql_benchmark: creating database benchmark.
PID: 1251 [RUNTIME: 07-21-14 23:21:27] pg_nosql_benchmark: dropping mongo collection json_tables
PID: 1251 [RUNTIME: 07-21-14 23:21:27] pg_nosql_benchmark: creating json_tables collection in postgreSQL.
PID: 1251 [RUNTIME: 07-21-14 23:21:28] pg_nosql_benchmark: loading data in postgresql using sample.json.
PID: 1251 [RUNTIME: 07-21-14 23:21:28] pg_nosql_benchmark: droping json object in postgresql.
PID: 1251 [RUNTIME: 07-21-14 23:21:28] pg_nosql_benchmark: inserting data in postgresql using sample_pg_inserts.json.
PID: 1251 [RUNTIME: 07-21-14 23:21:28] pg_nosql_benchmark: creating index on postgreSQL collections.
PID: 1251 [RUNTIME: 07-21-14 23:21:28] pg_nosql_benchmark: testing mongoimport.
PID: 1251 [RUNTIME: 07-21-14 23:21:29] pg_nosql_benchmark: dropping mongo collection json_tables
PID: 1251 [RUNTIME: 07-21-14 23:21:29] pg_nosql_benchmark: testing inserts in mongo
PID: 1251 [RUNTIME: 07-21-14 23:21:30] pg_nosql_benchmark: creating index in mongodb.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing FIRST SELECT in postgresql.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing SECOND SELECT in postgresql.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing THIRD SELECT in postgresql.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing FOURTH SELECT in postgresql.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: calculating PostgreSQL collection size.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing mongo FIRST SELECT.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing mongo SECOND SELECT.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing mongo THIRD SELECT.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: testing mongo FOURTH SELECT.
PID: 1251 [RUNTIME: 07-21-14 23:21:31] pg_nosql_benchmark: calculating the size of mongo collection.
number of rows 1000
PG COPY (ns) 107488390
PG INSERT (ns) 514707930
PG SELECT (ns) 24235127
PG SIZE (bytes) 1490944
MONGO IMPORT (ns) 494436900
MONGO INSERT (ns) 1666742540
MONGO SELECT (ns) 80460677
MONGO SIZE (bytes) 5578752
================================================
FILE: pg_nosql_benchmark
================================================
#/bin/bash
#################################################################################
# Copyright (c) 2013-2014, EnterpriseDB Corporation
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation and/or
# other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#===============================================================================
#title : pg_nosql_benchmark.
#description : This script will help in benchmarking PostgreSQL (JSONB) and
# : MongoDB (BSON).
#author : Vibhor Kumar (vibhor.aim@gmail.com).
#date : July 17, 2014
#version : 2.0.1
#usage : bash pg_nosql_benchmark
#notes : Install Vim and Emacs to use this script.
#bash_version : GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
#===============================================================================
################################################################################
# set require variables.
################################################################################
DIRECTORY=$(dirname $0)
BASENAME=$(basename $0)
PGHOME="/usr/ppas-9.3"
PGHOST="172.17.0.4"
PGPORT="5432"
PGUSER="postgres"
PGPASSWORD="postgres"
PGDATABASE="benchmark"
PGBIN="/usr/ppas-9.3/bin"
################################################################################
# set mongo variables.
################################################################################
MONGO="/usr/bin/mongo"
MONGOIMPORT="/usr/bin/mongoimport"
MONGOHOST="172.17.0.3"
MONGOPORT="27017"
MONGOUSER="mongo"
MONGOPASSWORD="mongo"
MONGODBNAME="benchmark"
COLLECTION_NAME="json_tables"
SAMPLEJSON="sample.json"
PG_INSERTS="sample_pg_inserts.json"
MONGO_INSERTS="sample_mongo_inserts.json"
export PATH=$PGBIN:$PATH
################################################################################
# source library files
################################################################################
source ${DIRECTORY}/lib/pg_func_lib.sh
source ${DIRECTORY}/lib/mongo_func_lib.sh
################################################################################
# declare require arrays
################################################################################
declare -a json_rows=(10000000)
declare -a pg_size_time
declare -a pg_copy_time
declare -a pg_inserts_time
declare -a pg_select_time
# mongo specific arrays
declare -a mongo_size_time
declare -a mongo_copy_time
declare -a mongo_inserts_time
declare -a mongo_select_time
################################################################################
# main function
################################################################################
mongodb_version=$(mongo_version "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}"
)
pg_version=$(pg_version "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}"
)
process_log "MongoDB Version $mongodb_version"
process_log "PostgreSQL Version $pg_version"
for (( indx=0 ; indx < ${#json_rows[@]} ; indx++ ))
do
generate_json_rows "${json_rows[${indx}]}" \
"${SAMPLEJSON}"
pg_json_insert_maker "${COLLECTION_NAME}" \
"${json_rows[${indx}]}" \
"${PG_INSERTS}"
mongo_json_insert_maker "${COLLECTION_NAME}" \
"${json_rows[${indx}]}" \
"${MONGO_INSERTS}"
remove_pg_db "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}"
create_pg_db "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}"
drop_mongocollection "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}"
mk_pg_json_collection "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}"
pg_copy_time[${indx}]=$(pg_copy_benchmark "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}" \
"${SAMPLEJSON}"
)
pg_create_index_collection "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}"
mongo_copy_time[${indx}]=$(mongodb_import_benchmark "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}" \
"${SAMPLEJSON}"
)
mongodb_create_index "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}"
pg_select_time[${indx}]=$(pg_select_benchmark "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}"
)
pg_size_time[${indx}]=$(pg_relation_size "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}"
)
mongo_select_time[${indx}]=$(mongodb_select_benchmark "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}"
)
mongo_size_time[${indx}]=$(mongo_collection_size "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}"
)
drop_mongocollection "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}"
mongo_inserts_time[${indx}]=$(mongodb_inserts_benchmark "${MONGOHOST}" \
"${MONGOPORT}" \
"${MONGODBNAME}" \
"${MONGOUSER}" \
"${MONGOPASSWORD}" \
"${COLLECTION_NAME}" \
"${MONGO_INSERTS}"
)
delete_json_data "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}"
pg_inserts_time[${indx}]=$(pg_inserts_benchmark "${PGHOST}" \
"${PGPORT}" \
"${PGDATABASE}" \
"${PGUSER}" \
"${PGPASSWORD}" \
"${COLLECTION_NAME}" \
"${PG_INSERTS}"
)
done
print_result "number of rows" "${json_rows[@]}"
print_result "PG COPY (ns)" "${pg_copy_time[@]}"
print_result "PG INSERT (ns)" "${pg_inserts_time[@]}"
print_result "PG SELECT (ns)" "${pg_select_time[@]}"
print_result "PG SIZE (bytes)" "${pg_size_time[@]}"
print_result "MONGO IMPORT (ns)" "${mongo_copy_time[@]}"
print_result "MONGO INSERT (ns)" "${mongo_inserts_time[@]}"
print_result "MONGO SELECT (ns)" "${mongo_select_time[@]}"
print_result "MONGO SIZE (bytes)" "${mongo_size_time[@]}"
rm -rf ${SAMPLEJSON}*
rm -rf ${PG_INSERTS}
rm -rf ${MONGO_INSERTS}
gitextract_yk5gzwpp/ ├── LICENSE ├── README.md ├── lib/ │ ├── common_func_lib.sh │ ├── mongo_func_lib.sh │ └── pg_func_lib.sh ├── output/ │ └── sample_expected.out └── pg_nosql_benchmark
Condensed preview — 7 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (83K chars).
[
{
"path": "LICENSE",
"chars": 1307,
"preview": "pg_nosql_benchmark\n\nCopyright (c) 2013-2014, EnterpriseDB Corporation\n\nRedistribution and use in source and binary forms"
},
{
"path": "README.md",
"chars": 3624,
"preview": "pg_nosql_benchmark\n==================\n\nThis is tool for benchmarking Postgres (JSONB) and MongoDB (BSON)\n\nIntroduction\n-"
},
{
"path": "lib/common_func_lib.sh",
"chars": 34528,
"preview": "#!/bin/bash\n\n################################################################################\n# Copyright EnterpriseDB C"
},
{
"path": "lib/mongo_func_lib.sh",
"chars": 11622,
"preview": "#!/bin/bash\n\n################################################################################\n# Copyright EnterpriseDB C"
},
{
"path": "lib/pg_func_lib.sh",
"chars": 14313,
"preview": "#!/bin/bash\n\n################################################################################\n# Copyright EnterpriseDB C"
},
{
"path": "output/sample_expected.out",
"chars": 2846,
"preview": "PID: 1251 [RUNTIME: 07-21-14 23:21:24] pg_nosql_benchmark: MongoDB Version 2.6.3\nPID: 1251 [RUNTIME: 07-21-14 23:21:24] "
},
{
"path": "pg_nosql_benchmark",
"chars": 11472,
"preview": "#/bin/bash\n\n#################################################################################\n# Copyright (c) 2013-2014,"
}
]
About this extraction
This page contains the full source code of the EnterpriseDB/pg_nosql_benchmark GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 7 files (77.8 KB), approximately 18.8k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.