ODOO (Open ERP) AWS setup notes for your unforeseen startup


Setup ODOO on EC2 & RDS (Insecure)


Go to https://www.odoo.com and make sure you understand the needs for an ERP system at your startup. I would advice that you evaluate the pros & cons for using an ERP before jumping into this bureaucratic complication.

Security is prime and make sure you realize that I have not covered any aspect of securing the instance or the application here.
1. Intallation pre-requisites

a. Initialize a RDS PostgreSql service on AWS.

b. Initialize an EC2 instance with Amazon Linux in the same security group as RDS

yum install git libtool zlib devel automake pkgconfig gcc c++ curl make gcc-c++ libxml2-devel rsync
yum install openldap-devel libjpeg-devel python-devel vim
yum -y install babel libxslt-python pyparsing python-dateutil python-decorator python-docutils python-feedparser python-imaging python-jinja2 python-ldap python-lxml python-mako python-mock python-openid python-passlib python-psutil python-psycopg2 python-reportlab python-requests python-simplejson python-unittest2 python-vobject python-werkzeug python-yaml pytz
yum install icu xorg-x11-fonts-75dpi freetype freetype-devel 

> Setup Node
yum install nodejs npm
npm install -g less less-plugin-clean-css
ln -s /usr/local/bin/lessc /usr/bin/lessc
ln -s /usr/bin/nodejs /usr/bin/node

> CentOS 6
yum localinstall http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm
wget https://bitbucket.org/wkhtmltopdf/wkhtmltopdf/downloads/wkhtmltox-0.13.0-alpha-7b36694_linux-centos6-amd64.rpm
rpm -ivh wkhtmltox-0.13.0-alpha-7b36694_linux-centos6-amd64.rpm

> CentOS 7
yum localinstall https://yum.postgresql.org/9.3/redhat/rhel-7-x86_64/pgdg-redhat93-9.3-3.noarch.rpm
wget http://download.gna.org/wkhtmltopdf/0.12/
rpm -Uvh wkhtmltox-

yum install postgresql93-contrib postgresql93-devel postgresql93-plpython27
Now lets get the odoo source on the machine to start the setup
git clone https://github.com/odoo/odoo.git
Go to your odoo directory and install the python packages
easy_install -U setuptools
easy_install pip
pip install -r requirements.txt

2. Create a config file openerp-server.conf with the below content

; This is the password that allows database operations:
; admin_passwd = admin
db_host = admindb.xxxxxxx.us-west-2.rds.amazonaws.com
db_port = 5432
db_user = admin
db_password = admin123
addons_path = /usr/lib/python2.7/dist-packages/openerp/addons
logfile = /tmp/odoo-server.log
xmlrpc_port = 8069

Setup a RDS with postgres database server

psql -h admindb.xxxxxxx.us-west-2.rds.amazonaws.com -U admin admindb


3. Run the server with the configurations

chown -R ec2-user odoo 
./odoo-bin --addons-path=addons --config=openerp-server.conf

4. Configure the EC2 Instance

a.) IP routing on the instance. Not secure, but will provide the basic routing to start your system.

iptables -t nat -A OUTPUT -p tcp -d ec2-xxx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com --dport 80 -j REDIRECT --to-port 8069
iptables -t nat -I PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-ports 8069
Make sure you verify the iptables. 
sudo iptables -t nat -L -v

target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 8069
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere ip-xxx-xxx-xxx-xxx.us-west-2.compute.internal tcp dpt:http redir ports 8069
target prot opt source destination


b.) Security group config for port 80. Add exception for incoming HTTP traffic
5. Run the ODOO server as a service

echo "Killing any previous instance of odoo"
ps ax | grep odoo | grep -v grep | awk '{print $1}' | xargs sudo kill
echo "Starting the Odoo server"
nohup ./odoo-bin --addons-path=addons --config=openerp-server.conf -d oodb -u all &> /dev/null & disown
echo "Conf : ~/odoo/odoo/openerp-server.conf"
echo "Log : tail -f /tmp/odoo-server.log"

6. Login as admin / admin


Simplify Tomcat/JVM Monitoring with Mission Control (JMX) or VisualVM

A. JMX Mission Control

Oracle Java Mission Control enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data collected for normal adaptive dynamic optimization of the Java Virtual Machine (JVM). Besides minimizing the performance overhead, this approach eliminates the problem of the observer effect, which occurs when monitoring tools alter the execution characteristics of the system.

1. Server setup

> Provide the JMX configuration to Tomcat server 
> Create a setenv.sh file in $CATALINA_HOME/bin 
> Add the following entry to the script file 
   export CATALINA_OPTS="-Dcom.sun.management.jmxremote=true \
-Dcom.sun.management.jmxremote.port=3614 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
> This will enable JMX listener on port 3614 when tomcat is restarted
> Make sure that this port is open and accessible to outside world. 
  This may have security concerns hence its not advisable for production environment.
> Restart the server to allow the properties to be set and initialized.

2. Mission Control setup

Download: mission control
In my test I had used an eclipse plugin available at http://download.oracle.com/technology/products/missioncontrol/updatesites/base/5.5.0/eclipse/
> Just added this plugin to the eclipse using Install new Software
> Launch a new connection to the JVM and provide the IP and port on which the jmx remote system is running.



B. Alternate way is to use VisualVM

VisualVM is a visual tool integrating several commandline JDK tools and lightweight profiling capabilities.

Here as well we need to start jstatd daemon on the server which opens up connections for the visualvm client and is packaged with the JDK.

Download: http://visualvm.java.net/download.html
Reference: https://visualvm.java.net/applications_remote.html

1. Start the jstatd daemon

> Make sure the default RMI port is open as per the javase documentation
> Create a policy file named jstatd.all.policy and copy the following content to it
  grant codebase "file:${java.home}/../lib/tools.jar" {
  permission java.security.AllPermission;

> Start the daemon 
  jstatd -J-Djava.security.policy=jstatd.all.policy

> Alternate option to run this silently
  nohup jstatd -J-Djava.security.policy=jstatd.all.policy &>/dev/null &

2. Start the VisualVM Client

> Start the Visual VM client and add remote host using its IP
> You will be able to monitor the jvm on that machine




Java Integration with Kafka distributed message broker

Kafka is a major distributed, partitioned, replicated, commit log service used as a message  broker in the current tech industry open-sourced by Linked-In. It functions as a central messaging bus for various domains and frameworks specializing in big-data systems.

It works as a broker between the producers and consumers with a strong order guarantee than any traditional messaging system. It provides a single consumer abstraction model that generalizes the queuing and publish-subscribe model as a Consumer-Group.

Reference: http://kafka.apache.org/documentation.html#introduction

Github: https://github.com/vishwakarmarhl/kafka-sample

The API is provided by Kafka-clients version 

Here I am going through a sample that is capable of publishing a message over the kafka topic. There is a listener constantly waiting with a consumer group that works as a thread pool subscribed to the topic.

A. Setting up kafka is as simple as extracting the archive and starting the server.

Reference: http://kafka.apache.org/090/documentation.html

Download: https://www.apache.org/dyn/closer.cgi?path=/kafka/

Say you have a virtual machine as per my setup and the host IP as

  1. Download and extract the above kafka-2.11-0.9 build in /opt folder, and set the KAFKA_HOME variable
  2. Configure $KAFKA_HOME/config/server.properties

kafka broker internals

If you are into spring and java then this is a great introduction resource from SpringDeveloper. However in this tutorial I am covering a very basic implementation.


B. Walk-through of the Producer and Listener


The code has two main classes responsible for the send (KafkaSpeaker.java) and listen (KafkaListener.java) functionality. KafkaMain is responsible for launching the KafkaListener in a pre-configured consumer threadpool.


The producer is configured with the broker url and client identifier.

//The kafka producer is configured with the topic and broker address for sending the message as follows: 

 Properties props = new Properties();
 props.put("bootstrap.servers", prop.getProperty("producer.url"));
 props.put("client.id", prop.getProperty("producer.client.id"));
 props.put("acks", "all");
 props.put("retries", 0);
 props.put("batch.size", 16384);
 props.put("linger.ms", 1);
 props.put("buffer.memory", 33554432);
 props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
 props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
 this.producer = new KafkaProducer<>(props);

The producer can send the message on a topic using the send api

producer.send(new ProducerRecord<String, String>(topic,



The listener has the concept of consumer group that can listen on each partition and transact on the messages queued for consumption and further processing.


The consumer group is created using the threadpool executors as follows,

final ExecutorService executor = Executors.newFixedThreadPool(numConsumers);
final List<KafkaListener> consumers = new ArrayList<>();
for (int i = 0; i < numConsumers; i++) {
 KafkaListener consumer = new KafkaListener(i, prop, topics);

Configure the listener and create a consumer

props.put("bootstrap.servers", prop.getProperty("consumer.url"));
props.put("group.id", prop.getProperty("consumer.group.id")); 
props.put("client.id", prop.getProperty("consumer.client.id"));
props.put("enable.auto.commit", "true");
props.put("auto.commit.interval.ms", "2000");
props.put("session.timeout.ms", "30000");
props.put("key.deserializer", StringDeserializer.class.getName());
props.put("value.deserializer", StringDeserializer.class.getName());
this.consumer = new KafkaConsumer<>(props);
this.timeOut = Integer.parseInt( prop.getProperty("consumer.poll.timeout"));

Subscribe and listen infinitely on the topic until the shutdown hook is executed for this thread pool.

while (true) {
 ConsumerRecords<String, String> records = consumer.poll(this.timeOut);
 for (ConsumerRecord<String, String> record : records) {
    log.info(record.topic()+" - "+this.id + ", Data: " + record.value());


The code available in github is self-sufficient and simple enough and was very simple to create.

Thanks to the great article that set me of on this trial project.

Spring Rest API and Mongodb with GridFS

Reference: https://github.com/vishwakarmarhl


Hello draft of the Mongo DB based file data store service


There are two more branches apart from the master branch which have the file upload functionality

  1. Checkout the branch
  2. Pull the current branch
    git pull
  3. After making changes in any of them index and commit
    git add .
    git commit -m “Updated the code and added a message”
  4. Push changes to the github repository
    git push origin FileUploadGridFSSpring
    git push origin MultiPartFileUpload


Download : http://www.mongodb.org/dr/downloads.mongodb.org/win32/mongodb-win32-x86_64-2.4.5.zip/download

  1. Unzip the zip contents in C:\mongodb\
  2. create C:\data\db
  3. Execute the C:\mongodb\bin\mongod.exe –dbpath C:\data\db


//The following command simply pre-allocates a 2 gigabyte, uncapped collection named people. db.createCollection(“files”, { size: 2147483648 }) db.files.save( { fileId: ‘1235’, fileName: ‘V_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123342, fileExtensionType: ‘EXE’ })


  1. Start the mongod.exe standalone server
  2. Import the source as a maven project in Eclipse STS IDE
  4. Deploy on a tomcat instance to see the data from mongodb
  5. An alternate plugin for tomcat enables maven based initialization. mvn tomcat7:run
  6. Open up http://localhost:8088/RDataServe to checkout the grid for file data


Mongo Shell Cmds

show dbs show collections

//This command creates a collection named file with a maximum size of 5 megabytes and a maximum of 5000 documents. db.createCollection(“files”, { capped : true, size : 5242880, max : 5000 } )

//The following command simply pre-allocates a 2 gigabyte, uncapped collection named people. db.createCollection(“files”, { size: 2147483648 })

//Drop a collection capped db.files.drop()

//Insert db.files.insert( { _id: 1, fileId: 1234, fileName: ‘R_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123412, fileExtensionType: ‘EXE’ })

db.files.save( { fileId: ‘1235’, fileName: ‘V_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123342, fileExtensionType: ‘EXE’ })

//Query db.files.find({fileId:1234})

vRODBC setup for DistributedR and Vertica database cluster


In this particular case there is vertica database server installed with Vertica-R-Udtf setup. The R runtime is being provided from within vertica hence the environment variables changed as below.

Reference: http://www.vertica.com/wp-content/uploads/2014/06/vRODBC-Installation-Guide.pdf

1. Add the following variables in the .bashrc of root for library linking and source it.

export R_HOME=/opt/vertica/R
export DR_HOME=/opt/hp/distributedR
export LD_LIBRARY_PATH=/usr/local/lib/:/opt/vertica/lib64/:/usr/lib/jvm/java-1.6.0-openjdk-$DR_HOME/lib:$DR_HOME/third_party/lib:$DR_HOME/third_party/lib/atomicio/:$LD_LIBRARY_PATH

2. Install using the  DistributedR-master/third_party/unixODBC-2.3.1.tar.gz

$ tar -xvzf unixODBC-2.3.1.tar.gz
$ cd unixODBC-2.3.1
$ ./configure && make
$ make install

3. Install the vRODBC available in the DistributedR-master/ source folder


4. Create the config files for vRODBC

# Create a directory for the config files
$ mkdir /home/dbadmin/vRODBC/

5. Add the following content to this folder in vertica.ini

 DriverManagerEncoding = UTF-16
 ODBCInstLib = /usr/local/lib/libodbcinst.so
 ErrorMessagesPath = /opt/vertica/lib64
 LogLevel = 0
 LogPath = /tmp

6. Add the following content to this folder in odbc.ini

 Description = vRODBC test client
 Driver = /opt/vertica/lib64/libverticaodbc.so
 Database = test
 Servername =
 UserName = dbadmin
 Password = test
 Port = 5433
 ConnSettings =
 Locale = en_US

7. Use the following environment variables in the users .bashrc and source it

 export R_HOME=/opt/vertica/R
 export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-
 export DR_HOME=/opt/hp/distributedR
 export PATH=/opt/vertica/bin:/opt/vertica/R/bin:$DR_HOME/bin:$PATH
 export LD_LIBRARY_PATH=/usr/local/lib/:/opt/vertica/lib64/:/usr/lib/jvm/java-1.6.0-openjdk-$DR_HOME/lib:$DR_HOME/third_party/lib:$DR_HOME/third_party/lib/atomicio/:$LD_LIBRARY_PATH

 # ODBC Configurations
 export VERTICAINI=/home/dbadmin/vRODBC/vertica.ini
 export ODBCINI=/home/dbadmin/vRODBC/odbc.ini

8. Open the R console and test the connections using the following script

connect <- odbcConnect("Test")
segment <- sqlQuery(connect, "SELECT * FROM role")

Setup open source Distributed R on a three node cluster with R and execute tests on workers


———————-              DISTRIBUTED R              ————————-



  1. Pre requisite packages
#Install dependencies
$ sudo yum install libtool zlib devel automake pkgconfig gcc c++ curl 
$ sudo yum install -y make gcc gcc-c++ libxml2-devel rsync

# Install R
$ curl -O http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ sudo rpm -i epel-release-latest-7.noarch.rpm
$ sudo yum update
$ sudo yum install R R-devel

2. Move the installation archive to target. copied from github https://github.com/vertica/DistributedR

scp -r disvert.tar disvert@
ssh disvert@

3. Remove any older version of the package and verify R installation

# Connect to the R console and make sure to remove any old versions

# Go to the source code of Distrib R and make clean 
make clean
whereis R
make distclean

# Remove any old installation
rm -rf /opt/hp/distributedR/

4. Update the environment for execution. This can be done towards the end of the installation.

Make sure you have password-less access to other nodes to your cluster nodes.

# Add the R runtime to the path bin just in case its installed separately 
ln -s /opt/disvert/R/bin/R /bin/R
ln -s /opt/disvert/R/bin/R /sbin/R
# Update the environment variables in ~/.bashrc file for the libraries and executables path
export R_HOME=/opt/disvert/R
export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-
export DR_HOME=/opt/hp/distributedR
export PATH=/opt/disvert/bin:/opt/disvert/R/bin:$DR_HOME/bin:$PATH
export LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-$DR_HOME/lib:$DR_HOME/third_party/lib:$DR_HOME/third_party/lib/atomicio/:$LD_LIBRARY_PATH

5. Install the following from the $DR_HOME/third party lib folder of the github distribution.

Press tab to autocomplete the version as per the package archive name in the folder.

R CMD INSTALL randomForest_
R CMD INSTALL data.table_

6. Build dependencies. Go to the DistributedR-master/third_party/ directory and make -j4

make -j4 all

7. Build and install the actual code in the DistributedR-master

make -j4
make install

8. Test the library execution in the R console

distributedR_start() # start DR
getpartition(B) # collect darray data
distributedR_shutdown() # stop DR

9. Cluster configuration for the nodes are available at /opt/hp/distributedR/conf/cluster_conf.xml

  • node0001 =,/home/disvert
  • node0002 =,/home/disvert
  • node0003 =,/home/disvert

Following configuration is for the node0001 and will be replicated on other nodes with the server info configuration.


This will get you started on the distributed R tests. I hope such a cluster configuration will be handy for any data crunching that you may want to do with R.

Secure an instance with SSH RSA based on a public and private key pair for access

Secure a public instance with SSH-2 RSA. This would be done for provisioning private key based SSH access to a user test on the remote machine.


A larger reference is available in the above link. I have tried the multi node password less test user setup.

  1. Create the user for secure access on the remote machine. In case its a cluster then on all the nodes of the cluster.
 $ adduser test
 $ passwd  test

2. Generate a public(id_rsa.pub)/private(id_rsa) KeyPair without a pass. The path shall be /home/test/.ssh

 $ ssh-keygen -t rsa

3. Add the public key string which looks something like below, in id_rsa.pub to the OpenSSH keys file at /home/test/.ssh/authorized_keys

ssh-rsa AAAAB3NzaC1 ... ... ...3PGVu4D+37RA0expQUJX1p++JtLlaw== rsa-key-20150623-test-user

4. Move the generated keys id_rsa, id_rsa.pub and authorized_keys to all the test nodes we want password-less access to.

Make sure the files are readable with the right permissions on the remote machines
 chmod 700 ~/.ssh
 chmod 600 ~/.ssh/*

5. Access the nodes from another machine with the private key in the .ssh folder

$ ssh -v test@ # should log in without a password

6. Share the key with another user in a ppk file for putty based access.
Move the private key to another system and use windows puttygen to load it and save the private key as a ppk file . Use this private key file in pageant to access this instance as team user