Repository Management with Nexus 3 for your Mavenized project, including release and snapshot distribution

Like the Nexus documentation says;

Stop developing in the Dark Ages, read this book, and start using a repository manager. Trust us, once you start using a Nexus Repository Manager, you’ll wonder how you ever functioned without it.

Reference:

A. Download the archive from https://www.sonatype.com/download-oss-sonatype

B. Unzip it into a folder and run it as follows

cd ~\nexus\nexus-3.3.1-01\bin
nexus.exe /run 

If the log shows the following that means the server is up  
-------------------------------------------------
 Started Sonatype Nexus OSS 3.3.1-01
-------------------------------------------------

C. Server starts by default on http://localhost:8081

username: admin  
password: admin123 
Use the above credentials to login as the default administrator

D. Add the following configuration to the ~\USER_HOME\.m2\settings.xml

Make sure you remove the code tags before using this configuration, which is used here for wordpress content formatting only.

<settings>
	  <mirrors>
		<mirror>
		  <!--This sends everything else to /public -->
		  <id>nexus</id>
		  <mirrorOf>*</mirrorOf>
		  <url>http://localhost:8081/repository/maven-public/</url>
		</mirror>
	  </mirrors>
	  <profiles>
		<profile>
		  <id>nexus</id>
		  <!--Enable snapshots for the built in central repo to direct -->
		  <!--all requests to nexus via the mirror -->
		  <repositories>
			<repository>
			  <id>central</id>
			  <url>http://central</url>
			  <releases><enabled>true</enabled></releases>
			  <snapshots><enabled>true</enabled></snapshots>
			</repository>
		  </repositories>
		 <pluginRepositories>
			<pluginRepository>
			  <id>central</id>
			  <url>http://central</url>
			  <releases><enabled>true</enabled></releases>
			  <snapshots><enabled>true</enabled></snapshots>
			</pluginRepository>
		  </pluginRepositories>
		</profile>
	  </profiles>
	  <activeProfiles>
		<!--make the profile active all the time -->
		<activeProfile>nexus</activeProfile>
	  </activeProfiles>
	   <servers>
		<server>
		  <id>nexus</id>
		  <username>admin</username>
		  <password>admin123</password>
		</server>
	  </servers>
	</settings>

E. Release and snapshot artifacts should be configured in the projects pom as distributionManagement

  <distributionManagement>
    <repository>
      <id>nexus</id>
      <name>Releases</name>
      <url>http://localhost:8081/repository/maven-releases</url>
    </repository>
    <snapshotRepository>
      <id>nexus</id>
      <name>Snapshot</name>
      <url>http://localhost:8081/repository/maven-snapshots</url>
    </snapshotRepository>
  </distributionManagement>

F. The clean and deploy goal in your Java project will build and upload the artifacts to the repository using the server credentials tag from settings.xml

mvn clean deploy -DskipTests

NexusRepository

G. Add a proxy repository

You can add a new proxy repository to your Nexus instance using the following steps

  1. Create a repository from the repositories admin page
  2. Select the maven2 recipe since JBOSS is a maven like repository
  3. Provide a name like “jboss-nexus-repository”
  4. Add this repository to the group you have defaulted your maven to, so that maven can use this as a part of the group it is defaulted to.

NexusProxyRepository

H. Adding your custom jars into the repository

  1. Create a repository with maven2 hosted recipe
  2. Obtain the created repository URL and run the following maven deploy command on your jar file
 mvn deploy:deploy-file 
-Durl=http://localhost:8081/repository/project-customs/ 
-DrepositoryId=nexus -DgroupId=com.oracle 
-DartifactId=ojdbc6 -Dversion=11.2.0.4 
-Dpackaging=jar -Dfile=C:/Users/vishwaka/.m2/ojdbc6.jar 
-DgeneratePom=true
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar (1942 KB at 583.2 KB/sec)
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom (392 B at 0.1 KB/sec)
Downloading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml
Downloaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml (302 B at 0.2 KB/sec)
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml (302 B at 0.1 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.768 s
[INFO] Finished at: 2017-06-15T11:29:59-07:00
[INFO] Final Memory: 11M/245M
[INFO] ------------------------------------------------------------------------

NexusHostedRepository

You should be able to see this in your repositories assets once the upload is successful. The upload deploy uses credentials from your server.xml configuration so make sure that is available.

3. Upon doing this we need to add the project-custom repository as a member to the maven-public group of repositories

NexusMemberRepository

I. Test by running a clean build of your maven project

  • Delete the folder containing the jar files in the path \.m2\repository\com\oracle\ojdbc6\11.2.0.4
  • Rerun the maven build using mvn clean compile
  • Verify the following logs in the build
Downloading: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom
Downloaded: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom (392 B at 2.8 KB/sec)
Downloading: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar
Downloaded: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar (1942 KB at 10846.1 KB/sec)

Continuous Integration in Pipeline as Code Environment with Jenkins, JaCoCo, Nexus and SonarQube

Github Link for the source code: https://github.com/vishwakarmarhl/jenkinstest

Here we discuss the setup for a Continuous integration pipeline. This is for mavenized Spring boot build with JaCoCo coverage reports and Sonar metrics. I used a windows machine with Tomcat 8 for hosting jenkins, but similar setup can be done on any OS where Sonar server can run on the same system.

A. Get the following artifacts on the system

  1. Tomcat server with Java JDK – Configure the server.xml to run on port 8099
  2. Setup Maven & other build utilities on your machine
  3. Access to Github source code
  4. Source code should have the Jenkinsfile in project root to be used by the pipeline
  5. Source should have the sonar-project.properties in project root for the SonarQube project linkage & source paths

JenkinsFile

Jenkinsfile and sonar-project.properties snapshot

B. Setup & Startup SonarQube

  1. Download the SonarQube package from https://www.sonarqube.org/#downloads
  2. Start sonar server: SONAR_HOME\bin\windows-x86-32\StartSonar.bat (for 32 bit Windows)
  3. Open Sonar admin page “http://localhost:9000“. Default credentials – admin/admin
  4. Create user in security tab and generate an access token, 50997f4a8c26d5698cccee30cf398c0ed9b98de0
  5. Create a project SPRINGBOOT with a key
  6. Download SonarQube scanner from https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner
  7. Additional configuration from https://docs.sonarqube.org/display/SCAN/Advanced+SonarQube+Scanner+Usages

C. Setup & Startup Tomcat

  1. Download jenkins.war from https://jenkins.io/download
  2. Put the jenkins.war file in webapps folder of Tomcat home
  3. Set Environment Variables as follows,
  4. SET JENKINS_HOME=”C:/Users/vishwaka/Documents/Workspace/git/jenkinstest/cisetup/jenkins_home”
  5. SET CATALINA_OPTS=”-DJENKINS_HOME=C:/Users/vishwaka/Documents/Workspace/git/jenkinstest/cisetup/jenkins_home”
  6. Start the server using startup.bat

JenkinsHome

Initial launch of Jenkins

D. Initialize Jenkins

  1. Access Jenkins at http://localhost:8099/jenkins
  2. Provide the initial credentials from jenkins_home/secrets/initialPassword*
  3. Install the default set of plugins and proceed
  4. Create a user for this installation
  5. Use “New Item” for creating a pipeline and provide the Jenkinsfile pipeline script from Git SCM for this

JenkinsCreatePipeline

Create pipeline project

E. Plugin & Configuration to Jenkins

  1. Add the “JaCoCo plugin” through the Manage Jenkins > Manage Plugins and install without restart
  2. Add “SonarQube Scanner for Jenkins” through the same Plugin Manager as above
  3. Go to the Manage Jenkins > Configure system and provide the credentials for Sonar Server
  4. Add the “SonarQube Server” name running on URL http://localhost:9000 alongwith user authentication key generated in SonarQube Server user administration page
  5. Remove the auto install option and add the “Sonar Scanner” env variable SONAR_RUNNER_HOME installation path as $JENKINS_HOME/sonar-scanner-3.0.3.778-windows through “Global Tool Configuration”
  6. Make sure the Sonar scanner path is configured properly as its path is hard coded in Jenkinsfile.

JenkinsGlobalProperties

Global Tool Configuration

F. Run the Build now for this pipeline

  1. The pipeline is at http://localhost:8099/jenkins/job/JENKINS-BOOT/JenkinsStatusPipeline
  2. Checkout the coverage report within the pipeline reports JenkinsJacoco
  3. You can also look at the Sonar reports at http://localhost:9000/dashboard?id=JENKINSBOOT JenkinsToSonar
  4. If you have many such projects then its better to execute all your Job Pipelines from a parent Job Pipeline. You can create one and call it “BUILD-ALL-JOBS”. It can be configured using the below pipeline script to run your JENKINS-BOOT job described in the example above as well as any other fictitious job call JENKINS-BOOT-XXX.
node {
    stage('JENKINS-BOOT-STAGE-A') {
        build job: 'JENKINS-BOOT'
    }
    stage('JENKINS-BOOT-STAGE-B') {
        build job: 'JENKINS-BOOT-XXX'
    }
}

There are plugins to build jobs in parallel as well but that depends on what workflow you want to build in your system.

G. Adding Nexus repository management capability to your CI environment from my blog

Click on the text link below:

Repository Management with Nexus 3 for your Mavenized project, including release and snapshot distribution

H. Finally put everything into a script that can run it all

Pardon my naive & careless script, considering my setup is on a local windows development workstation.

@echo off
echo "--------------------------------------------------------------------------"
echo "------------------------- CI STARTUP SCRIPT ------------------------------"
echo "--------------------------------------------------------------------------"

echo "Startup SonarQube Server"
echo "------------------------"
START CMD /C "cd c:\Dock\ci\sonar\sonarqube-6.4\bin\windows-x86-64 & CALL StartSonar.bat"
echo "Sonar may be up on http://localhost:9000/"

echo "Startup Nexus Repository Manager"
echo "--------------------------------"
START CMD /C "cd c:\Dock\ci\nexus\nexus-3.3.1-01\bin & nexus.exe /run"
echo "Nexus may be up on http://localhost:8081/"

echo "Startup Jenkins on Tomcat"
echo "-------------------------"
START CMD /C "cd c:\Dock\ci\jenkins\apache-tomcat-8.5.15\bin & startup.bat"
echo "Jenkins may be up on http://localhost:8099/jenkins"

echo "-------------------------------- END -------------------------------------"

 

Thanks.

 

Dockerification of your local virtual instance with SSH, XFCE & VNC

1. Docker: the client-server application made up of the Docker daemon, a REST API that specifies interfaces for interacting with the daemon, and a command line interface (CLI) client that talks to the daemon (through the REST API wrapper). Docker Engine accepts docker commands from the CLI, such as docker run , docker ps to list running containers, docker images to list images, and so on.

2. Docker Machine: a tool for provisioning and managing your Dockerized hosts (hosts with Docker Engine on them). Typically, you install Docker Machine on your local system. Docker Machine has its own command line client docker-machine and the Docker Engine client, docker. You can use Machine to install Docker Engine on one or more virtual systems.

engine-components-flow

We will be using Virtualbox based virtualization in docker which is supported on windows and mac.

A setup for linux instance is also available

I will be using docker toolbox as for my docker installation on windows.

 

1. Install the docker-toolbox using the default options and verify the versions from the Docker Quickstart Terminal.

$ docker-machine --version
  docker-machine.exe version 0.8.2, build e18a919

$ docker --version
  Docker version 1.12.2, build bb80604

 

2. Perfect, now we move to the docker machine toolbox which hosts the docker engine and give it a kickoff.

$ docker-machine rm default
$ docker-machine create --driver virtualbox --virtualbox-disk-size "500100" default
$ docker-machine start default
$ docker-machine env default
$ eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env default)

 

3. Lets work with docker engine CLI for management of the docker image

$ docker ps
$ docker info
$ docker --help
$ docker-machine ls
  NAME    ACTIVE DRIVER     STATE   URL                       SWARM DOCKER ERRORS
  default *      virtualbox Running tcp://192.168.99.100:2376       v1.12.3

 

4. Pull an image from docker hub and set it up locally

-- Verify the images that are downloaded, which  will be empty initially
$ docker images
-- Search for an image in docker hub and pull it
$ docker search ubuntu
$ docker pull ubuntu 
$ docker run ubuntu

 

5. Initialize and connect to bash for this image

-- Run a container and connect to its term
-- Also expose the ports and maps it to relevant exposed port in the image
$ docker run -it -p 52022:22 -p 52023:5900 -p 52021:80 ubuntu /bin/bash
-- Verify the version 
root@c:/# cat /etc/lsb-release
          apt-get update
          apt-get install -y build-essential openssh-server
          ip addr show
          service ssh restart

-- Exit from virtual host. This will also drop the changes if its uncommitted.
-- Keep this instance alive and go through the step 6 for persisting the changes.
root@c:/# exit

-- Get the IP from another console while keeping the image running 
$ docker ps
  CONTAINER ID IMAGE  COMMAND     CREATED       STATUS       PORTS NAMES
  dee57b8bba0e ubuntu "/bin/bash" 5 minutes ago Up 5 minutes 0.0.0.0:52022->22/tcp, 0.0.0.0:52021->80/tcp, 0.0.0.0:52023->5900/tcp sick_darwin 

$ docker inspect 
-- Look for the network configuration in the result
Obtained "IPAddress": "172.17.0.2"

 

6. Setup your new Ubuntu for vnc based desktop access

# apt-get update
-- Install the xfce desktop environment
# apt-get install -y build-essential xfonts-base xfce4 xfce4-goodies xubuntu-desktop
-- Install the vncserver
# apt-get install -y build-essential tightvncserver sudo vim openssh-server
# service ssh restart
-- Add a user for access to the instance
# adduser crusader
# usermod -aG sudo crusader
# su crusader
-- Initialize the VNC server for access
# export USER=crusader
# vncserver -geometry 1440x900 -rfbport 5900
# ps -eaf | grep vnc

 

7. Commit, persist and manage the image changes

-- Makes sure you commit the changes to docker and add a tag to it.
$ docker commit <container_id> vishwakarmarhl/ubshinydev:v01

-- Run a container and connect to its term
$ docker run -it -p 52022:22 -p 52023:5900 -p 52021:80 vishwakarmarhl/ubshinydev:v01 /bin/bash

-- Run the committed container image as a daemon, restart sshd and open bash
$ docker run -d -p 52022:22 -p 52023:5900 -p 52021:80 vishwakarmarhl/ubshinydev:v01 /bin/sh -c "while true; do echo hello world; sleep 10; done"
$ docker exec -it <container_id> /bin/bash
  # service ssh restart
 
-- Now you shoule be able to connect to the instance from the host
$ ssh -p 52022 crusader@192.168.99.100

-- Stop a container
$ docker stop

-- Remove Image by name. This will permanently delete your image
$ docker rmi -f ubuntu

 

8. Push Image to docker hub

Docker Hub Link : https://hub.docker.com/u/vishwakarmarhl

$ docker images
$ docker commit 162f8f8c5f19  vishwakarmarhl/ubunitydesk:v01
$ docker login
$ docker push vishwakarmarhl/ubunitydesk

-- Pull the images from docker hub
$ docker pull vishwakarmarhl/ubunitydesk

You should make sure you commit the changes done on this instance to the docker repository for persistence. This will be used to share at the docker hub repository.

 

9. Dockerfile for similar setup on Github

Download the docker file from the provided github link

Github Link: https://github.com/vishwakarmarhl/dockers/blob/master/Xubuntu-16-Desk-DockerFile

-- Build and run the docker container
$ docker build -t vishwakarmarhl/ubunitydesk:v01 . -f DockerFile 
$ docker run -it -p 52022:22 -p 52023:5900 -p 52021:80 vishwakarmarhl/ubunitydesk:v01 /bin/bash

Here upon you can configure your machine with any package. Will continue to describe how to use this environment for development purposes. This may as well be my notes but helps all the time for a quick reference.

Agilefant for your Agile ways

What is Agilefant?

Agilefant is a lean transformation tool that empowers your organization to execute change faster.

Setup on AWS EC2 & RDS(Mysql) Service instance

Referenced from the guide at https://github.com/Agilefant/agilefant/wiki/Agilefant-installation-guide

MySql RDS service with the following credentials.

mysql -h awdb.xxxxxxx.us-west-2.rds.amazonaws.com -u agilefant -p'agilefant123' agilefant

Deployment: http://ec2-xxxx.us-west-2.compute.amazonaws.com/

1.) Use this guide to setup tomcat server

Tomcat setup and deployment

2.) Setup the Database

Connect to your database

create database agilefant;
grant all on agilefant.* to agilefant@'%' identified by 'agilefant123';
exit;

 

3.) Configure the DB connection in Agilefant

Edit /WEB-INF/agilefant.conf in the agilefant.war package as

database {
 jndi-name = "jdbc/agilefant"
 driver-class = "com.mysql.jdbc.Driver"
 username = "agilefant"
 password = "agilefant123"
 url = "jdbc:mysql://awdb.xxxxxxx.us-west-2.rds.amazonaws.com:3306/agilefant?relaxAutoCommit=true&autoReconnect=true&useUnicode=true&characterEncoding=utf-8&autoReconnectForPools=true"
 }

 

4.) Start the server and verify the app

http://ec2-xxxx.us-west-2.compute.amazonaws.com/agilefant
N.B.! The username is admin and the password is secret.

 

 

 

 

Reduce Insecurity for free, HTTPS now democratized by LetsEncrypt

 

how-to-install-lets-encrypt-ssl-certificate

I am typing this with my fingers crossed, that I could just get someone to acknowledge that HTTPS is not only prudent but damn easy to setup. Security is not my primary focus, yet I align with most of the InfoSec’s paranoia out there today. A dumb hacker millions of years ago said that, the minimum you could do in security is to use SSL encryption in your communication.

Now that I have uncrossed my fingers. Below is my rough note for setting up a secure instance for which I assume you have an elastic IP in EC2 instance and a DNS pointing using the A-host configuration to this IP. Below is a totally fake xyzminime.org domain name which I do not own and is just used for example. No offence to anyone who owns it, I just think its an awesome name.

URL: https://xyzminime.org

Email: info@xyzminime.org

# This is how I used to generate my insecure self-signed certificate earlier
keytool -genkeypair -dname "CN=xyzminime.org, OU=XYZ, O=XYZ, L=PaloAlto, ST=CA, C=US" -alias xyzminime -keyalg RSA -ext san=ip:xyzminime.org -keystore /opt/tomcat7/.keystore

Since I am not an authorized certificate signing authority, all the browsers just flags my certificate as unsecure and block it by default.

This is where LetsEncrypt came for help with their democratic certificate authority. There were some references that I drew inspiration from, to do this thing as a rough note and not a tutorial.

Ref:    https://certbot.eff.org/#centosrhel6-other
        https://certbot.eff.org/docs/using.html#webroot
        https://melo.myds.me/wordpress/lets-encrypt-for-tomcat-7-on-ds/

 

1.) Pre-requisite is to get the certbot client

# Installation taken care by the certbot-auto client
sudo yum install epel-release wget
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto

# Install the certbot on your instance
sudo ./certbot-auto

Certbot dumps its contents in a folder like below, in my ec2-user local path,
/home/ec2-user/.local/share/letsencrypt/bin/letsencrypt certonly

 

2.) Generate a certificate

Keep an email address for notification and validation handy for the enrolment with ACME

If you have a functional webserver that needs to be SSLified then use the webroot way otherwise --standalone is preffered
sudo ./certbot-auto certonly -n --rsa-key-size 2048 --agree-tos --email info@xyzminime.org --webroot -w /opt/tomcat7/webapps/ -d xyzminime.org
IMPORTANT NOTES:
– Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/xyzminime.org/fullchain.pem.
Your cert will expire on 2016-12-22. To obtain a new or tweaked
version of this certificate in the future, simply run certbot-auto
again. To non-interactively renew *all* of your certificates, run
“certbot-auto renew”
– If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
The certificates are written to /etc/letsencrypt/live/xyzminime.org/
export CERT_PATH="/etc/letsencrypt/live/xyzminime.org/"
 - cert.pem
 - chain.pem
 - fullchain.pem
 - privkey.pem

 

3.) Create a keystore for Tomcat

Basically there are only two steps required to get our fullchain.pem and privkey.pem inside a JKS. 
First we bundle both our fullchain and the private key in a PKCS12 keystore. 
We do this, because apparently Java’s keytool (which we use to create our JKS),
is not able to import pre-existing keys and certificates into a JKS, as described here.
sudo openssl pkcs12 -export -in "$CERT_PATH"fullchain.pem -inkey "$CERT_PATH"privkey.pem -out fullchain_and_key.p12 -name xyzminime -password pass:mini#123

Now that we have our PKCS12 keystore, we can use Java’s keytool to generate a JKS,
from our PKCS12 file like;
keytool -importkeystore -deststorepass mini#123 -destkeypass mini#123 -destkeystore xyzminime.jks -srckeystore fullchain_and_key.p12 -srcstoretype PKCS12 -srcstorepass mini#123 -alias xyzminime
 
# Backup and place your self-signed keystore in the tomcat home
mv /opt/tomcat7/.keystore .keystore_backup_1
sudo cp xyzminime.jks /opt/tomcat7/.keystore
 
Make sure that the 8443 conector configuration in the conf/server.xml is as follows
<Connector port="8443" 
           keystoreFile="${user.home}/.keystore" 
           keystorePass="mini#123" 
           keyAlias="xyzminime" 
           ...
Run the InstallCert utility for java security ca cert 
Compile the InstallCert using javac
java InstallCert xyzminime.org
sudo cp jssecacerts /usr/java/jdk1.8.0_73/jre/lib/security/

 

4.) Automating renewal

# A test run for renewal
certbot-auto renew --dry-run

# Add the following to the cron or systemmd that should run twice daily in case of any certificate invalidation
certbot-auto renew --quiet

 

Now your tomcat will be able to serve the content over SSL. Verify this by accessing the server on the below URL.

https://xyzminime.org

 

 

 

ODOO (Open ERP) AWS setup notes for your unforeseen startup

_______________________________________

Setup ODOO on EC2 & RDS (Insecure)
_______________________________________

 

Go to https://www.odoo.com and make sure you understand the needs for an ERP system at your startup. I would advice that you evaluate the pros & cons for using an ERP before jumping into this bureaucratic complication.

Security is prime and make sure you realize that I have not covered any aspect of securing the instance or the application here.
1. Intallation pre-requisites

a. Initialize a RDS PostgreSql service on AWS.

b. Initialize an EC2 instance with Amazon Linux in the same security group as RDS

yum install git libtool zlib devel automake pkgconfig gcc c++ curl make gcc-c++ libxml2-devel rsync
yum install openldap-devel libjpeg-devel python-devel vim
yum -y install babel libxslt-python pyparsing python-dateutil python-decorator python-docutils python-feedparser python-imaging python-jinja2 python-ldap python-lxml python-mako python-mock python-openid python-passlib python-psutil python-psycopg2 python-reportlab python-requests python-simplejson python-unittest2 python-vobject python-werkzeug python-yaml pytz
yum install icu xorg-x11-fonts-75dpi freetype freetype-devel 

> Setup Node
yum install nodejs npm
npm install -g less less-plugin-clean-css
ln -s /usr/local/bin/lessc /usr/bin/lessc
ln -s /usr/bin/nodejs /usr/bin/node

> CentOS 6
yum localinstall http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-1.noarch.rpm
wget https://bitbucket.org/wkhtmltopdf/wkhtmltopdf/downloads/wkhtmltox-0.13.0-alpha-7b36694_linux-centos6-amd64.rpm
rpm -ivh wkhtmltox-0.13.0-alpha-7b36694_linux-centos6-amd64.rpm

> CentOS 7
yum localinstall https://yum.postgresql.org/9.3/redhat/rhel-7-x86_64/pgdg-redhat93-9.3-3.noarch.rpm
wget http://download.gna.org/wkhtmltopdf/0.12/0.12.2.1/wkhtmltox-0.12.2.1_linux-centos7-amd64.rpm
rpm -Uvh wkhtmltox-0.12.2.1_linux-centos7-amd64.rpm

yum install postgresql93-contrib postgresql93-devel postgresql93-plpython27
Now lets get the odoo source on the machine to start the setup
git clone https://github.com/odoo/odoo.git
Go to your odoo directory and install the python packages
easy_install -U setuptools
easy_install pip
pip install -r requirements.txt

2. Create a config file openerp-server.conf with the below content

[options]
; This is the password that allows database operations:
; admin_passwd = admin
db_host = admindb.xxxxxxx.us-west-2.rds.amazonaws.com
db_port = 5432
db_user = admin
db_password = admin123
addons_path = /usr/lib/python2.7/dist-packages/openerp/addons
logfile = /tmp/odoo-server.log
xmlrpc_port = 8069

Setup a RDS with postgres database server

psql -h admindb.xxxxxxx.us-west-2.rds.amazonaws.com -U admin admindb

CREATE USER admin WITH PASSWORD 'admin123';
ALTER USER admin CREATEDB;


3. Run the server with the configurations

chown -R ec2-user odoo 
./odoo-bin --addons-path=addons --config=openerp-server.conf

4. Configure the EC2 Instance

a.) IP routing on the instance. Not secure, but will provide the basic routing to start your system.

iptables -t nat -A OUTPUT -p tcp -d ec2-xxx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com --dport 80 -j REDIRECT --to-port 8069
iptables -t nat -I PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-ports 8069
Make sure you verify the iptables. 
sudo iptables -t nat -L -v

Chain PREROUTING (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 8069
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere ip-xxx-xxx-xxx-xxx.us-west-2.compute.internal tcp dpt:http redir ports 8069
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

 

b.) Security group config for port 80. Add exception for incoming HTTP traffic
5. Run the ODOO server as a service

echo "Killing any previous instance of odoo"
ps ax | grep odoo | grep -v grep | awk '{print $1}' | xargs sudo kill
echo "Starting the Odoo server"
nohup ./odoo-bin --addons-path=addons --config=openerp-server.conf -d oodb -u all &> /dev/null & disown
echo "Conf : ~/odoo/odoo/openerp-server.conf"
echo "Log : tail -f /tmp/odoo-server.log"

6. Login as admin / admin
http://ec2-xxx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com

Simplify Tomcat/JVM Monitoring with Mission Control (JMX) or VisualVM

A. JMX Mission Control

Oracle Java Mission Control enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data collected for normal adaptive dynamic optimization of the Java Virtual Machine (JVM). Besides minimizing the performance overhead, this approach eliminates the problem of the observer effect, which occurs when monitoring tools alter the execution characteristics of the system.

1. Server setup

> Provide the JMX configuration to Tomcat server 
> Create a setenv.sh file in $CATALINA_HOME/bin 
> Add the following entry to the script file 
   export CATALINA_OPTS="-Dcom.sun.management.jmxremote=true \
-Dcom.sun.management.jmxremote.port=3614 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.autodiscovery=true"
> This will enable JMX listener on port 3614 when tomcat is restarted
> Make sure that this port is open and accessible to outside world. 
  This may have security concerns hence its not advisable for production environment.
> Restart the server to allow the properties to be set and initialized.

2. Mission Control setup

Download: mission control
In my test I had used an eclipse plugin available at http://download.oracle.com/technology/products/missioncontrol/updatesites/base/5.5.0/eclipse/
 
> Just added this plugin to the eclipse using Install new Software
> Launch a new connection to the JVM and provide the IP and port on which the jmx remote system is running.

MissionControl-Jmx

 

B. Alternate way is to use VisualVM

VisualVM is a visual tool integrating several commandline JDK tools and lightweight profiling capabilities.

Here as well we need to start jstatd daemon on the server which opens up connections for the visualvm client and is packaged with the JDK.

Download: http://visualvm.java.net/download.html
Reference: https://visualvm.java.net/applications_remote.html

1. Start the jstatd daemon

> Make sure the default RMI port is open as per the javase documentation
> Create a policy file named jstatd.all.policy and copy the following content to it
  grant codebase "file:${java.home}/../lib/tools.jar" {
  permission java.security.AllPermission;
  };

> Start the daemon 
  jstatd -J-Djava.security.policy=jstatd.all.policy

> Alternate option to run this silently
  nohup jstatd -J-Djava.security.policy=jstatd.all.policy &>/dev/null &

2. Start the VisualVM Client

> Start the Visual VM client and add remote host using its IP
> You will be able to monitor the jvm on that machine

VisualVM