Spring Rest API and Mongodb with GridFS

Reference: https://github.com/vishwakarmarhl

RDataServer

Hello draft of the Mongo DB based file data store service

GIT USAGE

There are two more branches apart from the master branch which have the file upload functionality

  1. Checkout the branch
  2. Pull the current branch
    git pull
  3. After making changes in any of them index and commit
    git add .
    git commit -m “Updated the code and added a message”
  4. Push changes to the github repository
    git push origin FileUploadGridFSSpring
    git push origin MultiPartFileUpload

SETUP & START MONGO DB WIN x64

Download : http://www.mongodb.org/dr/downloads.mongodb.org/win32/mongodb-win32-x86_64-2.4.5.zip/download

  1. Unzip the zip contents in C:\mongodb\
  2. create C:\data\db
  3. Execute the C:\mongodb\bin\mongod.exe –dbpath C:\data\db

CREATE SAMPLE DATA

//The following command simply pre-allocates a 2 gigabyte, uncapped collection named people. db.createCollection(“files”, { size: 2147483648 }) db.files.save( { fileId: ‘1235’, fileName: ‘V_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123342, fileExtensionType: ‘EXE’ })

PROJECT USAGE

  1. Start the mongod.exe standalone server
  2. Import the source as a maven project in Eclipse STS IDE
  3. MVN PACKAGE
  4. Deploy on a tomcat instance to see the data from mongodb
  5. An alternate plugin for tomcat enables maven based initialization. mvn tomcat7:run
  6. Open up http://localhost:8088/RDataServe to checkout the grid for file data

APPENDIX/REFERENCE

Mongo Shell Cmds

show dbs show collections

//This command creates a collection named file with a maximum size of 5 megabytes and a maximum of 5000 documents. db.createCollection(“files”, { capped : true, size : 5242880, max : 5000 } )

//The following command simply pre-allocates a 2 gigabyte, uncapped collection named people. db.createCollection(“files”, { size: 2147483648 })

//Drop a collection capped db.files.drop()

//Insert db.files.insert( { _id: 1, fileId: 1234, fileName: ‘R_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123412, fileExtensionType: ‘EXE’ })

db.files.save( { fileId: ‘1235’, fileName: ‘V_XXX.EXE’, filePath: ‘/opt/storage/rhldata’, fileSizeInKB: 123342, fileExtensionType: ‘EXE’ })

//Query db.files.find({fileId:1234})

AspectJ component for my services audit logger

This is a blog that will leverage the advantages of AspectJ aspect oriented programming concept in solving a very basic problem of auditing the visitors to your service and the response times. These parameters are very important when we want to do some operations around these.

Pre-requisite for AspectJ in your pom.xml

 <!-- Spring AspectJ -->
 <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-aspects</artifactId>
    <version>${org.springframework-version}</version>
    <scope>compile</scope>
 </dependency>
 <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
    <version>${org.aspectj-version}</version>
    <scope>compile</scope>
 </dependency>
 <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjrt</artifactId>
    <version>${org.aspectj-version}</version>
 </dependency>
 <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-aop</artifactId>
    <version>${org.springframework-version}</version>
 </dependency>
 <dependency>
    <groupId>cglib</groupId>
    <artifactId>cglib</artifactId>
    <version>3.1</version>
 </dependency>

In your application context add the configuration as per the standard aop proxy. You should also include component-scan and annotation-driven.

<!-- Aspects -->
 <aop:aspectj-autoproxy proxy-target-class="true"/>

Here is the actual code that will intercept around the public calls  in controller. It will print the service url, method, parameters and arguments. We can also obtain the user from the session context.

/**
 * @author Rahul Vishwakarma
 * This class will log all the service calls made to the Generic Application 
 * relying on the @Around advice
 * Ref: http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/aop.html#aop-understanding-aop-proxies 
 */
@Component
@Aspect
public class GenericLoggerAspect {
 /** Logger for this class and subclasses */
 private static final Logger log = LoggerFactory.getLogger(RhlLoggerAspect.class);
 
 public static ConcurrentHashMap<String,RequestData> responseTime = new ConcurrentHashMap<String,RequestData>();
 
 @Autowired 
 Utility utility;
/**
 * Inner class for request info
 * @author Rahul
 *
 */
 public class RequestInfo{
 public int responseTimeMills = 0;
 public Date accessTime = null;
 public String urlPath;
 public String requestType;
 public String args;
 }
 /**
 * Inner class for Data capturing request information
 * @author Rahul
 *
 */
 public class RequestData {
   public RequestInfo requestInfo;
   public String api;
   public UserInfo userInfo;
 
   public RequestData(RequestInfo requestInfo,UserInfo userInfo,String methodSignature){
   this.requestInfo = requestInfo;
   this.userInfo = userInfo;
   this.api = methodSignature;
   }
 }

/**
 * User and client related info
 * @author Rahul
 *
 */
 public class UserInfo{
   public String userName;
   public int userId;
   public String sessionId;
   public String role;
   public String clientIp;
 }
 
 private String getRepresentation(Object [] params){
   StringBuilder sb = new StringBuilder();
   if(params!=null)
   {
   String value = null;
   for(int i=0;i<params.length;i++){
   value = params[i] + ",";
   if(value.contains("@")){
     value = "";
   }
     sb.append(value);
   }
    if(sb.length()>0)
    return sb.substring(0, sb.length() - 1);
  }
  return sb.toString();
}
enum IssueType{
  ISSUE_URL_SUFFIX, OTHER, NONE
}
@Around("execution(@*..RequestMapping * * (..))")
public Object log_around(ProceedingJoinPoint pjp) throws Throwable {
 
  Object obj = null;
  IssueType issueType = IssueType.NONE;
  String error="Error in GenericLoggerAspect";
 try{
  String methodSignature = pjp.getSignature()+"";
  StringBuffer args = new StringBuffer(); //getRepresentation(pjp.getArgs());
 
  //Append arguments
  Object[] arg = pjp.getArgs();
  for (int i = 0; i < arg.length; i++) {
    args.append(arg[i]).append(",");
  }
  if (arg.length > 0) {
   args.deleteCharAt(args.length() - 1);
  }
 
  log.info("\tSTART {}-{}", methodSignature+" ["+Thread.currentThread().getId()+"]",args.toString());

  UserInfo userInfo = new UserInfo();
  UserSecure userSecure = utility.getUserInfo();
  if(userSecure != null){
   userInfo.clientIp = userSecure.getClientIp();
   userInfo.userId = userSecure.getUserId();
   userInfo.sessionId = userSecure.getSessionId();
   userInfo.userName = userSecure.getUserName();
  if(userSecure.getAuthorities() != null)
   userInfo.role = userSecure.getAuthorities().toString();
  }
  ServletRequestAttributes sra = (ServletRequestAttributes)RequestContextHolder.getRequestAttributes();
 
  String urlPath=""; 
 
  if(sra!=null){
    HttpServletRequest req = sra.getRequest();
  if(req!=null){
    urlPath = req.getServletPath();
  } 
 }
 
 long start = System.currentTimeMillis();
 if(urlPath.endsWith("/")){
   issueType = IssueType.ISSUE_URL_SUFFIX;
   obj = null;
 } else
   obj = pjp.proceed();
 
 String requestType = "ajax";
 if(obj!=null && (obj instanceof ModelAndView)){
   requestType = "page";
 }
 int elapsedTime = (int) (System.currentTimeMillis() - start); 
 RequestInfo requestInfo = new RequestInfo();
 requestInfo.accessTime = (new Date());
 requestInfo.requestType = (requestType);
 requestInfo.responseTimeMills = (elapsedTime);
 requestInfo.urlPath = (urlPath);
 requestInfo.args = (args.toString());
 
 RequestData requestData = new RequestData(requestInfo, userInfo,methodSignature);
 log.info(userInfo.sessionId + " REQ {} by "+userInfo.userName +"@"+userInfo.clientIp+", time {} mills; args ["+args.toString()+"]", urlPath + " ["+ Thread.currentThread().getId() +"]" , elapsedTime);
 responseTime.put(methodSignature, requestData);
 //addUserAuditLog(requestData);
 
 }catch(Exception ex){
 issueType = IssueType.OTHER;
 error = ex.getMessage();
 log.error(ex.getMessage());
}
 
if(obj==null){
 switch(issueType){
 case ISSUE_URL_SUFFIX:throw new GenericException("URL ends with trailling /"); 
 case OTHER:throw new GenericException(error);
 default:
 break;
 } 
}
return obj;
}
/**
 * @param exceptions
 */
@AfterThrowing(pointcut="execution(public * com.generic.controller.*.*(..))",throwing="ex") 
 public void MethodError(Exception ex){ 
   log.error("@Exception {}", ex.toString()); 
 } 
 
@Pointcut("execution(public * *(..))")
private void anyPublicOperation() {
   log.info("Testing the public Execution call");
 }
}

When we run the system we expect the following response in the log which does the @Around joint point.

Before:

17-Jul-2014 03:25:35,067-INFO – GenericLoggerAspect:130 –       START List com.humesis.generic.controller.UserController.getUsers(HttpServletResponse)

After:

17-Jul-2014 03:25:35,106-INFO – GenericLoggerAspect:176 – 674B8F114926E5A3BB143E7126D828C7 REQ /users [28] by rahul@0:0:0:0:0:0:0:1, time 36 mills; args [HttpSessionSecurityContextRepository]

This data can Asynchronously be audited or logged for generating the access pattern or hotspots in the service access.

 

CAS-ify and Implement Single sign on in your application, Oh what a rellief…

 

Ah, there comes a time in a developers life when the application they develop requires to be actually used and in this particular case I am talking about multiple applications in the eco-system. Now this developer in discussion appears to be an enterprise scale geek. In order to use these applications, people need to be scrutinized by a central or single sign on like security entity. Here comes a Central Authentication Service to rescue your CASe.

 

I will take you through a set of steps here,

 

A. Configure CAS single sign on Server on a Tomcat with SSL configured.

Steps are detailed here.

B.. Create a service application to actually authenticate with this CAS server and service your request.

RESTful client for CAS secured services. A sample example is available here

 

C. Generate your SSL trusted certificates so that This CAS Server and your Service application can actually interact

Setup Certificates:   The SSL related certificates used for development are self-signed in nature and are restricted to IP on which server and services are running. The keytool command provisioned by the JDK is used for this purpose.

Self-Signed Certificate Setup steps for CAS:

  1. Configure: https://wiki.jasig.org/display/CASUM/RESTful+API
  2. Test : Using commons http client
  3.  Workflow,
  • - Get the TicketGrantingTicket from server = "https://localhost:8443/cas/v1/tickets";
  • - Get the ServiceTicket service = "https://localhost:8443/cas-sample/secure";
  • - Based on the service ticket GET access to the secured REST API service

D. Certification Path Exception for SSL handshake:
Reference: http://www.mkyong.com/webservices/jax-ws/suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/
Source: https://github.com/vishwakarmarhl/javahelper/blob/master/InstallCert.java
Command: java InstallCert localhost:8443 // Also add trust for the service and cas_server IP

E. Use a Http Test client to authenticate and call the service.

 

Show off your spring web services through swagger, Just like that…

A backend blackhole services developer can become a shooting star with swagger.  This is like a new super power that can take you to a whole new level. I have been trying to get deeper into it. I think i can discuss some basic stuff to kick start you on this.

Nirvana hits when you realize that you can even try out your web services right through this user interface.Holiness all the way.

Swagger: http://swagger.wordnik.com/

Based on the beautiful and easy to strap spring-mvc work from MartyPitt:

https://github.com/martypitt/swagger-springmvc-example

Step 1: Maven dependencies

<!-- Swagger for API Documentation -->
 <dependency>
 <groupId>org.projectlombok</groupId>
 <artifactId>lombok</artifactId>
 <version>0.10.8</version>
 <scope>provided</scope>
 </dependency>
 <dependency>
 <groupId>com.mangofactory</groupId>
 <artifactId>swagger-springmvc</artifactId>
 <version>0.6.6</version>
 </dependency>
 <dependency>
 <groupId>com.fasterxml.jackson.core</groupId>
 <artifactId>jackson-databind</artifactId>
 <version>2.1.1</version>
 </dependency>
 <dependency>
 <groupId>com.fasterxml.jackson.core</groupId>
 <artifactId>jackson-annotations</artifactId>
 <version>2.1.1</version>
 </dependency>
 <dependency>
 <groupId>com.fasterxml.jackson.core</groupId>
 <artifactId>jackson-core</artifactId>
 <version>2.1.1</version>
 </dependency>
 <dependency>
 <groupId>joda-time</groupId>
 <artifactId>joda-time</artifactId>
 <version>2.1</version>
 </dependency>
 <dependency>
 <groupId>org.springframework.hateoas</groupId>
 <artifactId>spring-hateoas</artifactId>
 <version>0.3.0.RELEASE</version>
 </dependency>
 </dependencies>

2. Include a swagger.properties file in your resources folder

documentation.services.basePath=http://localhost:8080/myspringwebapp 
documentation.services.version=1.0

3. Add the configuration for the swagger to discover all the a=spring based annotations and create the api docs service

<context:component-scan base-package="com.myspringwebapp.generic" />
 <mvc:annotation-driven/>
 <context:annotation-config/>
<bean id="propertyPlaceholderConfigurer"
 class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
 <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
 <property name="locations">
 <list>
 <value>classpath:swagger.properties</value>
 </list>
 </property>
 </bean>
<mvc:default-servlet-handler/>
<!-- To enable @RequestMapping process on type level and method level -->
 <bean class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping" />
 
 <!-- creates a controller at /api-docs from this uri, which serves swagger's raw documentation in JSON format. -->
 <bean id="documentationConfig" class="com.mangofactory.swagger.configuration.DocumentationConfig"></bean>


4. Add the swagger-ui dependencies and the relevant css, js, and html pages. I had to update the swagger-ui javascript but am sure the one shared by martypitt also works as i tried it too.

https://github.com/martypitt/swagger-springmvc-example/tree/master/src/main/webapp

5. Voila, just run this application in the container you are in love with

Test it out locally
http://localhost:8080/myspringwebapp/api-docs
The above url should return a list of controllers available in your spring mvc project
Doc UI: 
http://localhost:8080/myspringwebapp/index.html
The html has all that it takes to read the api and beautify it, such that you can read, play and do everything you could with it.

Swagger was smooth and as a project it was amazing to use and showcase the beautiful API doc to stakeholders. If you are just curious then take a peek at the beauty in the attached screen shot of a GET and a POST interface on my REST user spring controllers.

A beautiful proof of the working api doc

A beautiful proof of the working api doc

Going International with Spring i18n Internationalization or say Localization

This will cover the springframework internationalization feature in order to be able to show various messages using the language resource bundle file.

I had taken help from MKYONG blog and packaged a good working example to demonstrate it. This app even transitions pages and shows  that locale is maintained even when we navigate away from the page where locale was modified.

The WebTestJEEi18n project covers most of these configuration comprehensively.

Link to the GITHUB project: https://github.com/vishwakarmarhl/SpringInternational.git

Will be adding more on springs technology progressively.