Using Mule Studio to read Data from REST based Web Services and store only the value part in a PostgreSQL database - Step by Step approach

Introduction

In this article, we will look into as how to read Data from REST based Web Services and store only the value part in a PostgreSQL database

What we need to have for doing the experiment?

We need

1. PostgreSQL
2. PostgreSQL JDBC driver
3. Download Mule ESB

4. A database to be created in PostgreSQL say "TestDB" where atleast one Table say "tblJson" should be created as under

   		-- Table Creation Script
   		CREATE TABLE "tblJson"
   		(
   		   "JsonDataStorage" text
   		) 
   		

5. A REST service (I have made it using Dotnet.You are free to make it in your favorite language) that will return a json file as under

   

6. Everything should be up and running properly

Let us start the experiment

Step1:Open Mule Studio

So now we are all set to do our first example using Mule Studio.Execute the "MuleStudio.exe" and enter a location for creating the workspace. Workspace is the location where we need to work for the project.

Click on "OK" button to receive the Mule Studio window.Click on "Go to Mule Studio". We will receive the Mule studio editor.

Click File > New > Mule Project.From the "New Mule Project" window that appears, populate the "Name" field with "MuleRESTExperiment".

Click "Next" button and "New Mule Flow" window appears.Click "Next" button and "New Mule Flow" window appears.Leave it as it is.Click "Next" button and "Create New Java Project" window appears. Use proper JRE version that you are using.

Click "Next" button and "Java Settings" window appears.Leave it as it is.Click "Finish" button.We will be presented with the following

Step2:Import the Postgre SQL JDBC Driver

(a)In the Package Explorer, right-click over the Project folder ( in this case "jdbc-postgresql-mule-experiment)->BuildPath->Add External Archives…

(b)Choose the location where the "postgresql-9.2-1002.jdbc4.jar" jar file is located in the hard drive and then click on the "Open" button.

(c)Now we can find the jar file being present in the "Referenced Libraries" of the package explorer.

This will allow us to create an instance of the Object driver

Step3:Now the time for creating Data Source

(a)Switch to the "Global Elements" tab

(b)Click on the "Create" button.Look for Data Sources > PostgreSQL Data Source and click the OK button.

(c)The PostgreSQL Data Source Window appears

(d)In that, we need to specify the Database name in the URL section, PostgreSQL User Name in the "User" field and Password in "Password" field and click OK.

Step4:Let us create the Connector now

So we have successfully created a Data Source with it's proper driver.Now we need a Connector.

(a)Click on the "Create" button.Look for Connector > Database.Then click OK.

(b)From the Database Window that appears, click on the "Database Specific" dropdown and choose "PostgreSQL_Data_Source". It is the Data-Source that we created in the previous step

(c)Switch to the "Queries" tab and click on the button ("Add Query"). First we need to specify the "Query Key"(which is "InsertRecord" here) and then the query as shown under.Click "OK" button.

INSERT INTO "tblJson"("JsonDataStorage") VALUES (#[message.payload])

Finally, we will be presented with the below screen if everything has gone right so far

If at this stage we view the "Configuration XML" tab, it will look like

<jdbc:postgresql-data-source name="PostgreSQL_Data_Source" user="postgres" password="niladri_123" url="jdbc:postgresql://localhost:5432/TestDB" transactionIsolation="UNSPECIFIED" doc:name="PostgreSQL Data Source"/>
<jdbc:connector name="PostgreSQL_JDBC_Connector" dataSource-ref="PostgreSQL_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database">
	<jdbc:query key="InsertRecord" value="INSERT INTO "tblJson"("JsonDataStorage") VALUES (#[message.payload])"/>
</jdbc:connector>

Step5:Let us create flow now

Switch to the "Configuration XML" tab first and add the below piece of code just after the "<jdbc:postgresql-data-source....>"

<flow name="MuleRESTExperimentFlow1" doc:name="MuleRESTExperimentFlow1">
    
	<!-- Step 1: This will start the service.This may be replaced with an actual input page like a Html page  -->
	<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8082" doc:name="HTTP"/>
	
	<!-- Step 2: This will read the REST service data  -->
	<http:rest-service-component httpMethod="GET" serviceUrl="http://localhost:35798/RestServiceImpl.svc/Authors" />
	
	<!-- Step 3: Transform the JSON entity to a hierarchy of Java Map structure -->
	<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
	
	<!-- Step 4: Extract the record list -->        
	<expression-transformer expression="#[message.payload.AuthorListResult]" doc:name="Expression"/>
	
	<!-- Step 5: Dump into the destination Database -->	    
	<jdbc:outbound-endpoint exchange-pattern="one-way" queryKey="InsertRecord" queryTimeout="-1" connector-ref="PostgreSQL_JDBC_Connector" doc:name="Destination"/>
</flow>

We are basically using a "Http inbound endpoint" that allows Mule application to connect to web resources through the HTTP transport protocol.This we are introducing so as to start the application.In reality, this will be replace with a propoer view like Html page etc.

Next, we are using "rest-service-component" that will help to read the content from the REST based application. We will receive a Json object here

By using "json-to-object-transformer", we are transforming the JSON entity to a hierarchy of Java Map data structure

In the "expression-transformer", we are extracting the value from the JSON object by specifying the key (message.payload.AuthorListResult)

In the last step , we push the records to the PostgreSQL Database which is the "Database" component (Outbound).

The final flow is

The full configuration XML should look like

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:json="http://www.mulesoft.org/schema/mule/json"
	xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:jdbc="http://www.mulesoft.org/schema/mule/jdbc" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd 
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd 
http://www.mulesoft.org/schema/mule/jdbc http://www.mulesoft.org/schema/mule/jdbc/current/mule-jdbc.xsd 
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd 
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
    <jdbc:postgresql-data-source name="PostgreSQL_Data_Source" user="postgres" password="niladri_123" url="jdbc:postgresql://localhost:5432/TestDB" transactionIsolation="UNSPECIFIED" doc:name="PostgreSQL Data Source"/>
    <jdbc:connector name="PostgreSQL_JDBC_Connector" dataSource-ref="PostgreSQL_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database">
        <jdbc:query key="InsertRecord" value="INSERT INTO "tblJson"("JsonDataStorage") VALUES (#[message.payload])"/>
    </jdbc:connector>
    <flow name="MuleRESTExperimentFlow1" doc:name="MuleRESTExperimentFlow1">
    
    	<!-- Step 1: This will start the service.This may be replaced with an actual input page like a Html page  -->
        <http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8082" doc:name="HTTP"/>
        
        <!-- Step 2: This will read the REST service data  -->
        <http:rest-service-component httpMethod="GET" serviceUrl="http://localhost:35798/RestServiceImpl.svc/Authors" />
        
        <!-- Step 3: Transform the JSON entity to a hierarchy of Java Map structure -->
        <json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
        
        <!-- Step 4: Extract the record list -->        
	    <expression-transformer expression="#[message.payload.AuthorListResult]" doc:name="Expression"/>
	    
	    <!-- Step 5: Dump into the destination Database -->	    
        <jdbc:outbound-endpoint exchange-pattern="one-way" queryKey="InsertRecord" queryTimeout="-1" connector-ref="PostgreSQL_JDBC_Connector" doc:name="Destination"/>
    </flow>
</mule>

We are now almost done.Next we need to run the application.Right click on "MuleRESTExperiment.mflow" and from the context menu, choose Run As->Mule Application.

We will encounter the below screen at some point of time

Because it is looking for the input data.So open up the browser, and type "http://localhost:8082/".Press Enter.

Come back to the mule studio and we can observer the below in the console window

If we now look into the tblJson, we will find that 1 record has been inserted

Conclusion

Hope you enjoyed the step by step tutorial of using Mule Studio to read Data from REST based Web Services and store only the value part in a PostgreSQL database.I will write more on the same as and when I will perform some experiment.Thanks for reading.The configuration-xml file and WCF REST service file is attached in the zipped format.