How to disable JMS transport for ESB manager node in Cluster environment - ESB 4.9.0

Environment 

WSO2 ESB 4.9.0 Cluster Setup (Manager  and Worker Pattern)

Description

After creating the cluster  setup in ManagerWorker Pattern, You may require  to keep Manager node only for the deployments and configurations. (Where you don't need to process JMS messages inside the Manager node)

Solution 

You can find a new feature called “JMS inbound endpoints” with the ESB 4.9.0 which can use to handle this kind of scenarios.

Firstly, you have to follow ESB 4.9.0 clustering document [1] and check whether your cluster is in the Manager - Worker pattern. The main difference in the configuration of Manager and Worker is followed.

IN AXIS2.XML

The manager should have the parameter
<property name="subDomain" value="mgt"/>

Worker Should have the parameter

<property name="subDomain" value="worker"/>

IN CARBON.XML

The manager should have configured with both AutoCommit = true and AutoCheckout = true

The worker should have configured with AutoCommit = false and AutoCheckout = false

After confirming the Manager-Worker pattern we can use “JMS inbound endpoints ” which can be deployed using manager node and only the workers will enable to use the JMS endpoint. (So manager node will not able to use JMS endpoint and consume any messages)

You can create JMS inbound endpoints using Manager management console or WSO2 Developer Studio 3.8.0 [2]

Following is a sample JMS inbound which listens to an activemq message queue and forward the messages to a sequence called “request” [3].

 <inboundEndpoint xmlns="http://ws.apache.org/ns/synapse" name="jms_inbound" sequence="request" onError="fault" protocol="jms" suspend="false">  
    <parameters>  
      <parameter name="interval">1000</parameter>  
      <parameter name="transport.jms.Destination">ordersQueue</parameter>  
      <parameter name="transport.jms.CacheLevel">1</parameter>  
      <parameter name="transport.jms.ConnectionFactoryJNDIName">QueueConnectionFactory</parameter>  
      <parameter name="sequential">true</parameter>  
      <parameter name="java.naming.factory.initial">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>  
      <parameter name="java.naming.provider.url">tcp://localhost:61616</parameter>  
      <parameter name="transport.jms.SessionAcknowledgement">AUTO_ACKNOWLEDGE</parameter>  
      <parameter name="transport.jms.SessionTransacted">false</parameter>  
      <parameter name="transport.jms.ConnectionFactoryType">queue</parameter>  
    </parameters>  
   </inboundEndpoint>  

Beauty of Inbound Endpoints

The JMS inbound endpoint can bypass the inbound side axis2 layer and directly inject messages to a given sequence. So it is not required to enable JMS listeners in both manager and worker nodes. But If you are using JMS producer then you need to define  JMS senders in worker nodes. This blog post [4] shows more details on working with inbound endpoints.

References 

[1] https://docs.wso2.com/display/CLUSTER44x/Clustering+ESB+4.9.0
[2]http://wso2.com/products/developer-studio/
[3]https://docs.wso2.com/display/ESB490/JMS+Inbound+Protocol
[4]http://jagathsa.blogspot.com/2015/06/http-inbound-coordination-with-wso2-esb.html

RabbitMQ - WSO2 ESB 4.9.0 Secure Communication Channel Implementation part 1

Environment

• WSO2 ESB 4.9.0
• RabbitMQ 3.5.6
• Erlang 17.3
• JAVA 1.7

Description

 In this blog post I'm going to share my knowledge on Securing the connection between RabbirMQ Message Broker and WSO2 ESB.

The use case here going to explain is, ESB is going to act as a consumer and consumes messages from RabbitMQ server. So simply ESB will act as a Client and RabbitMQ Server Will act as the Server.

As the first step lets create certificates and keys which required for the RabbitMQ Server and the ESB to create SSL communication.

I'm following the same steps in official RabbitMQ site to create server and client certificates and keys. 

We have to follow the following steps in order to achieve

1. Create own certificate Authority
2. Generate CA signed certificates to the server
3. Generate CA signed certificates to the client

Creating  the Certificate Authority

First, go in to desirable directory and execute following commands. (Note – remove the # mark and execute each line separately)

 # mkdir testca  
 # cd testca  
 # mkdir certs private  
 # chmod 700 private  
 # echo 01 > serial  
 # touch index.txt  

 Then create a new file using the following command, inside the tesca directory.

 #gedit openssl.cnf  

Add following content to the openssl.cnf file and save the file

 [ca]  
 default_ca = testca  
 [ testca ]  
 dir = .  
 certificate = $dir/cacert.pem  
 database = $dir/index.txt  
 new_certs_dir = $dir/certs  
 private_key = $dir/private/cakey.pem  
 serial = $dir/serial  
 default_crl_days = 7  
 default_days = 365  
 default_md = sha1  
 policy = testca_policy  
 x509_extensions = certificate_extensions  
 [ testca_policy ]  
 commonName = supplied  
 stateOrProvinceName = optional  
 countryName = optional  
 emailAddress = optional  
 organizationName = optional  
 organizationalUnitName = optional  
 [ certificate_extensions ]  
 basicConstraints = CA:false  
 [ req ]  
 default_bits = 2048  
 default_keyfile = ./private/cakey.pem  
 default_md = sha1  
 prompt = yes  
 distinguished_name = root_ca_distinguished_name  
 x509_extensions = root_ca_extensions  
 [ root_ca_distinguished_name ]  
 commonName = hostname  
 [ root_ca_extensions ]  
 basicConstraints = CA:true  
 keyUsage = keyCertSign, cRLSign  
 [ client_ca_extensions ]  
 basicConstraints = CA:false  
 keyUsage = digitalSignature  
 extendedKeyUsage = 1.3.6.1.5.5.7.3.2  
 [ server_ca_extensions ]  
 basicConstraints = CA:false  
 keyUsage = keyEncipherment  
 extendedKeyUsage = 1.3.6.1.5.5.7.3.1  

So now we can create key and the certificate for the Certificate Authority. Still inside the tesca directory, execute following commands.

 # openssl req -x509 -config openssl.cnf -newkey rsa:2048 -days 365 -out cacert.pem -outform PEM -subj /CN=MyTestCA/ -nodes  
 # openssl x509 -in cacert.pem -out cacert.cer -outform DER  

So now we have successfully created our Certificate Authority. Then we need to create keys and certificates for the server and the client.

Generating certificate and key for the Server

Assuming you are still inside the testca directory, execute the following commands.

 # cd ..  
 # ls  
 # mkdir server  
 # cd server  
 # openssl genrsa -out key.pem 2048  
 # openssl req -new -key key.pem -out req.pem -outform PEM -subj /CN=$(hostname)/O=server/ -nodes  
 # cd ../testca  
 # openssl ca -config openssl.cnf -in ../server/req.pem -out ../server/cert.pem -notext -batch -extensions server_ca_extensions  
 # cd ../server  
 # openssl pkcs12 -export -out keycert.p12 -in cert.pem -inkey key.pem -passout pass:MySecretPassword  

Generating certificate and key for the client

 # cd ..  
 # ls  
 # mkdir client  
 # cd client  
 # openssl genrsa -out key.pem 2048  
 # openssl req -new -key key.pem -out req.pem -outform PEM -subj /CN=$(hostname)/O=client/ -nodes  
 # cd ../testca  
 # openssl ca -config openssl.cnf -in ../client/req.pem -out ../client/cert.pem -notext -batch -extensions client_ca_extensions  
 # cd ../client  
 # openssl pkcs12 -export -out keycert.p12 -in cert.pem -inkey key.pem -passout pass:MySecretPassword  

We are done here, now we have Certificate Authority, Server and Client Certificates and Keys. So lets move to configure the RabbitMQ server.

Configuring RabbiMQ Server

I have already written a blog post on configuring the RabbitMQ. You can follow this link to get details about it. I prefer to use generic Linux package, then you can access resources in one location. (logs, config files etc.)

You need to know some important resources and their location before proceeding ahead.

• Rabbitmq log file – [Rabbit_Home]/var/log (There are two log files available, required one is rabbit@user-machine.log (not the sasl.log file))
• Rabbitmq configuration file – [Rabbit_Home]/etc/rabbitmq (by default there is no configuration file available. You can create new configuration file with name of rabbitmq.config)

Also use following commands to start and stop the RabbitMQ server (Assuming you are in [RabbitMQ_Home]/sbin directory)

• To start - ./rabbitmq-server start
• To stop - ./rabbitmqctl stop

I think you have successfully installed the RabbitMQ plugins and the Management console is up and running.

After you start the server, you can see following log entries in rabbit.log file.

=INFO REPORT==== 9-Nov-2015::14:57:52 ===

started TCP Listener on [::]:5672

=INFO REPORT==== 9-Nov-2015::14:58:00 ===

Management plugin started. Port: 15672

RabbitMQ server is listening on TCP port 5672 which is the default port and we can't see any ports activated for the SSL connections.

Configuring rabbitmq.config file we can enable SSL listener with various SSL options. Following is sample rabbitmq.config file.

  [  
        { rabbitmq_management,[{listener, [ {port, 15672},   
                               {ssl, true},   
                               {ssl_opts, [{cacertfile,"/home/krishan/testca/cacert.pem"},  
                                       {certfile, "/home/krishan/server/cert.pem"},  
                                      {keyfile, "/home/krishan/server/key.pem"}  
                                      ]  
                               }  
                             ]  
                       }]  
        },  
        {ssl, [{versions, ['tlsv1.2']}]},  
        {rabbit,[  
             {tcp_listeners, [5672]},  
             {ssl_listeners, [5671]},  
             {ssl_options, [{cacertfile,"/home/krishan/testca/cacert.pem"},  
                      {certfile, "/home/krishan/server/cert.pem"},  
                      {keyfile, "/home/krishan/server/key.pem"},  
                      {verify, verify_none},  
                      {fail_if_no_peer_cert,false},  
                      {versions, ['tlsv1.2']}                       
                     ]  
               }  
              ]  
        }  
  ].  

After adding the rabbitmq.config file, restart the server. Then you can see the following entries in rabbit.log file.

=INFO REPORT==== 9-Nov-2015::16:57:47 ===

started TCP Listener on [::]:5672

=INFO REPORT==== 9-Nov-2015::16:57:47 ===

started SSL Listener on [::]:5671

So you can see that the RabbitMQ server has started SSL listner on port 5671.

We are now almost done with configuring the RabbitMQ broker and lets see the configuration of the WSO2 ESB in my next post.

RabbitMQ - WSO2 ESB 4.9.0 Secure Communication

Hi All, I'm going to explain some major question you may get during thinking about SSL communication between RabbitMQ message broker and WSO2 ESB 4.9.0

1) Does ESB 4.9.0 support for the SSL communication with RabbitMQ?

WSO2 ESB 4.9.0 is supported for the SSL communication with RabbitMQ. To enable SSL support in RabbitMQ, you need to configure the transport listener with the parameters required to enable SSL as well as the parameters that provide information related to keystores and truststores. You can find more details on ESB 4.9.0 official document. 

2) How to specify SSL protocol versions supported by RabbitMQ in WSO2 ESB 4.9.0?

We can define the SSL protocol version from the following parameter.

<parameter name="rabbitmq.connection.ssl.version"locked="false">SSL</parameter>

The parameters which you can specify are mentioned in this Document which describes SSL Context Alogrithms

3) Which truststore need to be used to store RabbitMQ certificate?

We can use any truststore (client-trustore.jks etc.)  but we have to define the truststore path in the transport reciver configuration by using the following parameter.
<parameter name="rabbitmq.connection.ssl.truststore.location"locked="false">ssl/rabbitstore</parameter>

4) Do WSO2 ESB 4.9.0 support for the cipher suite setup according to the RabbitMQ?
At the movement ESB 4.9.0 is not supported for this feature.

So guys, These are my little findings on this topic and I'm now working on creating an SSL communication with RabbitMQ 3.3.4 and WSO2 ESB 4.9.0.

I'm hoping  to see you with a working project in my next post.

Loading Governance Registry Entry into WSO2 Property Mediator and manipulate data

We can save resources in the Governance registry and use them in synapse configurations. In this post I'm going to share how we can load registry saved endpoint data into the message context and manipulate them using XPath queries.

First, click on the Registry resources and create a folder structure as governance/data/end points.

Create new address endpoint with sample stock quote service url
(http://localhost:9000/services/SimpleStockQuoteService). If you want to set up sample stock quote backed service, please refer this link.  Save this endpoint in registry by giving above created folder structure.

After creating the endpoint you can see that resource is appeared in endpoint collection.

 Let's create a Pass Through Proxy which refer the endpoint saved in the registry. Click on the "pick from Registry" radio button and browse the endpoint from Governance registry and create the proxy service.

Now we have created the proxy service which refers the register saved endpoint. Let's see how we
Can get the endpoint resource content into message context and manipulate them.

So the end point xml as follows

 <endpoint xmlns="http://ws.apache.org/ns/synapse" name="EP">  
   <address statistics="disable" trace="disable" uri="http://localhost:8280/services/echo">  
     <timeout>  
       <duration>0</duration>  
       <responseAction>discard</responseAction>  
     </timeout>  
     <markForSuspension>  
       <retriesBeforeSuspension>0</retriesBeforeSuspension>  
       <retryDelay>0</retryDelay>  
     </markForSuspension>  
     <suspendOnFailure>  
       <initialDuration>0</initialDuration>  
       <maximumDuration>0</maximumDuration>  
       <progressionFactor>1.0</progressionFactor>  
     </suspendOnFailure>  
   </address>  
 </endpoint>  

If we need to load this endpoint from registry to the message context and read the address URI and log it on the console. You can simply follow the following proxy configuration.

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
     name="testProxy1"  
     transports="https http"  
     startOnLoad="true"  
     trace="disable">  
   <description/>  
   <target endpoint="gov:/apps/custom/endpoint/dlr/EP.xml">  
    <inSequence>  
      <property name="resultOM"  
           expression="get-property('registry','gov:/apps/custom/endpoint/dlr/EP.xml')"  
           scope="default"  
           type="OM"/>  
      <property name="value" expression="$ctx:resultOM//file/name/@uri"/>  
      <log level="full"/>  
      <log level="custom" separator=",">  
       <property name="resultOM" expression="$ctx:resultOM"/>  
       <property name="resultOM1" expression="$ctx:value"/>  
      </log>  
    </inSequence>  
    <outSequence>  
      <send/>  
    </outSequence>  
   </target>  
 </proxy>  

You can load registry entry using following property.

 <property name="resultOM"   
       expression="get-property('registry','gov:/apps/custom/endpoint/dlr/EP.xml')"   
       scope="default"   
       type="OM"/>   

Then you can perform the XPath queries like following and retrieve attributes or values.

 <property name="value" expression="$ctx:resultOM//file/name/@uri"/>   

WSO2 ESB Sending Mails With Multiple Attachments

WSO2 ESB is mainly used for message mediation purposes.  It supports message routing, inter-mediation, transformation, logging, load balancing, failover routing, task scheduling, Eventing and much more.

There may be special cases where we need to implement business logic in the ESB even though it is not the ESB's task. Lets think we want to send emails with many attachments. So we have to implement business logic in ESB to do this right.

How can we send mails with multiple attachments?  I found TWO ways to do this.

1. Use gmail connector
2. Use own class mediator

You can find details about the gmail connector from the official wso2 documentation so I think you can easily catch the usage of it.

In this post I'm going to explain how we can use own class mediator to send mails with multiple attachments. So in this case you can use any mail service provide like yahoo, gmail etc.

So this is the sample class which uses to send mails with multiple attachments.
Note - this class depends on java-mail-1.4 so you have download and import to the project.

 import java.io.IOException;  
 import java.util.Date;  
 import java.util.Properties;  
 import javax.mail.Authenticator;  
 import javax.mail.Message;  
 import javax.mail.MessagingException;  
 import javax.mail.Multipart;  
 import javax.mail.PasswordAuthentication;  
 import javax.mail.Session;  
 import javax.mail.Transport;  
 import javax.mail.internet.AddressException;  
 import javax.mail.internet.InternetAddress;  
 import javax.mail.internet.MimeBodyPart;  
 import javax.mail.internet.MimeMessage;  
 import javax.mail.internet.MimeMultipart;  
 /*  
  * Create connection with the relevant SMTP server.  
  * Create new email with message body part and attachments(multi-parts)  
  * Send email to the address endpoint  
 */  
 public class EmailAttachmentSender extends AbstractMediator {  
      private String host; // smtp server address  
      private String port; // smtp port no  
      private String userName; // sender mail address  
      private String password; // sender password  
      private String toAddress; // receiver mail address  
      private String subject; // mail subject  
      private String message; // email message body   
      private String files; // files URI   
      public boolean mediate(MessageContext context) {  
           try {  
                sendMailWithAttachment(context);  
                System.out.println("message sent");  
           } catch (AddressException e) {  
                // TODO Auto-generated catch block  
                e.printStackTrace();  
                System.out.println("message not sent");  
                System.out.println(e.toString());  
           } catch (MessagingException e) {  
                // TODO Auto-generated catch block  
                e.printStackTrace();  
                System.out.println("message not sent");  
                System.out.println(e.toString());  
           } catch (Exception e) {  
                e.printStackTrace();  
                System.out.println("message not sent");  
                System.out.println(e.toString());  
           }  
           return true;  
      }  
      private void sendMailWithAttachment(MessageContext mc)  
                throws AddressException, MessagingException {  
           // sets SMTP server properties  
           Properties properties = new Properties();  
           properties.put("mail.smtp.host", host);  
           properties.put("mail.smtp.port", port);  
           properties.put("mail.smtp.auth", "true");  
           properties.put("mail.smtp.starttls.enable", "true");  
           properties.put("mail.user", userName);  
           properties.put("mail.password", password);  
           // creates a new session with an authenticator  
           Authenticator auth = new Authenticator() {  
                public PasswordAuthentication getPasswordAuthentication() {  
                     return new PasswordAuthentication(userName, password);  
                }  
           };  
           Session session = Session.getInstance(properties, auth);  
           // creates a new e-mail message  
           Message msg = new MimeMessage(session);  
           msg.setFrom(new InternetAddress(userName));  
           InternetAddress[] toAddresses = { new InternetAddress(toAddress) };  
           msg.setRecipients(Message.RecipientType.TO, toAddresses);  
           msg.setSubject(subject);  
           msg.setSentDate(new Date());  
           // creates message part  
           MimeBodyPart messageBodyPart = new MimeBodyPart();  
           messageBodyPart.setContent(message, "text/html");  
           // creates multi-part  
           Multipart multipart = new MimeMultipart();  
           multipart.addBodyPart(messageBodyPart);  
           //adding file URIs to the array  
           String[] attachFiles = files.split(",");  
           for (String string : attachFiles) {  
                System.out.println(string);  
           }  
           // adds attachments  
           if (attachFiles != null && attachFiles.length > 0) {  
                for (String filePath : attachFiles) {  
                     MimeBodyPart attachPart = new MimeBodyPart();  
                     try {  
                          attachPart.attachFile(filePath);  
                     } catch (IOException ex) {  
                          ex.printStackTrace();  
                     }  
                     multipart.addBodyPart(attachPart);  
                }  
           }  
           // sets the multi-part as e-mail's content  
           msg.setContent(multipart);  
           // sends the e-mail  
           Transport.send(msg);  
      }  
      public String getHost() {  
           return host;  
      }  
      public void setHost(String host) {  
           this.host = host;  
      }  
      public String getPort() {  
           return port;  
      }  
      public void setPort(String port) {  
           this.port = port;  
      }  
      public String getUserName() {  
           return userName;  
      }  
      public void setUserName(String userName) {  
           this.userName = userName;  
      }  
      public String getPassword() {  
           return password;  
      }  
      public void setPassword(String password) {  
           this.password = password;  
      }  
      public String getToAddress() {  
           return toAddress;  
      }  
      public void setToAddress(String toAddress) {  
           this.toAddress = toAddress;  
      }  
      public String getSubject() {  
           return subject;  
      }  
      public void setSubject(String subject) {  
           this.subject = subject;  
      }  
      public String getMessage() {  
           return message;  
      }  
      public void setMessage(String message) {  
           this.message = message;  
      }  
      public String getFiles() {  
           return files;  
      }  
      public void setFiles(String files) {  
           this.files = files;  
      }  
 }  

Let see the java source code and understand what's going on as a summarize. First, make a connection with the SMTP server and create the new email message according to variable values. (These values can be defined as properties inside class mediator in synapse config.)
Then set the message body as multipart and added the message body part (text message). After that, create the mime body part and add files to it (can add multiple files). Finally, add the mime body part to the message body and send the mail using created connection.
In order to setup this class mediator, you can simply add the EmailSendMediator.jar (you have to build the source project and get the jar file) and java-mail-1.4.jar into [ESB_HOME]\repository\components\lib and restart the server.

You can add a proxy service with the class mediator. Following is a sample proxy service.

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
     name="customProxy"  
     transports="https,http"  
     statistics="disable"  
     trace="disable"  
     startOnLoad="true">  
   <target>  
    <inSequence>  
      <log level="full"/>  
      <class name="com.wso2.test.EmailAttachmentSender">  
       <property name="port" value="587"/>  
       <property name="message" value="I have some attachments for you."/>  
       <property name="host" value="smtp.gmail.com"/>  
       <property name="subject" value="New email with attachments"/>  
       <property name="userName" value="user@gmail.com"/>  
       <property name="password" value="password"/>  
       <property name="toAddress" value="receiver@gmail.com"/>  
                <property name="files" value="C:/Demo/File/In/a.xml,C:/Demo/File/In/excel.xls,C:/Demo/File/In/pdf-sample.pdf,C:/Demo/File/In/formHTML.html"/>  
      </class>  
      <log level="full"/>  
      <drop/>  
    </inSequence>  
    <outSequence/>  
   </target>  
   <description/>  
 </proxy>  

You can just trigger this service by sending request and you can see the message has been sent successfully.