Speedup unspeedable

    A few days ago i have got some well know e-commerce platform to speed up it. Application use hibernate, solr, spring and spring mvc with apache velocity as view and look like very over optimised, cached everything, that possible. It cached solr results, hibernate use second level cache, web tier use etag, etc. I was frustrated, because no easy way to boost performance. After several hours of investigation some bright idea came to me - if solution cache the result of named query, why not cache the result of page rendering for most very popular pages product and category views ? Of cause not all pages can be cached !
   To achive good result need to hack the VelocityLayoutView. Create a derived class put rendering result into cache. Solution became two times faster. Guess not a bad result.

   1:  public class DerivedVelocityLayoutView extends VelocityLayoutView {

   2:   

   3:  // skipped

   4:   

   5:      private oncurrentMap<String, String> cache;

   6:   

   7:  // skipped

   8:   

   9:      /**

  10:       * Default constructor.

  11:       */

  12:      public DerivedVelocityLayoutView() {

  13:          super();

  14:          this.requestHelper = new RequestHelperImpl();

  15:          this.cache = new MapMaker()

  16:              .concurrencyLevel(16)

  17:              .softValues()

  18:              .expiration(3, TimeUnit.MINUTES)

  19:              .makeMap();

  20:      }

  21:   

  22:  // skipped

  23:   

  24:  /**

  25:       * The resulting context contains any mappings from render, plus screen content.

  26:       * @param velocityContext context.

  27:       * @throws Exception in case of exception

  28:       */

  29:      private void renderScreenContent(final Context velocityContext) throws Exception {

  30:   

  31:          if (logger.isDebugEnabled()) {

  32:              logger.debug("Rendering screen content template [" + getUrl() + "]");

  33:          }

  34:   

  35:          final HttpServletRequest servletRequest = (HttpServletRequest) velocityContext.get("request");

  36:          final String key = getKey(velocityContext);

  37:   

  38:          String renderedPage = cache.get(key);

  39:          if (renderedPage == null) {

  40:              StringWriter sw = new StringWriter();

  41:              Template screenContentTemplate = getTemplate(getUrl());

  42:              screenContentTemplate.merge(velocityContext, sw);

  43:              renderedPage = sw.toString();

  44:              if (isCacheAllowed(servletRequest.getRequestURI())) {

  45:                  cache.put(key, renderedPage);

  46:                  if (logger.isDebugEnabled()) {

  47:                      logger.debug("Cache miss  " + key);

  48:                  }

  49:              }

  50:          } else {

  51:              if (logger.isDebugEnabled()) {

  52:                  logger.debug("Cache hit   " + key);

  53:              }

  54:          }

  55:          // Put rendered content into Velocity context.

  56:          velocityContext.put(this.screenContentKey, renderedPage);

  57:      }

  58:   

  59:  // skipped

  60:   

  61:  }

  62:   

Hornetq JMS integration with Tomcat. Spring configuration. Part 2 of 2

In second part i want to show spring configuration, that allow to use hornetq from spring beans via well know spring JmsTemplate.

    <bean name="mbeanServer" 
          class="java.lang.management.ManagementFactory" 
          factory-method="getPlatformMBeanServer"/>

    <bean name="hornetQSecurityManagerImpl" 
          class="org.hornetq.spi.core.security.HornetQSecurityManagerImpl"/>

    <bean name="fileConfiguration" 
          class="org.hornetq.core.config.impl.FileConfiguration" 
          init-method="start" 
          destroy-method="stop"/> 

    <bean name="hornetQServerImpl" 
          class="org.hornetq.core.server.impl.HornetQServerImpl">
        <constructor-arg ref="fileConfiguration"/>
        <constructor-arg ref="mbeanServer"/>
        <constructor-arg ref="hornetQSecurityManagerImpl"/>
    </bean>

    <bean name="jmsServerManagerImpl" 
          class="org.hornetq.jms.server.impl.JMSServerManagerImpl" 
          init-method="start" 
          destroy-method="stop"> 
        <constructor-arg ref="hornetQServerImpl"/>
    </bean>

    <bean id="inVmConnectionFactory" 
          class="org.hornetq.jms.client.HornetQConnectionFactory">
        <constructor-arg ref="inVmTransportConfiguration"/>
    </bean>

    <bean id="inVmTransportConfiguration" 
          class="org.hornetq.api.core.TransportConfiguration">
        <constructor-arg index="0" value="org.hornetq.core.remoting.impl.invm.InVMConnectorFactory"/>
    </bean>

    <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
        <property name="environment">
            <props>
                <prop key="java.naming.factory.initial">org.apache.naming.java.javaURLContextFactory</prop>
                <prop key="java.naming.factory.url.pkgs">org.apache.naming</prop>
            </props>
        </property>
    </bean>

    <bean id="jmsQueueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiTemplate" ref="jndiTemplate"/>
        <property name="jndiName" value="HQConnectionFactory"/>
    </bean>


    <bean id="customerRegistrationQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiTemplate" ref="jndiTemplate"/>
        <property name="jndiName" value="queue/CustomerRegistrationQueue"/>
    </bean>
    <bean id="customerRegistrationJMSTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="jmsQueueConnectionFactory"/>
        <property name="defaultDestination" ref="customerRegistrationQueue"/>
    </bean>

Configured JmsTemplate can be use in your spring bean. Actually i use jms notification in several system aspects. It allow to have fully decoupled system parts.

My base aspect class is following:

/** 
 * 
 * Base class for notification aspects. 
 * 
 */ 
public class BaseNotificationAspect {

    private final JmsTemplate jmsTemplate;

    /** 
     * Construct base notification aspect class. 
     * @param jmsTemplate jms teplate to use 
     */ 
    public BaseNotificationAspect(final JmsTemplate jmsTemplate) {
        this.jmsTemplate = jmsTemplate; 
    } 

    /** 
     * Send registration notification. 
     * @param serializableMessage  object to send 
     */ 
    protected void sendNotification(final Serializable serializableMessage) {
        if (jmsTemplate != null) {
            jmsTemplate.send(new MessageCreator() { 
                public Message createMessage(final Session session) throws JMSException {
                    return session.createObjectMessage(serializableMessage);
                } 
            }); 
        } 
    } 
}

And derived class has following description as sprind bean:

    <bean id="customerRegistrationAspect" class="com.some.product.name.service.domain.aspect.impl.CustomerRegistrationAspect">
        <constructor-arg index="0" ref="customerRegistrationJMSTemplate"/>
    </bean>

Hornetq JMS integration with Tomcat. Configure jms provider. Part 1 of 2

How to integrate hornetq jms provider with tomcat and apache/tomcat JNDI service.

If you are here, so you realy need to have ability to process some event in asynchronous fashion in your web application on tomcat or other lightweight web server. I dont want to use dedicated jms service , so i need in JVM communications only.

First of all lets check hornetq and tomcat jndi server jar dependency:

 
  <dependency> 
    <groupId>tomcat</groupId> 
    <artifactId>naming-factory</artifactId> 
    <version>5.5.23</version> 
    <scope>test</scope> 
  </dependency> 
  <dependency> 
    <groupId>tomcat</groupId> 
    <artifactId>naming-resources</artifactId> 
    <version>5.5.23</version> 
    <scope>test</scope> 
  </dependency> 
  <dependency> 
     <groupId>org.hornetq</groupId> 
     <artifactId>hornetq-core</artifactId> 
     <version>2.0.0.GA</version> 
     <scope>compile</scope> 
  </dependency> 
  <dependency> 
     <groupId>org.hornetq</groupId> 
     <artifactId>hornetq-jms</artifactId> 
     <version>2.0.0.GA</version> 
     <scope>compile</scope> 
  </dependency> 
  <dependency> 
     <groupId>org.hornetq</groupId> 
     <artifactId>hornetq-logging</artifactId> 
     <version>2.0.0.GA</version> 
     <scope>compile</scope> 
  </dependency> 
  <dependency> 
     <groupId>org.hornetq</groupId> 
     <artifactId>hornetq-transports</artifactId> 
     <version>2.0.0.GA</version> 
     <scope>compile</scope> 
  </dependency> 
  <dependency> 
     <groupId>org.jboss.netty</groupId> 
     <artifactId>netty</artifactId> 
     <version>3.1.0.GA</version> 
  </dependency> 
  <dependency> 
     <groupId>org.jboss.javaee</groupId> 
     <artifactId>jboss-jms-api</artifactId> 
     <version>1.1.0.GA</version> 
     <scope>compile</scope> 
  </dependency> 

As you can see there is no dependency from jboss jnpserver, because im gonne to use tomcat JNDI service.

Configuration in jndi.properties

java.naming.factory.initial=org.apache.naming.java.javaURLContextFactory 
java.naming.factory.url.pkgs=org.apache.naming 

#java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory 
#java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces 
#java.naming.provider.url=jnp://localhost:1099 

My hornetq-jms.xml is following:

<?xml version="1.0" encoding="UTF-8"?>  
<configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"     
    xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">  
    <connection-factory name="InVMConnectionFactory">
      <connectors>
         <connector-ref connector-name="in-vm"/>  
      </connectors>
      <entries> 
          <entry name="/HQConnectionFactory"/>
      </entries>
   </connection-factory>  
   <queue name="DLQ"><entry name="/queue/DLQ"/></queue>
   <queue name="ExpiryQueue"><entry name="/queue/ExpiryQueue"/>
   </queue><queue name="CustomerRegistrationQueue"><entry name="/queue/CustomerRegistrationQueue"/></queue>
</configuration>

Only in JVM connection factory is used and it binded to read/write JNDI context with HQConnectionFactory name.
At this point some can say, that context in tomcat is read only, but if you are have the brain you can perform small investigation and found, that you can bind to jndi context in tomcat, but not in java:/ context. For more information check the method checkWritable in org.apache.naming.NamingContext. If you ac try to use java:/ConnectionFactory name you will get the "javax.naming.NamingException: Context is read only".



File hornetq-configuration.xml

<configuration xmlns="urn:hornetq"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
               xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd"> 

   <connectors>
       <connector name="in-vm"> 
         <factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class> 
      </connector>
   </connectors>

   <acceptors>
       <acceptor name="in-vm"> 
         <factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class> 
         <param key="server-id" value="0"/>         
      </acceptor>
   </acceptors>

   <security-settings> 
      <security-setting match="#">
          <permission type="createDurableQueue" roles="guest"/>
          <permission type="deleteDurableQueue" roles="guest"/>
          <permission type="createTempQueue" roles="guest"/>
          <permission type="deleteTempQueue" roles="guest"/>
          <permission type="send" roles="guest"/>
          <permission type="consume" roles="guest"/>
          <permission type="manage" roles="guest"/>
      </security-setting> 
   </security-settings> 

   <address-settings> 
      <!--default for catch all--> 
      <address-setting match="#">
         <dead-letter-address>jms.queue.DLQ</dead-letter-address> 
         <expiry-address>jms.queue.ExpiryQueue</expiry-address> 
         <redelivery-delay>0</redelivery-delay> 
         <max-size-bytes>-1</max-size-bytes> 
         <page-size-bytes>10485760</page-size-bytes> 
         <message-counter-history-day-limit>10</message-counter-history-day-limit> 
      </address-setting> 
   </address-settings> 

   <paging-directory>${hornetq.data.dir}/paging</paging-directory> 
   <bindings-directory>${hornetq.data.dir}/bindings</bindings-directory> 
   <journal-directory>${hornetq.data.dir}/journal</journal-directory> 
   <large-messages-directory>${hornetq.data.dir}/large-messages</large-messages-directory> 
</configuration>

In servlet container wwe will need the hornetq-users.xml because of security issues.

<configuration xmlns="urn:hornetq"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="urn:hornetq ../schemas/hornetq-users.xsd "> 
    <defaultuser name="guest" password="guest">
        <role name="guest"/>
    </defaultuser>
</configuration>    

Thats all with HornetQ and Tomcat. Next article will be with spring configuration files, that allow you to use hornetq via spring

No destination with id 'serviceName' is registered with any service.

WOW ! "No destination with id 'serviceName' is registered with any service." !
First of all check the spring config file. Is bean definition has a <flex:remoting-destination/> ?

"Statement is closed" SQLExeption or OutOfMemory exception during dbunit testing.

Do you have a problem with "Statement is closed" SQLExeption during dbunit testing ?
I had, because wrong dataSource configuration, so check the maxOpenPreparedStatements parameter. Your dataSource should looks like described below:

  1 <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
  2         <property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
  3         <property name="url" value="jdbc:hsqldb:mem:testnpadb"/>
  4         <property name="username" value="sa"/>
  5         <property name="password" value=""/>
  6         <property name="initialSize" value="5"/>
  7         <property name="maxActive" value="100"/>
  8         <property name="poolPreparedStatements" value="true"/>
  9         <property name="maxOpenPreparedStatements" value="100"/>
 10         <property name="defaultAutoCommit" value="true"/>
 11     </bean>

OutOfMemory ? Check the tear down method, guess you have to shutdown spring context.

Second level cache in hibernate

Do you use hibernate ? I guess - yes, if you are here.

Do you use second level cache and query cache with hibernate? Are you happy with performance? I dont know... probably. Actually I am not happy with hibernate performance with query cache and 2nd level cache. Lets look inside ;)

First of all you have to understand, that 2nd level cache used when you perform lookup entity by id. Sound good, but by default entity will be cache without collections. The first solution - enable cache for collection in configurations, it will works for bags, lists, collections, sets, maps.

Second - query cache. It cache only the ids and must be used in conjunction second level cache.

Everything not bad at surface, but when you create hibernate and ehcache configuration and run you solution you will be surprised. Hibernate still hit you db :( Root cause - collection cache will not works for map, that has a map-key, and you db will be hitted a lot of times per single entity.

How to solve this ? Throw away the hibernate cache and use cache aspect in your application, it save a lot of you times, IO operations and CPU. It more predictable way that hibernate cache and allow to cache that you really need :)