среда, 26 января 2011 г.

Speedup unspeedable

    A few days ago i have got some well know e-commerce platform to speed up it. Application use hibernate, solr, spring and spring mvc with apache velocity as view and look like very over optimised, cached everything, that possible. It cached solr results, hibernate use second level cache, web tier use etag, etc. I was frustrated, because no easy way to boost performance. After several hours of investigation some bright idea came to me - if solution cache the result of named query, why not cache the result of page rendering for most very popular pages product and category views ? Of cause not all pages can be cached !
   To achive good result need to hack the VelocityLayoutView. Create a derived class put rendering result into cache. Solution became two times faster. Guess not a bad result.


   1:  public class DerivedVelocityLayoutView extends VelocityLayoutView {
   2:   
   3:  // skipped
   4:   
   5:      private oncurrentMap<String, String> cache;
   6:   
   7:  // skipped
   8:   
   9:      /**
  10:       * Default constructor.
  11:       */
  12:      public DerivedVelocityLayoutView() {
  13:          super();
  14:          this.requestHelper = new RequestHelperImpl();
  15:          this.cache = new MapMaker()
  16:              .concurrencyLevel(16)
  17:              .softValues()
  18:              .expiration(3, TimeUnit.MINUTES)
  19:              .makeMap();
  20:      }
  21:   
  22:  // skipped
  23:   
  24:  /**
  25:       * The resulting context contains any mappings from render, plus screen content.
  26:       * @param velocityContext context.
  27:       * @throws Exception in case of exception
  28:       */
  29:      private void renderScreenContent(final Context velocityContext) throws Exception {
  30:   
  31:          if (logger.isDebugEnabled()) {
  32:              logger.debug("Rendering screen content template [" + getUrl() + "]");
  33:          }
  34:   
  35:          final HttpServletRequest servletRequest = (HttpServletRequest) velocityContext.get("request");
  36:          final String key = getKey(velocityContext);
  37:   
  38:          String renderedPage = cache.get(key);
  39:          if (renderedPage == null) {
  40:              StringWriter sw = new StringWriter();
  41:              Template screenContentTemplate = getTemplate(getUrl());
  42:              screenContentTemplate.merge(velocityContext, sw);
  43:              renderedPage = sw.toString();
  44:              if (isCacheAllowed(servletRequest.getRequestURI())) {
  45:                  cache.put(key, renderedPage);
  46:                  if (logger.isDebugEnabled()) {
  47:                      logger.debug("Cache miss  " + key);
  48:                  }
  49:              }
  50:          } else {
  51:              if (logger.isDebugEnabled()) {
  52:                  logger.debug("Cache hit   " + key);
  53:              }
  54:          }
  55:          // Put rendered content into Velocity context.
  56:          velocityContext.put(this.screenContentKey, renderedPage);
  57:      }
  58:   
  59:  // skipped
  60:   
  61:  }
  62:   

суббота, 2 октября 2010 г.

пятница, 27 августа 2010 г.

Hornetq JMS integration with Tomcat. Spring configuration. Part 2 of 2

In second part i want to show spring configuration, that allow to use hornetq from spring beans via well know spring JmsTemplate.

    <bean name="mbeanServer" 
class="java.lang.management.ManagementFactory"
factory-method="getPlatformMBeanServer"/>

<bean name="hornetQSecurityManagerImpl"
class="org.hornetq.spi.core.security.HornetQSecurityManagerImpl"/>

<bean name="fileConfiguration"
class="org.hornetq.core.config.impl.FileConfiguration"
init-method="start"
destroy-method="stop"/>

<bean name="hornetQServerImpl"
class="org.hornetq.core.server.impl.HornetQServerImpl">
<constructor-arg ref="fileConfiguration"/>
<constructor-arg ref="mbeanServer"/>
<constructor-arg ref="hornetQSecurityManagerImpl"/>
</bean>

<bean name="jmsServerManagerImpl"
class="org.hornetq.jms.server.impl.JMSServerManagerImpl"
init-method="start"
destroy-method="stop">
<constructor-arg ref="hornetQServerImpl"/>
</bean>

<bean id="inVmConnectionFactory"
class="org.hornetq.jms.client.HornetQConnectionFactory">
<constructor-arg ref="inVmTransportConfiguration"/>
</bean>

<bean id="inVmTransportConfiguration"
class="org.hornetq.api.core.TransportConfiguration">
<constructor-arg index="0" value="org.hornetq.core.remoting.impl.invm.InVMConnectorFactory"/>
</bean>


    <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
<prop key="java.naming.factory.initial">org.apache.naming.java.javaURLContextFactory</prop>
<prop key="java.naming.factory.url.pkgs">org.apache.naming</prop>
</props>
</property>
</bean>

<bean id="jmsQueueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jndiTemplate"/>
<property name="jndiName" value="HQConnectionFactory"/>
</bean>


<bean id="customerRegistrationQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jndiTemplate"/>
<property name="jndiName" value="queue/CustomerRegistrationQueue"/>
</bean>
<bean id="customerRegistrationJMSTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsQueueConnectionFactory"/>
<property name="defaultDestination" ref="customerRegistrationQueue"/>
</bean>



Configured JmsTemplate can be use in your spring bean. Actually i use jms notification in several system aspects. It allow to have fully decoupled system parts.

My base aspect class is following:
/** 
*
* Base class for notification aspects.
*
*/

public class BaseNotificationAspect {

private final JmsTemplate jmsTemplate;

/**
* Construct base notification aspect class.
* @param jmsTemplate jms teplate to use
*/

public BaseNotificationAspect(final JmsTemplate jmsTemplate) {
this.jmsTemplate = jmsTemplate;
}

/**
* Send registration notification.
* @param serializableMessage object to send
*/

protected void sendNotification(final Serializable serializableMessage) {
if (jmsTemplate != null) {
jmsTemplate.send(new MessageCreator() {
public Message createMessage(final Session session) throws JMSException {
return session.createObjectMessage(serializableMessage);
}
});
}
}
}


And derived class has following description as sprind bean:
    <bean id="customerRegistrationAspect" class="com.some.product.name.service.domain.aspect.impl.CustomerRegistrationAspect">
<constructor-arg index="0" ref="customerRegistrationJMSTemplate"/>
</bean>

среда, 25 августа 2010 г.

Hornetq JMS integration with Tomcat. Configure jms provider. Part 1 of 2

How to integrate hornetq jms provider with tomcat and apache/tomcat JNDI service.

If you are here, so you realy need to have ability to process some event in asynchronous fashion in your web application on tomcat or other lightweight web server. I dont want to use dedicated jms service , so i need in JVM communications only.

First of all lets check hornetq and tomcat jndi server jar dependency:
 
<dependency>
<groupId>tomcat</groupId>
<artifactId>naming-factory</artifactId>
<version>5.5.23</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>tomcat</groupId>
<artifactId>naming-resources</artifactId>
<version>5.5.23</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hornetq</groupId>
<artifactId>hornetq-core</artifactId>
<version>2.0.0.GA</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.hornetq</groupId>
<artifactId>hornetq-jms</artifactId>
<version>2.0.0.GA</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.hornetq</groupId>
<artifactId>hornetq-logging</artifactId>
<version>2.0.0.GA</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.hornetq</groupId>
<artifactId>hornetq-transports</artifactId>
<version>2.0.0.GA</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.jboss.netty</groupId>
<artifactId>netty</artifactId>
<version>3.1.0.GA</version>
</dependency>
<dependency>
<groupId>org.jboss.javaee</groupId>
<artifactId>jboss-jms-api</artifactId>
<version>1.1.0.GA</version>
<scope>compile</scope>
</dependency>


As you can see there is no dependency from jboss jnpserver, because im gonne to use tomcat JNDI service.

Configuration in jndi.properties
java.naming.factory.initial=org.apache.naming.java.javaURLContextFactory 
java.naming.factory.url.pkgs=org.apache.naming

#java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
#java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
#java.naming.provider.url=jnp://localhost:1099




My hornetq-jms.xml is following:

<?xml version="1.0" encoding="UTF-8"?>  
<configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<connection-factory name="InVMConnectionFactory">
<connectors>
<connector-ref connector-name="in-vm"/>
</connectors>
<entries>
<entry name="/HQConnectionFactory"/>
</entries>
</connection-factory>
<queue name="DLQ"><entry name="/queue/DLQ"/></queue>
<queue name="ExpiryQueue"><entry name="/queue/ExpiryQueue"/>
</queue><queue name="CustomerRegistrationQueue"><entry name="/queue/CustomerRegistrationQueue"/></queue>
</configuration>


Only in JVM connection factory is used and it binded to read/write JNDI context with HQConnectionFactory name.
At this point some can say, that context in tomcat is read only, but if you are have the brain you can perform small investigation and found, that you can bind to jndi context in tomcat, but not in java:/ context. For more information check the method checkWritable in org.apache.naming.NamingContext. If you ac try to use java:/ConnectionFactory name you will get the "javax.naming.NamingException: Context is read only".



File hornetq-configuration.xml

<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">

<connectors>
<connector name="in-vm">
<factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class>
</connector>
</connectors>

<acceptors>
<acceptor name="in-vm">
<factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class>
<param key="server-id" value="0"/>
</acceptor>
</acceptors>

<security-settings>
<security-setting match="#">
<permission type="createDurableQueue" roles="guest"/>
<permission type="deleteDurableQueue" roles="guest"/>
<permission type="createTempQueue" roles="guest"/>
<permission type="deleteTempQueue" roles="guest"/>
<permission type="send" roles="guest"/>
<permission type="consume" roles="guest"/>
<permission type="manage" roles="guest"/>
</security-setting>
</security-settings>

<address-settings>
<!--default for catch all-->
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<max-size-bytes>-1</max-size-bytes>
<page-size-bytes>10485760</page-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
</address-setting>
</address-settings>

<paging-directory>${hornetq.data.dir}/paging</paging-directory>
<bindings-directory>${hornetq.data.dir}/bindings</bindings-directory>
<journal-directory>${hornetq.data.dir}/journal</journal-directory>
<large-messages-directory>${hornetq.data.dir}/large-messages</large-messages-directory>
</configuration>



In servlet container wwe will need the hornetq-users.xml because of security issues.

<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq ../schemas/hornetq-users.xsd ">
<defaultuser name="guest" password="guest">
<role name="guest"/>
</defaultuser>
</configuration>




Thats all with HornetQ and Tomcat. Next article will be with spring configuration files, that allow you to use hornetq via spring

пятница, 13 августа 2010 г.

Wicket and Groovy integration

Forgot to add to "Wicket and Groovy integration" :
in your wicket application init method must have


getApplicationSettings().setClassResolver(new GroovyClassResolver(this));

пятница, 6 августа 2010 г.

Wicket and Groovy integration.

Perfect idea to integrate wicket and groovy. From one side the wicket way - strong separation between code and markup, from other side groovy as code behind the markup. And to get extra flexibility i can use the wicket auto components with groovy behind ! Cool !!!

Nice try for wicket 1.3 and "old" groovy 1.6 you can find here http://bigheadco.blogspot.com/2007/03/wicket-and-groovy.html BTW wicket-contrib-groovy was moved to attic in svn repository.

So for wicket 1.4 and groovy 1.7.4 need only two classes from stuff above GroovyWarClassLoader and GroovyClassResolver. The rest is obsolete, because groovy 1.7 support anonymous inner classes and nested classes. Original GroovyWarClassLoader and GroovyClassResolver need to be modified for wicket 1.4.

 
/*
* $Id: GroovyWarClassLoader.java 524 2006-01-08 09:30:40Z jdonnerstag $
* $Revision: 524 $ $Date: 2006-01-08 01:30:40 -0800 (Sun, 08 Jan 2006) $
*
* ==================================================================== Licensed
* under the Apache License, Version 2.0 (the "License"); you may not use this
* file except in compliance with the License. You may obtain a copy of the
* License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/

package wicket.contrib.groovy;

import groovy.lang.GroovyClassLoader;

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.codehaus.groovy.control.CompilationFailedException;


/**
* I had some issues with the GroovyClassLoader (1.0.7-beta) which failed to
* load a groovy files contained in the same war file. E.g.
*
* <pre>
* wicket-examples.war!/web-inf/classes/groovy/A.groovy
* wicket-examples.war!/web-inf/classes/groovy/B.groovy
*
* public class A {
* public A() {
* B b = new B()
* }
* }
* </pre>
*
* GroovyClassLoader did not load Groovy class B.
*
* @author Juergen Donnerstag
*/

public class GroovyWarClassLoader extends GroovyClassLoader
{
/** Logger */
final private static Log log = LogFactory.getLog(GroovyWarClassLoader.class);

/**
* Constructor
*
* @param loader
* The servlet's class loader
*/

public GroovyWarClassLoader(ClassLoader loader)
{
super(loader);
}

/**
* First try parent's implementation. If it fails, it'll throw a
* ClassCastException. Catch it and than apply additional means to load the
* class.
*
* @param name
* Class name incl. package
* @return The Class loaded, if found
* @throws ClassNotFoundException,
* if class could not be loaded
*/

protected Class findClass(final String name) throws ClassNotFoundException
{
try
{
// Try Groovies default implementation first
return super.findClass(name);
}
catch (ClassNotFoundException ex)
{
log.debug("class name: " + name);

// classname => filename
//String filename = Strings.replaceAll(name, ".", "/") + ".groovy";
String filename = name.replace('.','/') + ".groovy";

// File exists?
final URL url = getResource(filename);
if (url != null)
{
try
{
// Get Groovy to parse the file and create the Class
final InputStream in = url.openStream();
if (in != null)
{
Class clazz = parseClass(in);
if (clazz != null)
{
return clazz;
}
}
else
{
log.warn("Groovy file not found: " + filename);
}
}
catch (CompilationFailedException e)
{
throw new ClassNotFoundException("Error parsing groovy file: "
+ filename, e);
}
catch (IOException e)
{
throw new ClassNotFoundException("Error reading groovy file: "
+ filename, e);
}
catch (Throwable e)
{
throw new ClassNotFoundException("Error while reading groovy file: "
+ filename, e);
}
}

throw ex;
}
}
}


----------------------------------------------------------------------------------------------------

 
/*
* $Id: GroovyClassResolver.java 548 2006-01-19 23:59:30Z joco01 $
* $Revision: 548 $ $Date: 2006-01-19 15:59:30 -0800 (Thu, 19 Jan 2006) $
*
* ==============================================================================
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/

package wicket.contrib.groovy;

import java.io.InputStream;
import java.io.IOException;
import java.util.*;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.codehaus.groovy.control.CompilationFailedException;

import org.apache.wicket.Application;
import org.apache.wicket.application.IClassResolver;
import org.apache.wicket.WicketRuntimeException;
import org.apache.wicket.application.DefaultClassResolver;
import org.apache.wicket.util.listener.IChangeListener;
import org.apache.wicket.util.resource.IResourceStream;
import org.apache.wicket.util.watch.IModificationWatcher;

import java.util.concurrent.ConcurrentHashMap;
import java.net.URL;

/**
* Extends the default Page Factory to allow for Groovy based classes.
* Modifications to groovy files are tracked and files are reloaded if modified.
*
* @author Juergen Donnerstag
*/

public class GroovyClassResolver implements IClassResolver
{
/** Logging */
private static final Log log = LogFactory.getLog(GroovyClassResolver.class);

/**
* Caching map of class name to groovy class; not sure if GroovyClassLoader
* does it as well
*/

//private final Map classCache = new ConcurrentReaderHashMap();
private final Map classCache = new ConcurrentHashMap();

/** Default class resolver */
private final IClassResolver defaultClassResolver = new DefaultClassResolver();

/** Application */
private final Application application;

/**
* Constructor
*
* @param application
* Application
*/

public GroovyClassResolver(final Application application)
{
this.application = application;
}

/**
*
* @see org.apache.wicket.application.IClassResolver#getResources(java.lang.String)
*/

public Iterator<URL> getResources(String name)
{
HashSet<URL> loadedFiles = new HashSet<URL>();
try
{
// Try the classloader for the wicket jar/bundle
Enumeration<URL> resources = Application.class.getClassLoader().getResources(name);
loadResources(resources, loadedFiles);

// Try the classloader for the user's application jar/bundle
resources = Application.get().getClass().getClassLoader().getResources(name);
loadResources(resources, loadedFiles);

// Try the context class loader
resources = Thread.currentThread().getContextClassLoader().getResources(name);
loadResources(resources, loadedFiles);
}
catch (IOException e)
{
throw new WicketRuntimeException(e);
}

return loadedFiles.iterator();
}

/**
*
* @param resources
* @param loadedFiles
*/

private void loadResources(Enumeration<URL> resources, Set<URL> loadedFiles)
{
if (resources != null)
{
while (resources.hasMoreElements())
{
final URL url = resources.nextElement();
if (!loadedFiles.contains(url))
{
loadedFiles.add(url);
}
}
}
}




/**
* Resolve the class for the given classname. First try standard java
* classes, then groovy files. Groovy file name must be
* &lt;classname&gt;.groovy.
*
* @param classname
* The object's class name
* @return The class
* @see IClassResolver#resolveClass(String)
*/

public Class resolveClass(final String classname) throws ClassNotFoundException
{
try
{
return defaultClassResolver.resolveClass(classname);
}
catch (WicketRuntimeException ex)
{
; // default resolver failed. Try the groovy specific loader next
}
catch (ClassNotFoundException ex)
{
; // default resolver failed. Try the groovy specific loader next
}

// If definition already loaded, ...
Class groovyPageClass = (Class) classCache.get(classname);
if (groovyPageClass != null)
{
return groovyPageClass;
}

// Else, try Groovy.
final IResourceStream resource = application.getResourceSettings()
.getResourceStreamLocator().locate(getClass(),
classname.replace('.', '/'), null, null, ".groovy");

if (resource != null)
{
try
{
// Load the groovy file, get the Class and watch for changes
groovyPageClass = loadGroovyFileAndWatchForChanges(classname, resource);
if (groovyPageClass != null)
{
return groovyPageClass;
}
}
catch (WicketRuntimeException ex)
{
throw new WicketRuntimeException("Unable to load class with name: "
+ classname, ex);
}
}
else
{
throw new WicketRuntimeException("File not found: " + resource);
}

throw new WicketRuntimeException("Unable to load class with name: " + classname);
}


/**
* Load a Groovy file, create a Class object and put it into the cache
*
* @param classname
* The class name to be created by the Groovy filename
* @param resource
* The Groovy resource
* @return the Class object created by the groovy resouce
*/

private final Class loadGroovyFile(String classname, final IResourceStream resource)
{
// Ensure that we use the correct classloader so that we can find
// classes in an application server.
ClassLoader cl = Thread.currentThread().getContextClassLoader();
if (cl == null)
{
cl = GroovyClassResolver.class.getClassLoader();
}

final GroovyWarClassLoader groovyClassLoader = new GroovyWarClassLoader(cl);
Class clazz = null;

try
{
final InputStream in = resource.getInputStream();
if (in != null)
{
clazz = groovyClassLoader.parseClass(in);
if (clazz != null)
{
// this is necessary because with groovy the filename can be
// different from the class definition included.
if (false == classname.equals(clazz.getName()))
{
log.warn("Though it is possible, the Groovy file name and "
+ "the java class name defined in that file SHOULD "
+ "match and follow the java rules");
}
classname = clazz.getName();
}
}
else
{
log.warn("Groovy file not found: " + resource);
}
}
catch (CompilationFailedException e)
{
throw new WicketRuntimeException("Error parsing groovy file: " + resource, e);
}
catch (Throwable e)
{
throw new WicketRuntimeException("Error while reading groovy file: "
+ resource, e);
}
finally
{
if (clazz == null)
{
// Groovy file not found; error while compiling etc..
// Remove it from cache
classCache.remove(classname);
}
else
{
// Put the new class definition into the cache
classCache.put(classname, clazz);
}
}

return clazz;
}

/**
* Load the groovy file and watch for changes. If changes to the groovy
* happens, than reload the file.
*
* @param classname
* @param resource
* @return Loaded class
*/

private Class loadGroovyFileAndWatchForChanges(final String classname,
final IResourceStream resource)
{
// Watch file in the future
final IModificationWatcher watcher = application.getResourceSettings()
.getResourceWatcher(true);

if (watcher != null)
{
watcher.add(resource, new IChangeListener()
{
public void onChange()
{
try
{
log.info("Reloading groovy file from " + resource);

// Reload file and update cache
final Class clazz = loadGroovyFile(classname, resource);
log.debug("Groovy file contained definition for class: "
+ clazz.getName());
}
catch (Exception e)
{
log.error("Unable to load groovyy file: " + resource, e);
}
}
});
}

log.info("Loading groovy file from " + resource);
return loadGroovyFile(classname, resource);
}
}