четверг, 15 ноября 2012 г.

Wicket, nginx, ssl, proxy

HI there !

This post has answer to question - "How to configure reverse proxy and tomcat for Wicket application". Wicket itself has declarative instructions, what pages should be secured, and what - not. This configured via appropriate annotation @RequireHttps on page class and application configuration , like this:

final HttpsConfig httpsConfig = new HttpsConfig(

             8080,8443
            );
final HttpsMapper httpsMapper = new HttpsMapper(getRootRequestMapper(), httpsConfig);

setRootRequestMapper(httpsMapper);

//to be correct need to use 80 and 443 ports for this article


How it works - in case if page with @RequireHttps annotation is openet via not secured port, wicket send http error code 302 with url, which point  to secure url for requested page. And vise versa if not secure page opening via secure url wicket redirect to unsecure version. This sophisticated behavior not always aligned with usual web application behavior, so to support it need correct configuration on ngnix, tomcat.

Configure nginx to be an reverse proxy for apache tomcat with ssl termination very simple and fast.

First step generate certificate and key
openssl req -new -x509 -days 2000 -nodes -out cert.pem -keyout cert.key

Second step configure nginx

server {

        listen 443;

        ssl on;
        ssl_session_timeout 5m;
        ssl_protocols  SSLv3 TLSv1;
    ssl_certificate /var/cert/cert.pem;
        ssl_certificate_key /var/cert/cert.key;
        #ssl_session_cache shared:SSL:10m; #Not works in win 7
        location / {
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
       proxy_set_header X-Forwarded-Proto https;
            proxy_redirect off;
            proxy_connect_timeout      240;
            proxy_send_timeout         240;
            proxy_read_timeout         240;
            # note, there is not SSL here! plain HTTP is used
            proxy_pass http://localhost:8080/;
        }
     }

    server {
        listen       80;
        server_name  localhost;

        location / {
       proxy_pass http://localhost:8080/;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

Be sure, than X-Forwarded-Proto present in your configuration in server record for 443 port

According to this configuration ssl will be terminated on ngnix and forwarded to unsecure port 8080 on tomcat, but wicket will perform check, described about and send redirect to secure url and loop will be created. To avoid redirect looping need add some configuration line to tomcat config



            unpackWARs="true" autoDeploy="true">
... skipped ...
...skipped ...

This valve analyse the X-Forwarded-Proto header, and it it set , in our case by nginx, valve set schema and secure flag in http request to https and true. 




воскресенье, 7 октября 2012 г.

Wicket behind a front-end proxy with https support


Original acrticle located here https://cwiki.apache.org/WICKET/wicket-behind-a-front-end-proxy.html but, as usual, some important parts are missing. How to support https in wicket. I`ll provide step by steps instructions with apache httpd, mod_proxy_ajp, tomcat and wicket. This was originally done for yes-cart project http://code.google.com/p/yes-cart under windows, so my local pathes are provided.

First of all need to create ssl certificate for apache httpd server, apache for windows in wamp comes with preinstalled openssl. So jump to apache bin directory and run following commands:

openssl req -new -config ../conf/openssl.cnf > yes-shop.csr
openssl rsa -in privkey.pem -out yes-shop.key
openssl x509 -in yes-shop.csr -out yes-shop.cert -req -signkey yes-shop.key -days 365

Do not forget the password.

Create folders under.
D:\dev\wamp\bin\apache\apache2.2.22\conf\extra
mkdir certs
mkdir crl
mkdir newcerts
mkdir private

Copy yes-shop.cert yes-shop.csr yes-shop.key  files from apache bin folder to certs
Copy  .rnd privkey.pem to private folder

Now lets configure ssl in apache httpd
Open D:\dev\wamp\bin\apache\apache2.2.22\conf\httpd.conf and load modules
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so

Include extra config file Include conf/extra/httpd-vhosts.conf  all my virtual hosts located here, as well as ssl instructions for this example. So edit extra/httpd-vhosts.conf  and add lines

SSLSessionCache "shmcb:D:/dev/wamp/bin/apache/apache2.2.22/logs/ssl_scache(512000)"
SSLMutex default
SSLCertificateFile "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/certs/yes-shop.cert"
SSLCertificateKeyFile "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/certs/yes-shop.key"
SSLCARevocationPath "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/crl"

Locate openssl.cnf file and edit line to
dir = D:/dev/wamp/bin/apache/apache2.2.22/conf/extra # Where everything is kept

Configure virtual hosts, so my file looks like


NameVirtualHost *:80
 
<VirtualHost *:80>
 ServerName localhost
 ProxyRequests Off
 ProxyPreserveHost On
 <Proxy *>
         Order deny,allow
         Allow from all
 </Proxy>
 ProxyPass / ajp://localhost:8009/
 ProxyPassReverse / ajp://localhost:8009/
 <Location />
         Order allow,deny
         Allow from all
 </Location>
</VirtualHost>
 
 
SSLSessionCache "shmcb:D:/dev/wamp/bin/apache/apache2.2.22/logs/ssl_scache(512000)"
SSLMutex default
SSLCertificateFile "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/certs/yes-shop.cert"
SSLCertificateKeyFile "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/certs/yes-shop.key"
SSLCARevocationPath "D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/crl"
 
Listen 443
 
NameVirtualHost *:443
<VirtualHost *:443>
    SSLEngine On
    SSLCertificateFile D:/dev/wamp/bin/apache/apache2.2.22/conf/extra/certs/yes-shop.cert
    ProxyPreserveHost On
    ProxyPass / ajp://localhost:8009/
    ProxyPassReverse / ajp://localhost:8009/
</VirtualHost>
 


Configure wicket application

/**
     * {@inheritDoc}
     */
    protected void init() {
.....
            final HttpsConfig httpsConfig = new HttpsConfig(
                    80,
                    443
            );

            final HttpsMapper httpsMapper = new HttpsMapper(getRootRequestMapper(), httpsConfig);

            setRootRequestMapper(httpsMapper);
}

Tomcat configured to accept ajp connection
<Connector port="8009" enableLookups="false" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8"/>

Proxy from 443 to 8443 will not work 

понедельник, 6 августа 2012 г.

XSL Grouping data for sum, avg, etc

It it easy to calculate summary values at the end of document using something like this:
<fo:table-cell border="solid 1px black" text-align="right" font-weight="bold">
    <fo:block>
        <xsl:value-of   select="format-number(sum(./yes-report/object-array/big-decimal[position() = 2]), '##0.00')"/>
    </fo:block>
</fo:table-cell>
But sometimes need to have intermediate results for some data group, for example annual monthly based sales report per each departments or payments operation via different payment gateways with different currencies. Given example point to fact, than need to have nested group. "Well know" method to acheive this - use Steve Muench method:

1. Define grouping key
<xsl:key name="currencyGroup" match="org.yes.cart.payment.persistence.entity.impl.CustomerOrderPaymentEntity"
             use="concat(orderCurrency, transactionOperation, transactionGatewayLabel)"/>
use concatination to define complex key instead one node key.

2. Add appropriate sorting
<xsl:for-each select="//org.yes.cart.payment.persistence.entity.impl.CustomerOrderPaymentEntity[generate-id(.)=generate-id(key('currencyGroup', concat(orderCurrency, transactionOperation, transactionGatewayLabel))[1])]">
     <xsl:sort select="orderCurrency"/>
     <xsl:sort select="transactionOperation"/>
     <xsl:sort select="transactionGatewayLabel"/>


3. Calculate summary at the end of each group
<xsl:if test="position() = last()">
        <fo:table-row>
            <fo:table-cell border="solid 1px black"
                           number-columns-spanned="11" font-weight="bold">
                <fo:block>Summary</fo:block>
            </fo:table-cell>
            <fo:table-cell border="solid 1px black" text-align="right"
                           font-weight="bold">
                <fo:block>
                    <xsl:value-of
                            select="format-number(sum(
                        key('currencyGroup', concat(orderCurrency, transactionOperation, transactionGatewayLabel))/paymentAmount
                        ), '##0.00')"/>
                </fo:block>
            </fo:table-cell>
        </fo:table-row>
        <fo:table-row>
            <fo:table-cell border="none"
                           number-columns-spanned="12" font-weight="bold">
                <fo:block><fo:leader /></fo:block>
            </fo:table-cell>
        </fo:table-row>
    </xsl:if>


</xsl:for-each>


Result example with xsl-fo is following:


Source files located here paymentreport.zip including xml, xslf


пятница, 27 июля 2012 г.

How to work with huge output in Jersey RESTful Web services.

How to work with huge output in Jersey RESTful Web services.
The typical usecase for Jersey + JAXB - work with whole XML document and single JAXB objects tree. But sometimes the set of objects very huge and output can not be created, because of OutOfMemoryError or other reasons. In this case need to send output as stream. Code if following:
package az.edu.hugedoc;

/**
 * User: Igor Azarny iazarny@yahoo.com
 * Date: 7/27/12
 * Time: 1:56 PM
 */

import az.edu.data.model.ObjectFactory;
import az.edu.data.model.Person;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.StreamingOutput;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;
import java.io.IOException;
import java.io.OutputStream;
import java.util.UUID;


@Path("xmlrepo")
public class HugeDocAsXmlStream {

    private ObjectFactory of;
    private Marshaller marshaller;

    public HugeDocAsXmlStream() {
        try {
            JAXBContext jc = JAXBContext.newInstance(Person.class);
            marshaller = jc.createMarshaller();
            marshaller.setProperty(Marshaller.JAXB_FRAGMENT, true);
            of = new ObjectFactory();
        } catch (JAXBException e) {
            e.printStackTrace(); // impossible
        }

    }

    @GET
    @Produces({"text/xml"})
    public StreamingOutput getHugeDocAsXmlStreamOutput() {
        return new StreamingOutput() {
            public void write(OutputStream output) throws IOException, WebApplicationException {
                XMLStreamWriter xsw = null;
                try {
                    xsw = XMLOutputFactory.newInstance().createXMLStreamWriter(output);
                    xsw.writeStartElement("personrow");

                    /**
                     *
                     * Original idea - to use hibernate scrollable result of objects.
                     *
                     * ScrollableResults results = hibernateSession.createCriteria(persistentClass.class)
                     * .setFetchSize(BATCH_SIZE)
                     * .scroll(ScrollMode.FORWARD_ONLY);
                     *
                     * while (results.next()) {
                     *     index++;
                     *     T entity = (T) results.get(0);
                     *
                     */

                    for (int i = 0; i < 1000; i++) {
                        JAXBElement<Person> personJAXBElement = of.createPerson(
                                new Person(i, "Ivan " + UUID.randomUUID().toString(), "KillTesterOff " + UUID.randomUUID().toString())
                        );
                        marshaller.marshal(personJAXBElement, xsw);
                        xsw.flush();    // Force flush the stream

                    }
                    xsw.writeEndElement();
                    xsw.writeEndDocument();
                } catch (Exception e) {
                    throw new WebApplicationException(e);
                } finally {
                    if (xsw != null) {
                        try {
                            xsw.close();
                        } catch (XMLStreamException e) {
                            e.printStackTrace();
                        }
                    }
                }
            }
        };
    }

}

суббота, 21 июля 2012 г.

There will always be things that I do not understand

Code found in JAXB
/**
     * Makes sure that we are running with 2.1 JAXB API,
     * and report an error if not.
     */
    static {
        try {
            XmlSchema s = null;
            s.location();
        } catch (NullPointerException e) {
            // as epxected
        } catch (NoSuchMethodError e) {
            // this is not a 2.1 API. Where is it being loaded from?
            Messages res;
            if(XmlSchema.class.getClassLoader()==null)
                res = Messages.INCOMPATIBLE_API_VERSION_MUSTANG;
            else
                res = Messages.INCOMPATIBLE_API_VERSION;

            throw new LinkageError( res.format(
                Which.which(XmlSchema.class),
                Which.which(ModelBuilder.class)
            ));
        }
    }

суббота, 14 апреля 2012 г.

Force reindex single entity with hibernate search.

Hibernate search documentations is weak area. In case if you catch the  "IllegalStateException: Could not get property value"  try to unproxy entity.

FullTextSession fullTextSession = Search.getFullTextSession(sessionFactory.getCurrentSession());
fullTextSession.index(HibernateHelper.unproxy(entity));

пятница, 3 февраля 2012 г.

Hibernate search migration to new version.

    Have you got "SearchException: Unable to perform work. Entity Class is not @Indexed nor hosts @ContainedIn: class java.lang.String" ? I guess  yes, so try to use FieldBridge to sovle you problems with embeded indexes.

    I am migrate from 3.1.0.GA to 3.4.1.Final and collect a lot of errors. I am pretty sure, that hibernate search team dont event think about backward compatibility. Hibernate search unit test cover only simple cases , documentation is also weak.