Tuesday, July 21, 2009

Can Glassfish (v3) saturate 1 GBit/s?


Server specs
Dual Nehalem 5520 (4x2.26 Ghz. HT makes for a total of 16 virtual cpu's)
12 GB DDR 3 memory.
2x250 GB SATA RAID 1

What was tested
Using siege to randomly request files ranging from 1024 to 1048576 bytes.
The Siege "server" uses the same hardware.

Glassfish startup settings

-XX:+UnlockDiagnosticVMOptions -XX:MaxPermSize=192m -XX:NewRatio=2 -XX:+LogVMOutput -XX:LogFile=/home/glassfish/glassfishv3/glassfish/domains/domain1/logs/jvm.log -Xmx8000m -Djdbc.drivers=org.apache.derby.jdbc.ClientDriver -Djavax.net.ssl.trustStore=/home/glassfish/glassfishv3/glassfish/domains/domain1/config/cacerts.jks -Djavax.net.ssl.keyStore=/home/glassfish/glassfishv3/glassfish/domains/domain1/config/keystore.jks -Djava.security.policy=/home/glassfish/glassfishv3/glassfish/domains/domain1/config/server.policy -Dcom.sun.aas.instanceRoot=/home/glassfish/glassfishv3/glassfish/domains/domain1 -Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory -Djava.security.auth.login.config=/home/glassfish/glassfishv3/glassfish/domains/domain1/config/login.conf -Djava.endorsed.dirs=/home/glassfish/glassfishv3/glassfish/lib/endorsed -Dcom.sun.aas.installRoot=/home/glassfish/glassfishv3/glassfish -Djava.ext.dirs=/usr/java/jdk1.6.0_14/lib/ext:/usr/java/jdk1.6.0_14/jre/lib/ext:/home/glassfish/glassfishv3/glassfish/domains/domain1/lib/ext:/home/glassfish/glassfishv3/javadb/lib


top - 06:40:49 up 30 days, 17:22, 2 users, load average: 1.23, 0.81, 0.53
Tasks: 281 total, 1 running, 280 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.1%sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12290980k total, 5732924k used, 6558056k free, 266296k buffers
Swap: 6289436k total, 0k used, 6289436k free, 4548592k cached

10865 root 20 0 8597m 428m 10m S 70.9 3.6 15:14.55 java


** SIEGE 2.66
** Preparing 512 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 491907 hits
Availability: 100.00 %
Elapsed time: 1510.97 secs
Data transferred: 168817.77 MB
Response time: 1.57 secs
Transaction rate: 325.56 trans/sec
Throughput: 111.73 MB/sec
Concurrency: 511.71
Successful transactions: 491907
Failed transactions: 0
Longest transaction: 4.57
Shortest transaction: 0.01

Bandwidth proof:


Glassfish was not even stressing a single core on this machine and I'm fairly confident that even a simple dual core is enough to saturate a 1 Gbit port as long as the IO system can keep up. It will be interesting to see if you can approach these speeds when implementing servlets (probably not)
Also interesting to see is how during the benchmark the CPU usage went down from 120% to about 70% showing that the hotspot compiler does indeed optimize at runtime.

When I get some more time I'm going to extend the test by including everything from x=1 to 64 instead of just 32. This should also put more stress on the IO system and provide for a more realistic benchmark.

Tuesday, July 7, 2009

Just do the math and keep statistics

Today I came across a post from the MySQL performance blog titled "Just do the math!" which talks about something that should be very basic for every DBA/developer. Go and read it, as it's a helpful article.
The ability to estimate query performance based on metrics and expected data loads is very important and the example given shows how the query and reporting will have a hard time scaling. Other than trying to redo your hardware, optimize queries (however, at the end of the day you can not optimize away bulk)

So how else can you speed this example up?
Well first off all, in the example the data is being imported from Apache logs, my general recommendation is to create smaller workloads. Instead of importing the logs daily, import them hourly (or every 5 minutes). You should also do the same for reports and run them automatically and have them record their statistics in a table, which can then be queried for the interactive batch jobs.

This will still require the same kind of machine horsepower as it does not make processing disappear, but it does include some optimizations; Sorts, merges, joins can be done in memory on smaller chunks of data and thus lower the overall IO requirement.

Sunday, June 21, 2009

Glassfish + JFastCGI = Java + PHP together at last

I've been running PHP under fast-cgi with Sun webserver before and it had me wondering if this was possible in Java as well. Some searching on finding the Fast CGI specs brought up a reference to jFastCGI and lo and behold, all the heavy lifting has been done!

The code is simple and to the point and includes simple documentation. Here are some short steps you need to take to do this for yourself.

Ensure you have PHP with FastCGI support enabled.

php-cgi -i | grep fcgi

This should give you a result like:


This means fcgi support is compiled into the php-cgi binary, which is all you need on the php side of things. I'll see about including a simple project that does a bit more than just show phpinfo.

What you need is to add a Servlet to your webapplication using the following addition to your web.xml
XML removed, thanks a lot Blogger for crapping over the contents of my post. The XML can be found in the PDF I linked
Then just add the JAR files to your deployment (or place them in the correct folder on the server) and start your php-cgi (php-cgi -b6666)
All the PHP files in your web application will now be sent to the native cgi php version and the results be sent back to you.

A good sample would be:

Massive kudos to Jullien Railland for working on this!

So now the question rises, why would you rather do this than using Quercus?
Well not everything is supported natively under quercus and you also don't want to run PHP inside your JVM (or anything else native for that matter)
Worst that can happen now is the FastCGI process going down.

I'll try to post more information on doing a good setup of FastCGI which includes restarts on failure, user segregation and thread pooling.

Saturday, June 20, 2009

Lighter-Lighter Java / Web

Many times I've wondered if one even needs the likes of Spring / JEE (5) to build good web applications. Spring is a nice framework, born mostly out of unhappiness with previous versions of the Java EE stack and with good reason; J2EE was horribly complex and required many artifacts to even build the smallest EJB's.
Times have changed with Java EE 5 however and another artifact needs to bite the dust.
I'm sure you know what I'm talking about; it's the gratuitous use of XML by frameworks to configure your applications.

So why not use Java EE 5 together with your favorite web framework (Wicket in my case)?
Well my biggest problem is the added complexity in building and deploying, testing, etc.
All I really need is Wicket and a database enabled framework which allows me to work iteratively from an existing HTML design to a working application.
Java EE 6 will bring the next evolutionary step by eliminating the requirement to implement EAR's. Unfortunately it will be a while before Java EE 6 becomes ready for prime time, so for now I'm just going to use plain Java, JPA and Wicket. Yay for simplicity and those pushing to make developing complex applications easier

I'm not alone in this by the way

Monday, June 15, 2009

Upgrade your maven in Mac OSX

Under the category sweet and short.
You need to do the following to upgrade Maven 2 (2.0.6 is part of Leopard) to the latest so that you can use it from the command-line instead of the old version.

Without further ado, my bash history.

curl http://apache.hippo.nl/maven/binaries/apache-maven-2.1.0-bin.zip -o maven.zip
unzip maven.zip

cd apache-maven-2.1.0/
sudo mv /usr/bin/mvn /usr/bin/mvn-old

sudo ln -s /apache-maven-2.1.0/bin/mvn /usr/bin/mvn

Tuesday, June 9, 2009

Sun Webserver, perfect for the cloud

I've been working with Softlayer for a while and now more so as they are adding Cloud services. Their cloudservices have been off to a slightly rocky start as their development teams have surely been working hard on rolling this feature out.
I'm going to do a full review when I've put it into production use and when the last few kinks have been ironed out. To start off I can say their cloud offering performs very well and seems to retain the same performance at any moment of the day.

In my case their cloud is used to run both Java Virtual machines (Glassfish, Jboss, Tomcat) and Sun Systems Java Web server 7.0 which ties it all together.

Sun webserver is excellent for the cloud.
Unlike others you do not need to find a way to communicate configuration files to the different servers. You can do all configuration from a master node and push it to your webservers and if you push the content (rsync, NFS) as well you can fully automate your ability to scale up and down on their system.

I'm going to come back with a more in depth technical article on this one to help those who want to accomplish the same.

Sun's PHP addon 1.1 available

While you may have read my small article on how to setup PHP 5.2.9m Sun Web Server 7 and eAccelerator. Sun has come out with a Kenai project which brings the latest PHP Add-on for Solaris, Linux.

You can download them from here.

The instructions are pretty succinct:

This binary zip bundle includes PHP 5.2.9 (64-bit) for Web Server 7. Unzip this binary zip file under ws7-install-dir/plugins directory