Web Apps, Scalability, and French Real Estate

Using a low memory budget virtual machine to run a scalable web app is turning out to be a fun challenge.  To give a bit of context, I am two months from graduating Appalachian State University. I’m currently in a class where we are suppose to show off everything we learned in one cumulative project. I’ve chosen to build a web app for finding real estate in France.

The project has two sides, including a Django powered web platform running on Apache to display to users, as well as a backend java web crawler. The django side has been fairly good at keeping up with demands of the growing dataset. The java side however has been quite touchy. I’ve been running my jar file under these conditions.

java -Xms256m -Xmx256m -XX:MaxPermSize=96m -XX:+CMSClassUnloadingEnabled

While I’m not sure if that’s the most effective combination, it does keep my server under the guaranteed RAM limit of 1024MB. Accompanying this low memory JVM is a lot of memory overflows.

I found the JDBC driver for MySQL to be one of the biggest memory hogs. The real issue there was me not closing the resources when I was finished. Turns out for every Statement and ResultSet, they must be closed when finished or they will sit around longer than wanted.

Since discovering that, most of the memory issues have been alleviated. Now my limited 60GB HDD is running full of images. I’ll have to find a good solution for that within 3 weeks or it will be full up.

You can check out the development version of my senior project here: http://www.coinerd.com/