Saturday, July 15, 2006

 

AMD64: java.lang.OutOfMemoryError

Today I discovered that AMD64 version of Sun Java VM 1.5.0 has some memory problems.

I spent last couple of weeks profiling a Java web application (running with Tomcat). The application crashed after running for 2-3 days with java.lang.OutOfMemoryError (even with heap size 4GB). My development machine is 32-bit and production machine 64-bit which confused me more as profiling was not able to find anything suspicious. Finally I tried using i586 JRE on our 64-bit production machines and things worked fine. Probably amd64 garbage collector has some bug. Lesson: Java HotSpot amd64 VM 1.5 has memory management problems! use 32-bit VM instead.

To reproduce this effect, I have following simple program:

#File: JavaMemoryProblems.java
import java.util.HashMap;
import java.util.Map;
public class JavaMemoryProblems {
   public static void main(String[] args) {
       Map<String, Integer> map = new HashMap<String, Integer>();
       int max = 1000000;
       for (int i=0; i<max; i++) {
           map.put("key"+i, new Integer(i));
       }
       System.out.println("Finished with a map of size "+map.size());
   }
}

When I run this program with i586 and amd64 versions of Java VM 1.5.0_07, I get the following results:

  • 32-bit: java -Xmx120m JavaMemoryProblems. With 120MB memory it works fine.
  • 64-bit: java -Xmx170m -XX:+UseSerialGC JavaMemoryProblems. Anything less than 180MB throws OutOfMemory (requires 50% more than i586 version).
  • 64-bit: java -Xmx200m -XX:+UseParallelGC JavaMemoryProblems. With 200MB but with ParallelGC (default on server class machines) hangs and needs to be killed with kill -9. In this case the parallel GC thread keeps calling garbage collector repeatedly. Lesson: Use -XX:+UseSerialGC if running low on memory, ParallelGC takes too much time and CPU (and even hangs) before it figures out it can not continue.

After switching to i586 VM I am saving a lot of memory (30-40%). Earlier a simple search program required 250MB heap size to search a Lucene index with around 15 million docs (10GB index size), now it needs just 150MB. Maintaining statistics consisting of around 100 Tries each with roughly 1 million entries required over 1050MB with amd64 VM, now I can do the same with 650MB. Memory required for parsing a simple 55MB XML from weblogs using JDOM with SAXBuilder with i586 VM is 290MB memory compared to 440MB for amd64. Similiar things are also mentioned by sotua in Ubuntu forums.

Anyway I have learned many interesting things about Java VM and garbage collection in last 2 weeks:


Comments:
Doesn't the memory increase in the 64 bit case make sense? That is shouldn't an Integer in the 64bit VM be a 64bit Integer instead of a 32bit Integer, which would require more memory?
 
Hi - thanks for the help. I had the same issues on suse with eclipse - memory and then after 5 mins of using I got a stackdump. I changed -Xmx settings but then got jumpOverMethodBody errors. Eventually got it fixed by adding
~/.hotspot_compiler -
http://wbeaton.blogspot.com/2005/09/eclipse-in-64-bits.html
nigel
 
Post a Comment



<< Home

This page is powered by Blogger. Isn't yours?