Infinite 2D world

I want to know how to avoid “Java heap space” error in big 2D worlds. If i have more than 1000 chunks i get java heap space error. I tried to save chunks and delete them from game (keep data about them in computer hard drive). but when i need to load them my FPS drops to 1 for few seconds.