I wonder if this recent blog about installing and running "xrestop" might help: http://blog.kagesenshi.org/2009/01/viewing-xorg-memory-consumption-xrestop.html