You are using XSL FO for the page layout model, used to determine page numbers.
I haven't noticed any issue caused by fo.XMLWhiteSpaceHandler in the past.
You could allocate more memory to the VM. How much are you allocating? How big are your documents?
Or, if you are using v3.3.x, you could remove the docx4j-export-fo jar from your classpath, so that the commercial PDF Converter is used instead to calculate the page numbers. This is faster, more resource efficient and more accurate, but is native code specific to whatever OS you are running.