Intro

how can I investigate an OOME?

Metaspace

XX:MaxMetaspaceSize=[size] we can set the Metaspace upper bounds.

However, in case the MetaspaceSize extrapolates an OutOfMemory error.

GC only removes objects that are no longer in use. The GC process is triggered when the max size is reached. Metaspace by default auto increases and keeps meta-information about loaded classes. Instances of the class Class are in the regular heap.

Investigating Metaspace OOME

How to investigate:
~Do a heap dump and analyze it with Eclipse MAT. Look at the classes you have loaded. Check if there's something unexpected, especially duplicate classes. It also has a classloader explorer.

~https://github.com/mjiderhamn/classloader-leak-prevention

Issues on Metaspace ~ OOME
======
The main cause for the java.lang.OutOfMemoryError: Metaspace are one of the following:

~ too many classes:
   For dynamic languages, this might happen.

~too big classes being loaded to the Metaspace:
    A classloader leak happens when an application is redeployed, but there are lingering instances of the previous deployment’s classes left.

You can use MAT for investing those kinds of issue, and acquiring a heap dump to analyze. But on this case, the heap should be dumped after at least one ClassLoader instance has leaked. You can then analyze it with Eclipse Memory Analyzer ~ MAT.
There is a through explanation on details [1].

References
https://java.jiderhamn.se/2011/12/11/classloader-leaks-i-how-to-find-classloader-leaks-with-eclipse-memory-analyser-mat/

REFs

Classloader leaks I – How to find classloader leaks with Eclipse Memory Analyser (MAT)