Dataminer error: java.lang.OutOfMemoryError: Java heap space

The BSAE 9.2 dataminer can show a Java OOM (Out of Memory) error on the DMTransactionMiner when processing a large SA (Server Automation) transaction from the lcrep table:

Exception in thread "Timer-1" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuffer.append(StringBuffer.java:224)
at com.opsware.cmdb.dataminer.DMMultiMasterTruthImpl.applyValueAndScope(DMMultiMasterTruthImpl.java:973)
at com.opsware.cmdb.dataminer.DMMultiMasterTruthImpl.collectClob(DMMultiMasterTruthImpl.java:931)
at com.opsware.cmdb.dataminer.DMMultiMasterTruthImpl.collectData(DMMultiMasterTruthImpl.java:735)
at com.opsware.cmdb.dataminer.DMMultiMasterTruthImpl.processTransaction(DMMultiMasterTruthImpl.java:103)
at com.opsware.cmdb.dataminer.DMMultiMasterTruthImpl.processTransaction(DMMultiMasterTruthImpl.java:41)
at com.opsware.cmdb.dataminer.DMTransactionMiner.loadTransactions(DMTransactionMiner.java:411)
at com.opsware.cmdb.dataminer.DMTransactionMiner.mineData(DMTransactionMiner.java:176)
at com.opsware.cmdb.dataminer.DMTransactionMiner.run(DMTransactionMiner.java:94)
at com.opsware.cmdb.dataminer.DataMiner$6.run(DataMiner.java:1304)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)

This error can be seen on the DMTransactionMiner when processing a large SA transaction from the lcrep table.

To move past this problem is by lowering the number of transactions in an XML file. This is done using the DataFileChunkSize parameter in the dataminer. It defaults to 1500, and setting it lower might get past this problem. Determining a working value will probably be an iterative process. To change it, use ./dmconfig.sh on the BSAE server:

./dmconfig.sh –update –name <SA dataminer name> –settings DataFileChunkSize=<chunk size>

You can see the value it is currently set to with:

./dmconfig.sh –listsettings –name <SA dataminer name>

If it’s not listed then it is the default value, 1500.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s