WebLogic Plan file merger with webdescriptor

Ever seen this error when a plan file is applied to an application on weblogic10.3?

<[WebAppModule(test:test.war)] Error parsing descriptor in Web appplication "C:\webapp\test.war"
weblogic.descriptor.DescriptorValidateException: The following failures occurred:
-- EnvEntryType is a required value that has not been specified.
at weblogic.descriptor.internal.AbstractDescriptorBeanHelper.validateSubTree(AbstractDescriptorBeanHelper.java:113)

This is because of some wierd whitespace issue that weblogic is not able to parse the webdescriptor file or the plan file.
The same plan and application works on wls11g (Either wls11g is more forgiving or does a better parsing)
You are better off to redo the plan xml or webdescriptor file, if you encounter the above error.

Oracle Enterprise Manager perl Syntax

We needed to enable metric "Log File Pattern Matched Line Count" from OEM.
We were looking for java.lang.OutOfMemory string within our weblogic log file.
As first time user, I went ahead and added the string name and really did not care to check the "Ignore Pattern in Perl" that had "%" by default.



This was telling the EM to ignore virtually everything and no wonder we did not receive alerts.
The right to look for a string "java.lang.OutOfMemoryError" was:

Jrockit JRCMD

If you are one of those shops where remote access from JRMC (Jrockit Management Console) is not possible, jrcmd comes to rescue.
All we need is access through SSH.
This came in real handy for our environments where remote access is not allowed.

$ ./Jrockit_home/bin/jrcmd PID heap_diagnostics
This would print heap summary, Memory layout (similar to pmap in solaris) and details object statistics

If you are new to Jrockit and familiar with other JVM tools like Jhat or MAT,and would need to do an offline analysis, then a heap dump can be generated in HPROF format.

$ ./Jrockit_home/bin/jrcmd PID hprofdump filename=heapdump.hprof
(heapdump.hprof is created from JVM process spawning directory. For weblogic it will be under domain directory - assuming startscript is triggred from there)
This hprof can be further analyzed with tools like jhat or MAT (Eclipse Memory analyzer tool).
MAT would provide all sorts of info from leak suspects to Accumulated Objects by Class as shown above.

Another neat feature from JRMD is baselining the heap and native memory.
$./Jrockit_home/bin/jrcmd PID print_memusage scale=M baseline
This prints base line of memory usage (both java and native heap) something like:



Subsequent runs of same command would baseline to first snapshot and shows the added java & native heaps.



Reference:
https://www.packtpub.com/oracle-jrockit-the-definitive-guide/book

WebLogic & Oracle RAC

It's a common scenario to have weblogic hosted apps connect to Oracle RAC in the backend. From weblogic perspective, there are two ways to connect to Oracle RAC.

1)   Use WebLogic Multipool:   This would mean all the failover/loadbalancing of JDBC calls will be taken care by weblogic JDBC framework - built within application server.
2)   Connect-time failover :   This would mean, all failover/loadbalancing of jdbc calls will be taken care by Oracle driver.

Now what is better approach?   Based on testing and feedback from Oracle, multipool approach is a better choice.

In case of connect-time failover scenario, failover can take lot of time (max would depend on tcp timeout).
Oracle FCF (fast connection failover) would help to some extent in reducing the failover time, but still the failover would be connection by connection instead of whole pool atonce.
Failover in case of multipool is faster compared to FastConnection Failover feature from Oracle DB.
Also, XA connections are better taken care of w.r.t multipools since weblogic framework would make sure all JDBC calls within an XA transaction would reach the same Oracle server.(which is not the case with Oracle thin driver)