Finding memory problems

-

Finding memory problems:

When performance testing a java application and want to identify memory problems, below steps will help:

1.   Don’t call it “memory leak” until you are 100% sure-make your program manager jump up and down. (A better term “unaccountable heap usage”, etc.)

2.   There is probably some “unaccountable heap size” from each release, there is no need and no time to look for small ones. Concentrate on the large ones which could cause problem down the road.

3.   There are some caches which is not leakage-watch out for their growth in size. Also check for some session data, which might be cleaned up after user logoff.

4.   Run 72-90 hour stability test under load for each release, monitoring java heap size. Check java heap GC frequency and old gen size. (cat “Full GC” myServer.log |awk ‘/Full GC/ {getnextline; getnextline;getnextline;print $0}’ | awk ‘{print $4}’)

5.   Enable GC Logging in JAVA_OPTIONS by adding appropriate flags:
"-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:~/$HOSTNAME/logs/gc-log-file.log"
               
1)-XX:+PrintGCDetails we activate the “detailed” GC logging mode which differs depending on the GC algorithm used
2)-XX:+PrintGCTimeStamps timestamp reflecting the real time passed in seconds since JVM start GC
3) -verbose:gc activates the “simple” GC logging mode         
4) -Xloggc we can specify location of an output file. By default it is at stdout.

6.   When running stability test for 90 hours, keep all the counters running in the background-(such as, sar –A –o mylogfile 120 100000 >/dev/null 2>&1 &, while [ 1 ];do netstat –s |egrep “inpack|outpack” >> myNetstatlog;sleep 600 done & ) You will need to find out why things go wrong in the middle of the night.

7.   Check all the logs to make sure there are no exceptions in the logs. If there are exceptions, memory can grow. It should be a clean run.

8.   If there is a problem, such as out of memory exception, use your favorite memory profiler-JProbe, JProfiler ..etc.

9.   Look at the source code, see if this is actual “leakage”. If you are not sure, talk to the developer, provide a screen shot.


10.            In profiler, use the filter to look for your application classes first, before you look at the Java classes or 3rd party classes. Usually it is our code problem, not Java or middleware.

Comments

Post a Comment

Popular posts from this blog

Steps to Analyze AWR Report in Oracle

-
Image
Steps to Analyze AWR Report in Oracle AWR -   Automatic   workload repository is a collection of persistent   system performance   statistics   owned by SYS. It resides in SYSAUX   tablespace . By default snapshot are generated once every 60 min and maintained for 7 days . Each snapshot has a unique ID know as "snap_id". Snapshot detail can be found in " dba_hist_snapshot " view.   If we have Database performance issue and not the Database machine, then AWR Report is the place to look at. AWR is not used for real-time performance monitoring like the v$ tables. It is used for historical analysis of performance. AWR complements, but doesnot replace real-time monitoring. Once AWR Report is generated in Oracle, the next task is to analyze it. By going through the AWR Report we can easily solve issues like slow database, high wait events, slow query and many more issues. Even though the report is lengthy, Analyzing or Reading re...
Read more

Vmstat Output explained

-
vmstat : vmstat is a tool in Unix/Linux which is used to Report virtual memory statistics. It shows how much virtual memory there is, how much is free and paging activity. Most important, you can observe page-ins and page-outs as they happen.   vmstat   reports   information about processes, memory, paging, block IO, traps, and cpu activity.   > vmstat procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----   r   b    swpd    free    buff   cache         si    so     bi     bo    in    cs us sy id wa st   0   0   21688 12797396 850716 15372668     0     0      3     35     0     0   1   0 99   0   0   0   0   21688 12789292 850716 15372668 ...
Read more

Verifications and Error Handling in LoadRunner *Web_reg_find and Web_reg_save_param*

-
Verification's and Error Handling Verification points must be inserted into the scripts to verify the application being tested with the load. Verification function that are used in our application are:  Web_reg_find, web_reg_save_param These verification should be outside towards the end of the transaction. Mandatory verification points: ■          Login – whenever there is a login, verification points must be added to verify that the user logged in correctly. ■          Update Transactions – transactions that update the database must be verified that they are successful. ■          Each page should be verified using Web_reg_find function Using Web _ reg_find for verifications. Example: web_reg_find("Text= Main Package","SaveCount=Verify_Count1",LAST); Verification: if (strcmp(lr_eval_string("{Verify_Count1}"),"0")==0) { ...
Read more