Troubleshooting ASA high memory issues

Posted on November 23, 2010

0



Troubleshooting memory issues requires TAC support to get accurate result.

1. To identify a memory leak on pix, get the periodic (hourly) output of “show memory detail”.
Send this to the TAC, they will see the memory utilisation divided per block size.

Example:

pixfirewall(config)# show memory detail
Free memory:                     201811608 bytes (75%)
Used memory:
Allocated memory in use:     44248216 bytes (16%)
Reserved memory:             22375632 bytes ( 8%)
—————————–   —————-
Total memory:                    268435456 bytes (100%) 

—– allocated memory statistics —–

fragment size       count          total
(bytes)                        (bytes)
—————-  ———-  ————–
40             1             40
48           116           5568
56          1487          83272
64           265          16960
72           114           8208
80            87           6960
88            61           5368
96          1529         146784
104            34           3536
112            50           5600
120            42           5040
128            30           3840

2. Then, we have to identify which block size the memory keeps increasing for, by verifying the periodic outputs.
Compare the “show memory detail” outputs and check at which memory block size the counter increasing.

3. If you can identify at which memory block size the counter keeps increasing, Issue the “show memory binsize ” and send the output to the TAC.

Example:
Let’s say it is in the 80 byte block size, then we will have to periodically issue “show memory binsize 80” as well:

pixfirewall(config)# show memory binsize 80
pc = 0x0044462e, size = 32 , count = 1
pc = 0x0068c52a, size = 108 , count = 3
pc = 0x007ce037, size = 33 , count = 1
pc = 0x00b5b6c8, size = 520 , count = 13
pc = 0x0095da1e, size = 128 , count = 4
pc = 0x00c56660, size = 40 , count = 1
pc = 0x00288770, size = 113 , count = 3
pc = 0x00959480, size = 648 , count = 17
pc = 0x0019573b, size = 80 , count = 2

Where we can see which PC (program counter) is associated with the process that consumes a certain amount of memory.

4. Identify the line with the highest number at the “count” field from “show memory binsize ” output.
Again, we will see the memory size increasing for one process. That PC can be decoded with the PIX/ASA decoder* to find out which process is leaking memory.

Summary :
1. Get periodic “show mem detail” and identify the size of memory that is being leaked,
2. Get periodic “show mem binsize x” for the affected block size, and try to identify the process that is leaking memory.

*PIX/ASA decoder is for Cisco employees only.

Advertisements