Troubleshooting memory issues requires TAC support to get accurate result.
1. To identify a memory leak on pix, get the periodic (hourly) output of “show memory detail”.
Send this to the TAC, they will see the memory utilisation divided per block size.
Example:
pixfirewall(config)# show memory detail Free memory: 201811608 bytes (75%) Used memory: Allocated memory in use: 44248216 bytes (16%) Reserved memory: 22375632 bytes ( 8%) —————————– —————- Total memory: 268435456 bytes (100%) … —– allocated memory statistics —– fragment size count total |
2. Then, we have to identify which block size the memory keeps increasing for, by verifying the periodic outputs.
Compare the “show memory detail” outputs and check at which memory block size the counter increasing.
3. If you can identify at which memory block size the counter keeps increasing, Issue the “show memory binsize ” and send the output to the TAC.
Example:
Let’s say it is in the 80 byte block size, then we will have to periodically issue “show memory binsize 80” as well:
pixfirewall(config)# show memory binsize 80 pc = 0x0044462e, size = 32 , count = 1 pc = 0x0068c52a, size = 108 , count = 3 pc = 0x007ce037, size = 33 , count = 1 pc = 0x00b5b6c8, size = 520 , count = 13 pc = 0x0095da1e, size = 128 , count = 4 pc = 0x00c56660, size = 40 , count = 1 pc = 0x00288770, size = 113 , count = 3 pc = 0x00959480, size = 648 , count = 17 pc = 0x0019573b, size = 80 , count = 2 |
Where we can see which PC (program counter) is associated with the process that consumes a certain amount of memory.
4. Identify the line with the highest number at the “count” field from “show memory binsize ” output.
Again, we will see the memory size increasing for one process. That PC can be decoded with the PIX/ASA decoder* to find out which process is leaking memory.
Summary :
1. Get periodic “show mem detail” and identify the size of memory that is being leaked,
2. Get periodic “show mem binsize x” for the affected block size, and try to identify the process that is leaking memory.
*PIX/ASA decoder is for Cisco employees only.
Posted on November 23, 2010
0