Checkpoint firewall debugging basics

To debug a checkpoint firewall is not a big deal, but to understand the output is in many cases imposible for those NOT working at Checkpoint.
I write here not about the exact analysation with debugging, just a ‘how to collect the required informations’ that may speed up the troubleshooting.

1. Reset the debugs to the default.

In case someone changed the setting in the past and since then the firewall was not rebooted we should set all back to the defaults.

# fw ctl debug 0 Defaulting all kernel debugging options

2. Check the module list we can use and choose the requied ones

With the following command you can see the modules with their options.

[Expert@sgLondon]# fw ctl debug -m debug: option requires an argument — m Usage: fw ctl debug [-d ] [-s “”] [-v (“”|all)] [-x] [-m ] [-e expr | -i | -u] [+|-] Or: fw ctl debug [-t (NONE|ERR|WRN|NOTICE|INFO)] [-f (RARE|COMMON)] Or: fw ctl debug -buf [buffer size] -h – for helpWhere possible options are:Module: kiss Kernel debugging options: error warning ioctl memory misc chain driver pools handles vbuf pm rem sm dfa pmdump pmint htab bench ghtab mtctx queue thread Messaging threshold set to type=Info freq=CommonModule: kissflow Kernel debugging options: error warning memory pm compile dfa Messaging threshold set to type=Info freq=CommonModule: fw Kernel debugging options: error warning cookie crypt domain ex driver filter hold if install ioctl kbuf ld log machine memory misc packet q xlate xltrc conn synatk media sip vm chain bridge tcpstr scv packval sync ipopt link nat cifs drop route citrix misp portscan leaks mgcp sock mail spii chainfwd msnms wire balance dynlog smtp wap content mrtsync sam sock malware cmi aspii dos advp multik netquota monitor monitorall dfilter integrity epq cvpnd cptls ftp Messaging threshold set to type=Info freq=CommonModule: h323 Kernel debugging options: error init h225 h245 ras decode align cpas Messaging threshold set to type=Info freq=Common Module: multik Kernel debugging options: error conn packet api message state packet_err counter event quota ioctl lock clb Messaging threshold set to type=Info freq=Common Module: BOA Kernel debugging options: fatal error warning info stat memory analyzer spider flow stream disasm lock Messaging threshold set to type=Info freq=Common Module: WS Kernel debugging options: fatal error warning info timestamp connection session parser body global stat memory address policy pfinder regexp coverage report_mgr spii uuid ioctl module mem_pool pkt_dump subject sslt cookie stream vs event Messaging threshold set to type=Info freq=Common Module: CI Kernel debugging options: fatal error warning info timestamp coverage subject memory module session address vs regexp ioctl policy profile filter uf av crypto stat Messaging threshold set to type=Info freq=Common Module: CPAS Kernel debugging options: error warning tcp api glue events conns pkts timer tcpinfo http ftp skinny notify sync Messaging threshold set to type=Info freq=Common Module: VPN Kernel debugging options: driver err packet policy sas rdp pcktdmp queue init sr mem comp xl counters mspi cphwd ref vin cluster nat l2tp tnlmon warn tcpt tagging ike ifnotify resolver gtp topology multik multicast Messaging threshold set to type=Info freq=Common

3. Set the buffer Size

The size of the buffer depends on how much modules and options you choose and how much the are going to generate.

# fw ctl debug -buf 9600 Initialized kernel debugging buffer to size 9600K

4. Choose the modules an their options for the debug

A) If we do not use any options than the default is error and warning or just error (or just error multicast, like at VPN):

# fw ctl debug -m fw Kernel debugging buffer size: 9600KB Module: fw Enabled Kernel debugging options: error warning Messaging threshold set to type=Info freq=Common

B) if you would set your own options, then just type them after the module name:

# fw ctl debug -m fw packet drop sam Updated kernel’s debug variable for module fw# fw ctl debug -m fw Kernel debugging buffer size: 9600KB Module: fw Enabled Kernel debugging options: packet drop sam Messaging threshold set to type=Info freq=Common

C) To add another option type “+”:

# fw ctl debug -m fw + route Updated kernel’s debug variable for module fw# fw ctl debug -m fw Kernel debugging buffer size: 9600KB Module: fw Enabled Kernel debugging options: packet drop route sam Messaging threshold set to type=Info freq=Common

D) To delete the option you have set and start it again type “- all”:

# fw ctl debug -m fw – all Updated kernel’s debug variable for module fw# fw ctl debug -m fw Kernel debugging buffer size: 9600KB Module: fw Enabled Kernel debugging options: None

5. Start writing debug into a file:

Basic command is the “fw ctl kdebug”. The syntax is the following:

One of the most important parameter is the ‘t’ or ‘T’, that write a timestamp to the entries. This is always required.

# fw ctl kdebug ? Usage: fw ctl kdebug [-i | [-f] -o ] [-b ] [-t|-T] [-p fld1[,fld2..] [-m num [-s size]] -t/-T to print the time field (seconds/microseconds) -p to print specific fields all|proc|pid|date|mid|type|freq|topic|time|ticks|tid|text|err|host|vsid|cpu -m – number of cyclic files, -s – size of each

A) Write it in a file:

# fw ctl kdebug -T -f > samrules.txt(-o filename makes binary file, that is why I use the “>” sign)

B) Or write in a file in background and to the output as well, but give back the cursor:

# fw ctl kdebug -T -f | tee samdebug.txt &

tee command:
http://en.wikipedia.org/wiki/Tee_%28command%29

It can happen that the debugging process eats up all the cpu and we loose the control and maybe sessions are dropped.
Before this command ist started it is useful to create a crontab entry or a single script that kills this process after a couple of minutes.

Lets do a bash script that kills debugging after 10 second:
This example script writes in a file samrules.txt and stops after 10 seconds. Those values can be changed if you want.

# vi debug.sh timeout=10 # in seconds fw ctl kdebug -T -f > samrules.txt & cmdpid=$! # Command to terminate # Start “watchdog” process to terminate the command # after $timeout seconds: (sleep $timeout; kill -9 $cmdpid) & watchdogpid=$! wait $cmdpid # wait for command kill $watchdogpid >/dev/null 2>&1

Change the file permissions and we can start it:

# chmod +x debug.sh # ./debug.sh

The time in script should be as long as the reproduction time of the problem. It has no use if it is shorter.