Palo Alto troubleshooting commands

Posted on December 10, 2013

2



Its now a beautiful saturday afternoon outside and I sit here in this boring room and made this post about useful palo alto commands, that can help us in case of a problem arises.
There is no wind, I wanted today go to windsurf…
With my requirements for any networking layer 3 security device I collected the basic commands that you have to know or you will not be able to troubleshoot your device.
The commands show tha same things that I experienced from fortigate cisco asa and checkpoint (must admit checkpoint has not as useful cli as the others).

1. show the uptime and the active sessions
2.1 show the interface state (speed/duplex/state/mac)
2.2. show interface HW settings
2.3. show interface zone settings
2.4. show interface counters
2.5. show interface counter – not documented, but shows more in case of interface errors.
3. show routing table
4.1. show CPU usage
4.2. show CPU eaters, the linux “top” command
5. show temperature
6. show counters for everything
7. show a specific session
8. show the statistics on application recognition
9. show policy match for specific session
10. debug packet flow
11. show policy of a firewall managed through panorama

1. show the uptime and the active sessions

 

paroot@pa-firewall(active)> show system statistics session
System Statistics: ('q' to quit, 'h' for help)                                                                                      

Device is up          : 28 days 23 hours 19 mins 1 sec 
Packet rate           : 668/s
Throughput            : 990 Kbps
Total active sessions : 4911
Active TCP sessions   : 3414
Active UDP sessions   : 1497
Active ICMP sessions  : 0

2.1 show the interface state (speed/duplex/state/mac)

 

paroot@pa-firewall(active)> show interface all

total configured hardware interfaces: 9

name                    id    speed/duplex/state        mac address       
--------------------------------------------------------------------------------
ethernet1/1             16    1000/full/up              00:1b:17:00:01:10 
ethernet1/2             17    1000/full/up              00:1b:17:00:01:11 
ethernet1/3             18    1000/full/up              00:1b:17:00:01:12 
ethernet1/15            30    1000/full/up              00:1b:17:2f:b9:1e 
ethernet1/16            31    1000/full/up              00:1b:17:2f:b9:1f 
ethernet1/17            32    1000/full/up              00:1b:17:00:01:20 
vlan                    1     [n/a]/[n/a]/up            00:1b:17:00:01:01 
loopback                3     [n/a]/[n/a]/up            00:1b:17:00:01:03 
tunnel                  4     [n/a]/[n/a]/up            00:1b:17:00:01:04 

aggregation groups: 0

total configured logical interfaces: 14

name                id    vsys zone             forwarding               tag    address                                         
------------------- ----- ---- ---------------- ------------------------ ------ ------------------
ethernet1/1         16    1    dmz2             vr:default               0      10.23.12.126/25  
ethernet1/2         17    1    dmz1             vr:default               0      192.168.1.2/24  
ethernet1/3         18    1                     N/A                      0      N/A               
ethernet1/3.1102    258   1    dmz3             vr:default               1102   3.3.3.3/27 
ethernet1/3.1111    259   1    dmz4             vr:default               1111   192.168.3.1/26   
ethernet1/15        30    0                     ha                       0      1.1.1.2/30        
ethernet1/16        31    0                     ha                       0      N/A               
ethernet1/17        32    1                     N/A                      0      N/A               
ethernet1/17.401    260   1    dmz5             vr:default               401    10.31.3.3/23   
ethernet1/17.403    262   1    dmz6             vr:default               403    192.168.4.1/23  
ethernet1/17.411    261   1    inside           vr:default               411    10.31.2.1/23   
vlan                1     1                     N/A                      0      N/A               
loopback            3     1                     N/A                      0      N/A               
tunnel              4     1                     N/A                      0      N/A

2.2. show interface HW settings

paroot@pa-paris>  show interface hardware

total configured hardware interfaces: 6

name                    id    speed/duplex/state        mac address       
--------------------------------------------------------------------------------
ethernet1/1             16    1000/full/up              58:49:3b:1d:de:10 
ethernet1/2             17    ukn/ukn/down(power-down)  58:49:3b:1d:de:11 
ethernet1/4             19    1000/full/up              58:49:3b:1d:de:13 
vlan                    1     [n/a]/[n/a]/up            58:49:3b:1d:de:01 
loopback                3     [n/a]/[n/a]/up            58:49:3b:1d:de:03 
tunnel                  4     [n/a]/[n/a]/up            58:49:3b:1d:de:04 

aggregation groups: 0

2.3. show interface zone settings

Interface to zone and to virtual-router allocation inclusive IP:

paroot@pa-paris> show interface logical

total configured logical interfaces: 11

name                id    vsys zone             forwarding               tag    address                                         
------------------- ----- ---- ---------------- ------------------------ ------ ------------------
ethernet1/1         16    1    Transfer         vr:RTR1                  0      172.172.66.250/24 
ethernet1/2         17    1    Tap1             tap                      0      N/A               
ethernet1/4         19    1                     N/A                      0      N/A               
ethernet1/4.101     256   1    mydmz99          vr:RTR1                  101    192.168.1.250/24  
ethernet1/4.102     257   1    mydmz100         vr:RTR1                  102    192.168.2.250/24  
ethernet1/4.104     258   1    mydmz98          vr:RTR1                  104    192.168.4.250/22  
ethernet1/4.108     260   1    mydmz97          vr:RTR1                  108    192.168.8.250/22  
ethernet1/4.112     259   1    mydmz96          vr:RTR1                  112    192.168.12.250/24 
vlan                1     1                     N/A                      0      N/A               
loopback            3     1                     N/A                      0      N/A               
tunnel              4     1                     N/A                      0      N/A               

2.4. show interface counters

paroot@pa-paris> show interface ethernet1/1

--------------------------------------------------------------------------------
Name: ethernet1/1, ID: 16
Link status:
  Runtime link speed/duplex/state: 1000/full/up
  Configured link speed/duplex/state: auto/auto/auto            
MAC address:
  Port MAC address 58:49:3b:1d:de:10
Operation mode: layer3
Untagged sub-interface support: no
--------------------------------------------------------------------------------
Name: ethernet1/1, ID: 16
Operation mode: layer3
Virtual router RTR1
Interface MTU 1500
Interface IP address: 172.172.66.250/24
Interface management profile: N/A
Service configured: 
Zone: Transfer, virtual system: vsys1
Adjust TCP MSS: no
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Physical port counters read from MAC:
--------------------------------------------------------------------------------
rx-broadcast                  148034
rx-bytes                      559000882171
rx-multicast                  764294
rx-unicast                    454531003
tx-broadcast                  16
tx-bytes                      69206540263
tx-multicast                  0
tx-unicast                    298909821                
--------------------------------------------------------------------------------

Hardware interface counters read from CPU:
--------------------------------------------------------------------------------
bytes received                           557231551345
bytes transmitted                        68009781436
packets received                         455595765
packets transmitted                      298909837
receive errors                           264070
packets dropped                          0        
--------------------------------------------------------------------------------

Logical interface counters read from CPU:
--------------------------------------------------------------------------------
bytes received                           557184222899
bytes transmitted                        68009781436
packets received                         455366594
packets transmitted                      298909837
receive errors                           0
packets dropped                          832996
packets dropped by flow state check      47879
forwarding errors                        0
no route                                 0
arp not found                            7258
neighbor not found                       0
neighbor info pending                    0
mac not found                            0
packets routed to different zone         0
land attacks                             0
ping-of-death attacks                    0
teardrop attacks                         0
ip spoof attacks                         0
mac spoof attacks                        0
ICMP fragment                            0
layer2 encapsulated packets              0
layer2 decapsulated packets              0
--------------------------------------------------------------------------------

2.5. show interface counter – not documented, but shows more in case of interface errors.

paroot@pa-paris> show system state filter-pretty sys.s1.p*

sys.s1.p1.capability: [ 
  auto,
  10Mb/s-half,
  10Mb/s-full,
  100Mb/s-half,
  100Mb/s-full,
  1Gb/s-half,
  1Gb/s-full,
]
sys.s1.p1.cfg: { 
  farloop: False,
  mode: Autoneg,
  mru: 1522,
  nearloop: False,
  pause-frames: True,
  setting: auto,
}
sys.s1.p1.detail: { 
  pkts1024tomax_octets: 0x17745efd,
  pkts128to255_octets: 0x1349942,
  pkts256to511_octets: 0xf5eec3,
  pkts512to1023_octets: 0xcfacaa,
  pkts64_octets: 0x65aae3c,
  pkts65to127_octets: 0xc3e0f5a,
}
sys.s1.p1.mirror: { 
  rx-en: False,
  tx-en: False,
}
sys.s1.p1.phy: { 
  link-partner: { },
  media: CAT5,
  type: Ethernet,
}
sys.s1.p1.state: board_port_autoneg_enabled
sys.s1.p1.stats: { 
  rx-broadcast: 148079,
  rx-bytes: 559742599720,
  rx-multicast: 764573,
  rx-unicast: 455172732,
  tx-broadcast: 16,
  tx-bytes: 69466620847,
  tx-multicast: 0,
  tx-unicast: 299368876,
}
sys.s1.p1.status: { 
  farloop: False,
  link: Up,
  mode: Autoneg,
  mru: 1522,
  nearloop: False,
  pause-frames: False,
  setting: 1Gb/s-full,
  type: RJ45,
}
sys.s1.p2.capability: [ 
  auto,
  10Mb/s-half,
  10Mb/s-full,
  100Mb/s-half,
...

3. show routing table

paroot@pa-paris> show routing fib virtual-router RTR1

total virtual-router shown :              1


--------------------------------------------------------------------------------
virtual-router name: RTR1
interfaces:
   ethernet1/1 ethernet1/4.101 ethernet1/4.102 ethernet1/4.104
   ethernet1/4.108 ethernet1/4.112


route table:
flags: u - up, h - host, g - gateway

maximum of fib entries for device:                 1250
maximum of IPv4 fib entries for device:            1250
maximum of IPv6 fib entries for device:            1250
number of fib entries for device:                  13
maximum of fib entries for this fib:               1250
number of fib entries for this fib:                13
number of fib entries shown:                       13

id      destination           nexthop            flags  interface          mtu 
--------------------------------------------------------------------------------
39      0.0.0.0/0             172.172.66.1       ug     ethernet1/1        1500
38      172.172.66.0/24       0.0.0.0            u      ethernet1/1        1500
37      172.172.66.250/32     0.0.0.0            uh     ethernet1/1        1500
32      192.168.4.0/22        0.0.0.0            u      ethernet1/4.104    1500
28      192.168.1.0/24        0.0.0.0            u      ethernet1/4.101    1500
30      192.168.2.0/24        0.0.0.0            u      ethernet1/4.102    1500
27      192.168.1.250/32      0.0.0.0            uh     ethernet1/4.101    1500
29      192.168.2.250/32      0.0.0.0            uh     ethernet1/4.102    1500
31      192.168.4.250/32      0.0.0.0            uh     ethernet1/4.104    1500
34      192.168.8.0/22        0.0.0.0            u      ethernet1/4.108    1500
36      192.168.12.0/24       0.0.0.0            u      ethernet1/4.112    1500
33      192.168.8.250/32      0.0.0.0            uh     ethernet1/4.108    1500
35      192.168.12.250/32     0.0.0.0            uh     ethernet1/4.112    1500
--------------------------------------------------------------------------------

4.1. show CPU usage

There is no a simple number that can tell us the CPU state…

paroot@pa-paris> show running resource-monitor
&gt day      Per-day monitoring statistics
&gt hour     Per-hour monitoring statistics
&gt minute   Per-minute monitoring statistics
&gt second   Per-second monitoring statistics
&gt week     Per-week monitoring statistics
  |        Pipe through a command
  <Enter>  Finish input
paroot@pa-paris> show running resource-monitor

Resource monitoring sampling data (per second):

CPU load sampling by group:
flow_lookup                    :    11%
flow_fastpath                  :    11%
flow_slowpath                  :    11%
flow_forwarding                :    11%
flow_mgmt                      :     1%
flow_ctrl                      :     1%
nac_result                     :    11%
flow_np                        :     8%
dfa_result                     :    11%
module_internal                :    11%
aho_result                     :    11%
zip_result                     :    11%
pktlog_forwarding              :    10%
lwm                            :     0%
flow_host                      :     1%

CPU load (%) during last 60 seconds:
core   0   1   2   3
       0   1  12  10
       0   1  11  10
       0   1   9   9
       0   1   8   8
       0   1  13  14
       0   1   8   8
       0   1  10   9
       0   1   5   8
       0   1   7  11
       0   1  11  14
       0   1  11   8
       0   1  12  12
       0   1  15  18
       0   2  26  27
       0   1   9   9
       0   1  10   9
...

4.2. show CPU eaters, the linux “top” command

paroot@pa-paris> show system resources follow 
top - 15:13:10 up 16 days,  4:11,  1 user,  load average: 0.33, 0.14, 0.04
Tasks:  92 total,   1 running,  91 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.7%us,  0.2%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   1970840k total,  1880324k used,    90516k free,    46904k buffers
Swap:  2008084k total,   184136k used,  1823948k free,   406388k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                
 2832 root      20   0 89832  13m 2816 S    1  0.7 170:12.49 dnsproxyd                                                              
23322 paroot    20   0  4528 1144  904 R    1  0.1   0:00.11 top                                                                    
    7 root      20   0     0    0    0 S    0  0.0   3:26.16 events/0                                                               
 2332 root      15  -5 41028 3364 2304 S    0  0.2 161:08.05 sysd                                                                   
 2642 root      20   0 92820 3920 2848 S    0  0.2   6:27.22 cryptod                                                                
 2823 root      20   0  743m 444m 6064 S    0 23.1  74:51.62 logrcvr                                                                
 2827 root      17  -3 66392 3084 2480 S    0  0.2   6:22.43 ha_agent                                                               
 2847 nobody    20   0  116m 4660 3248 S    0  0.2   9:33.20 appweb3                                                                
    1 root      20   0  1832  560  536 S    0  0.0   0:22.53 init                                                                   
    2 root      20   0     0    0    0 S    0  0.0   0:00.01 kthreadd                                                               
    3 root      RT   0     0    0    0 S    0  0.0   0:02.10 migration/0                                                            
    4 root      20   0     0    0    0 S    0  0.0   0:01.33 ksoftirqd/0                                                            
    5 root      RT   0     0    0    0 S    0  0.0   0:02.57 migration/1                                                            
    6 root      20   0     0    0    0 S    0  0.0   0:01.74 ksoftirqd/1                                                            
    8 root      20   0     0    0    0 S    0  0.0   6:34.77 events/1                                                               
    9 root      20   0     0    0    0 S    0  0.0   0:00.05 khelper                                                                
   12 root      20   0     0    0    0 S    0  0.0   0:00.00 async/mgr                                                              
  111 root      20   0     0    0    0 S    0  0.0   0:02.47 sync_supers                                                            
  113 root      20   0     0    0    0 S    0  0.0   0:03.94 bdi-default                                                            
  114 root      20   0     0    0    0 S    0  0.0   0:05.12 kblockd/0                                                              
  115 root      20   0     0    0    0 S    0  0.0   0:04.83 kblockd/1                                                              
  124 root      20   0     0    0    0 S    0  0.0   0:00.00 ata/0                                                                  
  125 root      20   0     0    0    0 S    0  0.0   0:00.00 ata/1 
...

The following command shows the free Pool / Pool max values. If the left side nears to 0 then your pool is full and your firewall has heavy performance.

paroot@pa-firewall(active)> debug dataplane pool statistics 

Hardware Pools
[ 0] Packet Buffers            :    57217/57344    0x80000000b1400000
[ 1] Work Queue Entries        :   229289/229376   0x80000000b8400000
[ 2] Output Buffers            :     1011/1024     0x8000000007f00000
[ 3] DFA Result                :     2048/2048     0x80000000ba000000
[ 4] Timer Buffers             :     4092/4096     0x80000000ba200000
[ 5] PAN_FPA_LWM_POOL          :     1024/1024     0x8000000000ebca00
[ 6] PAN_FPA_ZIP_POOL          :     1023/1024     0x80000000ba600000
[ 7] PAN_FPA_BLAST_POOL        :     1024/1024     0x8000000008000000

Software Pools
[ 0] software packet buffer 0  :    16363/16384    0x80000000ba800680
[ 1] software packet buffer 1  :     8191/8192     0x80000000bb010700
[ 2] software packet buffer 2  :    16384/16384    0x80000000bb818780
[ 3] software packet buffer 3  :     4096/4096     0x80000000bd828800
[ 4] software packet buffer 4  :      304/304      0x80000000c5a2c880
[ 5] ZIP Results               :     1024/1024     0x80000000080b9978
[ 6] CTD Flow                  :    65037/65536    0x80000000d2f1a080
[ 7] CTD AV Block              :       32/32       0x80000000e4af5340
[ 8] SML VM Fields             :    69283/69632    0x80000000e4afd440
[ 9] SML VM Vchecks            :    32768/32768    0x80000000e4d614c0
[10] Detector Threats          :    65335/65536    0x80000000e4e01540
[11] CTD DLP FLOW              :    16366/16384    0x80000000e5d41608
[12] CTD DLP DATA              :     1024/1024     0x80000000e5f51688
[13] CTD DECODE FILTER         :    32768/32768    0x80000000e6052710
[14] Regex Results             :     2048/2048     0x80000000e6213088
[15] TIMER Chunk               :   131072/131072   0x80000000e9efeae0
[16] FPTCP segs                :    32768/32768    0x80000000ebf7eb60
[17] Proxy session             :     1024/1024     0x80000000ec01ebe0
[18] SSL Handshake State       :     1024/1024     0x80000000ec07bc60
[19] SSL State                 :     2048/2048     0x80000000ec19cce0
[20] SSL Handshake MAC State   :     2256/2256     0x80000000ec332d60
[21] SSH Handshake State       :       16/16       0x80000000ec37b920
[22] SSH State                 :      128/128      0x80000000ec397060
[23] TCP host connections      :       15/16       0x80000000ec3e6040

5. show temperature

paroot@pa-paris> show system environmentals

----Thermal----
Slot   Description                         Alarm    Degrees C  Min C   Max C  
 S0    Temperature at MP [U6]              False    33.50      5.00    50.00  
 S0    Temperature at DP [U7]              False    38.00      5.00    50.00  


----Fans----
Slot   Description                         Alarm     RPMs Min RPM
 S0    Fan #1 Operational                  False     True     1
 S0    Fan #2 Operational                  False     True     1

----Power----
Slot   Description                         Alarm    Volts  Min V  Max V 
 S0    1.05V Power Rail                    False    1.04   0.98   1.13  
 S0    1.1V Power Rail                     False    1.09   1.03   1.18  
 S0    1.2V Power Rail                     False    1.21   1.08   1.35  
 S0    1.8V Power Rail                     False    1.78   1.62   1.98  
 S0    2.5V Power Rail                     False    2.49   2.25   2.75  
 S0    3.3V Power Rail                     False    3.32   2.97   3.63  
 S0    5.0V Power Rail                     False    4.95   4.50   5.50  
 S0    3.0V RTC Battery                    False    2.49  

More info hier: https://live.paloaltonetworks.com/docs/DOC-1274

6. show counters for everything

paroot@pa-firewall(active)> show counter global filter

Global counters:
Elapsed time since last sampling: 64.514 seconds

name                                   value     rate severity  category  aspect    description
--------------------------------------------------------------------------------
pkt_recv                           1155768088      717 info      packet    pktproc   Packets received
pkt_sent                           815014585      507 info      packet    pktproc   Packets transmitted
pkt_alloc                          489073279      309 info      packet    resource  Packets allocated
session_allocated                   99404061       61 info      session   resource  Sessions allocated
session_freed                       99399316       63 info      session   resource  Sessions freed
session_installed                   99403892       61 info      session   resource  Sessions installed
session_discard                       290228        0 info      session   resource  Session set to discard by security policy check
flow_rcv_err                            2741        0 drop      flow      parse     Packets dropped: flow stage receive error
flow_rcv_dot1q_tag_err               1112122        0 drop      flow      parse     Packets dropped: 802.1q tag not configured
flow_no_interface                    1114236        0 drop      flow      parse     Packets dropped: invalid interface
flow_ipv6_disabled                      1943        0 drop      flow      parse     Packets dropped: IPv6 disabled on interface
flow_np_pkt_rcv                    883415495      538 info      flow      offload   Packets received from offload processor
flow_np_pkt_xmt                    796521895      497 info      flow      offload   Packets transmitted to offload processor
flow_policy_nofwd                         11        0 drop      flow      session   Session setup: no destination zone from forwarding
flow_policy_deny                      477803        0 drop      flow      session   Session setup: denied by policy
flow_tcp_non_syn                      293967        0 info      flow      session   Non-SYN TCP packets without session match

--------------------------------------------------------------------------------
Total counters shown: 124
--------------------------------------------------------------------------------

Global counters are good but they are not documented and there is about 1000 counters that you cant really understand without any information…
Palo Alto should publicate some documentation about them. I counted the counter types in groups that are specified from the names. Just the flow types has about 400 counter values. see the following outputs:

$ awk -F"_" '{print $1}' pa_show_counter_global_names.txt | sort | uniq -c | sort -n
      2   fpga
      2   ssh
      5   dlp
      6   uid
     14   url
     14   zip
     16   nat
     23   aho
     23   session
     23   tcp
     24   pkt
     27   dfa
     34   log
     42   appid
     42   ssl
     45   proxy
    123   ha
    157   ctd
    393   flow

Sorting with the second column too I see that the flow dos and the ctd sml are the smoothiest types. But what the ctd sml means?
Currently it is for me 35 unknown counter.

$ awk -F"_" '{print $1,$2}' pa_show_counter_global_names.txt | sort | uniq -c | sort -n
...
      8   nat dynamic
      8   proxy ssl
      9   appid ident
      9   flow arp
      9   flow scan
     10   ctd filter
     10   dfa fpga
     10   flow nd
     10   flow policy
     11   flow ipsec
     12   aho fpga
     13   ha update
     15  
     15   ha err
     16   ctd fwd
     16   flow ipfrag
     16   flow predict
     19   ssl hsm
     22   ha dos
     23   flow msg
     24   flow host
     24   flow tunnel
     25   flow fwd
     25   ha aa
     33   flow parse
     35   ctd sml
     81   flow dos

Palo Alto makes categories for us:

paroot@pa-paris> show counter global filter category
  aho       AHO match engine
  appid     Application-Identification
  ctd       Content-Identification
  dfa       DFA match engine
  dlp       DLP
  flow      Packet processing
  fpga      FPGA
  ha        High-Availability
  log       Logging
  nat       Network Address Translation
  packet    Packet buffer
  proxy     TCP proxy
  session   Session management
  ssh       SSH termination
  ssl       SSL termination
  tcp       TCP reordering
  uid       User Indentification
  url       URL filtering
  zip       ZIP processing
  <value>   Counter category

Or aspects. What is the difference between category and aspect?
(Something similar categorisation should I welcome from cisco asa for asp drops!)

paroot@pa-paris> show counter global filter aspect
  aa         HA Active/Active mode
  arp        ARP procesing
  dos        DoS protection
  forward    Packet forwarding
  ipfrag     IP fragment processing
  ipsec      IPSec transport mode procesing
  mgmt       Management-plane packet
  mld        MLD procesing
  nd         ND procesing
  offload    Hardware offload
  parse      Packet parsing
  pktproc    Packet processing
  qos        QoS enforcement
  resource   Resource management
  session    Session setup/teardown
  system     System function
  tunnel     Tunnel encryption/decryption
  <value>    Counter aspect

Palo Alto classifies the counters with severity as well. I thing the severity depends mostly on the level of the value and not just on the type.
Lets see in the future if that categorisation will be useful.

paroot@pa-paris> show counter global filter severity
  drop     Drop
  error    Error
  info     Informational
  warn     Warning
  <value>  Counter severity

There is a really cool value, it is called “rate”. The “sampling time” is the time interval that was calcultated for the rate.
This is a really good idea!

paroot@pa-paris> show counter global filter severity error

Global counters:
Elapsed time since last sampling: 2.991 seconds

name                                   value     rate severity  category  aspect    description
--------------------------------------------------------------------------------
flow_rcv_err                           22557        0 drop      flow      parse     Packets dropped: flow stage receive error
flow_rcv_dot1q_tag_err               5639000        3 drop      flow      parse     Packets dropped: 802.1q tag not configured
flow_no_interface                    5639000        3 drop      flow      parse     Packets dropped: invalid interface
flow_ipv6_disabled                   8640843       12 drop      flow      parse     Packets dropped: IPv6 disabled on interface
flow_policy_nofwd                         33        0 drop      flow      session   Session setup: no destination zone from forwarding
flow_policy_deny                     9505443       16 drop      flow      session   Session setup: denied by policy
flow_scan_drop                           160        0 drop      flow      session   Session setup: denied by scan detection
flow_tcp_non_syn_drop                2665825        2 drop      flow      session   Packets dropped: non-SYN TCP without session match
flow_fwd_l3_bcast_drop               1661664        1 drop      flow      forward   Packets dropped: unhandled IP broadcast
flow_fwd_l3_mcast_drop              12691304        9 drop      flow      forward   Packets dropped: no route for IP multicast
...
flow_host_service_deny                194361        1 drop      flow      mgmt      Device management session denied
flow_host_service_unknown              87234        0 drop      flow      mgmt      Session discarded: unknown application to control plane
ctd_filter_decode_failure_zip         548188        0 error     ctd       pktproc   Number of decode filter failure for zip
ctd_filter_decode_failure_chunk            1        0 error     ctd       pktproc   Number of decode filter failure for chunk
ctd_filter_decode_failure_qpdecode      1199        0 error     ctd       pktproc   Number of decode filter failure for qpdecode
url_request_pkt_drop                   96722        0 drop      url       pktproc   The number of packets get dropped because of waiting for url category request 
url_session_not_in_wait                 1323        0 error     url       system    The session is not waiting for url 
--------------------------------------------------------------------------------
Total counters shown: 42
--------------------------------------------------------------------------------

7. show a specific session

paroot@pa-paris> show session all filter
+ application         Application name
+ count               count number of sessions only
+ destination         destination IP address
+ destination-port    Destination port
+ destination-user    Destination user
+ egress-interface    egress interface
+ from                From zone
+ hw-interface        hardware interface
+ ingress-interface   ingress interface
+ min-kb              minimum KB of byte count
+ nat                 If session is NAT
+ nat-rule            NAT rule name
+ pbf-rule            Policy-Based-Forwarding rule name
+ protocol            IP protocol value
+ qos-class           QoS class
+ qos-node-id         QoS node-id value
+ qos-rule            QoS rule name
+ rematch             rematch sessions
+ rule                Security rule name
+ source              source IP address
+ source-port         Source port
+ source-user         Source user
+ ssl-decrypt         session is decrypted
+ start-at            Show next 1K sessions
+ state               flow state
+ to                  To zone
+ type                flow type
  |                   Pipe through a command
  <Enter>             Finish input

8. show the statistics on application recognition

paroot@pa-firewall(active)> show system statistics application
Top 20 Application Statistics: ('q' to quit, 'h' for help)                                                                          

Virtual System: vsys1
application                      sessions   packets      bytes
-------------------------------- ---------- ------------ ------------
ssl                              32208687   2621702162   2394119316296
youtube-base                     377        785717       743884280
unknown-tcp                      125        11188805     4917558883
flash                            3624       677147       606388608
http-video                       51         567913       544347901
paloalto-updates                 129        412014       531207347
symantec-av-update               519        344321       326874859
ldap                             106736     601099       296377922
ms-ds-smb                        24838      825350       265635071
tumblr-base                      1128       291235       265043705
kerberos                         37886      463510       141636027
facebook-base                    5181       234638       127381215
rtmp                             59         113566       108224342
google-maps                      614        107019       84628229
gmail-base                       245        126698       78117553
google-update                    360        70878        62608339
twitter-base                     5199       122036       52990855
itunes-base                      74         55465        52417148
sharepoint-base                  190        95025        49686715
itunes-appstore                  2          41337        39838469

9. show policy match for specific session

You can test a specific traffic and check the match with the rulebase or nat or policy based routes or whatever you want.

Lets see what you can define for your specific traffic. Its quite customizable, you can add more than just a source destination IP or ports or an Interface.
The to and from field mean the zones. It is not that good described if you hit the question mark:

paroot@pa-firewall(active)> test security-policy-match ?
+ application        Application name
+ category           Category name
+ destination        destination IP address
+ destination-port   Destination port
+ from               from 
+ protocol           IP protocol value
+ show-all           show all potential match rules
+ source             source IP address
+ source-user        Source User
+ to                 to 
  |                  Pipe through a command
              Finish input

Lets make an example for policy based routes. If you test policy based routing you have to definde the from zone or it shows wrong match. Sad :-(

paroot@pa-firewall(active)> test pbf-policy-match source 10.10.8.5 destination 4.2.2.2 protocol 6 destination-port 80 from DMZ-Zone1

DMZ-Zone1 {
        id 14;
        from DMZ-Zone1;
        source any;
        destination any;
        user any;
        application/service  any/any/any/any;
        action Forward;
        symmetric-return no;
        forwarding-egress-IF/VSYS ethernet1/1;
        next-hop 10.1.199.2;
        terminal no;
}

Lets make an example with proxono application.

paroot@pa-firewall(active)> test security-policy-match source 10.10.8.5 destination 4.2.2.2 show-all yes protocol 6 destination-port 80 category sports application proxono 

DMZ-Zone1-Apps-block {
        from DMZ-Zone1;
        source 10.10.8.0/22;
        source-region none;
        to Transfer;
        destination any;
        destination-region none;
        user any;
        category any;
        application/service [ socks/tcp/any/1080 ftp/tcp/any/21 tftp/tcp/any/69 tftp/udp/any/69 fasp/tcp/any/33001 fasp/tcp/any/22 fasp/udp
/any/33001-33500 http-proxy/tcp/any/80 http-proxy/tcp/any/443 http-proxy/tcp/any/1080 http-proxy/tcp/any/3128 http-proxy/tcp/any/8000 http-
proxy/tcp/any/8080 kazaa/tcp/any/1214 kazaa/tcp/any/1903 skydur/tcp/any/80 skydur/tcp/any/443 gnutella/tcp/any/any gnutella/udp/any/any sou
lseek/tcp/any/2230-2250 direct-connect/tcp/any/any ares/tcp/any/any ares/udp/any/any warez-p2p/tcp/any/6000 warez-p2p/tcp/any/6346-6351 war
ez-p2p/tcp/any/32285 warez-p2p/udp/any/6346-6351 emule/tcp/any/any emule/udp/any/any imesh/tcp/any/80 imesh/tcp/any/443 imesh/tcp/any/1863 
imesh/tcp/any/4000-4999 imesh/tcp/any/4661 imesh/tcp/any/4662 imesh/tcp/any/4671 imesh/tcp/any/6346-6351 imesh/udp/any/4665 imesh/udp/any/4
672 imesh/udp/any/6346-6351 bittorrent/tcp/any/any bittorrent/udp/any/any unknown-p2p/udp/any/any peerenabler/tcp/any/3529-3533 peerenabler
/udp/any/3529-3533 100bao/tcp/any/3468 100bao/tcp/        application/service(implicit) [ ssl/tcp/any/any ssl/tcp/any/80 ssl/tcp/any/443 ss
l/tcp/any/3690 ssl/tcp/any/9418 ssl/udp/any/any ssl/udp/any/17500 web-browsing/tcp/any/33333 web-browsing/tcp/any/49221 web-browsing/tcp/an
y/any web-browsing/tcp/any/80 web-browsing/tcp/any/443 web-browsing/tcp/any/3690 web-browsing/tcp/any/9418 web-browsing/udp/any/33338 web-b
rowsing/udp/any/33339 web-browsing/udp/any/49221 web-browsing/udp/any/any citrix-jedi/tcp/any/443 citrix-jedi/tcp/any/8200 ];
        action deny;
        terminal no;
}

DMZ-Zone1-Services {
        from DMZ-Zone1;
        source 10.10.8.0/22;
        source-region none;
        to Transfer;
        destination any;
        destination-region none;
        user any;
        category any;
        application/service [ any/tcp/any/443 any/tcp/any/80 any/tcp/any/8080 ];
        action allow;
        terminal yes;
}

If you check my matching rules, it will output you all matching rules. But if there is a deny first this will be the only matching rule since the firewall does a security policy lookup from top to bottom.
In the rule below with tha name DMZ-Zone1-Apps-block I have used an Application filter and set the category to networking and subcategory to proxy. The firewall will put then all the applications in the rule that belongs to this categories.
Warning with Application Updates! If you make an application update, this will automatically update the Application filters and adds or deletes the new apps to/from the rule that has the application filtered category.
Palo Alto suggests to use Application groups instead of filter but this can be a heavy work if you have to add manually a tons of applications to a group. I would use application filters and always read the release notes for Application Updates and check if my application filters are involved with the new release or not.

PA_objects

10. debug packet flow

debug flow from one source example 192.168.8.153

debug dataplane packet-diag set filter match source 192.168.8.153
debug dataplane packet-diag set filter on

debug dataplane packet-diag set log feature flow basic
#or for more debugs
debug dataplane packet-diag set log feature all

debug dataplane packet-diag clear log log
debug dataplane packet-diag set log on

capture into different files

debug dataplane packet-diag set capture stage firewall file ftp-fw
debug dataplane packet-diag set capture stage drop file ftp-drop
debug dataplane packet-diag set capture stage receive file ftp-rx
debug dataplane packet-diag set capture stage transmit file ftp-tx
debug dataplane packet-diag set capture on
debug dataplane packet-diag show setting

Initiate traffic and after that stop the debugging

debug dataplane packet-diag set log off
debug dataplane packet-diag set capture off

Check and copy all logs to ssh server (172.16.5.142). The file will be like management-plane_20140915_1217.tar.gz

debug dataplane packet-diag aggregate-logs
less dp-log pan_packet_diag.log
scp export log-file management-plane to daniela@172.16.5.142:/home/daniela/temp/

Delete the captures and delete debug and logs

delete debug-filter file ftp-fw
delete debug-filter file ftp-drop
delete debug-filter file ftp-rx
delete debug-filter file ftp-tx
debug dataplane packet-diag clear all

check debug state again to se its cleared

debug dataplane packet-diag show setting

11. show policy of a firewall managed through panorama

If the rulebases were created and pushed from panorama we cant see it in set cli mode, but in xml like mode. Its sad :-(
xml like policy can be filtered only with the command “show config running xpath“, xpath could be useful for pushed-shared-policy too.

paroot@pa-firewall(active)> show config pushed-shared-policy

policy {
  shared;
  panorama {
    address {
...
    address-group {
...
    external-list;
    region {
...
    service {
...
    service-group;
    application;
    application-filter {
...
    application-group {
...
    schedule;
    profile-group {
...
    tag {
...
    threats {
      vulnerability;
      spyware;
    }
    profiles {
      virus {
        Monitor {
          decoder {
            ftp {
...
      url-filtering {
...
      file-blocking {
...
      data-filtering;
      data-objects;
      hip-objects;
      hip-profiles;
      dos-protection {
...
    log-settings {
      profiles {
...
    pre-rulebase {
      security {
        rules {
...
      pbf {
        rules {
...
      nat {
        rules {
    post-rulebase;

To view the running configuration use the command show running with your config part, like nat-policy.

paroot@pa-firewall(active)> show running ?
> appinfo2ip                    Show application-specific IP mapping information
> application                   Show application info
> application-override-policy   Show currently deployed application override policy
> application-signature         Show application signature statistics
> captive-portal-policy         Show currently deployed captive-portal policy
> decryption-policy             Show currently deployed decryption policy
> dos-policy                    Show currently deployed DoS policy
> global-ippool                 Show global ippool status
> ippool                        Show ippool usage
> ipv6                          Show IPv6 information
> logging                       Show log and packet logging rate
> nat-policy                    Show currently deployed NAT policy
> nat-rule-cache                Show all NAT rules of all versions in cache
> nat-rule-ippool               Show specified NAT rule ippool usage
> pbf-policy                    Show currently deployed Policy-Based forwarding policy
> qos-policy                    Show currently deployed QoS policy
> resource-monitor              Show resource monitoring statistics
> rule-use                      Show used/non-used policy rules
> security-policy               Show currently deployed security policy
> ssl-cert-cn                   Show ssl certificate common name cache
> tcp                           Show tcp reassembly setup
> top-urls                      Show top-urls statistics
> tunnel                        Show runtime tunnel states
> url                           Show the category of the URL as in the url-cache
> url-cache                     Show all URLS in url-cache
> url-info                      Show categorization details of the URL as in the url-cache
> url-license                   Show url license information

# Make an example with the unused rule commands:

paroot@pa-firewall(active)> show running rule-use rule-base security type unused vsys vsys1

DMZ-Zone1-Apps-block_V6
DMZ-Zone2-Apps-block
DMZ-Zone1-Apps-allow

The templates are better. They can be reached in set cli mode.

paroot@pa-budapest> set cli config-output-format set
paroot@pa-budapest> configure 
Entering configuration mode
[edit]                                                                                                                                                                            
paroot@pa-budapest# show template network dhcp
set network dhcp interface ethernet1/4.101 server option dns primary 192.168.1.250
set network dhcp interface ethernet1/4.101 server option lease timeout 2880
set network dhcp interface ethernet1/4.101 server option gateway 192.168.1.250

If you would like to present your lovely command here, jus update me!

Monitor_pic

Last but not least you can monitor any of the commands output above if you want. It can be done with Palo Alto XML API.
Are you interested in it? Than visit my post, Custom Monitoring of Palo Alto with Perl and Cacti.
I have tried the same with prtg but prtg has fallen out with its shortcomings on HTTP XML/REST Value Sensor…

Advertisements