Extracting Information From a Hexadecimal Packet Dump
Contents
Sometimes we may obtain a hexadecimal dump of some part of a packet, and we may need to get the data into some format that allows us to do something with it, like analyzing it, rewriting or replaying it, etc. Network Expect has some built-in capabilities to handle hexadecimal packet dumps that are similar in spirit to Wireshark's text2pcap utility. This page provides a couple of examples of how to import hexadecimal packet dumps and make use of them.
Deciphering an OSPF Message
The first example shows how to dissect an OSPF message. We start only have a hexadecimal packet dump and a brief description that tells us that the packet is an OSPF message. By looking at the packet dump below it seems clear that it comes from tcpdump, but we could have received it via email, we may have seen it in some C or Python source code, etc.
12:27:53.704034 vlan 1, p 6, IP gw-ext.chapus.net > OSPF-ALL.MCAST.NET: OSPFv2, Hello, length 60
0x0000: c001 0800 45c0 0050 ea14 0000 0159 d86f ....E..P.....Y.o
0x0010: 0a0a 0c02 e000 0005 0201 0030 0a0a 0c02 ...........0....
0x0020: 0000 0000 946c 0000 0000 0000 0000 0000 .....l..........
0x0030: ffff ff00 000a 1201 0000 0028 0a0a 0c02 ...........(....
0x0040: 0a0a 0c01 0a0a 0c01 fff6 0003 0001 0004 ................
0x0050: 0000 0001 ....Note that tcpdump is telling us that this packet has a 802.1q tag (the "vlan 1, p 6" part) that precedes an IP packet (0x45, 0xc0, etc.)
Our objective is to dissect this packet dump in Network Expect. Note that we could easily use Wireshark's text2pcap to convert the dump to PCAP format and then open it in Wireshark, but the assumption is that we may have to do something more once we have the data in a format that Network Expect can use.
The first thing to do after we have the hexadecimal packet dump is to import the data into Network Expect. The best Network Expect data object to store the imported data in is a BArray object, which is a custom Tcl object that Network Expect implements to store binary data. Network Expect uses the barray command to manipulate BArrays, and this command has a hex-import option that we need to make use of. To import the above hexadecimal packet dump we use the barray command in this way:
netexpect> set dot1q [barray hex-import -ignore-prefix 1 -ignore-after-column 58 {
> 0x0000: c001 0800 45c0 0050 ea14 0000 0159 d86f ....E..P.....Y.o
> 0x0010: 0a0a 0c02 e000 0005 0201 0030 0a0a 0c02 ...........0....
> 0x0020: 0000 0000 946c 0000 0000 0000 0000 0000 .....l..........
> 0x0030: ffff ff00 000a 1201 0000 0028 0a0a 0c02 ...........(....
> 0x0040: 0a0a 0c01 0a0a 0c01 fff6 0003 0001 0004 ................
> 0x0050: 0000 0001 ....
>} ]
\xc0\x01\x08\x00\x45\xc0\x00\x50\xea\x14\x00\x00\x01\x59\xd8\x6f\x0a\x0a\x0c\x02\xe0\x00\x00\x05\x02\x01\x00\x30\x0a\x0a\x0c\x02\x00\x00\x00\x00\x94\x6c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\x00\x00\x0a\x12\x01\x00\x00\x00\x28\x0a\x0a\x0c\x02\x0a\x0a\x0c\x01\x0a\x0a\x0c\x01\xff\xf6\x00\x03\x00\x01\x00\x04\x00\x00\x00\x01Note the use of the -ignore-prefix and the -ignore-after-column options: the former is needed because the hexadecimal dump contains an offset field that we do not need, and the latter is needed because the dump contains an ASCII representation on the right of the data. -ignore-prefix 1 causes one field (word) at the beginning of each line to be ignored, and -ignore-after-column causes anything after character 58 in any input line to be ignored. The number of "prefix words" to ignore is easily determined by visual inspection. The column number at which we start to ignore input can be determined by pasting a sample input line into your favorite text editor, moving the cursor to the column after which all data needs to be ignored, i.e. right before the ASCII representation of the data, and then looking at your editor's column number.
The above command was obviously executed interactively, i.e. not from a script (notice the "netexpect>" prompt and the '>' line continuation characters). The hexadecimal dump was copied and pasted into the terminal window where Network Expect was being run.
The string "\xc0\x01\x08\x00\x45\xc0\x00..." is now stored in the variable $dot1q, which has a BArray type:
netexpect> set dot1q \xc0\x01\x08\x00\x45\xc0\x00\x50\xea\x14\x00\x00\x01\x59\xd8\x6f\x0a\x0a\x0c\x02\xe0\x00\x00\x05\x02\x01\x00\x30\x0a\x0a\x0c\x02\x00\x00\x00\x00\x94\x6c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\x00\x00\x0a\x12\x01\x00\x00\x00\x28\x0a\x0a\x0c\x02\x0a\x0a\x0c\x01\x0a\x0a\x0c\x01\xff\xf6\x00\x03\x00\x01\x00\x04\x00\x00\x00\x01
We can manipulate this variable using the barray Network Expect command. For example:
netexpect> barray slice dot1q 0l4 \xc0\x01\x08\x00
(This uses the barray command to extract four bytes starting at offset 0 of the $dot1q BArray variable.)
Once we have imported the dump into Network Expect our next task is to convert it to a Network Expect Packet object so we can dissect it. We do this using Network Expect's packet command:
netexpect> set pkt [packet new -y EN10MB ether(type = 0x8100)/'$dot1q'] 10.10.12.2 -> 224.0.0.5 OSPF Hello Packet
Note a couple of important points:
We used the -y switch of the packet new command to specify the data link type -- we do this because the packet dump starts with a 802.1q tag.
Because the packet dump starts with a 802.1q tag, the PDU definition in the packet new command uses "ether(type = 0x8100)". This gets prepended to the actual dump, which is stored in the $dot1q variable.
After using the packet new command and putting its result in the $pkt variable, $pkt now has the data in Packet format and we can use the packet command to dissect it:
netexpect> packet show pkt
---[ frame ]---
frame.time: "Aug 15, 2010 12:43:41.339683000 EDT"
frame.time_epoch: 1281890621.339683000
frame.time_delta: 0.2147483647
frame.time_delta_displayed: 0.2147483647
frame.time_relative: 0.2147483647
frame.number: 0
frame.len: 98
frame.cap_len: 98
frame.marked: 0
frame.ignored: 0
frame.protocols: eth:vlan:ip:ospf
---[ eth ]---
eth.dst: 00:00:00:00:00:00
eth.addr: 00:00:00:00:00:00
eth.ig: 0
eth.lg: 0
eth.src: 00:00:00:00:00:00
eth.addr: 00:00:00:00:00:00
eth.ig: 0
eth.lg: 0
eth.type: 33024
---[ vlan ]---
vlan.priority: 6
vlan.cfi: 0
vlan.id: 1
vlan.etype: 2048
---[ ip ]---
ip.version: 4
ip.hdr_len: 20
ip.tos: 192
ip.tos.precedence: 6
ip.tos.delay: 0
ip.tos.throughput: 0
ip.tos.reliability: 0
ip.tos.cost: 0
ip.flags: 0
ip.flags.rb: 0
ip.flags.df: 0
ip.flags.mf: 0
ip.len: 80
ip.id: 59924
ip.frag_offset: 0
ip.ttl: 1
ip.proto: 89
ip.checksum: 55407
ip.checksum_good: 1
ip.checksum_bad: 0
ip.src: 10.10.12.2
ip.addr: 10.10.12.2
ip.src_host: 10.10.12.2
ip.host: 10.10.12.2
ip.dst: 224.0.0.5
ip.addr: 224.0.0.5
ip.dst_host: 224.0.0.5
ip.host: 224.0.0.5
---[ ospf ]---
Text label: OSPF Header
Text label: OSPF Version: 2
ospf.msg: 1
ospf.msg.hello: 1
Text label: Packet Length: 48
ospf.srcrouter: 10.10.12.2
Text label: Area ID: 0.0.0.0 (Backbone)
Text label: Packet Checksum: 0x946c [correct]
Text label: Auth Type: Null
Text label: Auth Data (none)
Text label: OSPF Hello Packet
Text label: Network Mask: 255.255.255.0
Text label: Hello Interval: 10 seconds
ospf.v2.options: 18
ospf.v2.options.dn: 0
ospf.v2.options.o: 0
ospf.v2.options.dc: 0
ospf.v2.options.l: 1
ospf.v2.options.np: 0
ospf.v2.options.mc: 0
ospf.v2.options.e: 1
ospf.v2.options.mt: 0
Text label: Router Priority: 1
Text label: Router Dead Interval: 40 seconds
Text label: Designated Router: 10.10.12.2
Text label: Backup Designated Router: 10.10.12.1
Text label: Active Neighbor: 10.10.12.1
Text label: OSPF LLS Data Block
Text label: Checksum: 0xfff6
Text label: LLS Data Length: 12 bytes
Text label: Extended options TLV
Text label: Type: 1
Text label: Length: 4
ospf.lls.ext.options: 1
ospf.lls.ext.options.rs: 0
ospf.lls.ext.options.lr: 1The packet dump could also be replayed. Here is one possible way of accomplishing this:
netexpect> send_network -o hex data(data = [barray slice dot1q 4:]) Packet size: 80 bytes 00000000 45c0 0050 ea14 0000 0159 d86f 0a0a 0c02 E..P.....Y.o.... 00000010 e000 0005 0201 0030 0a0a 0c02 0000 0000 .......0........ 00000020 946c 0000 0000 0000 0000 0000 ffff ff00 .l.............. 00000030 000a 1201 0000 0028 0a0a 0c02 0a0a 0c01 .......(........ 00000040 0a0a 0c01 fff6 0003 0001 0004 0000 0001 ................ 00000050 ! 1
Note that here we are ignoring the first four bytes of the original packet dump (the 802.1q tag) and injecting the IP packet. In this case we do not need to specify an Ethernet header.
BGP UPDATE Message Affects Internet Routing Stability
On Friday August 27, 2010 around 8:41 UTC, an experiment involving BGP was conducted by RIPE NCC and researchers at Duke University. The experiment involved announcing a prefix with an unknown transitive attribute. Due to a bug in a router vendor's BGP implementation, the experiment caused BGP connection resets that affected the stability of the Internet during the duration of the experiment. The issue has been explained in detail here and here. The first link actually contains an exact description of the attribute that caused the problem.
This is relevant to this example because Cisco routers that reset BGP connections due to a bug in the peering router's BGP implementation logged syslog messages like this:
Aug 27 10:42:45.455: %BGP-3-NOTIFICATION: sent to neighbor 172.16.63.133 3/1 (update malformed) 188 bytes F0630BB8 00000000 00000000 00000000 00 Aug 27 10:42:45.455: BGP: 81.46.63.133 Bad attributes FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 00F4 0200 0000 D940 0101 0040 0208 0203 0D18 329C 316E 4003 0451 2E3F 85C0 0804 0D18 0078 F063 0BB8 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 185D AF90
Now, the question is: how do we go about decoding this hexadecimal dump to understand the experiment RIPE and Duke University performed on August 27, 2010? Well, we could use Network Expect's hexadecimal import and packet dissection capabilities:
First, we notice that there are 16 bytes at the beginning of the hexadecimal dump that have the same value 0xff. People familiar with BGP and RFC 1771 will notice that this is likely the BGP marker in the BGP header. So with that in mind we import this hexadecimal dump into a Tcl BArray variable using the barray hex-import command:
netexpect> set bgpmsg [barray hex-import {FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 00F4 0200 0000 D940 0101 0040 0208 0203 0D18 329C 316E 4003 0451 2E3F 85C0 0804 0D18 0078 F063 0BB8 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 185D AF90}]After importing the hexadecimal dump into a Tcl BArray variable, the variable itself looks like a long character string when the variable is used in a string context:
netexpect> set bgpmsg \xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\xf4\x02\x00\x00\x00\xd9\x40\x01\x01\x00\x40\x02\x08\x02\x03\x0d\x18\x32\x9c\x31\x6e\x40\x03\x04\x51\x2e\x3f\x85\xc0\x08\x04\x0d\x18\x00\x78\xf0\x63\x0b\xb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x18\x5d\xaf\x90
We then can create a Packet Tcl variable using the packet new command:
netexpect> set pkt [packet new ip(dst = 1.1.1.1)/tcp(dst = 179)/'$bgpmsg']
Note how we prefix the actual BGP message with layer 3 (IP) and layer 4 (TCP) headers, and the destination port for the TCP segment is 179, the BGP port.
After the above packet new command is run, the variable $pkt contains the BGP message that we are interested in dissecting. The Tcl string representation of the packet variable is actually a short description of the packet:
netexpect> set pkt 10.82.244.122 -> 1.1.1.1 BGP [TCP Retransmission] UPDATE Message
To actually dissect the packet we use the packet show command:
netexpect> packet show pkt
---[ frame ]---
frame.time: "Aug 31, 2010 21:58:04.218379000 EDT"
frame.time_epoch: 1283306284.218379000
frame.time_delta: 0.2147483647
frame.time_delta_displayed: 0.2147483647
frame.time_relative: 0.2147483647
frame.number: 0
frame.len: 284
frame.cap_len: 284
frame.marked: 0
frame.ignored: 0
frame.protocols: raw:ip:tcp:bgp
---[ raw ]---
Text label: No link information available
---[ ip ]---
ip.version: 4
ip.hdr_len: 20
ip.tos: 0
ip.tos.precedence: 0
ip.tos.delay: 0
ip.tos.throughput: 0
ip.tos.reliability: 0
ip.tos.cost: 0
ip.len: 284
ip.id: 256
ip.flags: 0
ip.flags.rb: 0
ip.flags.df: 0
ip.flags.mf: 0
ip.frag_offset: 0
ip.ttl: 64
ip.proto: 6
ip.checksum: 30734
ip.checksum_good: 1
ip.checksum_bad: 0
ip.src: 10.82.244.122
ip.addr: 10.82.244.122
ip.src_host: 10.82.244.122
ip.host: 10.82.244.122
ip.dst: 1.1.1.1
ip.addr: 1.1.1.1
ip.dst_host: 1.1.1.1
ip.host: 1.1.1.1
---[ tcp ]---
tcp.srcport: 1073
tcp.dstport: 179
tcp.port: 1073
tcp.port: 179
tcp.stream: 0
tcp.len: 244
tcp.seq: 0
tcp.nxtseq: 244
tcp.hdr_len: 20
tcp.flags: 0
tcp.flags.cwr: 0
tcp.flags.ecn: 0
tcp.flags.urg: 0
tcp.flags.ack: 0
tcp.flags.push: 0
tcp.flags.reset: 0
tcp.flags.syn: 0
tcp.flags.fin: 0
tcp.window_size: 8192
tcp.checksum: 25769
tcp.checksum_good: 1
tcp.checksum_bad: 0
Text label: SEQ/ACK analysis
tcp.analysis.bytes_in_flight: 244
tcp.analysis.rto: 8.975946000
tcp.analysis.rto_frame: 0
---[ bgp ]---
Text label: UPDATE Message
Text label: Marker: 16 bytes
Text label: Length: 244 bytes
bgp.type: 2
Text label: Unfeasible routes length: 0 bytes
Text label: Total path attribute length: 217 bytes
Text label: Path attributes
Text label: ORIGIN: IGP (4 bytes)
Text label: Flags: 0x40 (Well-known, Transitive, Complete)
Text label: 0... .... = Well-known
Text label: .1.. .... = Transitive
Text label: ..0. .... = Complete
Text label: ...0 .... = Regular length
Text label: Type code: ORIGIN (1)
Text label: Length: 1 byte
bgp.origin: 0
Text label: AS_PATH: 3352 12956 12654 (11 bytes)
Text label: Flags: 0x40 (Well-known, Transitive, Complete)
Text label: 0... .... = Well-known
Text label: .1.. .... = Transitive
Text label: ..0. .... = Complete
Text label: ...0 .... = Regular length
Text label: Type code: AS_PATH (2)
Text label: Length: 8 bytes
Text label: AS path: 3352 12956 12654
Text label: AS path segment: 3352 12956 12654
Text label: Path segment type: AS_SEQUENCE (2)
Text label: Path segment length: 3 ASs
Text label: Path segment value: 3352 12956 12654
bgp.as_path: 3352
bgp.as_path: 12956
bgp.as_path: 12654
Text label: NEXT_HOP: 81.46.63.133 (7 bytes)
Text label: Flags: 0x40 (Well-known, Transitive, Complete)
Text label: 0... .... = Well-known
Text label: .1.. .... = Transitive
Text label: ..0. .... = Complete
Text label: ...0 .... = Regular length
Text label: Type code: NEXT_HOP (3)
Text label: Length: 4 bytes
bgp.next_hop: 81.46.63.133
Text label: COMMUNITIES: 3352:120 (7 bytes)
Text label: Flags: 0xc0 (Optional, Transitive, Complete)
Text label: 1... .... = Optional
Text label: .1.. .... = Transitive
Text label: ..0. .... = Complete
Text label: ...0 .... = Regular length
Text label: Type code: COMMUNITIES (8)
Text label: Length: 4 bytes
Text label: Communities: 3352:120
Text label: Community: 3352:120
bgp.community_as: 3352
bgp.community_value: 120
Text label: Unknown (3004 bytes)
Text label: Flags: 0xf0 (Optional, Transitive, Partial, Extended Length)
Text label: 1... .... = Optional
Text label: .1.. .... = Transitive
Text label: ..1. .... = Partial
Text label: ...1 .... = Extended length
Text label: Type code: Unknown (99)
Text label: Length: 3000 bytes
Text label: Unknown (3000 bytes)
Text label: Network layer reachability information: 4 bytes
Text label: 93.175.144.0/24
Text label: NLRI prefix length: 24
bgp.nlri_prefix: 93.175.144.0And that's all -- by looking at the packet dissection we can know the details about the BGP update message that caused the problem, i.e. the Autonomous System the update came from (12654), the prefix (93.175.144.0/24), the path attribute (type 99), the size (3000 bytes), etc. Note that this confirms the description of the problem at http://seclists.org/nanog/2010/Aug/774, which states:
"The experimental attribute was part of an experiment conducted in collaboration with a group from Duke University. This involved announcing a large (3000 bytes) optional transitive attribute, using a modified version of Quagga. The attribute used type code 99. The data consisted of zeros. We used the prefix 93.175.144.0/24 for this and announced from AS 12654 on AMS-IX, NL-IX and GN-IX to all our peers."
For the curious, the impacted BGP implementation was on Cisco IOS XR (Cisco IOS was not affected) and Cisco published a Cisco Security Advisory on the same day: http://www.cisco.com/warp/public/707/cisco-sa-20100827-bgp.shtml.
Conclusion
These are simple examples, but they show how to quickly analyse hexadecimal packet dumps in a very short time and using very few commands and without the need to write specialized tools.