Network Expect Examples

Contents

  1. Network Expect Examples
    1. Doing a TCP three-way handshake by hand
    2. A simple TCP traceroute program
    3. A simple ping program
    4. Simple ARP scanner using the send_expect command
    5. Simple ARP scanner that uses send/expect commands
    6. Simple TCP SYN scanner
    7. A simple TCP traceroute using send_network and expect_network commands
    8. 0trace - traceroute on established connections
    9. PCAP File Re-writing and Replay
    10. Naptha TCP State Exhaustion Testing With Network Expect

This page contains useful examples of Network Expect scripts. Some of these scripts are included in the examples/ directory of the NetworkExpect distribution. If you write a script that you think could be useful to others please send it to the NetworkExpect developers for inclusion in the official examples or here.

Note: The scripts below were written for versions of NetworkExpect that do not use libwireshark for packet dissection. If the version of NetworkExpect that you are using uses libwireshark for packet dissection then some minor changes (mostly changes in variable names) are required. The examples distributed with the libwireshark-enabled versions of NetworkExpect already have these changes incorporated.

To run these scripts you will need to do something like: 

shell# nexp <script name> <script parameters>

For example: 

shell# nexp ping.nexp google.com

The examples need to be run with root privileges because of the low level operations (packet sniffing and crafting) that need to be done.

While NetworkExpect is usually used via scripts, it can also be used interactively by issuing commands from a prompt. A sample InteractiveSession session with NetworkExpect shows some of the most important features of NetworkExpect, so make sure you check that out too.

Doing a TCP three-way handshake by hand

Sometimes full control over a TCP session is required from the start. Maybe you need to send some special TCP options that the TCP stack of your operating system does not support. Maybe you need to establish the TCP session and then send some malformed TCP segment. Maybe you need to play tricks with TCP sequence numbers.

Establishing a TCP session is something that is normally done by the operating system's TCP stack. If you want to establish a TCP session by hand your options are limited, and there's always the risk of having the operating system's TCP stack getting on the way. The use raw sockets is an option, but it requires tedious programming.

NetworkExpect can be used to craft a TCP three-way handshake without much work. The following script will connect to a TCP server. The source IP address of the client is an unused address in the network (must be configured by changing the variable "myip"). The script handles ARP requests for this address. This allows to have complete control over the TCP session since the TCP stack of the host where NetworkExpect is running will not see anything, and therefore will not interfere. 

# Some useful constants
set SYN 0x02
set RST 0x04
set ACK 0x10

set target 10.10.10.1
set sport [random 20000:65535]
set dport 22
set interface [outif $target]
set window 4096

# We'll use a ghost IP. Make sure $myip is not being used...
set myip 10.10.10.123
set mymac [random mac]

# Spawn a listener for ARP requests
spawn_network -i $interface host $myip and {arp[6:2]} == 1

expect_network_before {1} {
    # Received an ARP request, send ARP reply
    send_network -o $interface \
        ether(src = $mymac, dst = $arp(sha) )/ \
        arp-reply(tha = $arp(sha), tip = $arp(sip), sha = $mymac, sip = $myip)
    nexp_continue
}

# Start TCP 3-way handshake

# Spawn a listener for TCP segments coming from the FTP server to us
spawn_network -i $interface "tcp and src host $target and dst host $myip and src port $dport and dst port $sport"

set retries 3
set isn [random]

# Send TCP SYN
send_network ip(src = $myip, dst = $target)/ \
             tcp(src = $sport, dst = $dport, \
                 window = $window, syn, seq = $isn, ack-seq = 0)

# Wait for response from the server
expect_network {$tcp(flags) == ($SYN | $ACK)} {
    # Got a SYN+ACK so we need to send the final segment of the 3-way HS
    send_network ip(src = $myip, dst = $target)/ \
                 tcp(src = $tcp(dstport), dst = $tcp(srcport), \
                     window = $window, ack, seq = $tcp(ack), \
                     ack-seq = [expr $tcp(seq) + 1])
} {$tcp(flags) & $RST} {
    puts "Connection refused"
    exit 1
} {1} {
    # Any other weird combination of TCP flags we respond to with a RST
    send_network ip(src = $myip, dst = $target)/ \
                 tcp(src = $tcp(dstport), dst = $tcp(srcport), rst)
    exit 1
} timeout {
    # Our SYN got lost in transit or it was filtered - perform exponential
    # backoff and retransmit the SYN...
    if {$retries > 0} {
        incr retries -1
        set timeout [expr $timeout*2]
        puts "SYN timeout, increasing timeout to $timeout"
        send_network ip(src = $myip, dst = $target)/ \
                     tcp(src = $sport, dst = $dport, \
                         window = $window, syn, \
                         seq = $isn, ack-seq = 0)
        nexp_continue
    } else {
        puts "Connection timed out"
        exit 1
    }
}

# We're done with the 3-way handshake. If we want to send more stuff
# we need to use correct sequence numbers. Our sequence number is
# $tcp(ack) and the server's is $tcp(seq) + 1.
#

puts Done.

A simple TCP traceroute program

#
# A very simple TCP traceroute program. Uses the send_expect command.
#
# How to use:
#
# shell# nexp ./tcp-traceroute.nexp google.com 1:25
#

if {$argc != 2} {
    puts "Usage: $argv0 <IP address> <TTL range>"
    exit 1
}

set target [lindex $argv 0]
set ttlrange [lindex $argv 1]

set verbose 1

set interface [outif $target]

# Spawn a listener. We don't really have to specify a filter because the
# send_expect command will intelligently match injected stimulus with
# received answers.
spawn_network -i $interface

send_expect -tries 2 -delay 0.010 \
    ip(id = random, dst = $target, ttl = $ttlrange)/ \
    tcp(src = random, dst = 80, syn)

foreach r $_(received) s $_(sent) {
    packet dissect r
    set source $ip(src)
    set pdu_type $pdu(1,type)

    packet dissect s

    puts [format "$ip(ttl) $source %.3f ms $pdu_type" [expr [packet tdelta r s]*1000] ]
}

A simple ping program

if {$argc != 1} {
    puts "Usage: $argv0 <hostname or IP address>"
    exit 1
}

set target [lindex $argv 0]

# Spawn a listener for ICMP messages from our target
spawn_network -i [outif $target] icmp and src host $target

puts "PING $target 20(48) bytes of data."

for {set id [pid]; set seq 0} {1} {incr seq} {
    send_network ip(dst = $target)/ \
                 icmp-echo(seq = $seq)/ \
                 raw(12345678901234567890)

    expect_network -timeout 1 {$icmp(type) == 0 && $icmp(id) == $id} {
        puts [format "$pdu(2,tot_len) bytes from $ip(src): icmp_seq=$seq ttl=$ip(ttl) time=%.3f ms" [expr [txdelta ip]*1000 ] ]
        sleep [expr 1.0 - [txdelta ip] ]
    }
}

Simple ARP scanner using the send_expect command

if {$argc != 1} {
    puts "Usage: $argv0 <interface>"
    exit 1
}

set interface [lindex $argv 0]
set network "$iface($interface,ip)/$iface($interface,netmask)"

set verbose 1

# Spawn a listener for ARP replies
spawn_network -i $interface {arp[6:2]} == 2

send_expect -o $interface -delay 0.001 -tries 2 \
    ether(dst = BROADCAST)/ \
    arp-request(tha = BROADCAST, \
                tip = '$network', \
                sha = $iface($interface,hw_addr), \
                sip = $iface($interface,ip) )

puts "\nFound [llength $_(received)] hosts alive:\n"

foreach r $_(received) {
    packet dissect r
    puts "$arp(sip) is at $arp(sha)"
}

Simple ARP scanner that uses send/expect commands

if {$argc != 1} {
    puts "Usage: $argv0 <interface>"
    exit 1
}

set interface [lindex $argv 0]
set network "$iface($interface,ip)/$iface($interface,netmask)"

set arprequest [pdu new -o $interface \
                    ether(dst = BROADCAST)/ \
                    arp-request(tha = BROADCAST, \
                                tip = '$network', \
                                sha = $iface($interface,hw_addr), \
                                sip = $iface($interface,ip) )]

# Spawn a listener for ARP replies
spawn_network -i $interface {arp[6:2]} == 2

for {set i 0} {$i < [pdu count arprequest]} {incr i} {
    # Send ARP request
    send_network -count 1 arprequest

    # Read ARP reply
    expect_network -timeout .05 {1} {
        puts "$arp(sip) is at $arp(sha)"
    }
}

Simple TCP SYN scanner

# Some useful constants
set SYN 0x02
set RST 0x04
set ACK 0x10

if {$argc != 2} {
    puts "Usage: $argv0 <hostname or IP address> <port range>"
    exit 1
}

set verbose 1
set ntries 3
set timeout 3
set delay 0.000100
set target [lindex $argv 0]
set ports [lindex $argv 1]

# Spawn a listener to get answers from our target
spawn_network -i [outif $target] src host $target

send_expect -tries $ntries -timeout $timeout -delay $delay \
    ip(dst = $target)/tcp(syn, dst = $ports)

foreach r $_(received) {
    packet dissect r
    if {$tcp(flags) & $RST} {
        # Do nothing to avoid printing excessive information, i.e. who
        # cares if what you're scanning has 65000+ ports closed?
    } elseif {$tcp(flags) == ($SYN | $ACK) } {
        puts "$tcp(srcport)/tcp open"
    } else {
        puts "$tcp(srcport)/tcp unknown"
    }
}

A simple TCP traceroute using send_network and expect_network commands

This script sends TCP SYN probes with IP time-to-live values that start at 1 and increment by 1 with each probe. After sending a probe an "expect_network" command waits for a response to the probe. If the response is an ICMP message it is assumed to be an ICMP time-exceeded message and probes continue to be sent. If the response is a TCP segment then it is assumed that we have reached the target. For each probe the device that generated the message, along the round-trip time of the response are printed. If a probe doesn't get a response then "timeout" is printed. 

if {$argc == 0} {
    puts "Usage: $argv0 <IP address> \[<port>]"
    exit 1
}

set target [lindex $argv 0]

incr argc -1
if {$argc == 1} {
    set port [lindex $argv 1]
} else {
    set port 80
}

set interface [outif $target]

spawn_network -i $interface src host $target or icmp

for {set reached_target 0; set ttl 1} {!$reached_target && $ttl <= 30} {incr ttl} {
    # Send probe
    send_network \
        ip(id = $ttl, dst = $target, ttl = $ttl)/ \
        tcp(src = random, dst = $port, syn)

    expect_network -timeout 2.0 {1} {
        if {[lsearch $pdu(list) icmp] == -1} {
            # No ICMP in received packet. This means we reached the target.
            set reached_target 1
        }

        puts [format "$ttl ${ip.src} %.3f ms" [expr [txdelta ip]*1000] ]
    } timeout {
        puts "$ttl timeout"
    }
}

0trace - traceroute on established connections

0trace is a proof of concept tool written by security researcher Michal Zalewski that rides an established TCP session to perform a traceroute. The technique and the tool were announced to several security mailing lists on January 7, 2007:

http://seclists.org/fulldisclosure/2007/Jan/0145.html

The original proof of concept written by Michal Zalewski uses: 1) a small C program that uses raw sockets to send custom TCP segments, 2) tcpdump to read network traffic, and 3) a shell script that glues everything together by using a lot of grep, awk, sed, tail, head and other Unix commands. To further complicate things, to use the tool the user needs to spawn a telnet session to the target from outside the tool, and then run the tool. It was a bit... hhhmmm, complicated and messy (after all it was a proof of concept, and it worked and proved the point.)

The 0trace technique and proof of concept tool is a great example of how the NetworkExpect framework can make things much simpler when designing, prototyping, and writing network tools: using Network Expect the whole process is simplified so the user doesn't need to manually use a tool (telnet) outside the main tool. Also, the resulting script is written in a single language (Network Expect), doesn't use any external programs (telnet, tcpdump, grep, sed, awk, cut, head, tail), and is very concise and easy to follow.

Here's the equivalent 0trace Network Expect script that performs the same function as the original proof of concept that Michal wrote: 

if {$argc != 2} {
    puts "Usage: $argv0 <IP address> <TTL range>"
    exit 1
}

set verbose 1

set target [lindex $argv 0]
set ttlrange [lindex $argv 1]
set interface [outif $target]
set port 80

# Open a TCP session to the target
spawn_network -tcp $target $port

# Obtain the actual IP address of the server we are connected to, as well
# as the local port number
set 4tuple [spawn_network -4tuple $listener_id]
set targetip [lindex $4tuple 2]
set myport [lindex $4tuple 1]

# Spawn a listener before we generate more traffic
spawn_network -i $interface (src host $targetip and dst port $myport) or icmp

# Generate some traffic over this TCP session, but do not do anything that
# would cause the server to tear down the session.
send_network {data('GET / HTTP/1.0\r\n')}

expect_network -timeout 1 {1} {
    # Keep reading data from target as long as it keeps sending. We will
    # leave this expect statement 1 second after there is no more data to
    # read.
    nexp_continue
}

# Figure out our sequence number and the server's.
set myseq ${tcp.ack}
set hisseq ${tcp.seq}

# Now send our probes while riding on the established TCP session
send_expect -tries 2 -delay 0.010 -o ip \
    ip(id = 1++, dst = $targetip, ttl = $ttlrange)/ \
    tcp(src = $myport, dst = $port, ack, seq = $myseq, \
        ack-seq = $hisseq, window = 65535)/ \
    data(\\x00)

# Process and display results
foreach r $_(received) s $_(sent) {
    packet dissect r
    set source ${ip.src}
    set proto $pdu(3,proto)

    packet dissect s

    puts [format "${ip.ttl} $source %.3f ms $proto" [expr [packet tdelta r s]*1000] ]
}

Here's an example of the tool in action: 

# ./0trace.nexp ebay.com 1:20
1 172.18.172.1 0.593 ms icmp
2 14.0.254.57 0.231 ms icmp
3 14.0.2.11 0.457 ms icmp
4 172.18.127.229 0.521 ms icmp
5 172.18.0.25 0.541 ms icmp
6 10.81.254.61 0.529 ms icmp
7 10.81.254.181 0.557 ms icmp
8 10.81.254.194 1.298 ms icmp
9 64.102.241.135 0.862 ms icmp
10 64.102.254.197 1.541 ms icmp
11 157.130.42.53 3.064 ms icmp
12 152.63.39.106 1.971 ms icmp
13 152.63.48.94 74.545 ms icmp
14 152.63.57.1 72.373 ms icmp
15 157.130.128.94 73.439 ms icmp
16 66.135.207.58 73.182 ms icmp
17 66.135.192.87 93.653 ms tcp
18 66.135.192.87 93.480 ms tcp
19 66.135.192.87 73.481 ms tcp
20 66.135.192.87 76.415 ms tcp
#

This shows that ebay.com was 17 hops away. A regular, UDP-based traceroute fails to show the last few hops: 

$ traceroute -n ebay.com
traceroute to ebay.com (66.135.192.87), 30 hops max, 38 byte packets
 1  172.18.172.1  0.675 ms  0.648 ms  0.664 ms
 2  14.0.254.57  0.280 ms  0.238 ms  0.233 ms
 3  14.0.2.11  0.689 ms  0.593 ms  0.592 ms
 4  172.18.127.229  0.717 ms  0.637 ms  0.626 ms
 5  172.18.0.25  0.708 ms  0.660 ms  0.663 ms
 6  10.81.254.61  0.692 ms  0.668 ms  0.666 ms
 7  10.81.254.181  0.726 ms  0.680 ms  0.677 ms
 8  10.81.254.194  1.096 ms  0.976 ms  0.930 ms
 9  64.102.241.135  1.471 ms  1.142 ms  1.412 ms
10  64.102.254.197  1.285 ms  1.326 ms  1.510 ms
11  157.130.42.53  2.803 ms  2.724 ms  2.252 ms
12  152.63.39.106  2.372 ms  3.318 ms  2.650 ms
13  152.63.48.94  72.617 ms  72.900 ms  73.257 ms
14  152.63.57.1  72.921 ms  72.656 ms  73.011 ms
15  * * *
16  * * *
[...]

The way the tool works is as follows:

  1. A TCP session to the target is established
  2. An incomplete HTTP request is sent
  3. This incomplete request allows to obtain the sequence numbers that both the server and the client are using
  4. One we know the sequence numbers a series of TCP segments that only carry one byte of data ('\x0') are sent to the server. Each segment has an IP TTL that starts at 1 and increments by one in each segment
  5. Normal traceroute behavior then follows, i.e. ICMP time exceeded received means that we hit a new hop and haven't reached the target yet

All this is implemented in the simple Network Expect script above.

PCAP File Re-writing and Replay

A separate wiki page has been written to discuss this topic. See RewriteAndReplay.

Naptha TCP State Exhaustion Testing With Network Expect

A separate wiki page has been written to discuss this topic. See Naptha.

Examples (last edited 2009-09-04 02:02:25 by EloyParis)