For several years I have been meaning to write a program to parse the output of tcpdump, which I like ot use to figure out how certain web transactions work (so I can automate them). A lot of other people have written these things too, but sometimes I just have to write it myself. Sometimes life is like that.
Up until now, I simply dealt with the hex dumps, which do not really bother me as much as the thought of reading a hex dump when it is so easy not to. In any case, I am use to reading these sorts of things because I use them to do things like figure out the iTunes Music Library format, and a lot of my time in academia was spent looking at binary formats (because everyone has their own).
For some reason, this output worries people. They get vary pained expressions on their faces, unless, like my cats, they just look at me as if to say "So when's lunch?"
sudo /usr/sbin/tcpdump -lX -i en1 -s 512 dst port 80 or src port 80
18:50:11.671997 10.0.1.4.50692 > useperl.org.http: P 1:619(618) ack 1 win 33984 (DF)
0x0000 4500 029e 67fe 4000 4006 8885 0a00 0104 E...g.@.@.......
0x0010 4223 faaf c604 0050 8f9a e018 8df5 2ab5 B#.....P......*.
0x0020 8018 84c0 d401 0000 0101 080a 0005 141e ................
0x0030 c49b d4f6 4745 5420 2f20 4854 5450 2f31 ....GET./.HTTP/1
0x0040 2e31 0d0a 486f 7374 3a20 7573 652e 7065 .1..Host:.use.pe
0x0050 726c 2e6f 7267 0d0a 5573 6572 2d41 6765 rl.org..User-Age
0x0060 6e74 3a20 4d6f 7a69 6c6c 612f 352e 3020 nt:.Mozilla/5.0.
0x0070 284d 6163 696e 746f 7368 3b20 553b 2050 (Macintosh;.U;.P
0x0080 5043 204d 6163 204f 5320 5820 4d61 6368 PC.Mac.OS.X.Mach
0x0090 2d4f 3b20 656e 2d55 533b 2072 763a 312e -O;.en-US;.rv:1.
0x00a0 302e 3129 2047 6563 6b6f 2f32 3030 3330 0.1).Gecko/20030
0x00b0 3330 3620 4361 6d69 6e6f 2f30 2e37 0d0a 306.Camino/0.7..
That is not so tough to read, but I wanted to use Perl to make it easier. First, I need to get rid of the (normal) human readable stuff. I just change the big X to a small x in the tcpdump switches. I could just grab the converted characters from the big X output, but then I have to jump through a lot of hoops to figure out where the newlines go.
This output, which is less busy, troubles people a bit more. I guess some people skipped out on that class on ASCII while in high school (or whatever form or level or whatever the rest of the world calls it).
sudo /usr/sbin/tcpdump -lx -i en1 -s 512 dst port 80 or src port 80
18:55:29.808364 10.0.1.4.50695 > useperl.org.http: P 1:619(618) ack 1 win 33984 (DF)
4500 029e 6b5a 4000 4006 8529 0a00 0104
4223 faaf c607 0050 16ba 2fc2 a1e6 e685
8018 84c0 aecf 0000 0101 080a 0005 169b
c49c 511d 4745 5420 2f20 4854 5450 2f31
2e31 0d0a 486f 7374 3a20 7573 652e 7065
726c 2e6f 7267 0d0a 5573 6572 2d41 6765
6e74 3a20 4d6f 7a69 6c6c 612f 352e 3020
284d 6163 696e 746f 7368 3b20 553b 2050
5043 204d 6163 204f 5320 5820 4d61 6368
2d4f 3b20 656e 2d55 533b 2072 763a 312e
302e 3129 2047 6563 6b6f 2f32 3030 3330
3330 3620 4361 6d69 6e6f 2f30 2e37 0d0a
After that, I only need to convert all of that back to the human readable characters. Easy-peasy. When SourceForge updates their CVS servers, you can see
the source for my httpsniffer program, whose output is a bit more friendly. People still look at me funny, but maybe I should get them
HTTP: The Definitive Guide for the holidays, just as a gentle nudge that they really should know how the world works, you know?
Time: 18:13:56 Src: 10.0.1.4.51750 Dst: service.cddb.com.http
POST http://service.cddb.com//service/2.0/product/1 HTTP/1.0
Host: service.cddb.com
Accept: application/octet-stream
User-Agent: SL CDDB 2.0
Content-Type: application/octet-stream
Content-Length: 573
-------------------------------------------------------------------------
Time: 18:13:57 Src: service.cddb.com.http Dst: 10.0.1.4.51750
HTTP/1.1 200 OK
Date: Sat, 29 Nov 2003 00:13:58 GMT
Server: KZSM/1.0
Content-length: 1205
Connection: close
Content-type: application/octet-stream
And Bob's your uncle.
Try ethereal
Fletch on 2003-12-02T16:43:53
It understands a good bit of many different protocols and can dump out something similar. It can also filter based on their contents (i.e. I just want to see HTTP traffic that's POST requests ...).
Ethereal.com
Re:Try ethereal
brian_d_foy on 2003-12-02T17:20:47
I saw that, and I wanted to install it on my FreeBSD machine, but my ports is out of date, so the ethereal version it knows about is gone. Not only that, but my version of cvsup no longer works, so I can't update ports until I update that.
Of course, these things really are not all that hard, but my ability to remember how to do all this stuff has greatly suffered this year. Heck, it took me 15 minutes last week to figure out where the Trash was on Mac OS X :)
Ethereal does look very cool though.
