How to analyse more than 1GB tcpdump file  

Posted by montel

1. To capture the whole network traffic, I'm using tcpdump . Yeah captured it all, since I've with me a 220 GB hardisk. So i've plenty of diskspace to play with.
A daily network traffic shows that the network traffics dump size is between 1G - 2G per day.

2. I tried using split . It can split the file, but the output can't be read by tcpdump.

3. i'm continue using wireshark. Crash. not even want to parse it using ethereal.

4. Switched to more powerful machine.a 4GB memory Dell XPS with Vista..yeah. Vista.. Download networkminer. cool tools, the analysis part are better than wireshark, ethereal and of course tcpdump.
http://sourceforge.net/projects/networkminer/

5. It took me about 12 hours to load 1G file. but, suddenly after the 13th hours, at around 4.30 am, the system crash. Damm.. 4.30 am. and it was only 63%. Damm ..Dammm

6. So the key for success is by breaking the tcpdump file into smaller size. Chop it so that it can be read by all the analyzer.
thank to my friend from the northen region who have given me this link
http://www.badpenguin.co.uk/main/content/view/46/1/ which introduce me to perl script pcap-lib.pl

7. I'm setting up a new PCBSD in VMware, since this notebook also still new.
http://pcbsd.org/

8. I've run the perl script but it come out with an error. The Perl guru have ask me to update the perl module from CPAN
#perl -MCPAN -e shell
cpan[1]> install Net::Pcap

then after all the installation done i managed to run the perl script that chop the file into whatever MB i want. In this case , I'm splitting the 1 GB file to individual 50MB.

9. walla..all the file now readable by tcpdump. But not by networkminer.

10. one last trick is to change the extension to .pcap instead of .tcpdump which have been generated by the pcap-lib.pl script.
11. On the same time I've reinstall a new XP box for the analysis, but the networkminer is not working there. Fall back using the XPS.

12. Today the log size is increasing to 1.8GB . apart from 1GB yesterday..

in my calculation, a day is only 24 hours...

13. hahha . Orang Utara kasi tau.. pi install dotnet dulu dalam xp tu. serious. After installed the dotnet, my networkminer is running fine. Now , i tried to load 1 GB file, see how laa..can stand or not.

This entry was posted on Thursday, February 12, 2009 at Thursday, February 12, 2009 . You can follow any responses to this entry through the comments feed .

1 comments

thanks for sharing your experience.

October 9, 2009 at 6:18 PM

Post a Comment