Hi,
We are using Snort on Linux in the binary packet capture mode (capture and log in tcpdump format). We find packet drops even at 5 Mbps bandwidth which we feel is very low for the hardware we are using. We would be grateful if you can provide any suggestions on the issue.
Hardware used: HP Proliant DL 140 G2. Dual processor, processor speed 2.8 GHz with 512MB RAM and 72 GB SATA HDD, Gigabit network card.
Operating system: Red Hat Enterprise Linux ES Version 3.
Snort version: Snort 2.3.0
The OS is a default installation. We are not running any software other than snort on the system.
Observations: We find that the drop is related to HDD writes.
If there are no hard disk writes, then there is no drop even at 80 Mbps. We tested this by using a rule in snort which rarely matches, so that snort hardly logs any packets.
We also found that the drop increases when the I/O is high, irrespective of whether it is being done by the same process (snort) or a totally unrelated one. We created a high I/O scenario by doing copy of a huge file (3GB) periodically while snort is running. Even this triggered packet drops.
So, to summarize, we see packet drops in sniffing whenever there is disk I/O happening. We do not suspect the HDD of the machine, as we were able to simulate the problem in two other totally different systems also.
Regards, Sonali
Sonali Gupta wrote:
Observations: We find that the drop is related to HDD writes.
If there are no hard disk writes, then there is no drop even at 80 Mbps. We tested this by using a rule in snort which rarely matches, so that snort hardly logs any packets.
these two observations above lead me to suspect the hardware (more rightly perhaps a driver/kernel related issue)
So, to summarize, we see packet drops in sniffing whenever there is disk I/O happening. We do not suspect the HDD of the machine, as we were able to simulate the problem in two other totally different systems also.
I am not sure which kernel : RHEL ES Version 3 uses...
Chances are its probably a conflict, but on multiple machines is really odd. Alternately try a recent kernel. Possibly something to do with the SATA controller... check for a kernel uprade.
Not sure how feasable is this considering your hardware... : try using another distro... just to check and verify the problem
if the server is not live yet, perhaps you could dump in a recent stock kernel (2.6.12 is out) and see if you get the same problems?
regards Erle
On 21/06/05 11:35 +0530, Sonali Gupta wrote:
Hi,
We are using Snort on Linux in the binary packet capture mode (capture and log in tcpdump format). We find packet drops even at 5 Mbps bandwidth which we feel is very low for the hardware we are using. We would be grateful if you can provide any suggestions on the issue.
Hardware used: HP Proliant DL 140 G2. Dual processor, processor speed 2.8 GHz with 512MB RAM and 72 GB SATA HDD, Gigabit network card.
May I suggest a RAID 10 setup with 15000 RPM SCSI disks instead? Alternatively, ask Sourcefire for what hardware they would recommend with Snort.
One of Marty Roesch's postings on an IDS list basically stated that IDS need hardware ==> fast CPU, gobs of RAM, and large numbers of very fast disks. You may find benchmarking with hardware RAID between RAID 5 and RAID 10 useful. Multiple spindles are required for I/O bound applications.
Devdas Bhagat
Devdas Bhagat devdas@dvb.homelinux.org wrote: On 21/06/05 11:35 +0530, Sonali Gupta wrote:
Hi,
We are using Snort on Linux in the binary packet capture mode (capture and log in tcpdump format). We find packet drops even at 5 Mbps bandwidth which we feel is very low for the hardware we are using. We would be grateful if you can provide any suggestions on the issue.
Hardware used: HP Proliant DL 140 G2. Dual processor, processor speed 2.8 GHz with 512MB RAM and 72 GB SATA HDD, Gigabit network card.
The test setup also makes a lot of difference. Can you elaborate on the test setup?It might just happen that the source of packets might be creating a bottleneck!. In other words either/or both sender/reciever of packets might be creating a bottleneck.
Regards,
Chirag
--------------------------------- Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone.
On 21/06/05 20:02 -0700, chirag radhakrishnan wrote:
On 21/06/05 11:35 +0530, Sonali Gupta wrote: Hi,
We are using Snort on Linux in the binary packet capture mode (capture and log in tcpdump format). We find packet drops even at 5 Mbps bandwidth which we feel is very low for the hardware we are using. We would be grateful if you can provide any suggestions on the issue.
Hardware used: HP Proliant DL 140 G2. Dual processor, processor speed 2.8 GHz with 512MB RAM and 72 GB SATA HDD, Gigabit network card.
The test setup also makes a lot of difference. Can you elaborate on the test setup?It might just happen that the source of packets might be creating a bottleneck!. In other words either/or both sender/reciever of packets might be creating a bottleneck.
Quoting Sonali's original mail again:
If there are no hard disk writes, then there is no drop even at 80 Mbps. We tested this by using a rule in snort which rarely matches, so that snort hardly logs any packets.
The problem is with the SATA drivers, or the disk itself. I am sorely tempted to blame SATA for something that needs tons of writes. SCSI RAID with battery backed cache is the way to go.
If you want to size disk intensive operations, see the server sizing at http://www.tpc-int.org/ (IIRC), the database benchmark site.
Devdas Bhagat
--- Devdas Bhagat devdas@dvb.homelinux.org wrote:
May I suggest a RAID 10 setup with 15000 RPM SCSI disks instead? Alternatively, ask Sourcefire for what hardware they would recommend with Snort.
If you have ooldes of old machines lying around, Distributed FileSystems might help..:D.
Trevor
RAID 10 useful. Multiple spindles are required for I/O bound applications.
Devdas Bhagat
|------|____________________________________|------| ( >- / Scaling FreeSoftware & OpenSource \ -< ) /~\ / In the Enterprise \ /~\ | ) \ | www.fsf.org | www.opensource.org | / (/ | |_|_ ____________________________________/ _|_|
__________________________________ Discover Yahoo! Find restaurants, movies, travel and more fun for the weekend. Check it out! http://discover.yahoo.com/weekend.html
Sonali wrote
We are using Snort on Linux in the binary packet capture mode (capture and log in tcpdump format). We find packet drops even at 5 Mbps bandwidth which we feel is very low for the hardware we are using. We would be grateful if you can provide any suggestions on the issue.
Good Afternoon Sonali,
Can you tell me how did you notice that you were getting packet loss at 5 MBPS??
Regards,
Keith
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
-
chirag radhakrishnan -
Devdas Bhagat -
Erle Pereira -
Keith Fernandez -
Sonali Gupta -
Trevor Warren