I seriously doubt you have a 100 mbps connection. That maybe the speed of your lan card, but not the speed of your internet. Your internet upstream speed is very important. If you live in the US, goto
Speedtest.net - The Global Broadband Speed Test and check your internet speed. If not, someone else can probably direct you to a site that can test your speed there.
I will say that I was suprised at how little bandwidth the game takes on average. I've had 40+ players on my server and the upstream averaged 450 KB/sec. This is the equivalent of about 4 mbits upstream with overhead. In fact, I probably wouldn't run it with less than 5 mbits at that level. You need some overhead room when serving games.
Let us know what your internet upstream/downstream speed is. The last thing you want to do is start up a substandard server.