Gigabit network performance

In this forum you are welcome to discuss the networking equipment and technologies from office network to home LAN, as well as network administration and security issues.

Moderators: CPUagnostic, MTX, Celt, Hammer_Time, Sauron_Daz, Tacitus, Anna

Gigabit network performance

Postby Stupify » Tue Jan 29, 2013 4:26 am

I am not getting the gigabit speed over all elements that are gigabit network in the big transfers (1GB+ files), rather I am getting 100mbps speed. Here is the setup:

NAS (Gigabit, PC) --> SwitchA (TP-Link TL-SG1005D 5-port Gigabit) --> Router (DLink DIR-655 Extreme: Gigabit) --> SwitchB (TP-Link TL-SG1005D 5-port Gigabit) --> Laptop (Gigabit)

Before I used to get 30MBps with Jumbo frames disabled when I had it like following (only one switch):
NAS (Gigabit, PC) --> Router (DLink DIR-655 Extreme: Gigabit) --> SwitchB (TP-Link TL-SG1005D 5-port Gigabit) --> Laptop (Gigabit)

I had enabled Jumbo frames (9k) on NAS and tested to get 10MBps. Although in this case the Laptop had Jumbo frames disabled.

In case it helps here is some extra information on what else is on the network:
SwitchA: MagicJack, Satellite and NAS
Router: Printer, SwitchA, SwitchB
SwitchB: PC, Laptop, Laptop, VOIP phone

Can anyone help/guide me on how to ensure that I get the full speed over the network or at least close to it (50MBps+).
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Hammer_Time » Tue Jan 29, 2013 6:13 am

Are you using CAT5e nw cable??

CAT5 cable is only rated to 100 Mbps

http://compnetworking.about.com/od/ethe ... cables.htm

CAT5e is rated up to 1000 Mbps ( Gigabit speed ).

Get some CAT5e cables :wink:

Also make sure your SATA hdd's are in enhanced/AHCI mode in mobo BIOS, and not in "IDE/compatibility" mode as that can slow down things as shown in this thread: http://www.tomshardware.com/forum/30532 ... it-network

I have a windows 7 computer, and am trying to back it up to my centos server, over LAN. I have a gigabit network, am using gigabit NIC cards, cat 5e cables, and a gigabit switch.

I am using samba to share files between the two computers, but the speed decrease is experienced over FTP and http as well.

The drivers are all up to date. RDC is disabled on windows.

The speed starts off at about 80MB/s, then drops within a second to 12MB/s. And remains there, or drops to about 3MB/s.

The odd thing is that when I use iperf, I get speeds at 100MB/s. So clearly the problem isnt in the network, but in either the settings on one of the computers or the protocol.



For anyone whose interested, I solved the problem. When I installed centos my bios settings for sata hard drives was as IDE compatible. Switching them to IDE enhanced enabled DMA.

My drives we /dev/hdX were now /dev/sdX. The root drive mounted itself. I had the other drives mounting to /home and /disk2 so I edited /etc/fstab.

After that, I got close to 75MB/s consistently to both drives.


Note ( for others reading this thread, I know Stupe is well aware of the difference ) he is talking about MegaBytes per second, not "bits"...
The richest man is not he who has the most, but he who needs the least. No good deed goes unpunished...

Image
User avatar
Hammer_Time
Rantmeister Mod
 
Posts: 33637
Joined: Wed Dec 31, 1969 4:00 pm
Location: Kitchener-Waterloo, Ontario, Mordor

Re: Gigabit network performance

Postby Stupify » Tue Jan 29, 2013 7:01 am

I doubt the IDE compatible is the culprit here as I have managed to get over 30MBps in the past without the new changes (2 switches) that was done last night.

I am more inclined to believe it is the cable that's the culprit here. The Cat5e cable that I had going to NAS now went to the switch and I took small cables that I had lying around and used them. If none of those small cables are cat5e or better, I will just have to pull out the crimper make me a new one from the brand spanking new 1000Ft Cat5e cable spool that I have.
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Tue Jan 29, 2013 10:44 am

I also would suggest checking the cables. You could try skipping one of the switches, at least to narrow it down.
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby veli05 » Tue Jan 29, 2013 1:14 pm

Stupify wrote:I am not getting the gigabit speed over all elements that are gigabit network in the big transfers (1GB+ files), rather I am getting 100mbps speed. Here is the setup:

NAS (Gigabit, PC) --> SwitchA (TP-Link TL-SG1005D 5-port Gigabit) --> Router (DLink DIR-655 Extreme: Gigabit) --> SwitchB (TP-Link TL-SG1005D 5-port Gigabit) --> Laptop (Gigabit)

Before I used to get 30MBps with Jumbo frames disabled when I had it like following (only one switch):
NAS (Gigabit, PC) --> Router (DLink DIR-655 Extreme: Gigabit) --> SwitchB (TP-Link TL-SG1005D 5-port Gigabit) --> Laptop (Gigabit)

I had enabled Jumbo frames (9k) on NAS and tested to get 10MBps. Although in this case the Laptop had Jumbo frames disabled.

In case it helps here is some extra information on what else is on the network:
SwitchA: MagicJack, Satellite and NAS
Router: Printer, SwitchA, SwitchB
SwitchB: PC, Laptop, Laptop, VOIP phone

Can anyone help/guide me on how to ensure that I get the full speed over the network or at least close to it (50MBps+).


:?: So you are or are not getting 100mbps Stupify and how does your goal of 50Mbps help you? Do you want to see network speeds at 50mbps on all devices on the network? Do you want more than 100mbps? I are confused
veli05
Full Member
 
Posts: 368
Joined: Thu Jan 20, 2011 7:59 pm

Re: Gigabit network performance

Postby Stupify » Tue Jan 29, 2013 3:44 pm

Ah... veli05 i guess you missed the lower and upper cases while reading my post, and also HT's note:

Hammer_Time wrote:Note ( for others reading this thread, I know Stupe is well aware of the difference ) he is talking about MegaBytes per second, not "bits"...


mbps = megabits per second
MBps = megabytes per second (1 byte = 8 bits)
So:
100mbps = 12.5MBps max theoretical but more like 10MBps due to network overhead and other items
1000mbps/gigabit = 125MBps max theoretical but more like 100MBps in real life

i am aiming for reasonable speeds, not the best in class, which is 50MBps (or 400mbps).

I have yet to check out the cables but I feel it and the fact that I had enabled 9K Jumbo frames on the NAS but was disabled on the laptop may have also added to the poor performance. And some part of me also say that because i was low on batter with Power profile set to Balanced may have added to the poor performance. I will play all three as soon as I get some time.

Note: with 8 months kid can really help occupy time, so don't hold your breath on my reply with results.
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Tue Jan 29, 2013 6:41 pm

Do you have jumbo frames enabled on the router? I'm not sure about the switches, but I'm pretty sure the router needs them enabled as well. Though you should still get much better performance than you are getting even without jumbo frames. I can hit 75-90 MBps without jumbo frames.
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby Stupify » Tue Jan 29, 2013 7:08 pm

so i just swapped the cable to Cat5e and still 11MBps is max. Did I need to reset the NAS (PC) for it to recognize the change or what?

Fuzz, the max I have seen so far is 30MBps or so. What could be limiting the performance so much between what you are getting and what I am getting? Could it have anything to do with the other network elements that are connected to each of the 3 devices (2 switches and the router)? The cables that I have running between the switches and the router are cat5e but seem a bit worn out. Even then I was able to hit 30MBps with one switch but the same set of cables (the cable connecting the other switch was connected to NAS directly before).
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Tiggerz » Tue Jan 29, 2013 8:16 pm

Generally I aim for 1/3 of rated, so 1gbs would be about 300mbs actual.

One thing I might suggest is don't mess with jumbo frames. One needs to have all devices supporting them in the loop for it to work or the frames may get dropped. Another suggestion, might want to check that your link is communicating in full duplex (async?)
My posts in X-Bit forums don't mean I condone racist or discriminatory comments made by forum members regarding any other individuals. I support the principles of the Universal Declaration of Human Rights & recognize that all people are born free & equal in dignity & rights & should be treated accordingly in the spirit of brotherhood, freedom, justice and peace in the world
User avatar
Tiggerz
X-bit Guru
 
Posts: 2700
Joined: Thu Apr 03, 2003 11:23 pm
Location: New Zealand

Re: Gigabit network performance

Postby Hammer_Time » Tue Jan 29, 2013 11:32 pm

Stupify wrote:so i just swapped the cable to Cat5e and still 11MBps is max. Did I need to reset the NAS (PC) for it to recognize the change or what?

Fuzz, the max I have seen so far is 30MBps or so. What could be limiting the performance so much between what you are getting and what I am getting? Could it have anything to do with the other network elements that are connected to each of the 3 devices (2 switches and the router)? The cables that I have running between the switches and the router are cat5e but seem a bit worn out. Even then I was able to hit 30MBps with one switch but the same set of cables (the cable connecting the other switch was connected to NAS directly before).


YES! You must enable Jumbo Frames on all your nw devices! :

http://www.maximumpc.com/article/howtos ... mbo_frames

Enabling jumbo frames can significantly increase your network’s throughput while consuming fewer CPU cycles (we’ll explain why in a moment). But before you configure your PCs to use jumbo frames, you should know that their value lies primarily in speeding up large file transfers within your network (versus to and from the Internet).

You should also be aware that enabling jumbo frames might cause problems with latency sensitive network applications, such as VoIP and online games. Lastly, jumbo frames are available only on gigabit networks, and every device in the path of the file transfer—all your switches (starting with the one in the router), your PC, server, and/or NAS—must all be equipped with gigabit Ethernet interfaces. What’s more, each of those devices must be capable of passing the same size jumbo frames. Okay, here’s another twist: There is no such thing as a standard-size jumbo frame.

Confused? Here’s a brief Ethernet primer: Data travels over an Ethernet network in frames, with each frame containing a 1,460-byte payload (the data being transferred) plus a 40-byte TCP/IP header. Add these together and you have Ethernet’s 1,500-byte maximum transmission unit (MTU). Now add the 12-byte Ethernet frame header, two bytes that identify the Ethernet type, and four bytes of error-detection data and you get a standard-sized Ethernet frame of 1,518 bytes.

A jumbo frame is simply an Ethernet frame that’s larger than 1,518 bytes. Since the header sizes remain the same, each frame can carry a larger payload. You’ll see the greatest benefit from enabling jumbo frames when you transfer large files across your network. Since fewer frames are needed to carry the same amount of data, transfer speeds go up and CPU utilization goes down. (CPU utilization probably isn’t an issue on your PC, but your server or NAS almost certainly has a fraction of that horsepower.)

The tricky part is determining the optimum size frame that everything supports, and that will take a little trial-and-error benchmarking to figure out. Gigabit Ethernet supports a maximum MTU of 9,000 bytes; but if you push your network too hard, performance might deteriorate as dropped and fragmented frames are constantly resent.

Image

Your PC's network interface probably came from the factory with jumbo frames disabled.
As we mentioned, the value of using jumbo frames manifests itself during large file transfers. So before you make any changes to your gear’s MTU settings, determine your network’s baseline performance by using a stopwatch to measure how long it takes to copy a large file—a DVD ISO image, for instance—from a client to a server or a NAS box across your network and back again.

On a PC running Windows 7, click the Start menu and then right-click on Computer and choose the Manage option. Click on Device Manager in the left-hand column, click on Network adapters in the center column, and then right-click on the network adapter you’re using to connect to your network and choose Properties. Click on the Advanced tab and look for Jumbo Frame in the Property window. Now, choose the MTU size you wish your NIC to operate with.

You’ll need to do the same thing with your server or NAS device (most NAS boxes have a Web interface for this purpose, so you should refer to its documentation for specifics). Although each device in the path of the transfer needs to be configured to use the same size jumbo frame, you probably won’t be able to manually configure the switch in your router or even a stand-alone switch if it’s an unmanaged model (the most common type in the consumer market). Fortunately, you won’t need to: You’re golden as long as the router’s and any stand-alone switch you use support jumbo frames.


Once you’ve tweaked the MTU values on each device’s network interface, measure how long it takes to copy that same file across the network. If you get a bump in speed with a 4K MTU, try 9K. If performance degrades at that size, dial it back until you find the optimum value.


So you have to mess around a bit to find the optimal frame size that gives you the fastest speed... might be 9k, might be less...try different settings till you get the fastest one...
The richest man is not he who has the most, but he who needs the least. No good deed goes unpunished...

Image
User avatar
Hammer_Time
Rantmeister Mod
 
Posts: 33637
Joined: Wed Dec 31, 1969 4:00 pm
Location: Kitchener-Waterloo, Ontario, Mordor

Re: Gigabit network performance

Postby Stupify » Wed Jan 30, 2013 5:28 am

what i was saying is that i have now disabled Jumbo Frames on all devices (original state) but did not reboot the systems, could that be why i did not see any performance change i.e. 10MBps only?
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Wed Jan 30, 2013 10:37 am

I'd suggest a reboot. you can test if jumbo frames are working by :

ping MACHINE_NAME -f -l 8972

I still don' think Jumbo frames are responsible for the discrepancy.
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby Stupify » Wed Jan 30, 2013 11:03 am

C:\Users\MY>ping tv -f -l 8972

Pinging TV [10.10.10.65] with 8972 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 10.10.10.65:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\MY>ping nas -f -l 8972

Pinging nas [10.10.10.66] with 8972 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 10.10.10.66:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),



Note:
1) nas is running on the HTPC computer (tv - win 7 64bit pro sp1) as a VirtualBox machine with its network setup as bridged so the NAS VM gets its own IP. The NAS (Xubuntu) has full RAW disk access to the 3TB drive which is where the data resides.
2) the laptop i used to do is a Windows 7 64bit pro sp1
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Wed Jan 30, 2013 11:11 am

looks like jumbo frames aren't working, if it is fragmenting the packet. Try another ping at 8000 or so, just to see if there is more packet overhead than 28bits. You can also try pinging each device to see where the problem starts. The router should respond without fragmenting. Are your switches managed? I had to enable jumbo frames on mine, so dumb switches may not handle them....

What speed can you copy files from the HTPC to the NAS VM?
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby Stupify » Wed Jan 30, 2013 11:44 am

i have disabled Jumbo frames on both the HTPC and the laptop (NAS was never modified). The switches are unmanaged desktop switch.

HTPC copy from NAS is stupidly slow now as well - 8MBps. Damn it I am really going to have to waste quite a bit of time to get this straighten out.
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Wed Jan 30, 2013 11:55 am

So last test. . laptop or PC to HTPC, what speed do you get?

It sounds like you will have to work out the slow VM transfer speed though.
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby Stupify » Wed Jan 30, 2013 7:19 pm

The guest OS <-> host OS connection ("virtual network adapter") is setup as 100mbps for some reason and a known thing, hence the 10MBps speeds. However, the guest OS/host OS <-> rest of the world is setup as 1gbps connection. I know that this is not a VirtualBox or the setup issue as I was able to get 30MBps before the switch was added and the wiring changes.

So I am still stuck at square one, that is I need to play the cables more and the switch, now that I have brought everything else back to original configurations.
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Fuzz » Wed Jan 30, 2013 7:57 pm

If I were trying to figure this out, I'd plug in the laptop at each step away from the NAS, and see where the seed degrades to pinpoint the slow device.
It's not the penguins I hate, so much as the idea of penguins.
Image
“I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they cross the road.” - Stephen Hawking
User avatar
Fuzz
X-bit Penguin Hater
 
Posts: 7886
Joined: Tue Jul 13, 2004 11:38 am
Location: Calgary, Canada

Re: Gigabit network performance

Postby Stupify » Thu Jan 31, 2013 3:24 am

okay things have now taken turn for the worse... i am now getting only 10mbps (~1MBps) speeds in the current setup. arrrgggh!!! I am not getting enough time to actually sort this out and so I am doing little small changes at a time but somehow it is making things worse I go along.
What goes around comes around with interest!
User avatar
Stupify
Moderator
 
Posts: 9195
Joined: Wed Dec 31, 1969 4:00 pm

Re: Gigabit network performance

Postby Tiggerz » Thu Jan 31, 2013 10:25 am

Stupify wrote:okay things have now taken turn for the worse... i am now getting only 10mbps (~1MBps) speeds in the current setup. arrrgggh!!! I am not getting enough time to actually sort this out and so I am doing little small changes at a time but somehow it is making things worse I go along.


Ahh, I see what you are trying to achieve....

You have gone from 1 gigabit down to 100 megabits down to 10 megabits and now are trying to achieve 0 megabits. This is meant as a joke but might I suggest purchasing two plastic cups and tieing them together with a bit of string for your next option ;)
My posts in X-Bit forums don't mean I condone racist or discriminatory comments made by forum members regarding any other individuals. I support the principles of the Universal Declaration of Human Rights & recognize that all people are born free & equal in dignity & rights & should be treated accordingly in the spirit of brotherhood, freedom, justice and peace in the world
User avatar
Tiggerz
X-bit Guru
 
Posts: 2700
Joined: Thu Apr 03, 2003 11:23 pm
Location: New Zealand

Next

Return to Networking & Security

Who is online

Users browsing this forum: No registered users and 1 guest

cron