Implementing Jumbo Frames
A lot has been said about jumbo frames. Some say it’s not worth the effort while others proclaim it to be an absolute requirement. Regardless of where you stand on the debate it is generally accepted that jumbo frames can yield performance improvements on isolated iSCSI networks.
Unfortunately, getting jumbo frames to work correctly is more complicated than just enabling it in VMware and your back end SAN. In order for jumbo frames to actually work it needs to be enabled on every NIC and every switch port between the source and the destination. Only then will jumbo frames actually work. In addition, if you don’t verify the connection, not only do you not know if you actually have jumbo frames working, but you may actually decrease overall performance or cripple your network if it’s only partially enabled.
To check the connection you can use a simple ping test to verify that jumbo frames are enabled from source to destination. From your source server you can use:
- ping destination_server -f -l 8000 (Windows)
- ping -s 8000 destingation_server (ESXi – using SSH)
Here destination_server would be the IP address of the back end SAN. If you do not get a reply, but do get replies when attempting a standard ping with no parameters then you do not have jumbo frames working. The most likely problem is your switch. Different switch vendors have different methods for enabling jumbo frames, however, so you should consult your switch vendor to verify the correct method for enabling jumbo frames, whether it’s on a switch or port basis (or sometimes both). Next verify that each NIC on your source and target devices have jumbo frames enabled and that you are using the correct path to communicate. Also verify that everything is configured for an MTU of at least 9000. If your source and destination are set for 9014, but your switch is only configured for 9000 you will likely have issues with connectivity. Switch ports must be set to at least the same size as your source and destination. Usually switches with jumbo frames enabled are set, by default, to an MTU of 9216, while NICs are set to 9000 or 9014. Once properly set you should be able to successfully ping with a length of 8000 bytes. If you can only ping with a length of 1472 bytes then you are still using a standard frame length.