Soon, 4.8 Gb/s USB 3.0 and 6 Gb/s SATA will be hitting the mainstream. But be careful when you buy your next mainstream motherboard; some don't handle these technologies very well. We compare three implementations and recommend best practice solutions.
This happens every few years when bottlenecks start impeding performance, but 2010 will be crowned by internal and external storage device bandwidth leaping forward to the point where we're, yet again, constrained by the speed of media and not a physical interface. SATA 3.0, running at 6 Gb/s, promises faster connectivity to the latest solid state drives and mechanical disks. Outside of the PC, we’ll be seeing more and more USB 3.0-based solutions able to move data at the storage devices’ maximum speed (as you likely already know, USB 2.0 holds back the storage performance of many devices in a big way). Existing bottlenecks will be gone—hopefully.
Zoom
USB 3.0 and SATA 6Gb/s Hardware
Add-in controllers enabling USB 3.0 and SATA 3.0 have been available for several months, and are now hitting the mainstream (in fact, AMD recently added 6 Gb/s SATA support to its SB850 southbridge). NEC was first to release a full-blown USB 3.0 controller (µPD720200). Cross-compatibility with USB 2.0 is something users take for granted, and we haven’t seen any USB 3.0 hardware that wouldn't work on prior-gen hardware. GDA and VIA offer USB 3.0 hub controllers, and more designs will certainly be following.
The situation is similar with SATA 3.0. Marvell’s 88SE9123 is the dominant add-in component as the storage industry focuses on transitioning from 3 to 6 Gb/s in 2010. However, not all platforms are able to give these higher-throughput subsystems the bandwidth needed to run unconstrained.
PCI Express Bandwidth Issues
The bandwidth issue is a product of chipsets with too little peripheral connectivity and motherboard vendors pressured to include copious value-added functionality. As long as USB 3.0 and SATA 6Gb/s aren’t built into Intel's and AMD's core logic, those controllers remain add-on devices that require an interface with ample throughput. The interface of choice, naturally, is typically PCI Express, which currently spans two generations of technology. PCI Express 2.0 offers 500 MB/s throughput per lane, while PCI Express 1.x is limited to 250 MB/s. Clearly, a single-lane link cannot saturate the 6 Gb/s peak bandwidth of SATA 3.0 or 4.8 Gb/s ceiling specified for USB 3.0. Rated at up to 500 MB/s, a second-gen PCIe x1 interface is considered adequate.
Second-gen PCI Express is most often used in 16-lane links, giving the latest high-performance GPUs ample bandwidth. As far as we know, every mainstream platform offers at least 16 lanes, whether through the northbridge (AMD 785G, for example) or the processor itself (Intel's Core i3 and Core i5 CPUs). Enthusiast chipsets like AMD's 790FX and Intel's X58 Express offer (at least) twice this amount. Unfortunately, all other PCI Express lanes remain at 250 MB/s. There is an interesting difference, though, in how AMD and Intel handle this connectivity.
AMD vs. Intel?
For some reason, Intel's mainstream chipsets only support PCI Express 2.0 on the primary links that are used for graphics. This applies to both LGA 775 and LGA 1156 platforms. While Intel claims PCI Express 2.0 support for its LGA 1156 based PCH, it limits throughput to PCI Express 1.1-class performance. This is naturally a problem when we start looking at the latest high-speed motherboard-down controllers.
AMD, on the other hand, upgraded the link speeds on its 700- and 800-series chipsets, which means that current AMD mainstream and enthusiast chipsets don’t create bandwidth bottlenecks for high-speed add-on devices.
We took three P55 motherboards from Gigabyte and MSI that all come with different solutions to offer USB 3.0 and SATA 6Gb/s connectivity. We analyzed SATA performance using Crucial’s new RealSSD-C300 and a Seagate Barracuda XT with support for the third-gen standard and found that not all solutions deliver ample bandwidth.
All AMD 700 and 800 chipsets (both northbridge and southbridge) fully support PCI Express 2.0, while Intel’s PCIe 2.0 support is limited to the northbridge/processor-based interfaceThis is why it's unlikely you'll encounter bandwidth bottlenecks on AMD platforms.
But let's say you're looking for a new P55-based motherboard, though. There are still a few options to pursue. Intel controllers typically utilize a single PCI Express lane for the sake of simplicity. Of course, performance bottlenecks could be solved by connecting to the host system using two or four lanes, but that's not the solution you'd want to rely on, since most consumer motherboards only offer x1 and x16 slots.
The first solution is simply to use existing PCIe 1.1 lanes to attach USB 3.0 and SATA 3.0 controllers. This yields a maximum bandwidth of 250 MB/s. Obviously, this approach should be avoided because it handicaps third-generation SATA below the performance levels of even SATA 3Gb/s. It also bottleneck's USB 3.0, too. For individual USB 3.0 drives, this might not be painfully apparent, but as soon as you operate two drives in parallel (or once SSDs exceed 300 MB/s throughput), this bottleneck will start to hurt a lot more.
If you want to see this challenge overcome properly, check out Asus' P7P55D Premium motherboard, which employs a PLX 8613 chip to condense the bandwidth of multiple PCIe 1.1 lanes into a PCIe 2.0 link. Latency-wise, this is not perfect, but it's still better than multiple PCIe 1.1. links. Unfortunately, it's also expensive, resulting in a board that costs more than many mainstream customers are willing to pay. At under $200, the P7P55D-E Pro is perhaps a more attractive mainstream offering that "does switching right."
The second approach to overcoming bandwidth limits is to link high-speed components, such as USB 3.0 or SATA 6Gb/s controllers, to the primary PCI Express 2.0 lanes. This will result in the 16 main graphics lanes sharing bandwidth with high-speed storage interfaces. Gigabyte’s P55A-UD6 uses this solution. Unfortunately, if you actually use the USB and SATA controllers, this knocks your graphics card down to x8 mode. Worse, if you run in CrossFire mode (employing two x8 links), the USB and SATA controller get knocked back to 2.5 Gb/s links. Either way, you're sacrificing performance in order to add support for the latest standards.
Finally, you can allocate available bandwidth in a more flexible way. This is done on Gigabyte’s P55A-UD7 motherboard, which works around the -UD6's limitations. The -UD7 includes an nForce 200 bridge, capable of multiplexing the processor's PCI Express connectivity to support a broader array of graphics configurations (not to mention adding SLI support). A PLX PEX 8608 switch is needed to enable the next-gen SATA and USB controllers found on-board, too.
Zoom
MSI and Gigabyte use the PLX 8608 PCI Express switch to allow dynamic bandwidth switching between available 16 PCI Express 2.0 lanes, multiple x16 slots, and high-speed devices, including USB 3.0 and SATA 6Gb/s controllers.
Zoom
The board's feature list starts with a 24-phase dynamic voltage regulator design offering significant power supply capabilities, and continues with a long list of connectivity options. These include CrossFire support, three x16 PCI Express slots, and JMicron JMB362, ITE 8213, and Marvell 88SE9128 controllers for eSATA, UltraATA/133 and SATA 6Gb/s, respectively. The benefit of three different chips is that each can be linked to a separate PCI Express 1.1 lane, resulting in better distributed bandwidth. Not least of all, there's an NEC controller for USB 3.0.
This and the SATA 3.0 controller are routed through a PCI Express pathway selector switch, allowing the two components to draw connectivity from the processor's second-gen links or the PCH's second-gen links (handicapped by Intel to first-gen throughput). You're either limiting graphics performance/connectivity or SATA/USB throughput. That's a tough choice to make either way.
Zoom
The motherboard can control PCIe lane assignment, or you can manually define whether USB 3.0 or SATA 3.0 should utilize maximum bandwidth. If you take the manual route, you won't be able to run dual graphics cards. Alternatively, if you install CrossFire, you'll face a serious performance impact on the SATA 3.0 chip (as we'll see shortly). This is not inherently Gigabyte’s fault. It's simply a matter of insufficient total bandwidth to accommodate the new high-speed interfaces and dual graphics from Intel's mainstream chipset.
Zoom
The P55-UD7 builds on the feature set of the -UD6 former flagship. The new UD7 is even more feature-laden than its predecessor, including a 24-phase voltage regulator design, comprehensive memory overclocking (Gigabyte claims DDR3-2600+ speeds), automatic phase switching for chipset, memory, and CPU circuitry, and all the "dual" items, like DualBIOS and dual gigabit Ethernet.
However, the UD7's key selling points are its PLX switch for on-board devices and Nvidia's nForce 200 bridge. The latter introduces additional PCI Express 2.0 lanes, multiplexed from the processor's 16 host lanes.
Zoom
As a result, both the USB 3.0 controller and the SATA 3.0 chip—again, NEC's µPD720200 and Marvell's SE9128—can utilize PCI Express bandwidth dynamically. The PLX chip is located below the water block of the hybrid cooling solution and provides PCI Express switching as needed. Although the overall throughput problem isn't solved (the LGA 1156 platform still only provides 16 total PCIe 2.0 lanes), this is the best way to balance loads, by simultaneously introducing even more PCIe flexibility.
Zoom
The third board in this roundup is MSI’s P55-GD85. With the exception of the company's Big Bang family, this is the highest-end LGA 1156 motherboard you can get from MSI. The board comes with dynamic voltage regulator switching (called APS, or active phase switching), the OC Genie overclocking assistant, dual gigabit Ethernet controllers, heatpipe-based cooling, and a handful of other enthusiast-oriented features.
Though it isn't loaded with as many extras as some of the other high-end P55-based motherboards out there, the P55-GD85 does sport some of the most important additions for a flagship platform trying to introduce next-gen storage peripheral interfaces. For instance, the same PLX 8608 PCI Express switch found on Gigabyte's P55A-UD7 makes an appearance. With MSI's P55-GD85, though, we have a fully-featured board with two x16 PCI Express 2.0 slots able to run on eight physical PCIe 2.0 lanes and still allow the USB 3.0 and SATA 6Gb/s controllers (again, from NEC and Marvell) to receive sufficient bandwidth through switching. In addition to these controllers, the board also sports a controller to enable eSATA.
The main difference between the MSI board and Gigabyte’s latest flagship is the absence of Nvidia's nForce 200 bridge chip. Nevertheless, MSI still has the SLI license needed to expose multi-GPU support using Nvidia-based graphics cards.
We're using Crucial’s RealSSD-C300 as a high-bandwidth device and a Seagate Barracuda XT 2TB drive, both of which are equipped with a SATA 6Gb/s interface.
Zoom
| System Hardware |
| Hardware |
Details |
Motherboard
(Socket LGA 1156) |
Gigabyte P55A-UD6 (Rev. 1.0)
Chipset: P55 Express; BIOS: 7d |
Motherboard
(Socket LGA 1156) |
Gigabyte P55A-UD7 (Rev. 1.0)
Chipset: P55 Express; BIOS: F3 |
Motherboard
(Socket LGA 1156) |
MSI P55-GD85 (Rev. 1.0)
Chipset: P55 Express; BIOS: 1.1 |
| CPU Intel |
Intel Core i5-661 (32nm, 3.33 GHz, 2 x 256KB L2 and 4MB L3 Cache, TDP 87W, Rev. B1) |
| RAM DDR3 (dual) |
2 x 2GB DDR3-1600 (OCZ OCZ3G2000LV4GK)
DDR3-1333 8-8-8-24 1T |
| Hard Drive |
Seagate Barracuda 7200.11, 500GB (ST3500320AS)
7,200 RPM, SATA/300, 32MB Cache |
| Graphics (2x) |
Sapphire Radeon HD 5850
GPU: Cypress (725 MHz); Graphics RAM: 1GB GDDR5 (2,000 MT/s); Stream Processors: 1,440 |
| Power Supply |
PC Power & Cooling, Silencer 750EPS12V 750W |
| System Software & Drivers |
| Operating System |
Windows 7 Ultimate 64-bit
Updated 2010-02-23 |
| Drivers and Settings |
| Intel Chipset Drivers |
Chipset Installation Utility Ver. 9.1.1.1025 |
| Intel Matrix Storage Manager |
Version 8.​9.​0.​1023 |
| ATI Graphics Drivers |
Radeon Version 10.1 |
| Benchmarks |
| Performance Measurements |
h2benchw 3.13 |
We didn’t run performance benchmarks on the three boards since the primary purpose of this review is to evaluate possible PCI Express bottlenecks when high-speed controllers, such as SATA 3.0 solutions, are used. Therefore we created a CrossFire setup using two Sapphire Radeon HD 5850 cards. This was more than enough to saturate the PCI Express 2.0 links and show which boards would take a hit on storage performance due to the lack of available bandwidth.
As expected, SATA performance using Marvell's SATA 3.0 controller dives when both x16 PCI Express slots are used in a CrossFire setup on Gigabyte’s P55A-UD6. The two other solutions utilize the PLX chip to dynamically allocate PCI Express 2.0 bandwidth, and don't take as significant of a performance hit.
The results are similar when running a mechanical hard drive. The Barracuda XT sports a SATA 6Gb/s interface, true. But it only delivers peak throughput when reading from or writing into the drive’s cache memory. The rest of the time, it's limited to more modest performance specifications. Only the solutions that use the PLX PCI Express switch are capable of delivering high bandwidth.
Today, the bandwidth issue isn't particularly pressing. After all, there still aren't any SATA-based drives able to move more than 300 MB/s. But it's important to take note of your motherboard's concessions, if there are any. For those who've purchased (or are looking to purchase) boards that divide PCI Express connectivity between graphics cards and storage controllers, you could be hurting performance in a very tangible way, even today. Moreover, as you start attaching next-gen devices to those controllers, you don't want to be dismayed by performance below what you were expecting.
Issues
Existing mainstream chipsets from do not provide sufficient PCI Express bandwidth for USB 3.0 or SATA 6Gb/s controllers because, while PCH-based PCI Express lanes supposedly offer a second-gen interface, they run at first-gen transfer rates (250 MB/s instead of 500 MB/s). Motherboard manufacturers can work around this by routing add-on components through PCIe switching logic or by physically wiring these controllers to PCI Express 2.0 lanes, which typically drive your graphics cards. AMD chipsets (starting with the 700-series) are fully PCI Express 2.0-compliant and consequently don’t exhibit such a limitation.
Motherboard solutions that reroute USB 3.0 and SATA 6Gb/s logic through the chipset’s PCI Express 1.1 links will lead to bottlenecked bandwidth. This also applies if you decide to install a x1 PCI Express USB 3.0 or third-gen SATA add-on card into any Intel platform or an older AMD-based machine. The 250 MB/s of PCIe 1.1 is the most you’ll get. Our benchmark results show that effective bandwidth may even be much lower.
Solutions
Let’s focus on AMD for a moment. The company beat Intel to the inclusion of SATA 6Gb/s support in its latest southbridge revision, which complements the 890GX platform. The chipset serves up six SATA 6Gb/s ports natively, requiring no add-on controller at all. USB 3.0 is not yet supported by any chipset, but hooking up a discrete USB 3.0 controller to a single 500 MB/s PCI Express 2.0 link is a common and definitely workable approach.
For Intel systems, we have to recommend that you pay much closer attention when it comes to motherboard selection. Since 16 PCI Express 2.0 lanes are more than enough for a single graphics card, PCIe switches like the PLX device used on Gigabyte’s P55A-UD7 and MSI's P55-GD85 can accommodate the bandwidth requirements of additional USB 3.0 and SATA 3.0 controllers by dynamically allocating bandwidth from the P55 PCH when a pair of graphics cards is already monopolizing the processor's available second-gen PCI Express. These two boards show that this flexible bandwidth allocation remains a good solution, even if two graphics cards on x8 PCI Express 2.0 connects tax the PCIe bandwidth in CrossFire mode. Any motherboard that comes with added USB 3.0 and SATA 3.0 hardware should be running a PCI Express switch to administer the available bandwidth efficiently.
Recommendations
Bandwidth limitations on Intel's side will require a chipset refresh. It's possible to manage existing bandwidth more efficiently through PCI Express switching, though, which leads us to our main recommendation. If you're truly concerned with buying a board now that includes USB 3.0 and and SATA 6Gb/s support, it makes sense to purchase a higher-end platform in order to make sure you get a configuration with switching capabilities, at least (assuming, of course, that you're in the market for an H55-, H57-, or P55-based setup). Should you instead opt for an X58-based machine, these potential bottlenecks won't be an issue. Of course, that's a viable option as well and, frankly, more attractive to performance enthusiasts.
Though the promise of SATA 6Gb/s is exciting for enthusiasts eager to adopt early, the performance benefits are still limited, and we think it'd be best to wait. The market simply lacks drives faster than 300 MB/s.
With USB 3.0, though, the situation is different. Even if you plug a USB 2.0 x1 PCIe add-on card into a PCIe 1.1 slot, you'll still receive up to 250 MB/s of bandwidth, or a minimum of about 160 MB/s, as you can see in our benchmark results. Compared to USB 2.0 performance (at an effective 30-35 MB/s), this is already well worth the upgrade. ***
Views: 3730 
