We had an older version of the NIC driver installed on these servers - basically whatever had shipped in the box. The later revisions of the Broadcom NIC drivers should fully support the TCP Chimney Offloading feature.
Microsoft introduced Update for Windows Server 2003 (KB948496), which disables the Microsoft Scalable Network features that were installed as either the Windows Server 2003 Scalable Networking Pack or as part of Windows Server 2003 SP2. The KB article has more information.
The configuration, specifically enabling the TCP Chimney Offloading feature, gave us no end of grief with practically every application we had recently deployed on Windows Server SP2. Turns out, the Broadcom Ethernet Adapters that ship as part of our Dell servers weren't on the VIP list for supporting TCP Chimney Offloading. Why enabling this feature as the default setting as part of SP2 was ever greenlighted one can only guess.
We wound up manually making several registry entries under the HKLM\CurrentControlSet\Services\Tcpip\Parameters branch to manually disable the features.
Yes. We did test, and we briefly had connectivity issues that were attributed to SynAttackProtect key. After making that change in our test environment we never had connectivity issues again - until production. The interesting thing was that it started with one application, then moved to another, and yet another. I can only attribute this to go-live procedures as we slowly ramped up capacity of each system to 100%. Under the production loads, things would often fail, repeatedly, and in somewhat spectacular ways I might add (in hindsight). After making the manual changes, things settled down into expected behavior.
The configuration you should LIKELY be running in is to have it disabled, unless you happen to have a network adapter that supports the feature. Good luck with that.