This Comparative Report is based on Mindcraft's Certified Performance Reports for Compaq's ProLiant 5000 and Sun's Ultra Enterprise 2 Model 2170, each running Netscape's Enterprise Server 2.0. Tests were run using the WebStone 2.0.1 test suite. Our test methodology is summarized in this Comparative Report and is provided in detail in the Certified Performance Report for each system.

This Comparative Performance Report was commissioned by Compaq Computer Corporation.

Peak Performance

The peak performance reported below is for comparable configurations of a ProLiant 5000 and an Ultra Enterprise 2.

Analysis

The Compaq ProLiant 5000 configurations tested outperformed the corresponding Sun Ultra Enterprise 2 configurations for all the workloads we ran. The Compaq configurations had a performance advantage of 300 to 500 per cent for the static HTML tests, and 40 to 70 per cent for the dynamic content (NSAPI and CGI) tests. Detailed benchmark results are available for both the ProLiant 5000 and the Ultra Enterprise 2.

100% Static HTML Workload

The WebStone 2.0.1 100% static HTML workload challenges a server system's ability to manage large numbers of active network connections. Since the Web server software is able to cache the relatively small set of files in the standard WebStone file set, the server operates out of memory rather than accessing files on disk. Thus the Web server is able to respond quickly to requests and to fill up the network queues with the files requested. WebStone adds load by increasing the number of clients, so communications performance is crucial and disk performance, for example, has less influence on the results.

The Compaq Proliant 5000 had 97% to 100% CPU utilization when performing the tests that generated the peak performance for a 100% HTML load. However, the Sun Ultra Enterprise 2 had only 55% to 60% CPU utilization at its peak performance. We were unable to drive the system to 100% CPU utilization with 600 clients, and only a small fraction of the available network bandwidth was used.

The maximum throughput observed in our testing of the ProLiant 5000 was 136.53 Mbits/s, which is close to the maximum that can be expected for two 100 Mbit/s half-duplex networks. Network bandwidth and associated operating system overhead were probably the limiting factor in our HTML testing on the ProLiant 5000. While the peak performance numbers were similar for the uniprocessor and the dual processor configurations of the ProLiant 5000, the performance for the uniprocessor builds up with increasing demand while the dual processor configuration saturates the network interfaces even with low numbers of WebStone clients. In the real world, this should mean that a multiprocessor system would provide better response under moderate load conditions.

We believe that the 100% HTML Web server performance we report here is limited by each computer's networking efficiency.

100% NSAPI Workload

The nsapi-send.c module supplied with Webstone 2.0.1 fills a buffer with pseudorandom data and passes the data back to the requesting program. The code uses a significant amount of processing power running the rand() function. This means that the NSAPI workload tells us about CPU speed as well as about the performance of the API. All configurations of both systems ran at 98% to 100% CPU utilization for these tests. For both systems the two-processor performance is significantly greater than the uniprocessor performance.

100% CGI Workload

The cgi-send.c module performs the same actions that the nsapi-send.c module does, and in addition allocates the memory it uses. There is also the process creation overhead necessary to execute a stand-alone program, over and above what is required for a linked-in NSAPI module. All configurations of both systems ran at 95% to 100% CPU utilization for these tests.

These numbers came in at about 25% to 30% of the NSAPI performance for all configurations.

Test Procedures

Mindcraft followed the standard WebStone 2.0.1 run rules with the following extensions:

• In addition to the standard run from 10 to 100 clients, we also ran tests for 200 to 600 clients in steps of 100 to show the capabilities of these systems.
• We modified the runbench and webmaster.c files to support clients running on Windows NT systems.

Code changes are shown in Appendix 1 of the ProLiant 5000 and the Ultra Enterprise 2 Certified Performance Reports.

The following basic set of procedures was used for performing these tests:

• The server was started automatically at boot time.
• The server log files were deleted after each run from 10 to 100 clients and after each run from 200 to 600 clients.
• All test runs were done with the server console idle (no user logged in).

Configurations of the Tested Systems

Every effort was made to provide comparable hardware and software configurations for the two systems. The configurations tested were as follows:

Mindcraft tried to obtain Sun's Solaris Internet Server Supplement (SISS) which was announced while our testing was under way. Sun declined to provide the SISS to Mindcraft in time to be used for these tests.

The ProLiant 5000 actually had four processors in it. The number of processors used and the uniprocessor versus multiprocessor kernel were selected by software at boot time. For the Ultra Enterprise 2 , we unplugged one processor and half the memory to do the single processor tests.

The tests described in this report were performed on isolated LANs that were quiescent except for the test traffic.