Our tests demonstrate that SiteMinder is a high-performance, scalable user management system capable of satisfying the performance needs of the most demanding Web sites.

Performance Analysis

We suggest that you review the details of the AuthMark Login Scenario Benchmark specified below in order to have the context to understand the discussion in this section.

Login Performance Analysis

The performance measurements shown in Figure 1 are for the warmed-up Login Scenario tests. These tests show SiteMinder's performance on three different server configurations: a single server with one CPU, a single server with two CPUs, and two servers with two CPUs. Table 1 show the performance scaling we measured for SiteMinder.

Table 1 : SiteMinder Performance Scaling

Server Configuration	Logins/Minute	Performance Scaling
1 Server, 1 CPU	6,149	N/A
1 Server, 2 CPUs	10,444	1.7 times 1 CPU
2 Servers, 2 CPUs	21,687	2.1 times 1 server

The CPU utilization of the various servers will give you insight into how to configure an environment to get the maximum performance from SiteMinder. Table 2  shows the CPU utilization on each of the servers during the one server, 1 CPU test. It shows that the SiteMinder policy server is saturated while the other servers are lightly loaded. So, if you want to use a single SiteMinder policy server, use one with the fastest CPU available to you so that it will be able to handle the load you need.

Table 2: Server CPU Utilization for a Single CPU SiteMinder Policy Server

Server	CPU Utilization
SiteMinder Policy Server	100%
Web Server #1	11% on both CPUs
Web Server #2	11% on both CPUs
Directory Server	15% on all CPUs

Table 3 shows the CPU utilization on each of the servers during the one server, two-CPU SiteMinder Policy server test. It shows that the SiteMinder policy server is fully utilized as performance scales up 70% from a single CPU server. This level of scalability means that you can economically justify adding a second CPU to a SiteMinder policy server.

Table 3 : Server CPU Utilization for a Two-CPU SiteMinder Policy Server

Server	CPU Utilization
SiteMinder Policy Server	100% on both CPUs
Web Server #1	23% on both CPUs
Web Server #2	23% on both CPUs
Directory Server	25% on all CPUs

Table 4 shows the CPU utilization on each of the servers during the two server, two-CPU SiteMinder Policy server test. The SiteMinder policy server CPU utilization shows that SiteMinder is load balancing across both policy servers, although it is not sharing the load evenly. However, the 110% performance increase over a single dual-CPU policy server shows that SiteMinder is able to make better use of the more heavily loaded system when it is load balancing with another server. Thus, if you need the highest possible authentication performance at your Web site, use multiple, load-balanced SiteMinder policy servers.

Table 4 : Server CPU Utilization for Two dual-CPU SiteMinder Policy Servers

Login Performance Conclusions

SiteMinder's performance for logins (authentications) scales extremely well as you increase the speed of the server CPU, add a CPU to the server, and add an additional server in a load-balancing configuration. The SiteMinder policy server configuration options let you balance the authentication server cost with the performance you need. 

Benchmark

The AuthMark Benchmark is designed to test the performance of products that provide authentication and authorization services in support of Web servers. Authentication is the process of verifying who a users is. Authorization is the process of verifying that an authenticated user is allowed to see or to use a particular resource; in the case of a Web server such resources include HTML files, graphic files, and programs that generate Web pages dynamically.

AuthMark simulates a large number of users accessing a Web server via their browsers. This approach permits AuthMark to test authentication and authorization performance independent of the technology used to provide those services.

AuthMark Login Scenario

The AuthMark Login Scenario focuses on testing authentication. We call it the Login Scenario because authentication is done the first time a user accesses a protected part of a Web site, just like a login. The HTTP 1.0 and 1.1 protocols define the steps a browser follows for authentication. Some of the steps are visible level to you and others are not. It is important to understand what happens during an authorization in order to understand what the Login Scenario measurements mean.

Authorization Process

The following simplified sequence will walk you through the authorization process and show you how what you see ties in with the authorization process:

1. When you click on a link or enter a URL in your browser your browser sends the requested URL to the Web server.
2. The Web server determines that you must be authenticated before it returns the resource at the requested URL. Typically, the authentication requirement is specified as part of the Web server's configuration or via an authentication/authorization product embedded in the Web server.
3. The Web server sends back a "401" HTTP response to your browser indicating that you are not authorized to see that requested resource (generally a Web page).
4. Your browser pops open a window and asks you to enter your user ID and a password.
5. After you enter your user ID and password, your browser stores them in memory and associates them with the protected space (called a realm ) containing the URL you requested.
6. Your browser then resends a request for the same URL but this time it includes an HTTP authorization header containing your user ID and password.
7. This time the Web server checks your user ID and password to see if they match the authentication information in the authentication system. If they do, you are authenticated.
8. Now that you have been authenticated, the authorization system checks whether or not you are authorized to access Web pages in the realm. If you are authorized, the Web server sends the Web page you requested.

Notice that the URL you clicked on or entered is actually sent twice (in steps 1 and 6). This means that the authentication system is used twice—first, it finds out that the requested URL requires the user be authenticated, then it processes the authorization header when the request is resent.

After the user is authenticated, the request is checked to see if the user is authorized to use the requested resource. The authorization check does not require any contact with the Web browser.

Once a user has been authenticated, the Web browser automatically sends the authorization header whenever the user requests a URL in a realm requiring authentication. So, the user will not see a window open up asking him or her to login again to a realm that has already been visited.

Login Scenario Parameters

The Login Scenario is based on the iLOAD MVP configuration parameter values in Table 5.

Table 5: AuthMark Login Scenario Configuration Parameters

Parameter	Value
Number of users in security directory/data base	1,000,000
Number of organizationalUnits or security groups	10
Total number of user sessions	100,000
Number of HTTP GETs per user session	1 (authentication actually causes 2 GETs to be issued)

Running the Login Scenario

The basic steps for running the Login Scenario are:

1. Generate the data to fill the security directory/database. iLOAD MVP provides a tool to generate realistic data for the LDAP V3 orgalizationalPerson schema and Netscape's inetOrgPerson schema.
2. Load the security directory/database with the user data.
3. Generate the test scripts for the Login Scenario. iLOAD MVP provides a tool to do this. These scripts drive iLOAD MVP to simulate user interaction with the Web server(s).
4. Load Web pages on the Web server(s). There are 100 Web pages each of which is 14 KB in size for the Login Scenario.
5. Load and configure the user management system or authentication/authorization system.
6. Run the benchmark.

The Login Scenario test script uses 100,000 users randomly selected from the directory/database of 1,000,000 users. The tester is free to select the number of client test systems and the number of iLOAD MVP client threads to use. These are called the load generators .

The tester selects the number of load generators to get the highest performance possible from the authentication/authorization system being tested. In order to obtain the peak performance from an authentication/authorization system, the tester may need to use multiple Web servers and directory/database servers.

The tester is permitted, but not required, to do a warm-up run of the test scenario in order to get the servers to a state that would more likely represent the state they would be in during normal operation. The test report must state whether the servers were warmed-up or not.

Products Tested

Server Configurations

Mindcraft used Sun Enterprise 450 servers for the Web servers, SiteMinder policy servers and the LDAP directory servers. Table 6 shows the server configurations we used.

Table 6: Sun Enterprise 450 Configurations

Hardware Configuration

• Used the Solaris psradm command to disable/enable processors to test uniprocessor and dual-processor configurations

Software Configuration

Solaris 7 Configuration

• Default configuration for the hardware configurations used except for the following:
  
  • On the SiteMinder policy servers and the directory server:

    Used ndd to set tcp_time_wait_interval to 60 seconds

  • On the Web servers:

    Used ndd to set tcp_time_wait_interval to 60 seconds

SiteMinder Configuration

• Used SiteMinder Version 3.51
• The Web agent configuration file we used is as follows:

  # WebAgent.conf - configuration file for SiteMinder Web Agent
  # Web Agent Version=351
  # Web Agent InstallPath=
     /opt # Do not remove or edit the preceding lines,
  or   you may not # be able to perform
  
  upgradesin
  thefuture
  hostname="net08,10.10.2.8"
  enablewebagent="YES"
  enablecookies="YES"
  persistentcookies="NO"
  enableaccounting="NO"
  enablefailover="NO"
  policyserver="10.10.1.5,44441,44442,44443"
  ignoreext=".gif,.jpg,.png,.class,.jpeg"
  maxsessiontimeout="7200"
  idlesessiontimeout="3600"
  cachetimeout="6600"
  maxcachesize="200"
  maxsessioncachesize="0"
  requesttimeout="120000"
  cookiedomain=".nety.com"
  logenabled="YES"
  logappend="NO"
  logtrace="NO"
  logname="../logs/webagent"
  agentkey="9ada9707238f2494"
  sharedsecret="9ada9707238f2494"
  disableservsesstest="YES"
  

• Policy server configuration:

  • Policies were bound to each organizationalUnit.

  • Policy store cache was on. Policy information that is normally stored in LDAP or SQL database can be cached in RAM. Any admininistrative changes to the policy information automatically update policy store caches on all policy servers.

  • Authorization cache was on. This dynamic cache tracks the relationship between resources and policies.

  • Verbose logging was off.

  • Error logging was on.

Netscape Enterprise Server (Web Server) Configuration

• Used Enterprise Server 3.61
• Default configuration used

Netscape Directory Server Configuration

• Used Directory Server 4.0
• Used inetOrgPerson schema with all attribute indexes off except for the following:
  • dn: equality and wildcard
  • cn: equality (this attribute was used as a unique ID)

The Test Lab Configuration

Mindcraft ran these tests using four identically configured Dell Optiplex GX1 client test systems (six systems were available to be used). Table 7 shows the configuration of the client systems.

Table 7: Configuration of the Client Test Systems

Switches were used for all network connections. Figure 3 shows a diagram of the complete test lab. We did not use all of the systems in the diagram.