Mindcraft^® developed the AuthMark™ Benchmark to test authentication and authorization performance for Web access control products. The AuthMark Benchmark has two test scenarios that we commonly use:

• The Login Scenario simulates users requesting and receiving the first Web page at a protected Web site.
• The Extranet Scenario simulates customers or suppliers logging into a private Web site and obtaining information they are authorized to get.

The AuthMark Benchmark is run using our iLOAD MVP™ tool.

iLOAD MVP Overview

iLOAD MVP is a general-purpose, script-driven capacity planning, benchmarking, and regression testing tool. The major components of iLOAD MVP are:

• A Control Center that manages load generator systems, controls test script execution, and reports on test results.
• Multithreaded client load generators that execute test scripts to simulate users accessing a server.
• Test script generation programs.
• Test data generation programs.

iLOAD MVP provides the capabilities needed to test high-performance servers with a small number of load generator systems. Its capabilities include:

• The ability to simulate a large number of simultaneous user sessions. The number of user sessions is limited only by the load generator OS, the amount of memory and the performance of the load generator systems.
• Support for HTTP 1.0 and 1.1.
• Support for authentication and authorization.
• Support for SSL.
• Custom test scripts.

The AuthMark Benchmark

The AuthMark Benchmark is designed to test the performance of products that provide authentication and authorization services in support of Web servers. Authentication is the process of verifying who a user is; it typically occurs when a user logs in. Authorization is the process of verifying that an authenticated user is allowed to see or to use a particular resource. In the case of a Web server such resources include HTML files, graphic files, and programs that generate Web pages dynamically.

AuthMark simulates a large number of users accessing Web servers via their browsers. This approach permits AuthMark to test authentication and authorization performance independent of the technology used to provide those services.

AuthMark consists of several test scenarios to determine various aspects of performance for authentication and authorization systems under different circumstances. The AuthMark Login and Extranet Scenarios are the ones most commonly used.

AuthMark Login Scenario

The AuthMark Login Scenario focuses on testing authentication. It simulates users requesting and receiving the first Web page at a protected Web site. The Login Scenario measures the combination of one user authentication and one authorization for access to a protected resource (called a Login). We report Logins/minute. Understanding what happens during a login will help you understand what the Login Scenario measurements mean.

Login Process

The following simplified sequence will walk you through the login process to show you how it works:

1. The iLOAD load generator (a thread of a script-driven program that emulates a web browser) sends a request to the Web server for a protected resource.
2. The Login Scenario supports both the standard HTTP authentication protocol and form-based authentication. The authentication process varies as follows:
   1. If the HTTP authentication protocol is used:
      1. The Web server sends back a "401" HTTP response indicating that the iLOAD load generator has not yet been authenticated.
      2. The iLOAD load generator then resends a request for the same protected resource but this time it includes an HTTP authorization header containing its user name and password.
   2. If form-based authentication is used, the process may vary by product but will be similar to the following:
      1. The Web server returns the login form typically requesting username and password as the login credentials.
      2. Using an HTTP POST operation, the load generator returns the login credentials to the Web server, which forwards them to the Product Under Test.
3. The Product Under Test checks the credentials against those in the user data repository (typically an LDAP directory or a database) to validate the user name and password. If authenticated, authorization is checked in the next step. Otherwise, an an authentication error is returned returned.
4. The Product Under Test checks if the user is authorized to access the requested resource. If so, the resource, which in this case is one of the 14 KB Web pages, is returned to the load generator along, typically with an encrypted session cookie.

Login Scenario Configuration

Table 1 shows the AuthMark Login Scenario configuration parameters we use.

Table 1: AuthMark Login Scenario Configuration Parameters

The number of user sessions active during a given test run is determined by the length of the test and the number of logins. Sessions are not logged out once created. Instead, each session remains quiescent after login. 

Running the Login Scenario

The basic steps for running the Login Scenario are:

1. Generate the data to fill the security directory. iLOAD MVP provides a tool to generate realistic data for the LDAP V3 organizationalPerson object class and Netscape's inetOrgPerson object class. It also includes tools to load the same data into an LDAP directory.
2. Load the security directory with the user data.
3. Generate the test scripts for the Login Scenario. iLOAD MVP provides a tool to do this. These scripts drive iLOAD MVP to simulate user interaction with the Web servers.
4. Load Web pages on the Web servers. There are 100 Web pages each of which is 14 KB in size for the Login Scenario.
5. Load and configure the authentication/authorization system.
6. Run the benchmark.

The Login Scenario test script selects users randomly from the user database (see Table 1 for the numbers we use). The tester is free to select the number of load generator systems and the number of iLOAD MVP load generator threads to use.

The tester selects the number of load generators to get the highest performance possible from the authentication/authorization product being tested. In order to obtain the peak performance from an authentication/authorization product, the tester may need to use multiple Web servers and data repositories (directory or database servers).

The tester is permitted, but not required, to do a warm-up run of the test scenario in order to get the servers to a state that would more likely represent the state they would be in during normal operation. We typically warm-up the servers by running the test script in its entirety.

Extranet Scenario

The Extranet Scenario simulates customers or suppliers logging into a private Web site and obtaining information they are authorized to get. It measures the combination of one user authentication and 10 authorizations for access to resources (these 11 Extranet operations constitute one Extranet sequence). We report the total operations per minute. The Extranet Scenario depicts a more complete and more realistic usage pattern than the Login Scenario.

The Extranet Scenario test execution starts with the same operation sequence as the Login Scenario (steps 1 - 4 above) and continues with the following operations:

5. A load generator requests a resource, sending the encrypted session cookie along with the request, if one was returned during the authentication.
6. The Product Under Test checks the validity of the user and that the user is authorized to have access to the resource.
7. If the user is authorized, the resource is returned.
8. The load generator then returns to Step 6 eight more times, for a total of 10 authorizations.

For the Extranet Scenario, we typically warm-up the servers by running the test script in its entirety.

If you would like more information about the Mindcraft AuthMark Benchmark, please contact us.