S3 Workload

The Amazon S3 constant protocol workload is designed to simulate a variety of applications that use the Amazon S3 APIs. The following Amazon S3 functionalities are supported:

  • Object “Read / Write” operations: PUT, POST, GET

  • Metadata operations for Buckets and Objects

  • Object Versioning

  • Lifecycle

  • Bucket / Object ACL

  • Bucket Policy

In addition, the S3 Workload also supports the ability to create and delete temporary short-lived objects during a workload run (to simulate applications that can generate a large number of objects for temporary use), and is more flexible in working with existing Buckets.

Access Pattern

Two options are available for Access Pattern configuration: simple configuration and command distribution.

2019-11-20_16-14-32.png

Simple configuration

Under simple configuration, you can specify the Read (GET) and Write (PUT, POST) distribution, and the overall Data versus Metadata operations. The Read / Write setting defines the Data operations. In the example above, the resulting workload will have 41.5% Read (GET), 41.5% Write (PUT), and 17% Metadata which is made up of other supported Amazon S3 commands that are not considered Data operations.

If you select the checkbox Create a dedicated object for each HEAD and PUT ACL operation, for each concurrent worker, then WorkloadWisdom creates one additional temporary object dedicated for each HEAD operation for each concurrent worker, and one additional temporary object dedicated for each PUT ACL operation for each concurrent worker. The effect of selecting this option is that you create many more Objects on the system under test than you configured in the Object Storage System section of this workload test, but allowing these different operations to be performed on different objects.

The effect of not selecting this option is that you will not end up creating many more Objects on the system under test than what is configured (except for the temporary Objects that must be created for DELETE operations, and additional short-lived temporary Objects if you enable the option Create and Delete temporary Buckets and Objects), but some Object systems under test might not be able to process HEAD or PUT ACL operations on the same object while that same object is receiving a PUT or GET operation.

Commands distribution

Under Commands distribution, you can individually control the percentage of each supported command. Below are the available commands:

  • Data: These are “read / write” commands that transfer payload data to or from Objects in the data store.

    • PUT: S3 PUT Object API

    • POST: S3 POST Object API

    • GET: S3 GET Object API

Retrieve older versions option: If set, then the specified percentage of GET commands will first issue a GET Bucket Object Versions command to retrieve the Version IDs of an Object, and then issue a GET Object with the most recent non-current (i.e. IsLatest = false) version of the Object.

  • Metadata: These are commands that do not transfer payload data to or from Objects / Buckets in the data store. Unless specified otherwise, these commands do not include any specific Version ID in the requests.

    • HEAD Object

    • GET Object ACL

    • PUT Object ACL: Uses Canned ACL. Sets x-amz-acl to public-read-write by default.

    • HEAD Bucket

    • GET Bucket ACL

    • PUT Bucket ACL: Uses Canned ACL. Sets x-amz-acl to public-read-write by default. If Number of Unique ACLs is greater than 1, then the x-amz-acl field cycles through public-read-write, public-read, authenticated-read, bucket-owner-full-control, private.

    • GET Bucket Policy

    • PUT Bucket Policy: Sets Permissions for Object Operations to true by default, and others to false. If Number of Unique Policies is greater than 1, then some of the Object Operations will be set to false to give some variation to the Policies set.

    • GET Bucket Lifecycle

    • PUT Bucket Lifecycle: Sets a rule to transition Objects to Glacier (after 3,650 days). If Number of Unique Lifecycle Rules is greater than 1, then the aging and storage class transition rules vary to give some variation to the Lifecycle Rules set.

    • GET Bucket Versioning

Object Storage System

Buckets

You can either use an existing Bucket to create the Objects needed for your workload, or create new Buckets for the Objects.

2019-11-14_17-30-24.png

To create new Buckets for the Objects, select Create new in the drop-down menu, and specify the number of Buckets that should be created per user account (i.e. per S3 credential in the Test Bed). To use existing Buckets, select Use existing, and specify the Bucket names that exist on your Object storage system to hold the Objects.

Maximum number of existing Buckets: 100

If you choose to create new Buckets to use, then you can choose auto generated or user defined, which gives you the ability to specify the Bucket names’ prefix and postfix. This is helpful later on when you need to delete Buckets and Objects in bulk.

2019-11-14_17-31-09.png

If you select the checkbox Use unique bucket names, then a unique ID will be inserted before the postfix. The unique ID will be generated per user account per test.

Optionally, you can Enable Versioning on the Buckets that you create. This option is not available if you select to Use existing Buckets.

Objects

The options available for creating new Objects are similar to those for creating new Buckets. In addition, you need to specify the Object sizes.

2019-11-14_17-31-54.png

As with file workloads, you can specify a constant object size or create your own distribution of object sizes with custom bins. To change the size distribution method, select the option from the drop-down menu.

Temporary Content

To generate short-lived Objects that are created and deleted throughout a workload run, click on the drop-down menu and change from Don’t create temporary Buckets and Objects to Create and delete temporary Buckets and Objects. Use caution if you have a high number of workers, because the settings from this section are applied to each worker, which can result in a much larger number of Buckets and Objects on the System Under Test than anticipated.

2019-11-14_17-32-40.png

You can control these numbers by count or by percentage, and the set values are applied to each worker. When you use percentage, it is a percentage of the total number of Objects defined in the Object Storage System section, except for Delete. Delete is a percentage of the temporary Objects created. The size of these Objects is the same as those configured in the Object Storage System section.

The location of the temporary Objects depends on your settings from the Object Storage System section as well. If you selected Create new Buckets, then each worker will create a new dedicated Bucket will be used to house the temporary Objects. If you selected Use existing Buckets, then a random existing Bucket will be used to house the temporary Objects. You can also specify the distribution of the operations performed on the temporary Objects.

For the temporary Objects created, you can specify the approximate life time for them. When the life time of the objects is up, the workload will begin deleting the temporary Objects in pseudo-random order.

Depending on many factors including workload settings and how the System Under Test responds throughout the test run, the actual observed life of the temporary Objects may differ from the configured value, and that some temporary Objects may not be completely deleted during the test run. To give an extreme example to illustrate this point, if you configure 1,000,000 temporary Objects with an average life of 5 seconds for the temporary Objects, and set the workload Duration to 5 minutes, and that the System Under Test has an average response time of 100ms, then the workload simply cannot run as expected.

A general recommendation is that the Approximate life of temporary Objects should be several times greater than Average Response Time x Temporary objects to create and delete, and set the workload run Duration to several times greater than the life time value as the workload performs operations on both the temporary Objects as well as the Objects that exist on the Object Storage System from the Pre-Test run.

Response Handling

By default, 400-level and 500-level HTTP Responses and considered errors, and a worker encountering these errors will terminate its workload. You can optionally select the checkbox Ignore errors gracefully to force the workload to continue to run when it receives 400-level and 500-level HTTP Responses.

2019-11-14_17-33-38.png

In addition, for 503 HTTP Responses, you can specify that the workload performs a Retry of the HTTP Request that resulted in the 503 HTTP Response. If this is set, then whenever a worker receives a 503 Response to a Request, it performs a retry after Retry Delay, until the number of retries exceeds Maximum Retries.

Pre-test parameters

The S3 Pre-Test populates the Objects, and optionally Buckets (depending on your workload settings), that are required for your workload to run. When you run an S3 workload on your SUT for the first time, you must run the pre-test at least once. Depending on your workload settings, subsequent runs may or may not require a pre-test run. For example, if you apply the setting Use unique object names, then you need to run pre-test because a unique identifier is created at run time and is used as part of the Object name.

The Max concurrent workers setting allows you to speed up the Object System population time by using multiple workers to create the configured Buckets and Objects simultaneously. It is important to note that all S3 credentials defined in the Test Bed can be used, so if you have strict access to the S3 SUT, then it is best to either reconfigure the SUT to allow any S3 credential to create new Buckets, or use a Test Bed with a single S3 credential that can create new Buckets, or just use an existing Bucket and create only new Objects (not create new Buckets).

2019-11-14_17-34-06.png

For example, if you have 1 S3 credential defined in the Test Bed, and set the pre-test concurrent workers to 4, then all 4 workers will use the same S3 credential during pre-test. If you have 8 S3 credentials defined in the Test Bed, and set the pre-test concurrent workers to 4, then, all else being equal, each worker will use 2 S3 credentials throughout the pre-test.

More information

For more information on designs and behaviors of the S3 workload that are not exclusively specific to the S3 protocol, see the Workload Tests Concepts section as it relates to the Constant Protocol Workloads section.