Skip to main content

Object Storage Workload (will become obsolete)

Note

The Object Storage Workload is an obsolete general purpose workload model for Object Storage protocols that include OpenStack Swift and Amazon S3. This workload was built before the new dedicated S3 workload described above, and it contains less functionality and flexibility.

The behavior is dependent on the metadata operations, block sizes, and other aspects, which are negotiated by the client and the object storage server. Unlike file or block workloads, the object enables you to select the protocol within the workload, or, if simple configuration is selected, it defaults to the protocol in the test bed against which it is run. As with file workloads, metadata commands differ between protocols.

WorkloadWisdom provides out of the box models for Amazon S3 and OpenStack-Swift in a single workload: Object Storage Workload. SNIA CDMI is supported by running TDE tests on WorkloadWisdom.

Access Patterns

You can specify access patterns in the following ways:

  • Simple configuration

  • Common configuration

  • Amazon S3 commands distribution

  • OpenStack-Swift commands distribution

Simple and common configuration automatically map object access concepts to the appropriate protocol command based on the service specified in the test bed. This enables a single workload to be run to compare Amazon S3 versus OpenStack-Swift or against a mixed environment.

Amazon S3 commands distribution and OpenStack-Swift commands distribution provide direct control over the protocol commands that are sent, but can only be used on test bed services for the selected protocol.

Simple Configuration

If you use the simple configuration, the protocol that runs is automatically determined by service configured in the test bed.

2019-11-14_17-36-09.png

To change the ratio, move the slider to reflect the desired percentage value. If you move the slide to the left for the read/write percentage the read percentage increases.

Simple configuration is the default. When you use the simple configuration option, you only need to specify read/write ratio and data/metadata ratio, but you cannot control the distribution of different metadata commands. Simple configuration will evenly distribute all available commands.

For example, assume you use simple configuration and have the following settings:

Read: 80%

Write: 20%

Data: 40%

Metadata: 60%

The resulting workload will have 32% read operations, 8% write operations, and 60% metadata operations, depending on whether you run this workload on an Amazon S3 test bed or an OpenStack Swift test bed, or use different HTTP methods or APIs. For example, if you run this on an Amazon S3 test bed, the supported “write” operations for Amazon S3 can be PUT NEW, PUT REPLACE, or POST APIs, so the 8% write operations are further divided evenly across the three methods (i.e., 2.67% PUT NEW, 2.67% PUT REPLACE, and 2.67% POST). Similarly, for the metadata operations, there are seven supported APIs, so the 60% of metadata operations are evenly divided by seven (i.e. 8.57% DELETE OBJECT, 8.57% HEAD BUCKET, 8.57% GET BUCKET ACL, etc.). However, in this specific example there are more DELETE OBJECT APIs than PUT NEW APIs which would result in erroneous conditions as there are not enough Objects to be deleted, so the percentage of DELETE OBJECT APIs is updated to match the percentage of PUT NEW APIs (i.e. 2.67% in this specific example).

The checkbox Create a dedicated object for each HEAD and PUT ACL operation, for each concurrent worker is applicable only to Amazon S3 test beds. If you select the checkbox, then WorkloadWisdom creates one additional temporary object dedicated for each HEAD operation for each concurrent worker, and one additional temporary object dedicated for each PUT ACL operation for each concurrent worker.

The effect of selecting this option is that you create many more Objects on the system under test than you configured in the Object Storage System section of this workload test, but allowing these different operations to be performed on different objects. The effect of not selecting this option is that you will not end up creating many more Objects on the system under test than what is configured (except for the temporary Objects that must be created for DELETE operations), but some Object systems under test might not be able to process HEAD or PUT ACL operations on the same object while that same object is receiving a PUT or GET operation.

Common Commands Configuration

Use Common commands configuration for more realistic control over the metadata traffic than simple configuration, while still enabling the protocol comparison and mixed protocol test bed use cases. This configuration is not used frequently unless you have a use case to compare how Amazon S3 and OpenStack Swift perform against each other when “similar comparable” Data and Metadata operations are used.

2019-11-14_17-37-11.png

The following table displays how commands are mapped to protocols when applied to a test bed with the specified protocol.

Table 43. Mapping of Common commands to Amazon S3 APIs and OpenStack Swift APIs

Access

S3

OpenStack-Swift

Read Object

Get

Retrieve Object

Write New

Put New

Write Object New

Write Replace

Put Replace

Write Object Replace

Delete Object

Delete

Delete Object

Get Object Metadata

Head Object and Get Object ACL

Retrieve Object Metadata

Update Object Metadata

Put Object ACL

Update Object Metadata

Get Container Info

Get Bucket ACL

Retrieve Container

Update Container Info

Put Bucket ACL

Write Container



Write New and Write Replace do not indicate different protocol commands but rather whether the object being written is one that was created during the pretest phase (replace) or a new object name. If Write New is greater than Delete Object, the amount of storage needed grows with each test run and is not be cleaned up at the end of the test. Write Replace might also result in increased storage depending on versioning configurations of the storage system.

Note

Using Get Object Metadata has more overhead on Amazon S3 than on OpenStack-Swift because two operations are performed instead of one. Depending on the workload this might not be what would happen in a real workload and is a worst-case scenario use case for Amazon S3.

Data Parameters

Object protocol workloads do not support the deduplication feature of data reduction.

Amazon S3 Commands Distribution

When using the Amazon S3 commands distribution configuration, the commands (those with at least 1%) are issued the same way as the simple configuration when applied on an Amazon S3 Test Bed. The only difference is the percentage of commands issued.

The donut chart displays the overall data (read/write) versus the metadata ratio. Gray is displayed if there is unused capacity.

2019-11-14_17-38-52.png

OpenStack-Swift Commands Distribution

When using the OpenStack-Swift commands distribution configuration, the commands (those with at least 1%) are issued the same way as the simple configuration when applied on an OpenStack Swift test bed. The only difference is the percentage of commands issued.

The donut chart displays the overall data (read/write) versus the metadata ratio. Gray is displayed if there is unused capacity.

2019-11-14_17-39-28.png

Commmand Reference

The access pattern specified can be mapped to those specified by: http://developer.openstack.org/api-ref-objectstorage-v1.html

OpenStack-Swift uses the keystone authentication service within OpenStack and this can be configured in the OpenStack-Swift service in the test bed.

Table 44. OpenStack Swift Command Mapping

Virtana Label

OpenStack Reference

Data

RETRIEVE OBJECT

Objects Get

WRITE OBJECT

Objects Put

Metadata

COPY OBJECT

Objects Copy

DELETE OBJECT

Objects Delete

RETRIEVE OBJECT METADATA

Objects Head

UPDATE OBJECT METADATA

Objects Post Metadata

DELETE ACCOUNT

Accounts Post to Delete account metadata

RETRIEVE ACCOUNT

Accounts Head

WRITE ACCOUNT

Accounts Post with updates

DELET CONTAINER

Containers Delete

RETRIEVE CONTAINER

Containers Head

WRITE CONTAINER

Containers Post with metadata