Storage Testing
Change in a production storage environment is inevitable. Change can be planned, such as technology change, vendor replacement, firmware upgrade, or it can be unexpected, for example, application change, user behavior change, or outages.
When a change occurs, there is a potential for performance degradation, as the performance of a storage infrastructure is highly dependent on the characteristics of the storage workloads running on it. The most reliable way to de-risk these changes is to measure the performance impact of these changes against your own, unique, workloads. Different organizations have different workload characteristics, especially in a shared SAN/NAS environment where multiple applications coexist.
To perform Storage Testing correctly for your own environment, we recommend the Best Practices for Evaluating Storage Using Workload Testing and Validation located athttp://www.virtualinstruments.com/cnt/uploads/2017/06/WP_LDX_HowTest.pdf.
The methodology can be briefly summarized into the following workflow:
Acquire your own production workload data. When possible, use your own production workload data as the starting point. No other organization’s storage workloads are exactly identical to yours. To acquire your own production workload data, you can either export the data that is recorded by the storage array or 3rd party monitoring tools into a .csv file, or use data that is collected from IO Probes.
Create a realistic workload test. Use your production workload data, import it into Storage Load Testing (SLT) using the Workload Data Importer[1]. The Workload Data Importer analyzes and creates a realistic baseline workload test based on your very own production workload data. Once you have the baseline workload test created, you can scale it up for future proofing, or modify other aspects of the workload test such as data content and many others.
Configure the target storage. Configure and optimize the storage array according to the vendor’s best practices. In addition to configuring the storage array itself, the network between the simulated initiators/clients and the storage also needs to be considered a part of the storage test environment, as it is a mandatory part of every production storage infrastructure.
Pre-condition the array. Pre-condition the SAN storage arrays prior to running workload Tests, especially for all flash arrays. Flash arrays need to be “broken in” to bring them into the state that they would be in production. Otherwise, the performance results are not realistic.
Run workload tests. Run the baseline workload test to make sure the test results are as expected. Once the baseline results are confirmed, perform subsequent workload tests by scaling up the load or modifying other parameters, such as block sizes and deduplication ratio. Carefully track the configuration and test results from each workload Test Run.
Analyze workload test results. With all workload Test Runs completed and their results identified, create a Report that compares the KPIs from each Test Run and use the data as one of input for change confirmation.
[1] While SLT makes the task of analyzing your own production workloads as easy as possible, it still requires certain domain expertise and knowledge. If it is your first time performing this task, it is strongly recommended that you consult Virtual Instruments Services team.