Implementing Flash Storage for SQL Server from Virident

Implementing SQL Server on
the Virident Drive

Earle F. Philhower, III

Table of Contents
Executive Summary................................................................................................................................... 1
FlashMAX Benefits .................................................................................................................................. 1
Implementation Requirements ................................................................................................................ 2
System....................................................................................................................................................... 2
Software ................................................................................................................................................... 2
General High-Performance Tunables ..................................................................................................... 2
Operating System Configurations ......................................................................................................... 2
Microsoft SQL Server Settings ............................................................................................................... 3
Virident FlashMAX Settings ................................................................................................................... 4
Placement of Databases on the Virident FlashMAX ............................................................................. 5
All Tables and Indexes on the FlashMAX .............................................................................................. 5
Individually Tiering Tables and Indexes ................................................................................................ 5
Placing Specific Indexes on FlashMAX ............................................................................................................ 6
Placing Specific Tables on FlashMAX............................................................................................................... 7
Placing Parts of Tables on FlashMAX .............................................................................................................. 8
Transaction Logs on FlashMAX............................................................................................................ 10
Tempdb on FlashMAX ........................................................................................................................... 11
Data Reliability ........................................................................................................................................ 12
Flash Management ................................................................................................................................ 12
RAID-1 Across Multiple Cards ............................................................................................................. 12
Data Availability ...................................................................................................................................... 13
SQL Server Mirroring ............................................................................................................................ 13
Transaction Log Shipping ...................................................................................................................... 14
Conclusion................................................................................................................................................ 14

2012, Virident Systems, Inc. All Rights Reserved.

Executive Summary
The Virident FlashMAX Drive delivers uncompromising performance to Microsoft SQL Server installations.
A single card provides up to 2.2 terabytes of database space in an industry-standard PCI Express half-height,
half-length form factor. For larger databases, up to eight cards can be employed for combined storage of
over 17TB.
FlashMAX’s incredibly high IOPS performance is a great match for OLTP-type applications, while its great
bandwidth allows it to replace racks of short-stroked disk drives in DSS-type workloads. FlashMAX can
increase the CPU core utilization on your database server, maximizing the investment in your hardware.
Best of all, FlashMAX is easy for the database administrator to implement. Utilizing standard optimization
techniques, FlashMAX can accelerate entire databases, specific indexes or tables, or even only portions of
tables without any user-level application changes.

FlashMAX Benefits
Easy to implement
 Utilizes Microsoft standard STORPORT architecture
 Allows administrators to use standard disk-management utilities, both Microsoft and third party
 Works in any industry-standard, server-class system
 Low memory usage
 More room for SQL Server buffers
High-speed DSS and OLTP workload support
 No need to choose one workload or another
No need to overprovision

 No short-stroking required for highest performance (unlike HDD arrays or other flash)
 Best utilization of storage investment
Reduced costs via consolidation
 Consolidation minimizes over-provisioning of storage and servers
 Predictable performance means meeting SLAs with fewer resources
New business intelligence
 Convert batch processing into interactive applications
 Make faster, more informed decisions

Page 1


Implementation Requirements
The Virident FlashMAX supports most industry-standard server installations, thanks to its small size
and adherence to Microsoft and industry standards.

System
Hardware

A single half-height, half-length PCI Express Gen 2 x8 slot in any server-class chassis
from 1-U to 4-U. No additional power is required for the card to operate.

Processor

Modern Intel Xeon or AMD Opteron dual- or quad-socket server. Hyper-threading has
been shown to be helpful in most workloads, but individual database administrators
should verify this.

Memory

8GB DRAM or more. The FlashMAX device driver requires approximately two gigabytes
for each terabyte of flash present in the system (i.e., a 2TB card requires around 4GB of
DRAM).

Software
Operating
System

Microsoft Windows Server 2008 R2 SP1, 64-bit edition

SQL Server

Microsoft SQL Server 2008 R2, SP1 or later. The latest service pack is required to ensure
that SQL server is able to deliver the highest performance on the Virident FlashMAX’s
4K physical sectors.

General High-Performance Tunables
All standard tunables within both Microsoft Windows and Microsoft SQL Server can apply with a Virident
FlashMAX.

Operating System Configurations
At the operating-system level, the most important tunable is the performance mode of the system. By
default servers are configured in a “Balanced” or “Power Savings” mode (via the Control Panel -> Power
Options panel). This mode trades off slower processor clock frequencies and longer latencies for a
reduction in server power.
When paired with a low-performing disk subsystem, this performance tradeoff makes sense because SQL
Server is often bottlenecked waiting for data from the slow disk. However, the Virident FlashMAX is able to
provide data to SQL Server at such a high rate that this IO waiting time is reduced almost to nothing. It thus
makes sense to use the “High Performance” setting to ensure the CPU is always running at the highest
speed.

Page 2


Microsoft SQL Server Settings
Several general settings and techniques are configurable from within Microsoft SQL Server to get the highest
performance possible with the Virident FlashMAX.
Multiple tempdb files should be used no matter where these files are stored, to avoid any bottlenecks on
tempdb allocations.
SQL Server should be given as much memory as practicable. With the lower memory requirements of the
Virident FlashMAX, it may be possible to increase the memory that is allocated to SQL Server.
Finally, because the FlashMAX performs best at the highest levels of IO parallelism, SQL Server should be
given a high setting for worker threads. The appropriate SP_CONFIGURE settings to achieve this are “max
worker threads” (set very high, in the range of 1,000 to 3,000) and “cost threshold for parallelism” (set to 0 to
encourage SQL server to parallelize query execution).
Command Example
Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.
PS C:UsersAdministrator> sqlcmd
1> -- Enable the advanced configuration options
2> sp_configure 'show advanced options', 1;
3> reconfigure with override
4> go
Configuration option 'show advanced options' changed from 1 to 1. Run the RECONFIGURE statement to install.
1> -- Make SQL server prefer parallelizing queries
2> sp_configure 'cost threshold for parallelism', 0
3> go
Configuration option 'cost threshold for parallelism' changed from 0 to 0. Run the RECONFIGURE statement to install.
1> sp_configure 'lightweight pooling', 1
2> go
Configuration option 'lightweight pooling' changed from 1 to 1. Run the RECONFIGURE statement to install.
1> sp_configure 'max worker threads', 3200
2> go
Configuration option 'max worker threads' changed from 3200 to 3200. Run the RECONFIGURE statement to install.
1> sp_configure 'priority boost', 1
2> go
Configuration option 'priority boost' changed from 1 to 1. Run the RECONFIGURE statement to install.
1> exit
PS C:UsersAdministrator>

Page 3


Optimization Note
It is often not necessary to change any SQL Server settings to see a significant performance impact with
the FlashMAX drive. It is often worthwhile to start benchmarking without any changes to SQL
configuration before individually tweaking specific settings.

Virident FlashMAX Settings
No special settings are required on the Virident FlashMAX card to achieve the highest performance.
However, to ensure that the card has been installed properly and the system properly configured, the
special “test.exe” utility provided by Virident should be run. This will allow any performance
configuration problems related to the card to be isolated and fixed before any SQL server tests are
performed.
Command Example
C:>test.exe
*** VIRIDENT PERFORMANCE TEST ***
(C) Copyright 2012 Virident Systems, Inc.
--------------------------------------------------------------------------Usage: TEST <drive letter>
ex: "test e"
C:>test.exe g
*** VIRIDENT PERFORMANCE TEST ***
(C) Copyright 2012 Virident Systems, Inc.
--------------------------------------------------------------------------Checking for administrator permissions...OK
Checking the Virident Driver is loaded...OK
Checking power settings...High performance mode not detected.
You may not get highest performance without this setting.
Would you like to enable this setting now, automatically? (y/n) y
Checking CPU configuration...Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz...OK
Checking number of CPU cores...32...OK
Creating 4GB test file...OK
--------------------------------------------------------------------------Running Max Read Bandwidth test... (20 seconds)
Max Read Bandwidth measured is 2554MB/s
Running Max Write Bandwidth test... (20 seconds)
Max Write Bandwidth measured is 1023MB/s
Running Max Read IOPS test... (20 seconds)
Max Read IOPS measured is 315K
Running Max Write IOPS test... (20 seconds)

Page 4


Max Write IOPS measured is 255K
Running Max 512b Read IOPS test... (20 seconds)
Max 512b Read IOPS measured is 761K

Placement of Databases on the Virident FlashMAX
There are two general implementation strategies for using the Virident FlashMAX to accelerate SQL
Server: either placing all tables and indexes on the drive, or selecting individual tables or indexes to place
on the drive, while still employing a slower and larger storage tier to hold the bulk of the database.

All Tables and Indexes on the FlashMAX
The fastest and most effortless way of implementing the Virident FlashMAX into a database application is
to place everything onto the FlashMAX drive. When the database is larger than the capacity of an
individual card, multiple cards (up to eight in a single server) can be installed and striped using standard
Windows Disk Manager tools.
If the entire database (tables, indexes, logs), including planned growth, fits in the server, this option is as
simple as possible for the database administrator and delivers the greatest performance gain possible.
Management and maintenance are no different in this case from that of any other direct-attached
database store.
Virident FlashMAX can reduce up-front investment by completely eliminating hardware such as batterybacked RAID cards and large, short-stroked disk arrays. Ongoing costs, such as power, rack space and
cooling, are also reduced the most in this configuration.
Optimization Note
One often overlooked benefit, beyond simply increasing transaction throughput with a FlashMAX drive,
is stability of performance. Thanks to its superior design, at any workload level the FlashMAX can
deliver a steadier, much more predictable level of service than any other HDD or SSD solution. This in
turn translates into a reduction in over-provisioning, with resultant CAPEX and OPEX savings.

Individually Tiering Tables and Indexes
In some cases it is not practical to place the entire database on Virident FlashMAX volumes. The dataset
itself could be too large to fit practically, there could be an existing SAN infrastructure that needs to be
retained, or high performance may only needed for individual tables (or sometimes even just parts of
tables). In these cases, users can undertake a more involved effort to place only portions of the database
on a Virident FlashMAX yet still derive quantifiable benefits.

Page 5


Placing Specific Indexes on FlashMAX
In general, table indexes have higher IO requirements (much more random, and many more, updates) than
the main table data files. Table indexes are also, in general, significantly smaller than the tables
themselves; even indexes of very large databases can fit on a single FlashMAX card. Finally, indexes may
have somewhat less stringent availability requirements (since they can be regenerated in case of storage
failure) and may be easier to migrate from a SAN to locally attached storage.

Command Example
Windows PowerShell
PS C:UsersAdministrator> SQLCMD
1> ALTER DATABASE test ADD FILEGROUP FlashMAX_index;
2> GO
1> ALTER DATABASE test ADD FILE ( NAME='flashidx', FILENAME='V:TESTflashidx.mdf',
SIZE=1000MB, FILEGROWTH=1000MB) TO FILEGROUP FlashMAX_index;
2> GO
1> USE test
2> GO
Changed database context to 'test'.
1> CREATE INDEX birthday_idx ON usertable(birthday) WITH ( FILLFACTOR=100,
SORT_IN_TEMPDB=ON ) ON FlashMAX_index;
2> GO

Optimization Note
Indexes may be placed onto the FlashMAX via the use of standard FILEGROUPS in SQL Server. Care
should be taken when moving clustered indexes to the FlashMAX, since moving a clustered index to a
filegroup will also move the data into that filegroup as well.
Choosing which indexes to place on a FlashMAX drive is a manual process, but input to that process can
be generated either by database administrators with the requisite intuition and application knowledge
or via Microsoft SQL Server Dynamic Management Views: “sys.dm_db_index_usage_stats “ and
“sys.dm_db_index_operational_stats.”

Page 6


Placing Specific Tables on FlashMAX
Like indexes, data tables themselves can be placed on FlashMAX. Criteria for choosing these tables should
include their table size (which should be large enough that the tables are not already completely cached in
buffers, but small enough to fit with expected growth on the FlashMAX), their update frequency (even
smaller tables with frequently updated contents can benefit from the higher random-IO performance of the
FlashMAX drive) and their criticality.
Optimization Note
SQL Profiler can be used to capture real-world traces of queries; the database administrator can then
examine the result to identify frequently accessed tables. A proxy for this would be the same Dynamic
Management Views mentioned previously, “sys.dm_db_index_usage_stats “ and
“sys.dm_db_index_operational_stats,” since index access also implies table accesses.

Command Example
Windows PowerShell
1> USE tpcc;
2> SELECT i.name, s.* FROM sys.dm_db_index_usage_stats s INNER JOIN
sys.indexes i ON i.object_id = s.object_id AND i.index_id = s.index_id;
3> GO
...
name
user_seeks user_scans user_lookups user_updates system_seeks system_scans system_lookups system_updates
NULL
0
0
0
130013
0
0
0
0
pkey_new_orders
261370
0
0
260699
0
14
0
0
pkey_orders
0
28
0
42
0
0
0
0
idx_orders
13001
0
0
130485
0
3
0
0
pkey_orders
1701272
0
0
261057
0
2
0
0
pkey_order_line
287146
0
0
1427685
0
4
0
0
pkey_warehouse
390511
0
0
130013
0
0
0
0
pkey_item
0
14
0
14
0
0
0
0
pkey_item
1298468
0
0
0
0
0
0
0
pkey_district
664010
0
0
260498
0
0
0
0
pkey_stock
5815022
0
0
1297113
0
1
0
0
pkey_customer
852009
0
7874
260585
0
0
0
0
idx_customer
171416
0
0
0
0
1
0
0
idx_customer
14
0
0
0
0
0
0
0

Page 7


Placing Parts of Tables on FlashMAX
Even more finely grained tuning of database placement is possible via the use of partitioned tables. By
partitioning a table (often done by date or sequence number), historic data can be manually tiered to more
slowly rotating media, while recent data can be stored on FlashMAX locally for frequent access and updates.
Note, however, that this method can require significant database administrator work, since these
partitioning schemes, and the management of the FILEGROUPS which contain these PARTITIONS,, we will
need to be continyally updated as the database access patterns vary overtime.

Optimization Note
Partitioning tables and placing frequently accessed partitions on the Virident FlashMAX can
significantly speed up application performance, but be sure that queries accessing these tables are not
also accessing indexes, other tables or other partitions of the same table that are not stored on the
FlashMAX.

Page 8


Command Example
Windows PowerShell
1> -- Make the two filegroups
2> ALTER DATABASE test ADD FILEGROUP data_2010
3> GO
1> ALTER DATABASE test ADD FILEGROUP data_2011
2> GO
1> -- Historic FG lives on the SAN...
2> ALTER DATABASE test ADD FILE (NAME = 'data_2010', FILENAME = 'S:datadata_2010.mdf',
SIZE = 1000000MB, FILEGROWTH = 1000MB) TO FILEGROUP data_2010
3> GO
1> -- Active FG lives on Virident...
2> ALTER DATABASE test ADD FILE (NAME = 'data_2011', FILENAME = 'V:datadata_2011.mdf',
SIZE = 300000MB, FILEGROWTH = 1000MB) TO FILEGROUP data_2011
3> GO
1> -- Define the partition function, only have 1 range point as anything past
2> CREATE PARTITION FUNCTION part_yearly(DATETIME) AS RANGE LEFT FOR VALUES
( '20111231 23:59:59.999' );
3> GO
1> -- If it's prior to first cutoff go to 2010, otherwise
2> CREATE PARTITION SCHEME partsch_yearly AS PARTITION part_yearly TO ( data_2010, data_2011 );
3> GO
1> -- Make a new partitioned table and populate it...
2> CREATE TABLE orders_part ( ... ) on partsch_yearly;
3> GO
1> -- Go get some coffee and a good book...
2> INSERT orders_part SELECT * FROM orders;
3> GO

Page 9


Transaction Logs on FlashMAX
Transaction logs are written in small blocks, sequentially. The size of a transaction log varies widely with the
database side, as does the activity logged therein. For OLTP workloads, where many updates and inserts
are performed, the transaction log can often become a limiting factor. Conversely, for DSS workloads,
which have much less frequent updates, the transaction log may not be particularly performance sensitive.
On rotating media, transaction logs are often stored on their own RAID mirrored drive set. If the logs were
stored on the main table volume set, the random-access nature of table and index access would destroy
their sequential-write access pattern. Since HDDs perform sequential-write accesses significantly faster
than random ones, the cost (in terms of money, power and space) of the additional log-drive pair is
outweighed by the performance improvement received.
No such design decision needs to be made with the Virident FlashMAX. Log and database can successfully
coexist on the same drive without any impact on performance. Multiple database logs can also be stored on
a single FlashMAX, without any of the negative performance implications such a configuration would have
on rotating media. The FlashMAX Drive provides enough raw IO horsepower to handle sequential
transaction logging without having an impact on the main database random read/write access, and vice
versa.

2.5

One Log

3.8

Two Logs

7.3

Four Logs
0X

2X

4X

6X

8X

Performance Relative to 15K RPM HDDs with BBU RAID

Optimization Note
SQL Server writes only a single log file per database. Specifying multiple log files per database on the
Virident FlashMAX, or any other storage, will not improve performance.

Page 10


Tempdb on FlashMAX
SQL Server uses tempdb to store intermediate query results; tempdb is also used during index creation and
updates. Writes to SQL Server are often large-block sequential in nature, while reads are commonly
random in nature. Data turnover is high, since the data inside each tempdb table is only valid for a single
query or index-generation step. Index creation and updates are an ideal use for the very high write
bandwidth and random IOPS.
Command Example
Windows PowerShell
PS C:UsersAdministrator> MKDIR V:tempdb
1> -- Move TEMPDB file to Virident, and add additional 7 for 8 total files
2> ALTER DATABASE tempdb MODIFY FILE (NAME=tempdev , FILENAME='V:tempdbtdb0.mdf', SIZE=500MB);
3> ALTER DATABASE tempdb ADD FILE (NAME=tempdev1 , FILENAME='V:tempdbtdb1.mdf', SIZE=500MB);
10>
11> -- Don't forget to move the log, too
12> ALTER DATABASE tempdb MODIFY FILE (NAME=templog , FILENAME='V:tempdbtdb.ldf', SIZE=100MB);
13>
14> GO
The file "tempdev" has been modified in the system catalog. The new path will be used the next time the database is
started.
The file "templog" has been modified in the system catalog. The new path will be used the next time the database is
started.
1> EXIT
PS C:UsersAdministrator> NET STOP MSSQLSERVER
The SQL Server (MSSQLSERVER) service is stopping.
The SQL Server (MSSQLSERVER) service was stopped successfully.
PS C:UsersAdministrator> NET START MSSQLSERVER
The SQL Server (MSSQLSERVER) service is starting.
The SQL Server (MSSQLSERVER) service was started successfully.

Page 11


Optimization Note
Standard best practices for tempdb should be followed. Ensure that tempdb is specified with as many
separate files as there are cores in your server to minimize any tempdb allocation bottlenecks. This is
especially important for highly utilized SQL Server installations, as it allows multiple queries to utilize
the tempdb in parallel.

Data Reliability
As with all direct attach storage, proper care should be taken of databases stored on the Virident FlashMAX
drive. As an easy start, any existing data protection and availability plans can be implemented identically on
the FlashMAX drive.

Flash Management
FlashMAX implements the highest level available of onboard flash wear leveling and of error detection and
correction. It actively monitors flash performance and can move data behind the scenes, transparently, to
SQL Server, to ensure continual data availability.

RAID-1 Across Multiple Cards
Multiple cards can be striped in a mirrored (RAID-1) volume using the standard Disk Manager. No special
setup or configuration is required.

Page 12


Command Example
Windows PowerShell
PS C:UsersAdministrator> DISKPART
Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: KANHA03
DISKPART> SELECT DISK=3
Disk 3 is now the selected disk.
DISKPART> CONVERT DYNAMIC
DiskPart successfully converted the selected disk to dynamic format.
DISKPART> SELECT DISK=4
Disk 4 is now the selected disk.
DISKPART> CONVERT DYNAMIC
DiskPart successfully converted the selected disk to dynamic format.
DISKPART> CREATE VOLUME MIRROR DISK=3,4
DiskPart successfully created the volume.
DISKPART> FORMAT FS=NTFS LABEL=Virident-R1 QUICK
100 percent completed
DiskPart successfully formatted the volume.
DISKPART> ASSIGN LETTER=V
DiskPart successfully assigned the drive letter or mount point.
DISKPART> EXIT
Leaving DiskPart...

Optimization Note
RAID is not a backup strategy. A software error or unexpected “DROP DATABASE” can destroy all user
data protected by either of these methods.

Data Availability
Microsoft SQL Server provides both mirroring and log-shipping modes to ensure high data availability in
cases of server failure.

SQL Server Mirroring
Microsoft SQL Server provides mirroring capabilities to allow a primary and secondary database server to
stay synchronized through a network connection. This mode supports asynchronous and synchronous
replication. The data-consistency guarantees for these two modes vary somewhat, and the proper mode
depends on your own requirements. Asynchronous mode gives the highest performance replication for
frequently updated databases.

Page 13


Whichever mode of replication is used, the lowest-latency network interconnect possible should be used.
While the 10-fold improvement in bandwidth available when moving from 1G Ethernet to 10G Ethernet
may not be useful, the reduction in the latency of smaller messages will impact replication traffic and
responsiveness.
Optimization Note
Servers at both ends of a mirroring operation should be of similar design. If the database at the primary
mirror is placed on a Virident FlashMAX, it should be on a FlashMAX on the secondary mirror as well.
Otherwise the performance of the system can become limited to that of the lowest-performing server,
not only in the case of failover but also in steady state.

Transaction-Log Shipping
Transaction-log shipping allows for a master database to periodically send updated transaction-log copies to
one or many secondary servers over shared storage. Logs are backed up at the primary server, then copied
and finally restored to secondary servers to present a delayed view of the primary database.
Little changes when implementing this replication on servers utilizing the Virident FlashMAX. The
additional bandwidth available for backing up or restoring transaction logs stored on the FlashMAX drive
can slightly reduce the load on any server during this operation.
Optimization Note
Transaction-log shipping to secondary servers is often used for reporting or analysis work, to avoid
making an impact on the main database server. By implementing a Virident FlashMAX on the main
server, however, the need for these secondary servers can be greatly diminished or eliminated entirely.
The limiting performance factor in most database servers is the storage subsystem. The FlashMAX has
enough IO performance and flexibility to support on-line analysis on live databases without the need
for secondary copies.

Conclusion
Adding the Virident FlashMAX to SQL Server architectures is the simplest, most cost-effective way of
increasing application performance. FlashMAX reduces capital and operating expenses by removing noisy,
power-hungry and failure-prone hard drives. With the additional IO power that FlashMAX provides, more
users can be served by fewer SQL Server installations.

Page 14


Implementing Flash Storage for SQL Server from Virident

Recommended

More Related Content

What's hot (17)

Similar to Implementing Flash Storage for SQL Server from Virident (20)

Recently uploaded (20)

Implementing Flash Storage for SQL Server from Virident