For developing, testing and troubleshooting against on-prem SharePoint it has become common practice to run Windows Server virtual environments with SharePoint running on them on your own laptop. Now, if you have ever looked at the system requirements of SharePoint 2013 (and if you have read this far then you probably have) then you know that SharePoint is a resource hog. The obvious solution 3 years ago when SP 2013 was published, was to get ridiculously powerful laptops to get your work done. You know, the kind you can use to heat your apartment, with a power brick the size of a, well, actual brick.
After three years of running around with this beast of a laptop (which even after three years is still really fast), I want to get rid of this burden, and try to find some other solution.
Running SharePoint on Azure virtual machines
There are a lot of articles on how to move a Hyper-V virtual machine to Azure for running it in the cloud, so I won't go into detail on that. Check out this article if you want to know more: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-create-upload-vhd-windows-server/. But DO NOT SYSPREP your SharePoint VM. In my tests, it did not survive that.
The problem? Azure Storage is slow. Really really slow. It is reasonably fast when reading and writing large, sequential pieces of data, and once your server has loaded DLLs and whatnot into RAM, your machine will be fast. For random reads and writes of small bits of data it is horribly slow though. For databases you can set up striped disks and get good IOPS from standard storage, but then... I don't want to set up development environments with striped disks for SQL databases and my Visual Studio installation and then have to copy an array of .vhd files around every time I need a new copy of the environment. Don't get me wrong, Azure is great for running VMs, but running a single server SharePoint dev environment with just one disk is a real pain.
I want a single .vhd, containing Windows Server, SharePoint, Visual Studio and whatever I want to use for development. And I want things to happen reasonably fast. Luckily, Microsoft has introduced Premium Storage, SSD backed storage with the only purpose of providing better performance for IOPS heavy workloads in virtual machines (and me playing around with SharePoint and Visual Studio is IOPS heavy).
If you have used Azure for anything recently, you might know that:
- Microsoft is replacing the old management UI (https://manage.windowsazure.com/) with a new portal (https://portal.azure.com/)
- Microsoft is moving from the "Classic" deployment model to a new "Resource Manager" deployment model
This means Azure is currently (as of January 7, 2016) a pretty confusing mix of things, and I had a hard time figuring out some things when trying to get my virtual machine into Premium Storage. So I thought this might be an ideal topic for a first post in my new blog.
What is Premium Storage
First of all, you can only use Premium Storage for disks. All other Storage services are backed by Standard Storage. There are currently 3 tiers in Premium Storage. The tier (if I understood correctly) is determined only by the size of the .vhd you put in there. So e.g. if your .vhd is 130 GB, it will be in the P20 tier. Check the Azure website for current pricing on Premium storage (https://azure.microsoft.com/en-us/pricing/details/storage/).
These are the Premium Storage tiers and their respective performance promises:
These are the Premium Storage tiers and their respective performance promises:
| Tier | P10 | P20 | P30 | Standard storage |
|---|---|---|---|---|
| Disk size | 128 GB | 512 GB | 1024 GB | max 1023 GB |
| IOPS per disk | 500 | 2300 | 5000 | max 500 |
| Thoughput per disk | 100 MB/s | 150 MB/s | 200 MB/s | ??? MB/s |
Now this is confusing, isn't it? The P10 tier offers the same 500 IOPS as a Standard Storage disk does. Or so you'd think. When you use Premium Storage, according to Microsoft, Azure "provisions for you a guaranteed number of IOPS as per the disk specification". I did not find any promise for Standard Storage. So lets find out if there is a difference.
Premium Storage can only be used with DS and GS Azure VMs. The pricing for DS and GS series is the same as for the equivalent D and G series. There are two main differences between the S and non-S machines:
- *S-tiers are enabled for Premium Storage
- *S-tiers allocate a large part of the local SSD temp storage in D and G series as cache, and leave twice the amount of RAM as temp storage (e.g. D2 has 7 GB of RAM and 100 GB of SSD temp storage, whereas DS2 has only 14 GB of SSD temp storage and uses the remaining 86 for caching of the persistent disks)
As you were probably not using the temp disk with your SharePoint VM anyway, this is not going to be a problem. (Or maybe you want to live a little, put your content databases there, or even better, your work in progress source code?)
How to set up a VM with premium storage
For the purpose of this post, I'll use the classic deployment model. I will have to learn the resource manager model, that's another thing I've not come around to do.
I mostly follow this guide for setting up my Premium VM: https://azure.microsoft.com/en-us/documentation/articles/storage-migration-to-premium-storage/
Before you start, I assume you have the following ready:
- Azure subscription (obviously)
- .vhd containing your VM in Azure Storage (the "Standard" kind)
Next we will:
- Create a Premium Storage account (and storage container)
- Copy a .vhd to the Premium Storage account
- Fire up a DS virtual machine from the existing .vhd
- Compare performance with a D series VM
Creating a Premium Storage account
To create a Premium Storage account in the Azure Portal, navigate to Storage accounts (I used the classic kind), select Add, give your storage account a name and select the Premium-LRS pricing tier. Easy. Then create a storage container and call it "vhds".
Alternatively, you can use Powershell, in my case the commands look like this:
New-AzureStorageAccount -StorageAccountName "yourstorageaccountname" -Type "Premium_LRS" -Location "North Europe"
Set-AzureSubscription -SubscriptionName "Your Subscription Name" -CurrentStorageAccountName "yourstorageaccountname"
New-AzureStorageContainer vhds -Permission Off
Make sure you have a fairly new version of Azure Powershell installed for this (I'm currently using 1.0.2, as of January 8, 2016). According to MSDN you can set the Resource Group when creating the storage account using PowerShell, but that did not work for me, so I set the Location to "North Europe" manually.
Copying VHDs to Premium Storage
Now you should use Powershell. Microsoft tells you, that you can also use AzCopy or Azure Storage Explorer, but I find Powershell to be the most effective and reliable by far. (Especially uploading large files to Azure Storage works much faster and is more reliable with Powershell than any other tool I've tried. But this is almost a topic for another post.)Copy your .vhd to the newly created Premium Storage account like this (make sure to set source and destination account names and keys according to your environment):
$destContext = New-AzureStorageContext -StorageAccountName $destAccountName -StorageAccountKey $destAccountKey
$srcContext = New-AzureStorageContext -StorageAccountName $sourceAccountName -StorageAccountKey $sourceAccessKey
Start-AzureStorageBlobCopy -SrcBlob "YourVHD.vhd" -SrcContainer "vhds" -DestContainer "vhds" -Context $srcContext -DestContext $destContext
Creating a VM from the uploaded .vhd
For this step you can again use either Powershell or the Azure Portal and Powershell .First, you need to "register" it as a Disk which can be attached to a VM. I did not find this operation in the Portal, so use Powershell:
$vhdLocation = "https://yourstorageaccountname.blob.core.windows.net/vhds/YourVHD.vhd"
Add-AzureDisk -DiskName "YourVHD" -MediaLocation $vhdLocation -Label "My SharePoint 2013 VM" -OS "Windows"
Now you can create a VM from this disk. If you use Powershell, make sure you set the InstanceSize to a DS series such as "Standard_DS2". If you have created VMs from disks in the old management UI, then you will have a hard time doing the same in the new Portal. Before, you went to create a new VM under Virtual Machines, and then selected the disk you wanted to create from. Now in the new Portal, you go to Disks, and under your disk, you'll find a button to create a new VM.
Now wait a few minutes, and you are ready to connect to you Premium VM! Now lets see how it compares to a Standard VM.
Performance comparison
For the sake of this blog post, I created two VMs from the same .vhd (from two separate copies obviously). A DS12 with Premium Storage and a D12 (or rap-machine, as my colleague likes to call them) with Standard Storage. For the sake of possibly saving some money, I also fired up a DS11 VM (which costs half as much as a D12), to see how it performs compared to a D12 VM.
First I measured disk speed using CrystalDiskMark. The performance difference is remarkable. For sequential reads the Standard Storage disk looses, but the real difference come with random, small reads and writes. For random 4kb reads, Premium Storage performed 30 times better than Standard Storage. P10 with a DS11 VM gets about half of the DS12 results shown below, because the storage bandwidth is throttled to 64 MB/s.
To see if there is any real-world performance benefit, I tried a few real-world tasks someone working with SharePoint will be facing often. Manual tasks performed through UI were measured a few times (by hand) and the result is an average. To measure the duration of creating site collections in Powershell, I used [System.Diagnostics.Stopwatch] (very handy by the way).
| Task | D12 | DS12 | DS11 |
|---|---|---|---|
| Creating a site collection from Central Admin |
1 min 59 s | 33 s | 35 s |
| Creating 10 site collections using Powershell |
16 min 56 s | 4 min 56 s | 5 min 18 s |
| Opening a team site after running IISRESET |
1 min 27 s | 50 s | 60 s |
That is a clear performance boost! Creating site collections takes 4 times longer on a Standard D12 VM than it does on a DS11 or DS12 machine. Everything else I tried, like installing Windows Updates, installing SharePoint CUs, using Visual Studio, felt a lot snappier than on the D12 VM.
Lessons learned
- 500 Premium IOPS is more than 500 Standard IOPS.
- If you are setting up a "single server" SharePoint development VM, use Premium Storage. It's probably worth it.
- DS11 with a P10 disk is in most cases probably the best "bang for your buck" solution.
- Don't even think about setting up a SharePoint VM using Basic Tier VMs. You'll grow old before you get anything done.
- I'm starting to get used to the new Azure Portal.
To-do list
- Try the P20 and P30 Premium Storage tiers
- Learn the resource manager model
No comments:
Post a Comment