Get Started With Storage Explorer and Azcopy
Get Started With Storage Explorer and Azcopy
Overview
Azure Storage Explorer is a standalone app that enables you to easily work with Azure
Storage data on Windows, macOS, and Linux. In this article, you learn several ways of
connecting to and managing your Azure storage accounts.
Prerequisites
Windows
macOS
Linux
Windows 10 (recommended)
Windows 8
Windows 7
In addition, you can work with storage accounts in global and national Azure:
If you don't have an Azure account, you can sign up for a free trial or activate your Visual
Studio subscriber benefits.
Panel.
2. The left pane now displays all the Azure accounts you've signed in to. To connect to
another account, select Add an account
3. If you want to sign into a national cloud or an Azure Stack, click on the Azure
environment dropdown to select which Azure cloud you want to use. Once you have
chosen your environment, click the Sign in... button. If you are signing in to Azure
Stack, see Connect Storage Explorer to an Azure Stack subscription for more
information.
4. After you successfully sign in with an Azure account, the account and the Azure
subscriptions associated with that account are added to the left pane. Select the Azure
subscriptions that you want to work with, and then select Apply (Selecting All
subscriptions: toggles selecting all or none of the listed Azure subscriptions).
the left pane displays the storage accounts associated with the selected Azure
subscriptions.
Starting with version 1.1.0, local storage emulator is supported on all platforms. Storage
Explorer can connect to any emulated service listening to its default local storage endpoints.
Note
Support for storage services and features may vary widely depending on your choice of
emulator. Make sure your emulator supports the services and features you intend to work
with.
Storage Explorer does not automatically start your emulator. You must start it
yourself.
3. In Storage Explorer, click the Add Account button. Select Attach to a local
emulator and click Next.
4. Enter the port numbers for the services you configured above (leave blank if you don't
intend to use that service). Click Next then Connect to create the connection.
5. Expand the Local & Attached > Storage Accounts > nodes, then expand the service
nodes underneath the node corresponding to your emulator connection.
You can use this node to create and work with local blobs, queues, and tables. To
learn how to work with each storage account type, refer to the following guides:
To share an external storage account, the owner of that account must first get the credentials
(account name and key) for the account and then share that information with the person who
wants to attach to said account. You can obtain the storage account credentials via the Azure
portal by doing the following steps:
To attach to an external storage account, you need the account's name and key. The "Get the
storage account credentials" section explains how to obtain these values from the Azure
portal. However, in the portal, the account key is called key1. So, when Storage Explorer
asks for an account key, you enter the key1 value.
3. Paste your account name in the Account name text box, and paste your account key
(the key1 value from the Azure portal) into the Account key text box, and then select
Next.
Note
To use a name and key from a national cloud, use the Storage endpoints domain:
dropdown to select the appropriate endpoints domain:
4. In the Connection Summary dialog box, verify the information. If you want to
change anything, select Back and reenter the desired settings.
5. Select Connect.
6. After the storage account has successfully been attached, the storage account is
displayed with (External) appended to its name.
1. Right-click the external storage account that you want to detach, and then select
Detach.
2. In the confirmation message, select Yes to confirm the detachment from the external
storage account.
To illustrate this scenario, let's say that UserA is an admin of an Azure subscription, and
UserA wants to allow UserB to access a storage account for a limited time with certain
permissions:
1. UserA generates a SAS connection string for a specific time period and with the
desired permissions.
2. UserA shares the SAS with the person (UserB, in this example) who wants access to
the storage account.
3. UserB uses Storage Explorer to attach to the account that belongs to UserA by using
the supplied SAS.
Generate a SAS connection string for the account you want to share
1. In Storage Explorer, right-click the storage account you want share, and then select
Get Shared Access Signature....
2. In the Generate Shared Access Signature dialog box, specify the time frame and
permissions that you want for the account, and then click the Create button.
3. Next to the Connection String text box, select Copy to copy it to your clipboard, and
then click Close.
In this context, a service can be a blob container, queue, table, or file share. To generate the
SAS for a listed service, see:
1. Find Local and Attached in the left tree, right-click Azure Cosmos DB Accounts,
choose Connect to Azure Cosmos DB...
2. Choose Azure Cosmos DB API, paste your Connection String, and then click OK to
connect Azure Cosmos DB account. For information on retrieving the connection
string, see Get the connection string.
As you type in the search box, the left pane displays all resources that match the search value
you've entered up to that point. For example, a search for endpoints is shown in the
following screenshot:
Note
Use the Account Management Panel to deselect any subscriptions that do not contain the
item you are searching for to improve the execution time of your search. You can also right-
click on a node and choose Search From Here to start searching from a specific node.
Manage Azure Blob Storage resources with
Storage Explorer
Overview
Azure Blob Storage is a service for storing large amounts of unstructured data, such as text or
binary data, that can be accessed from anywhere in the world via HTTP or HTTPS. You can
use Blob storage to expose data publicly to the world, or to store application data privately. In
this article, you'll learn how to use Storage Explorer to work with blob containers and blobs.
Prerequisites
To complete the steps in this article, you'll need the following:
The following steps illustrate how to create a blob container within Storage Explorer.
4. A text box will appear below the Blob Containers folder. Enter the name for your
blob container. See the Create the container and set permissions for information on
rules and restrictions on naming blob containers.
5. Press Enter when done to create the blob container, or Esc to cancel. Once the blob
container has been successfully created, it will be displayed under the Blob
Containers folder for the selected storage account.
The following steps illustrate how to view the contents of a blob container within Storage
Explorer:
The following steps illustrate how to delete a blob container within Storage Explorer:
The following steps illustrate how to copy a blob container from one storage account to
another.
5. Right-click the desired "target" storage account into which you want to paste the blob
container, and - from the context menu - select Paste Blob Container.
Get the SAS for a blob container
A shared access signature (SAS) provides delegated access to resources in your storage
account. This means that you can grant a client limited permissions to objects in your storage
account for a specified period of time and with a specified set of permissions, without having
to share your account access keys.
The following steps illustrate how to create a SAS for a blob container:
5. In the Shared Access Signature dialog, specify the policy, start and expiration dates,
time zone, and access levels you want for the resource.
6. When you're finished specifying the SAS options, select Create.
7. A second Shared Access Signature dialog will then display that lists the blob
container along with the URL and QueryStrings you can use to access the storage
resource. Select Copy next to the URL you wish to copy to the clipboard.
8. When done, select Close.
The following steps illustrate how to specify a public access level for a blob container.
5. In the Set Container Public Access Level dialog, specify the desired access level.
6. Select Apply.
The following steps illustrate how to manage the blobs (and folders) within a blob container.
2. In the Upload files dialog, select the ellipsis (…) button on the right
side of the Files text box to select the file(s) you wish to upload.
3. Specify the type of Blob type. See Create the container and set
permissions for more information.
4. Optionally, specify a target folder into which the selected file(s) will be
uploaded. If the target folder doesn’t exist, it will be created.
5. Select Upload.
o Upload a folder to a blob container
1. On the main pane's toolbar, select Upload, and then Upload Folder
from the drop-down menu.
2. In the Upload folder dialog, select the ellipsis (…) button on the right
side of the Folder text box to select the folder whose contents you
wish to upload.
3. Specify the type of Blob type. See Create the container and set
permissions for more information.
4. Optionally, specify a target folder into which the selected folder's
contents will be uploaded. If the target folder doesn’t exist, it will be
created.
5. Select Upload.
o Download a blob to your local computer
1. Select the blob you wish to download.
2. On the main pane's toolbar, select Download.
3. In the Specify where to save the downloaded blob dialog, specify the
location where you want the blob downloaded, and the name you wish
to give it.
4. Select Save.
o Open a blob on your local computer
1. Select the blob you wish to open.
2. On the main pane's toolbar, select Open.
3. The blob will be downloaded and opened using the application
associated with the blob's underlying file type.
o Copy a blob to the clipboard
1. Select the blob you wish to copy.
2. On the main pane's toolbar, select Copy.
3. In the left pane, navigate to another blob container, and double-click it
to view it in the main pane.
4. On the main pane's toolbar, select Paste to create a copy of the blob.
o Delete a blob
1. Select the blob you wish to delete.
2. On the main pane's toolbar, select Delete.
3. Select Yes to the confirmation dialog.
Transfer data with the AzCopy on Windows
AzCopy is a command-line utility designed for copying data to/from Microsoft Azure Blob,
File, and Table storage, using simple commands designed for optimal performance. You can
copy data between a file system and a storage account, or between storage accounts.
There are two versions of AzCopy that you can download. AzCopy on Windows offers
Windows style command-line options. AzCopy on Linux targets Linux platforms offering
POSIX style command-line options. This article covers AzCopy on Windows.
Download the latest preview version of AzCopy on Windows. This Preview version offers
significant performance improvements and packages .NET Core in the installation.
Table service is no longer supported in the latest version. If you use Table export feature,
download the stable version.
Built with .NET Core 2.1, and all .NET Core dependencies are now packaged in the
installation.
Significant performance improvements for both upload and download scenarios
Post-installation Step
After installing AzCopy on Windows using the installer, open a command window and
navigate to the AzCopy installation directory on your computer - where the AzCopy.exe
executable is located. If desired, you can add the AzCopy installation location to your system
path. By default, AzCopy is installed to %ProgramFiles(x86)%\Microsoft SDKs\Azure\
AzCopy or %ProgramFiles%\Microsoft SDKs\Azure\AzCopy.
AzCopy
AzCopy /Source:<source> /Dest:<destination> [Options]
The following examples demonstrate a variety of scenarios for copying data to and from
Microsoft Azure Blobs, Files, and Tables. Refer to the AzCopy Parameters section for a
detailed explanation of the parameters used in each sample.
Download blobs from Blob storage
Let's look at several ways to download blobs using AzCopy.
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /Pattern:"abc.txt"
Note that if the folder C:\myfolder does not exist, AzCopy creates it and download abc.txt
into the new folder.
AzCopy
AzCopy
/Source:https://myaccount-secondary.blob.core.windows.net/mynewcontainer
/Dest:C:\myfolder /SourceKey:key /Pattern:"abc.txt"
Note that you must have read-access geo-redundant storage enabled to access the secondary
region.
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /S
abc.txt
abc1.txt
abc2.txt
vd1\a.txt
vd1\abcd.txt
After the download operation, the directory C:\myfolder includes the following files:
C:\myfolder\abc.txt
C:\myfolder\abc1.txt
C:\myfolder\abc2.txt
C:\myfolder\vd1\a.txt
C:\myfolder\vd1\abcd.txt
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /Pattern:a /S
Assume the following blobs reside in the specified container. All blobs beginning with the
prefix a are downloaded:
abc.txt
abc1.txt
abc2.txt
xyz.txt
vd1\a.txt
vd1\abcd.txt
After the download operation, the folder C:\myfolder includes the following files:
C:\myfolder\abc.txt
C:\myfolder\abc1.txt
C:\myfolder\abc2.txt
The prefix applies to the virtual directory, which forms the first part of the blob name. In the
example shown above, the virtual directory does not match the specified prefix, so it is not
downloaded. In addition, if the option /S is not specified, AzCopy does not download any
blobs.
Set the last-modified time of exported files to be same as the source blobs
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /MT
You can also exclude blobs from the download operation based on their last-modified time.
For example, if you want to exclude blobs whose last modified time is the same or newer
than the destination file, add the /XN option:
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /MT /XN
If you want to exclude blobs whose last modified time is the same or older than the
destination file, add the /XO option:
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder /SourceKey:key /MT /XO
If the specified destination container does not exist, AzCopy creates it and uploads the file
into it.
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer/vd
/DestKey:key /Pattern:abc.txt
If the specified virtual directory does not exist, AzCopy uploads the file to include the virtual
directory in its name (e.g., vd/abc.txt in the example above).
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /S
Specifying option /S uploads the contents of the specified directory to Blob storage
recursively, meaning that all subfolders and their files are uploaded as well. For instance,
assume the following files reside in folder C:\myfolder:
C:\myfolder\abc.txt
C:\myfolder\abc1.txt
C:\myfolder\abc2.txt
C:\myfolder\subfolder\a.txt
C:\myfolder\subfolder\abcd.txt
After the upload operation, the container includes the following files:
abc.txt
abc1.txt
abc2.txt
subfolder\a.txt
subfolder\abcd.txt
If you do not specify option /S, AzCopy does not upload recursively. After the upload
operation, the container includes the following files:
abc.txt
abc1.txt
abc2.txt
C:\myfolder\abc.txt
C:\myfolder\abc1.txt
C:\myfolder\abc2.txt
C:\myfolder\xyz.txt
C:\myfolder\subfolder\a.txt
C:\myfolder\subfolder\abcd.txt
After the upload operation, the container includes the following files:
abc.txt
abc1.txt
abc2.txt
subfolder\a.txt
subfolder\abcd.txt
If you do not specify option /S, AzCopy only uploads blobs that don't reside in a virtual
directory:
C:\myfolder\abc.txt
C:\myfolder\abc1.txt
C:\myfolder\abc2.txt
AzCopy
AzCopy /Source:C:\myfolder\
/Dest:https://myaccount.blob.core.windows.net/myContainer/ /DestKey:key
/Pattern:ab /SetContentType:video/mp4
If you specify /SetContentType without a value, AzCopy sets each blob or file's content
type according to its file extension.
AzCopy
AzCopy /Source:C:\myfolder\
/Dest:https://myaccount.blob.core.windows.net/myContainer/ /DestKey:key
/Pattern:ab /SetContentType
Copy a single blob from one container to another within the same storage
account
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer1
/Dest:https://myaccount.blob.core.windows.net/mycontainer2
/SourceKey:key /DestKey:key /Pattern:abc.txt
When you copy a blob within a Storage account, a server-side copy operation is performed.
AzCopy
AzCopy /Source:https://sourceaccount.blob.core.windows.net/mycontainer1
/Dest:https://destaccount.blob.core.windows.net/mycontainer2
/SourceKey:key1 /DestKey:key2 /Pattern:abc.txt
When you copy a blob across Storage accounts, a server-side copy operation is performed.
Copy a single blob from the secondary region to the primary region
AzCopy
AzCopy
/Source:https://myaccount1-secondary.blob.core.windows.net/mynewcontainer1
/Dest:https://myaccount2.blob.core.windows.net/mynewcontainer2
/SourceKey:key1 /DestKey:key2 /Pattern:abc.txt
Note that you must have read-access geo-redundant storage enabled to access secondary
storage.
Copy a single blob and its snapshots from one storage account to another
AzCopy
AzCopy /Source:https://sourceaccount.blob.core.windows.net/mycontainer1
/Dest:https://destaccount.blob.core.windows.net/mycontainer2
/SourceKey:key1 /DestKey:key2 /Pattern:abc.txt /Snapshot
After the copy operation, the target container includes the blob and its snapshots. Assuming
the blob in the example above has two snapshots, the container includes the following blob
and snapshots:
abc.txt
abc (2013-02-25 080757).txt
abc (2014-02-21 150331).txt
Specifying option /S uploads the contents of the specified container recursively. See Upload
all blobs in a folder for more information and an example.
AzCopy by default copies data between two storage endpoints asynchronously. Therefore,
the copy operation runs in the background using spare bandwidth capacity that has no SLA in
terms of how fast a blob is copied, and AzCopy periodically checks the copy status until the
copying is completed or failed.
The /SyncCopy option ensures that the copy operation gets consistent speed. AzCopy
performs the synchronous copy by downloading the blobs to copy from the specified source
to local memory, and then uploading them to the Blob storage destination.
AzCopy
AzCopy /Source:https://myaccount1.blob.core.windows.net/myContainer/
/Dest:https://myaccount2.blob.core.windows.net/myContainer/ /SourceKey:key1
/DestKey:key2 /Pattern:ab /SyncCopy
/SyncCopy might generate additional egress cost compared to asynchronous copy, the
recommended approach is to use this option in an Azure VM that is in the same region as
your source storage account to avoid egress cost.
AzCopy
AzCopy
/Source:https://myaccount.file.core.windows.net/myfileshare/myfolder1/
/Dest:C:\myfolder /SourceKey:key /Pattern:abc.txt
If the specified source is an Azure file share, then you must either specify the exact file name,
(e.g. abc.txt) to download a single file, or specify option /S to download all files in the
share recursively. Attempting to specify both a file pattern and option /S together results in
an error.
AzCopy
AzCopy /Source:https://myaccount.file.core.windows.net/myfileshare/
/Dest:C:\myfolder /SourceKey:key /S
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key
/Pattern:abc.txt
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key /S
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key
/Pattern:ab* /S
AzCopy
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare1/
/Dest:https://myaccount2.file.core.windows.net/myfileshare2/
/SourceKey:key1 /DestKey:key2 /S
When you copy a file across file shares, a server-side copy operation is performed.
AzCopy
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare/
/Dest:https://myaccount2.blob.core.windows.net/mycontainer/ /SourceKey:key1
/DestKey:key2 /S
When you copy a file from file share to blob, a server-side copy operation is performed.
AzCopy
AzCopy /Source:https://myaccount1.blob.core.windows.net/mycontainer/
/Dest:https://myaccount2.file.core.windows.net/myfileshare/ /SourceKey:key1
/DestKey:key2 /S
When you copy a file from a blob to a file share, a server-side copy operation is performed.
You can specify the /SyncCopy option to copy data from File Storage to File Storage, from
File Storage to Blob Storage and from Blob Storage to File Storage synchronously, AzCopy
does this by downloading the source data to local memory and upload it again to destination.
Standard egress cost applies.
AzCopy
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare1/
/Dest:https://myaccount2.file.core.windows.net/myfileshare2/
/SourceKey:key1 /DestKey:key2 /S /SyncCopy
When copying from File storage to Blob storage, the default blob type is block blob; the user
can specify the option /BlobType:page to change the destination blob type.
Note that /SyncCopy might generate additional egress costs compared to asynchronous copy.
The recommended approach is to use this option in the Azure VM which is in the same
region as your source storage account to avoid egress cost.
Export a table
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\
myfolder\ /SourceKey:key
AzCopy writes a manifest file to the specified destination folder. The manifest file is used in
the import process to locate the necessary data files and perform data validation. The
manifest file uses the following naming convention by default:
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\
myfolder\ /SourceKey:key /Manifest:abc.manifest
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/mytable/ /Dest:C:\
myfolder /SourceKey:key /S /SplitSize:100
AzCopy uses a volume index in the split data file names to distinguish multiple files. The
volume index consists of two parts, a partition key range index and a split file index. Both
indexes are zero-based.
The partition key range index is 0 if the user does not specify option /PKRS.
For instance, suppose AzCopy generates two data files after the user specifies option
/SplitSize. The resulting data file names might be:
myaccount_mytable_20140903T051850.8128447Z_0_0_C3040FE8.json
myaccount_mytable_20140903T051850.8128447Z_0_1_0AB9AC20.json
Note that the minimum possible value for option /SplitSize is 32MB. If the specified
destination is Blob storage, AzCopy splits the data file once its sizes reaches the blob size
limitation (200GB), regardless of whether option /SplitSize has been specified by the user.
By default, AzCopy exports tables to JSON data files. You can specify the option
/PayloadFormat:JSON|CSV to export the tables as JSON or CSV.
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\
myfolder\ /SourceKey:key /PayloadFormat:CSV
When specifying the CSV payload format, AzCopy also generates a schema file with file
extension .schema.csv for each data file.
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\
myfolder\ /SourceKey:key /PKRS:"aa#bb"
AzCopy starts concurrent operations to export entities when the user specifies option /PKRS.
Each operation exports one partition key range.
Note that the number of concurrent operations is also controlled by option /NC. AzCopy uses
the number of core processors as the default value of /NC when copying table entities, even if
/NC was not specified. When the user specifies option /PKRS, AzCopy uses the smaller of the
two values - partition key ranges versus implicitly or explicitly specified concurrent
operations - to determine the number of concurrent operations to start. For more details, type
AzCopy /?:NC at the command line.
AzCopy
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/
/Dest:https://myaccount.blob.core.windows.net/mycontainer/
/SourceKey:key1 /Destkey:key2
AzCopy generates a JSON data file into the blob container with following naming
convention:
The generated JSON data file follows the payload format for minimal metadata. For details
on this payload format, see Payload Format for Table Service Operations.
Note that when exporting tables to blobs, AzCopy downloads the Table entities to local
temporary data files and then uploads those entities to the blob. These temporary data files
are put into the journal file folder with the default path "%LocalAppData%\Microsoft\
Azure\AzCopy", you can specify option /Z:[journal-file-folder] to change the journal file
folder location and thus change the temporary data files location. The temporary data files'
size is decided by your table entities' size and the size you specified with the option
/SplitSize, although the temporary data file in local disk is deleted instantly once it has been
uploaded to the blob, please make sure you have enough local disk space to store these
temporary data files before they are deleted.
Import a table
AzCopy
AzCopy /Source:C:\myfolder\
/Dest:https://myaccount.table.core.windows.net/mytable1/ /DestKey:key
/Manifest:"myaccount_mytable_20140103T112020.manifest"
/EntityOperation:InsertOrReplace
The option /EntityOperation indicates how to insert entities into the table. Possible values
are:
InsertOrSkip: Skips an existing entity or inserts a new entity if it does not exist in the
table.
InsertOrMerge: Merges an existing entity or inserts a new entity if it does not exist in the
table.
InsertOrReplace: Replaces an existing entity or inserts a new entity if it does not exist in
the table.
Note that you cannot specify option /PKRS in the import scenario. Unlike the export scenario,
in which you must specify option /PKRS to start concurrent operations, AzCopy starts
concurrent operations by default when you import a table. The default number of concurrent
operations started is equal to the number of core processors; however, you can specify a
different number of concurrent with option /NC. For more details, type AzCopy /?:NC at the
command line.
Note that AzCopy only supports importing for JSON, not CSV. AzCopy does not support
table imports from user-created JSON and manifest files. Both of these files must come from
an AzCopy table export. To avoid errors, please do not modify the exported JSON or
manifest file.
Assume a Blob container contains the following: A JSON file representing an Azure Table
and its accompanying manifest file.
myaccount_mytable_20140103T112020.manifest
myaccount_mytable_20140103T112020_0_0_0AF395F1DC42E952.json
You can run the following command to import entities into a table using the manifest file in
that blob container:
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer
/Dest:https://myaccount.table.core.windows.net/mytable /SourceKey:key1
/DestKey:key2 /Manifest:"myaccount_mytable_20140103T112020.manifest"
/EntityOperation:"InsertOrReplace"
The /XO and /XN parameters allow you to exclude older or newer source resources from
being copied, respectively. If you only want to copy source resources that don't exist in the
destination, you can specify both parameters in the AzCopy command:
/Source:http://myaccount.blob.core.windows.net/mycontainer /Dest:C:\
myfolder /SourceKey:<sourcekey> /S /XO /XN
/Source:C:\myfolder
/Dest:http://myaccount.file.core.windows.net/myfileshare /DestKey:<destkey>
/S /XO /XN
/Source:http://myaccount.blob.core.windows.net/mycontainer
/Dest:http://myaccount.blob.core.windows.net/mycontainer1
/SourceKey:<sourcekey> /DestKey:<destkey> /S /XO /XN
Note that this is not supported when either the source or destination is a table.
AzCopy
AzCopy /@:"C:\responsefiles\copyoperation.txt"
You can include any AzCopy command-line parameters in a response file. AzCopy processes
the parameters in the file as if they had been specified on the command line, performing a
direct substitution with the contents of the file.
Assume a response file named copyoperation.txt, that contains the following lines. Each
AzCopy parameter can be specified on a single line
/Source:http://myaccount.blob.core.windows.net/mycontainer /Dest:C:\
myfolder /SourceKey:<sourcekey> /S /Y
or on separate lines:
/Source:http://myaccount.blob.core.windows.net/mycontainer
/Dest:C:\myfolder
/SourceKey:<sourcekey>
/S
/Y
AzCopy fails if you split the parameter across two lines, as shown here for the /sourcekey
parameter:
http://myaccount.blob.core.windows.net/mycontainer
C:\myfolder
/sourcekey:
<sourcekey>
/S
/Y
/Source:http://myaccount.blob.core.windows.net/mycontainer
And a response file named dest.txt that specifies a destination folder in the file system:
/Dest:C:\myfolder
And a response file named options.txt that specifies options for AzCopy:
/S /Y
To call AzCopy with these response files, all of which reside in a directory C:\
responsefiles, use this command:
AzCopy
AzCopy /@:"C:\responsefiles\source.txt" /@:"C:\responsefiles\dest.txt"
/SourceKey:<sourcekey> /@:"C:\responsefiles\options.txt"
AzCopy processes this command just as it would if you included all of the individual
parameters on the command line:
AzCopy
AzCopy /Source:http://myaccount.blob.core.windows.net/mycontainer /Dest:C:\
myfolder /SourceKey:<sourcekey> /S /Y
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer1
/Dest:https://myaccount.blob.core.windows.net/mycontainer2
/SourceSAS:SAS1 /DestSAS:SAS2 /Pattern:abc.txt
AzCopy
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer1/?
SourceSASToken /Dest:C:\myfolder /S
Each time you issue a command to AzCopy, it checks whether a journal file exists in the
default folder, or whether it exists in a folder that you specified via this option. If the journal
file does not exist in either place, AzCopy treats the operation as new and generates a new
journal file.
If the journal file does exist, AzCopy checks whether the command line that you input
matches the command line in the journal file. If the two command lines match, AzCopy
resumes the incomplete operation. If they do not match, you are prompted to either overwrite
the journal file to start a new operation, or to cancel the current operation.
If you want to use the default location for the journal file:
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /Z
If you omit option /Z, or specify option /Z without the folder path, as shown above, AzCopy
creates the journal file in the default location, which is %SystemDrive%\Users\%username%\
AppData\Local\Microsoft\Azure\AzCopy. If the journal file already exists, then AzCopy
resumes the operation based on the journal file.
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key
/Z:C:\journalfolder\
This example creates the journal file if it does not already exist. If it does exist, then AzCopy
resumes the operation based on the journal file.
AzCopy
AzCopy /Z:C:\journalfolder\
This example resumes the last operation, which may have failed to complete.
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /V
If you specify option /V without providing a file path to the verbose log, then AzCopy creates
the log file in the default location, which is %SystemDrive%\Users\%username%\AppData\
Local\Microsoft\Azure\AzCopy.
AzCopy
AzCopy /Source:C:\myfolder
/Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key
/V:C:\myfolder\azcopy1.log
Note that if you specify a relative path following option /V, such as /V:test/azcopy1.log,
then the verbose log is created in the current working directory within a subfolder named
test.
Option /NC specifies the number of concurrent copy operations. By default, AzCopy starts a
certain number of concurrent operations to increase the data transfer throughput. For Table
operations, the number of concurrent operations is equal to the number of processors you
have. For Blob and File operations, the number of concurrent operations is equal 8 times the
number of processors you have. If you are running AzCopy across a low-bandwidth network,
you can specify a lower number for /NC to avoid failure caused by resource competition.
You can run AzCopy against the Azure Storage Emulator for blobs:
AzCopy
AzCopy /Source:https://127.0.0.1:10000/myaccount/mycontainer/ /Dest:C:\
myfolder /SourceKey:key /SourceType:Blob /S
AzCopy
AzCopy /Source:https://127.0.0.1:10002/myaccount/mytable/ /Dest:C:\myfolder
/SourceKey:key /SourceType:Table
AzCopy determines content type of a blob based on a JSON file that stores content type to
file extension mapping. This JSON file is named AzCopyConfig.json, and is located in the
AzCopy directory. If you have a file type that is not in the list you can append the mapping to
the JSON file:
{
"MIMETypeMapping": {
".myext": "text/mycustomtype",
.
.
}
}
AzCopy Parameters
Parameters for AzCopy are described below. You can also type one of the following
commands from the command line for help in using AzCopy:
/Source:"source"
Specifies the source data from which to copy. The source can be a file system directory, a
blob container, a blob virtual directory, a storage file share, a storage file directory, or an
Azure table.
Specifies the destination to copy to. The destination can be a file system directory, a blob
container, a blob virtual directory, a storage file share, a storage file directory, or an Azure
table.
/Pattern:"file-pattern"
Specifies a file pattern that indicates which files to copy. The behavior of the /Pattern
parameter is determined by the location of the source data, and the presence of the recursive
mode option. Recursive mode is specified via option /S.
If the specified source is a directory in the file system, then standard wildcards are in effect,
and the file pattern provided is matched against files within the directory. If option /S is
specified, then AzCopy also matches the specified pattern against all files in any subfolders
beneath the directory.
If the specified source is a blob container or virtual directory, then wildcards are not applied.
If option /S is specified, then AzCopy interprets the specified file pattern as a blob prefix. If
option /S is not specified, then AzCopy matches the file pattern against exact blob names.
If the specified source is an Azure file share, then you must either specify the exact file name,
(e.g. abc.txt) to copy a single file, or specify option /S to copy all files in the share
recursively. Attempting to specify both a file pattern and option /S together results in an
error.
AzCopy uses case-sensitive matching when the /Source is a blob container or blob virtual
directory, and uses case-insensitive matching in all the other cases.
The default file pattern used when no file pattern is specified is . for a file system location or
an empty prefix for an Azure Storage location. Specifying multiple file patterns is not
supported.
/DestKey:"storage-key"
/DestSAS:"sas-token"
Specifies a Shared Access Signature (SAS) with READ and WRITE permissions for the
destination (if applicable). Surround the SAS with double quotes, as it may contains special
command-line characters.
If the destination resource is a blob container, file share or table, you can either specify this
option followed by the SAS token, or you can specify the SAS as part of the destination blob
container, file share or table's URI, without this option.
If the source and destination are both blobs, then the destination blob must reside within the
same storage account as the source blob.
/SourceKey:"storage-key"
/SourceSAS:"sas-token"
Specifies a Shared Access Signature with READ and LIST permissions for the source (if
applicable). Surround the SAS with double quotes, as it may contains special command-line
characters.
If the source resource is a blob container, and neither a key nor a SAS is provided, then the
blob container is read via anonymous access.
/S
Specifies recursive mode for copy operations. In recursive mode, AzCopy copies all blobs or
files that match the specified file pattern, including those in subfolders.
Specifies whether the destination blob is a block blob, a page blob, or an append blob. This
option is applicable only when you are uploading a blob. Otherwise, an error is generated. If
the destination is a blob and this option is not specified, by default, AzCopy creates a block
blob.
/CheckMD5
Calculates an MD5 hash for downloaded data and verifies that the MD5 hash stored in the
blob or file's Content-MD5 property matches the calculated hash. The MD5 check is turned
off by default, so you must specify this option to perform the MD5 check when downloading
data.
Note that Azure Storage doesn't guarantee that the MD5 hash stored for the blob or file is up-
to-date. It is client's responsibility to update the MD5 whenever the blob or file is modified.
AzCopy always sets the Content-MD5 property for an Azure blob or file after uploading it to
the service.
/Snapshot
Indicates whether to transfer snapshots. This option is only valid when the source is a blob.
The transferred blob snapshots are renamed in this format: blob-name (snapshot-
time).extension
/V:[verbose-log-file]
/Z:[journal-file-folder]
If this option is not specified, or it is specified without a folder path, then AzCopy creates the
journal file in the default location, which is %LocalAppData%\Microsoft\Azure\AzCopy.
Each time you issue a command to AzCopy, it checks whether a journal file exists in the
default folder, or whether it exists in a folder that you specified via this option. If the journal
file does not exist in either place, AzCopy treats the operation as new and generates a new
journal file.
If the journal file does exist, AzCopy checks whether the command line that you input
matches the command line in the journal file. If the two command lines match, AzCopy
resumes the incomplete operation. If they do not match, you are prompted to either overwrite
the journal file to start a new operation, or to cancel the current operation.
Note that resuming an operation from a journal file created by a previous version of AzCopy
is not supported.
/@:"parameter-file"
Specifies a file that contains parameters. AzCopy processes the parameters in the file just as
if they had been specified on the command line.
In a response file, you can either specify multiple parameters on a single line, or specify each
parameter on its own line. Note that an individual parameter cannot span multiple lines.
Response files can include comments lines that begin with the # symbol.
You can specify multiple response files. However, note that AzCopy does not support nested
response files.
/Y
Suppresses all AzCopy confirmation prompts. This option also allows the use of write-only
SAS tokens for data upload scenarios, when /XO and /XN are not specified.
/L
AzCopy interprets the using of this option as a simulation for running the command line
without this option /L and counts how many objects are copied, you can specify option /V at
the same time to check which objects are copied in the verbose log.
The behavior of this option is also determined by the location of the source data and the
presence of the recursive mode option /S and file pattern option /Pattern.
AzCopy requires LIST and READ permission of this source location when using this option.
/MT
Sets the downloaded file's last-modified time to be the same as the source blob or file's.
/XN
Excludes a newer source resource. The resource is not copied if the last modified time of the
source is the same or newer than destination.
/XO
Excludes an older source resource. The resource is not copied if the last modified time of the
source is the same or older than destination.
/A
/IA:[RASHCNETOI]
Uploads only files that have any of the specified attributes set.
R = Read-only files
A = Files ready for archiving
S = System files
H = Hidden files
C = Compressed files
N = Normal files
E = Encrypted files
T = Temporary files
O = Offline files
I = Non-indexed files
/XA:[RASHCNETOI]
/Delimiter:"delimiter"
Indicates the delimiter character used to delimit virtual directories in a blob name.
By default, AzCopy uses / as the delimiter character. However, AzCopy supports using any
common character (such as @, #, or %) as a delimiter. If you need to include one of these
special characters on the command line, enclose the file name with double quotes.
/NC:"number-of-concurrent-operations"
AzCopy by default starts a certain number of concurrent operations to increase the data
transfer throughput. Note that large number of concurrent operations in a low-bandwidth
environment may overwhelm the network connection and prevent the operations from fully
completing. Throttle concurrent operations based on actual available network bandwidth.
/SourceType:"Blob" | "Table"
Specifies that the source resource is a blob available in the local development environment,
running in the storage emulator.
/DestType:"Blob" | "Table"
Specifies that the destination resource is a blob available in the local development
environment, running in the storage emulator.
Applicable to: Blobs, Tables
/PKRS:"key1#key2#key3#..."
Splits the partition key range to enable exporting table data in parallel, which increases the
speed of the export operation.
If this option is not specified, then AzCopy uses a single thread to export table entities. For
example, if the user specifies /PKRS:"aa#bb", then AzCopy starts three concurrent
operations.
Each operation exports one of three partition key ranges, as shown below:
[first-partition-key, aa)
[aa, bb)
[bb, last-partition-key]
/SplitSize:"file-size"
Specifies the exported file split size in MB, the minimal value allowed is 32.
If this option is not specified, AzCopy exports table data to a single file.
If the table data is exported to a blob, and the exported file size reaches the 200 GB limit for
blob size, then AzCopy splits the exported file, even if this option is not specified.
InsertOrSkip - Skips an existing entity or inserts a new entity if it does not exist in the table.
InsertOrMerge - Merges an existing entity or inserts a new entity if it does not exist in the
table.
InsertOrReplace - Replaces an existing entity or inserts a new entity if it does not exist in the
table.
/Manifest:"manifest-file"
Specifies the manifest file for the table export and import operation.
This option is optional during the export operation, AzCopy generates a manifest file with
predefined name if this option is not specified.
This option is required during the import operation for locating the data files.
/SyncCopy
Indicates whether to synchronously copy blobs or files between two Azure Storage endpoints.
AzCopy by default uses server-side asynchronous copy. Specify this option to perform a
synchronous copy, which downloads blobs or files to local memory and then uploads them to
Azure Storage.
You can use this option when copying files within Blob storage, within File storage, or from
Blob storage to File storage or vice versa.
/SetContentType:"content-type"
AzCopy sets the content type for a blob or file to application/octet-stream by default. You can
set the content type for all blobs or files by explicitly specifying a value for this option.
If you specify this option without a value, then AzCopy sets each blob or file's content type
according to its file extension.
/PayloadFormat:"JSON" | "CSV"
If this option is not specified, by default AzCopy exports table data file in JSON format.
When you copy blobs or files with AzCopy, keep in mind that another application may be
modifying the data while you are copying it. If possible, ensure that the data you are copying
is not being modified during the copy operation. For example, when copying a VHD
associated with an Azure virtual machine, make sure that no other applications are currently
writing to the VHD. A good way to do this is by leasing the resource to be copied.
Alternately, you can create a snapshot of the VHD first and then copy the snapshot.
If you cannot prevent other applications from writing to blobs or files while they are being
copied, then keep in mind that by the time the job finishes, the copied resources may no
longer have full parity with the source resources.
Enable FIPS-compliant MD5 algorithms for AzCopy when you "Use FIPS-
compliant algorithms for encryption, hashing and signing."
AzCopy by default uses .NET MD5 implementation to calculate the MD5 when copying
objects, but there are some security requirements that need AzCopy to enable FIPS-compliant
MD5 setting.
FIPS-compliant algorithms are disabled by default on Windows. You can change this policy
setting on your machine. In the Run window (Windows+R), type secpol.msc to open the
Local Security Policy window. In the Security Settings window, navigate to Security
Settings > Local Policies > Security Options. Locate the System cryptography: Use FIPS-
compliant algorithms for encryption, hashing and signing policy. Double-click on the
policy to see the value displayed in the Security Setting column.