Create a migration job

Database Migration Service uses migration jobs to migrate data from your source databases to the Cloud SQL destination instance. Migration jobs help you organize the source and destination connection profiles, define settings specific to the migration process, monitor the progress, and safely finalize the whole operation.

Creating a migration job includes performing the following tasks:

  • Selecting source and destination connection profiles.

  • Selecting the conversion workspace and specifying objects you want to migrate.

  • Performing a migration job test to ensure that Database Migration Service is able to connect to your data source and destination.

  • Starting the migration job, and monitoring the progress.

  • Promoting the migration job when you want to switch your application to the new instance.

Before you begin

  1. Ensure you meet the following requirements:
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Enable the Database Migration Service and Cloud SQL Admin APIs.

    Enable the APIs

Required roles

To get the permissions that you need to create a migration job for heterogeneous SQL Server migrations, ask your administrator to grant the required IAM roles on your project for the following accounts involved in the migration process:

For more information about granting roles, see Manage access in the Identity and Access Management documentation.

These predefined roles contain the permissions required to perform heterogeneous SQL Server migrations with Database Migration Service. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to perform heterogeneous SQL Server migrations with Database Migration Service:

  • datamigration.*
  • resourcemanager.projects.get
  • resourcemanager.projects.list
  • cloudsql.instances.create
  • cloudsql.instances.get
  • cloudsql.instances.list
  • cloudsql.databases.get
  • cloudsql.databases.delete
  • cloudsql.operations.get
  • compute.machineTypes.list
  • compute.machineTypes.get
  • compute.projects.get

You might also be able to get these permissions with custom roles or other predefined roles.

Define settings and create a migration job

To create a migration job, perform the following steps:

  1. In the Google Cloud console, go to the Migration jobs page.

    Go to Migration jobs

  2. Click Create migration job.

    The migration job configuration wizard page opens. This wizard contains multiple panels that walk you through each configuration step.

    You can pause the creation of a migration job at any point by clicking SAVE & EXIT. All of the data that you enter up to that point is saved in a draft migration job. You can finish your draft migration job later. See Update a draft migration job.

  3. On the Get started page, enter the following information:
    1. Migration job name: a human-readable name for your migration job. This value is displayed in the Google Cloud console.
    2. Migration job ID: a machine-readable identifier for your migration job. You use this value to work with migration jobs by using Database Migration Service Google Cloud CLI commands or API.
    3. From the Source database engine list, select your SQL Server source database.
    4. From the Destination database engine drop-down menu, select Cloud SQL for PostgreSQL.
    5. Optional: If you want to manage your own data encryption key for the migration, expand the Advanced encryption options and do the following:
      1. Select the Cloud KMS key option.
      2. Leave the default Cloud KMS option for the Key type setting.

        Database Migration Service doesn't support the Cloud KMS with autokey feature.

      3. From the Select a customer-managed key drop-down menu, select your encryption key. You can also manually enter the fully qualified identifier for your key.
  4. Click Save and continue.
  5. On the Define your source page, perform the following actions:
    1. From the Source connection profile drop-down menu, select your source connection profile.

    2. In this section, you can choose how to perform the full dump phase of your migration The default value is Automatic. For more information on available settings, expand the Full dump settings section:

      Full dump settings

      You can use the following settings for making the full dump:

      • Automatic

        When the migration is started, Database Migration Service pulls existing data in addition to newly changed data from the included objects. This is the default option.

      • Customer-managed

        When the migration is started, Database Migration Service pulls only newly changed data from the included tables.

        Choose this option if you want to perform the full dump on your own.

        In the CDC start position (LSN) field, provide a SQL Server LSN (Log Sequence Number) to start the CDC from.

    3. In the Source read settings section, configure how many concurrent connections Database Migration Service can make to your source instance. For more information on available settings, expand the Source read settings section:

      Source read settings

      Database Migration Service adjusts the number of connections to ensure the best performance within provided connection limits. Increasing the maximum connection limit can improve the migration speed, but introduces additional load on your source databases.

      You can use the following settings:

      • Maximum concurrent full dump connections defines the maximum number of concurrent full dump connections. The actual number of concurrent connections might be lower at any particular time.

        Default: 50 connections

        Custom: minimum 1, maximum 50 connections

      • Maximum concurrent CDC connections

        Default: 5 connections

        Custom: minimum 1, maximum 50 connections
    4. Click Save and continue.
  6. On the Define a destination page, from the Destination connection profile drop-down menu, select your destination connection profile.
  7. In the Customize destination configuration section, configure the following settings:
    Maximum concurrent destination connections

    Default value: 128

    Allowed values: minimum 2, maximum 256

    You can customize how many maximum concurrent connections Database Migration Service can make to your destination instance.

    Database Migration Service adjusts the number of connections to ensure the best performance within provided connection limit. Increasing the maximum connection limit can improve the migration speed, but introduces additional load on your destination databases.

    Transaction timeout

    Default value: 30

    Allowed values: minimum 30, maximum 300

    During the migration process, Database Migration Service can encounter certain issues that cause the transaction to time out. You can adjust the number of seconds that Database Migration Service waits for the transaction to complete before it is canceled.

    Canceled transactions don't cause your migration job to fail. The migration job continues to copy data, but moves to the Running with errors status. You can view migration job details to check what issues need to be addressed.

  8. Click Save and continue.
  9. On the Configure migration objects page, perform the following actions:
    1. From the Conversion workspace drop-down menu, select your conversion workspace.
    2. In the Select objects to migrate section, use the checkboxes to select which objects you want to include in this migration job. This list is populated based on what objects you converted to PostgreSQL schema in the conversion workspace.
  10. Click Save and continue.
  11. On the Test and create migration job, perform the following actions:
    1. (Optional) Click Test job to verify if Database Migration Service can successfully identify all the backup files and establish the necessary network connections.

      If the test fails, you can consult the error messages to address the problem, and run the test again. For more information on possible errors, see Diagnose issues.

    2. Click Create and start job to begin your migration.

      If you want to perform the migration job at a different time, click Save and return later to run the job. See Start a migration job.

What's next