Tutorial: Migrate from Azure Database for PostgreSQL - Single Server to Flexible Server with the migration service

Prerequisites (offline)

Before you start your migration with migration service in Azure Database for PostgreSQL, you must fulfill the following prerequisites, which apply to offline migration scenarios.

Verify the source version

Source PostgreSQL version should be >= 9.5. If the source PostgreSQL version is less than 9.5, upgrade the source PostgreSQL version to 9.5 or higher before migration.

Target setup

• Azure Database for PostgreSQL flexible server must be deployed and properly configured in Azure before you begin the migration process.

• The SKU chosen for the Azure Database for PostgreSQL should correspond with the specifications of the source database to ensure compatibility and adequate performance.

• For detailed instructions on creating a new Azure Database for PostgreSQL, refer to the following link: Quickstart: Create server.

Network setup

Network setup is crucial for the migration service to function correctly. Ensure that the source PostgreSQL server can communicate with the target Azure Database for PostgreSQL server. The following network configurations are essential for a successful migration.

For information about network setup, visit Network guide for migration service.

Enable extensions

To ensure a successful migration by using the migration service in Azure Database for PostgreSQL, you might need to verify extensions to your source PostgreSQL instance. Extensions provide functionality and features that might be required for your application. Make sure that you verify the extensions on the source PostgreSQL instance before you initiate the migration process.

In the target instance of Azure Database for PostgreSQL - Flexible Server, enable supported extensions that are identified in the source PostgreSQL instance.

For more information, see Extensions in Azure Database for PostgreSQL.

Check the server parameters

These parameters aren't automatically migrated to the target environment and must be manually configured.

• Match server parameter values from the source PostgreSQL database to the Azure Database for PostgreSQL by accessing the Server parameters section in the Azure portal and manually updating the values accordingly.

• Save the parameter changes and, if necessary, restart the Azure Database for PostgreSQL flexible server to apply the new configuration.

Disable high availability (reliability) and read replicas in the target

• Disabling high availability (reliability) and read replicas in the target environment is essential. These features should be enabled only after the migration has been completed.

• By following these guidelines, you can help ensure a smooth migration process without the added variables introduced by high availability and read replicas. Once the migration is complete and the database is stable, you can proceed to enable these features to enhance the availability and scalability of your database environment in Azure.

Configure your Azure Database for PostgreSQL flexible server

• Create the target flexible server. For guided steps, refer to the quickstart Create an Azure Database for PostgreSQL flexible server using the portal.

• Allowlist extensions whose libraries must be loaded at server start. It's essential that the extension is on the allowlist before you initiate a migration.

• Check if the data distribution among a database's tables is skewed, with most of the data present in a single (or few) tables. If its skewed, the migration speed could be slower than expected. In this case, the migration speed can be increased by migrating the large table in parallel.

Configure the migration task

The migration service comes with a simple, wizard-based experience on the Azure portal. Here's how to start:

1. Open your web browser and go to the portal. To sign in, enter your credentials. The default view is your service dashboard.

2. Go to your Azure Database for PostgreSQL Flexible Server target.

3. In the Overview tab of the Flexible Server, on the left menu, scroll down to Migration and select it.

   

4. Select the Create button to start a migration from a single server to a flexible server. If this is your first time using the migration service, an empty grid appears with a prompt to begin your first migration.

   

   If you've already created migrations to your Flexible Server target, the grid contains information about migrations that were attempted to this target from the Single Server.

5. You go through a wizard-based series of tabs to create a migration into this Flexible Server target from different possible sources. By default, Source server type is set to Azure Database for PostgreSQL Single Server, which is the one we're interested in for this scenario.

Alternatively, you can initiate the migration process from the Azure Database for PostgreSQL Single Server.

1. Open your web browser and go to the portal. To sign in, you must enter your credentials. The default view is your service dashboard.

2. Upon selecting the Single Server, you can observe a migration-related banner in the Overview tab. Select Migrate now to get started.

   

3. You're taken to a page with two options. If you've already created a Flexible Server and want to use that as the target, choose Select existing, and select the corresponding Subscription, Resource group, and Server name details. Once the selections are made, select Go to migration wizard and follow the instructions under the Setup section.

   

4. Should you choose to create a new Flexible Server, select Create new and select Go to create wizard. This action takes you through the Flexible Server creation process and deploys the Flexible Server.

   

After deploying the Flexible Server, follow the steps 3 to 5 under Configure the migration task.

Setup

The first tab is Setup. In case you missed it, allowlist necessary extensions as described in Configure your Azure Database for PostgreSQL flexible server, before you initiate a migration.

Migration name is the unique identifier for each migration to this Flexible Server target. This field accepts only alphanumeric characters and doesn't accept any special characters except for underscore (_) and hyphen (-). The name must start with an alphanumeric character. The name must also be unique for a target server, because no two migrations to the same Flexible Server target can have the same name.

Source server type indicates the source. In this case, it's Azure Database for PostgreSQL Single Server

Migration option allows you to perform validations before triggering a migration. You can pick any of the following options.

• Validate - Checks your server and database readiness for migration to the target.
• Migrate - Skips validations and starts migration.
• Validate and Migrate - Performs validation before triggering a migration. Migration gets triggered only if there are no validation failures.

It's always a good practice to choose Validate or Validate and Migrate option to perform premigration validations before running the migration.

Migration mode allows you to choose between an online and an offline migration, in this case it must be set to Offline.

Select the Next: Select Runtime Server button.

Runtime Server

The Migration Runtime Server is a specialized feature within the migration service in Azure Database for PostgreSQL, designed to act as an intermediary server during migration. It's a separate Azure Database for PostgreSQL - Flexible Server instance that isn't the target server but is used to facilitate the migration of databases from a source environment that is only accessible via a private network.

For more information about the Runtime Server, visit the Migration Runtime Server.

Select the Next: Connect to source button.

Connect to source

The Source section prompts you to give details related to the Single Server, which is the source of the databases.

After you make the Subscription and Resource Group selections, the dropdown list for server names shows Single Servers under that resource group across regions. Select the source that you want to migrate databases from.

After you choose the Single Server source, the Location, and PostgreSQL version boxes are populated automatically. Make sure you provide the credentials of an admin role, since that is required for the migration service to successfully migrate the databases.

The Custom FQDN/IP field is optional and can be used when the source is behind a custom DNS server or has custom DNS namespaces, making it accessible only via specific FQDNs or IP addresses. For example, this could include entries like singleserver.example.com, 198.1.0.2, or a PostgreSQL FQDN such as singleserver.postgres.database.chinacloudapi.cn, if the custom DNS server contains the DNS zone postgres.database.chinacloudapi.cn or forwards queries for this zone to 168.63.129.16, where the FQDN is resolved in the Azure public or private DNS zone.

After filling out all the fields, select the Connect to source link. This validates that the source server details entered are correct and that the source server is reachable.

Select the Next: Select migration target button to continue.

Select migration target

The Select migration target section displays metadata for the Flexible Server target, such as Subscription, Resource group, Server name, Location, and PostgreSQL version.

The Custom FQDN/IP field is optional and can be used when the target is behind a custom DNS server or has custom DNS namespaces, making it accessible only via specific FQDNs or IP addresses. For example, this could include entries like flexibleserver.example.com, 198.1.0.2, or a PostgreSQL FQDN such as flexibleserver.postgres.database.chinacloudapi.cn, if the custom DNS server contains the DNS zone postgres.database.chinacloudapi.cn or forwards queries for this zone to 168.63.129.16, where the FQDN is resolved in the Azure public or private DNS zone.

Make sure that the identity provided is that of the administrator user in the target server. After filling all required information, select the Connect to target link. This validates that the target server details entered are correct and target server is reachable.

Select the Next: Select database(s) for migration button to select the databases to migrate.

Select database(s) for migration

Under this tab, there's a list of user databases inside the Single Server. You can select and migrate up to eight databases in a single migration attempt. If there are more than eight user databases, the migration process is repeated between the source and target servers for the next set of databases. Selected databases that exist on the target server with the exact same names are overwritten.

Select the Next: Summary button to review the details.

Summary

The Summary tab summarizes all the details for creating the validation or migration. Review the details and select the Start Validation and Migration button.

Monitor the migration portal

After you start the migration, a notification appears to say that the validation or migration creation is successful. You're redirected automatically to the Migration page of Flexible Server. This has a new entry for the recently created validation or migration.

The grid that displays the migrations has these columns: Name, Status, Migration mode, Migration type, Source server, Source server type, Databases, Start time and Duration. The entries are displayed in the descending order of the start time with the most recent entry on the top.

You can use the Refresh button to refresh the status of the validation or migration.

You can also select the name of one particular migration in the grid to see the associated details.

When the validation or migration is created, it moves to the InProgress state and PerformingPreRequisiteSteps substate. The workflow takes 2-3 minutes to set up the migration infrastructure and network connections.

Let us look at how to monitor migrations for each migration option.

Validate

After the PerformingPreRequisiteSteps substate is completed, the validation moves to the substate of Validation in Progress where checks are done on the source and target server to assess the readiness for migration.

The validation moves to the Succeeded state if all validations are either in Succeeded or Warning state.

The validation grid has the following information:

• Validation details for instance and Validation details for databases sections, representing the validation rules used to check migration readiness.
• Validation Name - The name of each specific validation rule.
• Validation Status - Represents the result for each rule and can have any of the three values:
  • Succeeded - If no errors were found.
  • Failed - If there are validation errors.
  • Warning - If there are validation warnings.
• Duration - Time taken for the validation operation.
• Start time (UTC) and End time (UTC) - Start and end times of the validation operation in UTC.

The Validation status moves to Failed state if there are any errors in the validation. Select the Validation name or Database name validation that has failed, and a fan-out pane gives the details and the corrective action you should take to avoid this error.

Migrate

After the PerformingPreRequisiteSteps substate is completed, the migration moves to the substate of Migrating Data when the cloning/copying of the databases takes place. The time for migration to complete depends on the size and shape of the databases you're migrating. The migration is quick if the data is mostly evenly distributed across all the tables. Skewed table sizes take a relatively longer time.

When you select any of the databases in migration, a fan-out pane appears. It has all the table counts (copied, queued, copying, and errors) and also the database migration status.

The migration moves to the Succeeded state when the Migrating Data state finishes successfully. If there's an issue at the Migrating Data state, the migration moves into a Failed state.

Once the migration moves to the Succeeded state, schema and data migration from your Single Server to your Flexible Server target is complete. You can refresh the page to check the progress.

Validate and Migrate

In this option, validations are performed first before migration starts. After the PerformingPreRequisiteSteps substate is completed, the workflow moves into the substate of Validation in Progress.

• If validation has errors, the migration moves into a Failed state.
• If validation is complete without any error, the migration starts, and the workflow moves into the substate of Migrating Data.

You can see the results of Validate and Migrate once the operation is complete.

Prerequisites (online)

Before you start your migration with migration service in Azure Database for PostgreSQL, fulfilling the following prerequisites, which apply to offline migration scenarios is essential.

Verify the source version

Source PostgreSQL version should be >= 9.5. If the source PostgreSQL version is less than 9.5, upgrade the source PostgreSQL version to 9.5 or higher before migration.

Set up online migration parameters

For Online migration, the replication support should be set to Logical under replication settings of the source PostgreSQL server. In addition, the server parameters max_wal_senders and max_replication_slots values should be more than the number of Databases that need to be migrated. The parameters can be set in the Azure portal under Settings->Server Parameters or configured in the command line using the following commands:

• ALTER SYSTEM SET wal_level = logical;
• ALTER SYSTEM SET max_wal_senders = number of databases to migrate + 1;
• ALTER SYSTEM SET max_replication_slots = number of databases to migrate + 1;

Ensure that there are no long running transactions. Long running transactions don't allow creation of replication slots. The creation of a replication slot will succeed if all long running transactions are committed or rolled-back. You'll need to restart the source PostgreSQL server after completing all the Online migration prerequisites.

To prevent the Online migration from running out of storage to store the logs, ensure that you have sufficient tablespace space using a provisioned managed disk. To achieve this, disable the server parameter azure.enable_temp_tablespaces_on_local_ssd on the Flexible Server for the duration of the migration, and restore it to the original state after the migration.

Set up target

• Azure Database for PostgreSQL must be set up in Azure before migration.

• The SKU chosen for the Azure Database for PostgreSQL should correspond with the specifications of the source database to ensure compatibility and adequate performance.

• For detailed instructions on creating a new Azure Database for PostgreSQL, refer to the following link: Quickstart: Create server.

• The server parameter max_replication_slots should be more than the number of Databases that need to be migrated. It can be set in the Azure portal under Settings->Server Parameters or configured in the command line using the following command:

• ALTER SYSTEM SET max_replication_slots = number of databases to migrate + 1;

Set up Network

Network setup is crucial for the migration service to function correctly. Ensure that the source PostgreSQL server can communicate with the target Azure Database for PostgreSQL server. The following network configurations are essential for a successful migration.

For information about network setup, visit Network guide for migration service.

Enable extensions

To ensure a successful migration by using the migration service in Azure Database for PostgreSQL, you might need to verify extensions to your source PostgreSQL instance. Extensions provide functionality and features that might be required for your application. Make sure that you verify the extensions on the source PostgreSQL instance before you initiate the migration process.

In the target instance of Azure Database for PostgreSQL - Flexible Server, enable supported extensions that are identified in the source PostgreSQL instance.

For more information, see Extensions in Azure Database for PostgreSQL.

Server parameters

These parameters aren't automatically migrated to the target environment and must be manually configured.

• Match server parameter values from the source PostgreSQL database to the Azure Database for PostgreSQL by accessing the "Server parameters" section in the Azure portal and manually updating the values accordingly.

• Save the parameter changes and restart the Azure Database for PostgreSQL to apply the new configuration if necessary.

Disable high availability (reliability) and read replicas in the target

• Disabling high availability (reliability) and reading replicas in the target environment is essential. These features should be enabled only after the migration has been completed.

• By following these guidelines, you can help ensure a smooth migration process without the added variables introduced by HA and Read Replicas. Once the migration is complete and the database is stable, you can proceed to enable these features to enhance the availability and scalability of your database environment in Azure.

Configure your Azure Database for PostgreSQL flexible server

• Create the target flexible server. For guided steps, refer to the quickstart Create an Azure Database for PostgreSQL flexible server using the portal.

• Allowlist extensions whose libraries must be loaded at server start. It's essential that the extension is on the allowlist before you initiate a migration.

• Check if the data distribution among a database's tables is skewed, with most of the data present in a single (or few) tables. If its skewed, the migration speed could be slower than expected. In this case, the migration speed can be increased by migrating the large table in parallel.

Configure the migration task

The migration service comes with a simple, wizard-based experience on the Azure portal. Here's how to start:

1. Open your web browser and go to the portal. To sign in, enter your credentials. The default view is your service dashboard.

2. Go to your Azure Database for PostgreSQL Flexible Server target.

3. In the Overview tab of the Flexible Server, on the left menu, scroll down to Migration and select it.

   

4. Select the Create button to start a migration from a single server to a flexible server. If this is your first time using the migration service, an empty grid appears with a prompt to begin your first migration.

   

   If you've already created migrations to your Flexible Server target, the grid contains information about migrations that were attempted to this target from the Single Server.

5. You go through a wizard-based series of tabs to create a migration into this Flexible Server target from different possible sources. By default, Source server type is set to Azure Database for PostgreSQL Single Server, which is the one we're interested in for this scenario.

Alternatively, you can initiate the migration process from the Azure Database for PostgreSQL Single Server.

1. Open your web browser and go to the portal. To sign in, you must enter your credentials. The default view is your service dashboard.

2. Upon selecting the Single Server, you can observe a migration-related banner in the Overview tab. Select Migrate now to get started.

   

3. You're taken to a page with two options. If you've already created a Flexible Server and want to use that as the target, choose Select existing, and select the corresponding Subscription, Resource group, and Server name details. Once the selections are made, select Go to migration wizard and follow the instructions under the Setup section.

   

4. Should you choose to create a new Flexible Server, select Create new and select Go to create wizard. This action takes you through the Flexible Server creation process and deploys the Flexible Server.

   

After deploying the Flexible Server, follow the steps 3 to 5 under Configure the migration task.

Setup

The first tab is Setup. In case you missed it, allowlist necessary extensions as described in Configure your Azure Database for PostgreSQL flexible server, before you initiate a migration.

Migration name is the unique identifier for each migration to this Flexible Server target. This field accepts only alphanumeric characters and doesn't accept any special characters except for underscore (_) and hyphen (-). The name must start with an alphanumeric character. The name must also be unique for a target server, because no two migrations to the same Flexible Server target can have the same name.

Source server type indicates the source. In this case, it's Azure Database for PostgreSQL Single Server

Migration option allows you to perform validations before triggering a migration. You can pick any of the following options.

• Validate - Checks your server and database readiness for migration to the target.
• Migrate - Skips validations and starts migration.
• Validate and Migrate - Performs validation before triggering a migration. Migration gets triggered only if there are no validation failures.

It's always a good practice to choose Validate or Validate and Migrate option to perform premigration validations before running the migration.

Migration mode allows you to choose between an online and an offline migration, in this case it must be set to Online.

When Online migration mode is selected, Logical replication must be turned on in the source Single server. If it's not turned on, the migration service automatically turns on logical replication at the source Single server. Replication can also be set up manually by selecting the Replication item, under Settings in the resource menu of the Single server, and setting Azure replication support to LOGICAL. Either approach restarts the source single server.

Select the Next: Select Runtime Server button.

Runtime Server

The Migration Runtime Server is a specialized feature within the migration service in Azure Database for PostgreSQL, designed to act as an intermediary server during migration. It's a separate Azure Database for PostgreSQL - Flexible Server instance that isn't the target server but is used to facilitate the migration of databases from a source environment that is only accessible via a private network.

For more information about the Runtime Server, visit the Migration Runtime Server.

Select the Next: Connect to source button.

Connect to source

The Source section prompts you to give details related to the Single Server, which is the source of the databases.

After you make the Subscription and Resource Group selections, the dropdown list for server names shows Single Servers under that resource group across regions. Select the source that you want to migrate databases from.

After you choose the Single Server source, the Location, and PostgreSQL version boxes are populated automatically. Make sure you provide the credentials of an admin role, since that is required for the migration service to successfully migrate the databases.

After filling out all the fields, select the Connect to source link. This validates that the source server details entered are correct and that the source server is reachable.

Select the Next: Select migration target button to continue.

Select migration target

The Select migration target section displays metadata for the Flexible Server target, such as Subscription, Resource group, Server name, Location, and PostgreSQL version.

Make sure that the identity provided is that of the administrator user in the target server. After filling all required information, select the Connect to target link. This validates that the target server details entered are correct and target server is reachable.

Select the Next: Select database(s) for migration button to select the databases to migrate.

Select database(s) for migration

Under this tab, there's a list of user databases inside the Single Server. You can select and migrate up to eight databases in a single migration attempt. If there are more than eight user databases, the migration process is repeated between the source and target servers for the next set of databases. Selected databases that exist on the target server with the exact same names are overwritten.

Select the Next: Summary button to review the details.

Summary

The Summary tab summarizes all the details for creating the validation or migration. Review the details and select the Start Validation and Migration button.

Monitor the migration portal

After you start the migration, a notification appears to say that the validation or migration creation is successful. You're redirected automatically to the Migration page of Flexible Server. This has a new entry for the recently created validation or migration.

The grid that displays the migrations has these columns: Name, Status, Migration mode, Migration type, Source server, Source server type, Databases, Start time and Duration. The entries are displayed in the descending order of the start time with the most recent entry on the top.

You can use the Refresh button to refresh the status of the validation or migration.

You can also select the name of one particular migration in the grid to see the associated details.

When the validation or migration is created, it moves to the InProgress state and PerformingPreRequisiteSteps substate. The workflow takes 2-3 minutes to set up the migration infrastructure and network connections.

Let us look at how to monitor migrations for each migration option.

Validate

After the PerformingPreRequisiteSteps substate is completed, the validation moves to the substate of Validation in Progress where checks are done on the source and target server to assess the readiness for migration.

The validation moves to the Succeeded state if all validations are either in Succeeded or Warning state.

The validation grid has the following information:

• Validation details for instance and Validation details for databases sections, representing the validation rules used to check migration readiness.
• Validation Name - The name of each specific validation rule.
• Validation Status - Represents the result for each rule and can have any of the three values:
  • Succeeded - If no errors were found.
  • Failed - If there are validation errors.
  • Warning - If there are validation warnings.
• Duration - Time taken for the validation operation.
• Start time (UTC) and End time (UTC) - Start and end times of the validation operation in UTC.

The Validation status moves to Failed state if there are any errors in the validation. Select the Validation name or Database name validation that has failed, and a fan-out pane gives the details and the corrective action you should take to avoid this error.

Migrate

After the PerformingPreRequisiteSteps substate is completed, the migration moves to the substate of Migrating Data when the cloning/copying of the databases takes place. The time for migration to complete depends on the size and shape of the databases you're migrating. The migration is quick if the data is mostly evenly distributed across all the tables. Skewed table sizes take a relatively longer time.

When you select any of the databases in migration, a fan-out pane appears. It has all the table counts (copied, queued, copying, and errors) and also the database migration status.

The migration moves to the Succeeded state when the Migrating Data state finishes successfully. If there's an issue at the Migrating Data state, the migration moves into a Failed state.

Once the migration moves to the Succeeded state, schema and data migration from your Single Server to your Flexible Server target is complete. You can refresh the page to check the progress.

Validate and Migrate

In this option, validations are performed first before migration starts. After the PerformingPreRequisiteSteps substate is completed, the workflow moves into the substate of Validation in Progress.

• If validation has errors, the migration moves into a Failed state.
• If validation is complete without any error, the migration starts, and the workflow moves into the substate of Migrating Data.

You can see the results of Validate and Migrate once the operation is complete.

Cutover migration

For the Migrate and for the Validate and Migrate migration options, completion of the online migration requires the user to complete an additional step, which is to trigger the Cutover action. After the copy/clone of the base data is complete, the migration moves to the WaitingForUserAction state and the `WaitingForCutoverTrigger substate. In this state, user can trigger cutover from the portal by selecting the migration, and selecting the Cutover button.

Before initiating cutover, it's essential to ensure that:

• Writes to the source are stopped -Latency (minutes) parameter is 0 or close to 0 The Latency (minutes) information can be obtained from the migration details screen:

Latency (minutes) parameter indicates when the target last synced up with the source. For example, for the Orders database above, it's 0.33333. It means that the changes that occurred in the last ~0.3 minutes at the source are yet to be sent over to the target, for the Orders database. At this point, writes to the source can be stopped and cutover initiated. In case there's heavy traffic at the source, it's recommended to stop writes first so that Latency (minutes) can come close to 0, and only then initiate the cutover. The cutover operation applies all pending changes from the source to the target and completes the migration. If you trigger a cutover even with non-zero latency, the replication stops until that point in time. All the data on the source until the cutover point is then applied to the target. Say a latency was 15 minutes at the cutover point, so all the changed data in the last 15 minutes is applied to the target. Time taken depends on the backlog of changes that occurred in the last 15 minutes. Hence, it's recommended that the latency is zero or near zero, before triggering the cutover.

The migration moves to the Succeeded state as soon as the Migrating Data substate or the cutover (in online migration mode) finishes successfully. If there's a problem at the Migrating Data substate, the migration moves into a Failed state.

Possible migration states include:

• InProgress: The migration infrastructure setup is underway, or the actual data migration is in progress.
• Canceled: The migration is canceled or deleted.
• Failed: The migration has failed.
• Validation Failed : The validation has failed.
• Succeeded: The migration has succeeded and is complete.

Possible migration substates include:

• PerformingPreRequisiteSteps: Infrastructure set up is underway for data migration.
• Validation in Progress: Validation is in progress.
• MigratingData: Data migration is in progress.
• CompletingMigration: Migration is in final stages of completion.
• Completed: Migration has successfully completed.

Prerequisites (offline)

Before you start your migration with migration service in Azure Database for PostgreSQL, you must fulfill the following prerequisites, which apply to offline migration scenarios.

Verify the source version

Source PostgreSQL version should be >= 9.5. If the source PostgreSQL version is less than 9.5, upgrade the source PostgreSQL version to 9.5 or higher before migration.

Target setup

• Azure Database for PostgreSQL flexible server must be deployed and properly configured in Azure before you begin the migration process.

• The SKU chosen for the Azure Database for PostgreSQL should correspond with the specifications of the source database to ensure compatibility and adequate performance.

• For detailed instructions on creating a new Azure Database for PostgreSQL, refer to the following link: Quickstart: Create server.

Network setup

Network setup is crucial for the migration service to function correctly. Ensure that the source PostgreSQL server can communicate with the target Azure Database for PostgreSQL server. The following network configurations are essential for a successful migration.

For information about network setup, visit Network guide for migration service.

Enable extensions

To ensure a successful migration by using the migration service in Azure Database for PostgreSQL, you might need to verify extensions to your source PostgreSQL instance. Extensions provide functionality and features that might be required for your application. Make sure that you verify the extensions on the source PostgreSQL instance before you initiate the migration process.

In the target instance of Azure Database for PostgreSQL - Flexible Server, enable supported extensions that are identified in the source PostgreSQL instance.

For more information, see Extensions in Azure Database for PostgreSQL.

Check the server parameters

These parameters aren't automatically migrated to the target environment and must be manually configured.

• Match server parameter values from the source PostgreSQL database to the Azure Database for PostgreSQL by accessing the Server parameters section in the Azure portal and manually updating the values accordingly.

• Save the parameter changes and, if necessary, restart the Azure Database for PostgreSQL flexible server to apply the new configuration.

Disable high availability (reliability) and read replicas in the target

• Disabling high availability (reliability) and read replicas in the target environment is essential. These features should be enabled only after the migration has been completed.

• By following these guidelines, you can help ensure a smooth migration process without the added variables introduced by high availability and read replicas. Once the migration is complete and the database is stable, you can proceed to enable these features to enhance the availability and scalability of your database environment in Azure.

Get started

1. If you're new to Azure, create an account to evaluate the offerings.

2. Install the latest Azure CLI for your operating system from the Azure CLI installation page.

   If the Azure CLI is already installed, check the version by using the az version command. The version should be 2.50.0 or later to use the migration CLI commands. If not, update your Azure CLI version.

3. Run the az login command:

   az login
   

   A browser window opens with the Azure sign-in page. Provide your Azure credentials to do a successful authentication. For other ways to sign with the Azure CLI, refer this article.

Migration CLI commands (offline)

The migration service comes with easy-to-use CLI commands to do migration-related tasks. All the CLI commands start with az postgres flexible-server migration. It's important to allowlist the extensions before you initiate a migration.

For help with understanding the options associated with a command and with framing the right syntax, you can use the --help parameter:

az postgres flexible-server migration --help

Executing previous command returns the following output:

The output lists the supported migration commands, along with their actions. Let's look at these commands in detail.

Create a migration using the Azure CLI

The create command helps in creating a migration from a source server to a target server:

az postgres flexible-server migration create --help

Executing previous command returns the following output:

It lists the expected arguments and has an example syntax for successfully creating a migration from the source server to the target server. Here's the CLI command to create a new migration:

az postgres flexible-server migration create [--subscription]
                                             [--resource-group]
                                             [--name]
                                             [--migration-name]
                                             [--migration-mode]
                                             [--properties]

For example:

az postgres flexible-server migration create --subscription 11111111-1111-1111-1111-111111111111 --resource-group my-learning-rg --name myflexibleserver --migration-name migration1 --properties "./migrationBody.json" --migration-mode offline

The migration-name argument used in the create command is used in other CLI commands, such as update, delete, and show. It uniquely identifies the migration attempt in the corresponding actions in all those commands.

Finally, the create command needs a JSON file to be passed as part of its properties argument.

The structure of the JSON is:

{
	"properties": {
		"sourceDbServerResourceId": "/subscriptions/<subscriptionid>/resourceGroups/<sourceServerResourceGroup>/providers/Microsoft.DBforPostgreSQL/servers/<sourceServer>",
		"secretParameters": {
			"adminCredentials": {
				"sourceServerPassword": "<password>",
				"targetServerPassword": "<password>"
			},
			"sourceServerUserName": "<username>@<servername>",
			"targetServerUserName": "<username>"
		},
		"dbsToMigrate": [
			"<db1>",
			"<db2>"
		],
		"overwriteDbsInTarget": "true"
	}
}

The create parameters that go into the JSON file format are as shown below:

Note these essential points for the command response:

• When the create command is triggered, the migration moves to the InProgress state and the PerformingPreRequisiteSteps substate. The migration workflow takes a few minutes to deploy the migration infrastructure and set up connections between the source and target.
• After the PerformingPreRequisiteSteps substate is completed, the migration moves to the substate of Migrating Data, where the cloning/copying of the databases takes place.
• Each database migrated has its own section with all migration details, such as table count, incremental inserts, deletions, and pending bytes.
• The time that the Migrating Data substate takes to finish depends on the size of the databases that are migrated.
• The migration moves to the Succeeded state as soon as the Migrating Data substate finishes successfully. If there's a problem at the Migrating Data substate, the migration moves into a Failed state.

List the migrations

The list command lists all the migration attempts made to a Flexible Server target:

az postgres flexible-server migration list [--subscription]
                                           [--resource-group]
                                           [--name]
                                           [--filter]

The filter parameter has two options:

• Active: Lists the current active (in progress) migration attempts into the target server. It doesn't include the migrations that have failed, canceled, or succeeded.
• All: Lists all the migration attempts into the target server. This includes both the active and completed migrations, regardless of the state.

For more information about this command, use the --help parameter:

az postgres flexible-server migration list --help

Monitor the migration

The --show command helps you monitor ongoing migrations and gives the current state and substate of the migration.

az postgres flexible-server migration show [--subscription]
                                           [--resource-group]
                                           [--name]
                                           [--migration-name]

The migration_name parameter is the name assigned to the migration during the create command. Here's a snapshot of the sample response from the CLI command for showing details:

For more information about this command, use the --help parameter:

 az postgres flexible-server migration show --help

The following tables describe the migration states and substates.

Prerequisites (online)

Before you start your migration with migration service in Azure Database for PostgreSQL, fulfilling the following prerequisites, which apply to offline migration scenarios is essential.

Verify the source version

Source PostgreSQL version should be >= 9.5. If the source PostgreSQL version is less than 9.5, upgrade the source PostgreSQL version to 9.5 or higher before migration.

Set up online migration parameters

For Online migration, the replication support should be set to Logical under replication settings of the source PostgreSQL server. In addition, the server parameters max_wal_senders and max_replication_slots values should be more than the number of Databases that need to be migrated. The parameters can be set in the Azure portal under Settings->Server Parameters or configured in the command line using the following commands:

• ALTER SYSTEM SET wal_level = logical;
• ALTER SYSTEM SET max_wal_senders = number of databases to migrate + 1;
• ALTER SYSTEM SET max_replication_slots = number of databases to migrate + 1;

Ensure that there are no long running transactions. Long running transactions don't allow creation of replication slots. The creation of a replication slot will succeed if all long running transactions are committed or rolled-back. You'll need to restart the source PostgreSQL server after completing all the Online migration prerequisites.

To prevent the Online migration from running out of storage to store the logs, ensure that you have sufficient tablespace space using a provisioned managed disk. To achieve this, disable the server parameter azure.enable_temp_tablespaces_on_local_ssd on the Flexible Server for the duration of the migration, and restore it to the original state after the migration.

Set up target

• Azure Database for PostgreSQL must be set up in Azure before migration.

• The SKU chosen for the Azure Database for PostgreSQL should correspond with the specifications of the source database to ensure compatibility and adequate performance.

• For detailed instructions on creating a new Azure Database for PostgreSQL, refer to the following link: Quickstart: Create server.

• The server parameter max_replication_slots should be more than the number of Databases that need to be migrated. It can be set in the Azure portal under Settings->Server Parameters or configured in the command line using the following command:

• ALTER SYSTEM SET max_replication_slots = number of databases to migrate + 1;

Set up Network

Network setup is crucial for the migration service to function correctly. Ensure that the source PostgreSQL server can communicate with the target Azure Database for PostgreSQL server. The following network configurations are essential for a successful migration.

For information about network setup, visit Network guide for migration service.

Enable extensions

To ensure a successful migration by using the migration service in Azure Database for PostgreSQL, you might need to verify extensions to your source PostgreSQL instance. Extensions provide functionality and features that might be required for your application. Make sure that you verify the extensions on the source PostgreSQL instance before you initiate the migration process.

In the target instance of Azure Database for PostgreSQL - Flexible Server, enable supported extensions that are identified in the source PostgreSQL instance.

For more information, see Extensions in Azure Database for PostgreSQL.

Server parameters

These parameters aren't automatically migrated to the target environment and must be manually configured.

• Match server parameter values from the source PostgreSQL database to the Azure Database for PostgreSQL by accessing the "Server parameters" section in the Azure portal and manually updating the values accordingly.

• Save the parameter changes and restart the Azure Database for PostgreSQL to apply the new configuration if necessary.

Disable high availability (reliability) and read replicas in the target

• Disabling high availability (reliability) and reading replicas in the target environment is essential. These features should be enabled only after the migration has been completed.

• By following these guidelines, you can help ensure a smooth migration process without the added variables introduced by HA and Read Replicas. Once the migration is complete and the database is stable, you can proceed to enable these features to enhance the availability and scalability of your database environment in Azure.

Get started

1. If you're new to Azure, create an account to evaluate the offerings.

2. Install the latest Azure CLI for your operating system from the Azure CLI installation page.

   If the Azure CLI is already installed, check the version by using the az version command. The version should be 2.50.0 or later to use the migration CLI commands. If not, update your Azure CLI version.

3. Run the az login command:

   az login
   

   A browser window opens with the Azure sign-in page. Provide your Azure credentials to do a successful authentication. For other ways to sign with the Azure CLI, refer this article.

Migration CLI commands (online)

The migration service comes with easy-to-use CLI commands to do migration-related tasks. All the CLI commands start with az postgres flexible-server migration. It's important to allowlist the extensions before you initiate a migration.

For help with understanding the options associated with a command and with framing the right syntax, you can use the --help parameter:

az postgres flexible-server migration --help

Executing previous command returns the following output:

The output lists the supported migration commands, along with their actions. Let's look at these commands in detail.

Create a migration using the Azure CLI

The create command helps in creating a migration from a source server to a target server:

az postgres flexible-server migration create --help

Executing previous command returns the following output:

It lists the expected arguments and has an example syntax for successfully creating a migration from the source server to the target server. Here's the CLI command to create a new migration:

az postgres flexible-server migration create [--subscription]
                                             [--resource-group]
                                             [--name]
                                             [--migration-name]
                                             [--migration-mode]
                                             [--properties]

For example:

az postgres flexible-server migration create --subscription 11111111-1111-1111-1111-111111111111 --resource-group my-learning-rg --name myflexibleserver --migration-name migration1 --properties "./migrationBody.json" --migration-mode online

The migration-name argument used in the create command is used in other CLI commands, such as update, delete, and show. It uniquely identifies the migration attempt in the corresponding actions in all those commands.

Finally, the create command needs a JSON file to be passed as part of its properties argument.

The structure of the JSON is:

{
  "properties": {
    "sourceDbServerResourceId": "/subscriptions/<subscriptionid>/resourceGroups/<sourceServerResourceGroup>/providers/Microsoft.DBforPostgreSQL/servers/<sourceServer>",
    "secretParameters": {
      "adminCredentials": {
        "sourceServerPassword": "<password>",
        "targetServerPassword": "<password>"
      }
    },
    "sourceServerUserName": "<username>@<servername>",
    "targetServerUserName": "<username>",
    "dbsToMigrate": ["<db1>", "<db2>"],
    "overwriteDbsInTarget": "true"
  }
}

The create parameters that go into the JSON file format are as shown below:

Note these essential points for the command response:

• When the create command is triggered, the migration moves to the InProgress state and the PerformingPreRequisiteSteps substate. The migration workflow takes a few minutes to deploy the migration infrastructure and set up connections between the source and target.
• After the PerformingPreRequisiteSteps substate is completed, the migration moves to the substate of Migrating Data, where the cloning/copying of the databases takes place.
• Each database migrated has its own section with all migration details, such as table count, incremental inserts, deletions, and pending bytes.
• The time that the Migrating Data substate takes to finish depends on the size of the databases that are migrated.
• The migration moves to the Succeeded state as soon as the Migrating Data substate finishes successfully. If there's a problem at the Migrating Data substate, the migration moves into a Failed state.

Setup replication

For online migration mode, logical replication must be turned on in the source Single server. If it isn't turned on, the migration service automatically turns on logical replication at the source Single server when the setupLogicalReplicationOnSourceDBIfNeeded parameter is passed with a value of true in the accompanying JSON file. Replication can also be set up manually at the source after starting the migration using the command below. Either approach of turning on logical replication restarts the source Single server.

For example:

az postgres flexible-server migration update --subscription 11111111-1111-1111-1111-111111111111 --resource-group my-learning-rg --name myflexibleserver --migration-name CLIMigrationExample --setup-replication

This command is required to advance the migration when the flexible server is waiting in the WaitingForLogicalReplicationSetupRequestOnSourceDB state.

To perform online migration use:

az postgres flexible-server migration create --subscription 11111111-1111-1111-1111-111111111111 --resource-group my-learning-rg --name myflexibleserver --migration-name migration1 --properties "./migrationBody.json" --migration-mode online

List the migrations

The list command lists all the migration attempts made to a Flexible Server target:

az postgres flexible-server migration list [--subscription]
                                           [--resource-group]
                                           [--name]
                                           [--filter]

The filter parameter has two options:

• Active: Lists the current active (in progress) migration attempts into the target server. It doesn't include the migrations that have failed, canceled, or succeeded.
• All: Lists all the migration attempts into the target server. This includes both the active and completed migrations, regardless of the state.

For more information about this command, use the --help parameter:

az postgres flexible-server migration list --help

Monitor the migrations

The show command helps you monitor ongoing migrations and gives the current state and substate of the migration.

az postgres flexible-server migration show [--subscription]
                                           [--resource-group]
                                           [--name]
                                           [--migration-name]

The migration_name parameter is the name assigned to the migration during the create command. Here's a snapshot of the sample response from the CLI command for showing details:

For more information about this command, use the --help parameter:

 az postgres flexible-server migration show --help

The following tables describe the migration states and substates.

Cutover the migrations

In online migrations, after the base data migration is complete, the migration task moves to WaitingForCutoverTrigger substate. In this state, user can trigger cutover through CLI using the command below. The cutover can also be triggered from the portal by selecting the migration name in the migration grid.

For example:

az postgres flexible-server migration update --subscription 11111111-1111-1111-1111-111111111111 --resource-group my-learning-rg --name myflexibleserver --migration-name CLIMigrationExample --cutover

Before initiating cutover, it's essential to ensure that:

• Writes to the source are stopped -latency parameter is 0 or close to 0

latency parameter indicates when the target last synced up with the source. For example, here it's 201 and 202 for the two databases as shown in the picture below. It means that the changes that occurred in the last ~200 seconds at the source are yet to be synced to the target. At this point, writes to the source can be stopped and cutover initiated. In case there's heavy traffic at the source, it's recommended to stop writes first so that Latency can come close to 0, and then cutover is initiated. The Cutover operation applies all pending changes from the source to the Target and completes the migration. If you trigger a "Cutover" even with nonzero Latency, the replication stops until that point in time. All the data on the source until the cutover point is then applied to the target. Say a latency was 15 minutes at the cutover point, so all the change data in the last 15 minutes applies to the target. The time taken depends on the backlog of changes that occurred in the last 15 minutes. Hence, it's recommended that the latency goes to zero or near zero, before triggering the cutover. The latency information can be obtained using the migration show command.

Here's a snapshot of the migration before initiating the cutover:

After the cutover is initiated, all transactions that happened during the base copy are copied sequentially to the target, and migration is completed.

If the cutover isn't successful, the migration moves to the Failed state.

For more information about this command, use the --help parameter:

 az postgres flexible-server migration update --help