Introduction

WSO2 API Manager (APIM) is a comprehensive open-source platform that enables organizations to design, publish, and manage APIs in a secure and scalable manner. With the release of WSO2 APIM 3.2.0, users have been able to enjoy a wide range of features and improvements, including enhanced security, improved user experience, and better integration with other WSO2 products. In this article, we will discuss the migration process from WSO2 APIM 3.2.0 with PostgreSQL Database and Identity Server as Key Manager Pack to WSO2 APIM 4.2 with PostgreSQL Database and Identity Server as Key Manager Pack.

The migration process from WSO2 APIM 3.2.0 to WSO2 APIM 4.2 is a complex task that requires careful planning and execution. The migration process involves several steps, including data backup, database migration, configuration migration, and testing. One of the key components of the migration process is the database migration, which involves migrating the data from the old database to the new database.

In this article, we will focus on the migration process from WSO2 APIM 3.2.0 with PostgreSQL Database and Identity Server as Key Manager Pack to WSO2 APIM 4.2 with PostgreSQL Database and Identity Server as Key Manager Pack. We will provide a step-by-step guide on how to migrate the database and configuration settings from the old version to the new version.

We will also discuss the benefits of upgrading to the latest version of WSO2 APIM, including improved performance, enhanced security, and better integration with other WSO2 products. Additionally, we will provide some best practices and tips for ensuring a successful migration process.

By the end of this article, readers will have a clear understanding of the migration process from WSO2 APIM 3.2.0 to WSO2 APIM 4.2 with Postgres Database and Identity Server as Key Manager Pack, as well as the benefits and best practices of upgrading to the latest version of WSO2 APIM.

1.1 –  Configure WSO2 Identity Server as the Key Manager

First of all, download and install WSO2 Identity server. As per the compatibility matrix of WSO2 API Manager version 3.2, the compatible version of WSO2 Identity Server is 5.10.0.

In order to download and install the WSO2 Identity server, please follow these steps:

1. Go to the following link: https://wso2.com/identity-and-access-management/previous-releases/
2. From the drop down menu, select the 5.10.0 version.
3. Download the software from the provided link.

Make sure to take the latest U2 update if updates are available. Extract the downloaded folder and move to, /repository/conf/deployment.toml and add the following configuration to it,

[server]
Offset = 1

By adding the above line to the deployment.toml it will increase the default port of the product by one.

To configure the WSO2 Identity server with WSO2 API manager , you should download the WSO2-IS connector which is compatible with the WSO2 API-M version. Mostly it’s better to move forward with WSO2 IS Connector for the WSO2 API-M U2 updated version. Extract the downloaded zip file and move to <wso2is-extensions-1.0.16>/dropins folder and copy the,

• wso2is.key.manager.core-1.0.16.jar
• wso2is.notification.event.handlers_1.0.16.jar

Into the  <IS_HOME>/repository/components/dropins directory. And also move to <wso2is-extensions-1.0.16>/webapps directory and copy the keymanager-operations.war into   the <IS_HOME>/repository/deployment/server/webapps directory.After that add the below configurations into the <IS_HOME>/repository/conf/deployment.toml to configure the traffic manager endpoints as below,

[[event_listener]]
id = "token_revocation"
type = "org.wso2.carbon.identity.core.handler.AbstractIdentityHandler"
name = "org.wso2.is.notification.ApimOauthEventInterceptor"
order = 1

[[resource.access_control]]
context = "(.)/keymanager-operations/user-info/claims(.)"
secure = true
http_method = "GET"
permissions = "/permission/admin/manage/identity/usermgt/list"
scopes = "internal_user_mgt_list"

[[resource.access_control]]
context = "(.*)/keymanager-operations/user-info/claims/generate"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/usermgt/list"
scopes = "internal_user_mgt_list"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/applicationmgt/create"
scopes = "internal_application_mgt_create"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "GET"
permissions = "/permission/admin/manage/identity/applicationmgt/view"
scopes = "internal_application_mgt_view"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "DELETE"
permissions = "/permission/admin/manage/identity/applicationmgt/delete"
scopes = "internal_application_mgt_delete"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "PUT"
permissions = "/permission/admin/manage/identity/applicationmgt/update"
scopes = "internal_application_mgt_update"

[[resource.access_control]]
context = "(.)/keymanager-operations/dcr/register(.)"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/applicationmgt/update"
scopes = "internal_application_mgt_update"

[tenant_context.rewrite]
custom_webapps = ["/keymanager-operations/"]

Then configure the event listener endpoint to publish controller events to the traffic manager.

[event_listener.properties]
notification_endpoint = "https://<tm.wso2.com>:9443/internal/data/v1/notify"
username = "${admin.username}"
password = "${admin.password}"
'header.X-WSO2-KEY-MANAGER' = "WSO2-IS"

1.2 –  Configuring WSO2 API-M with WSO2-IS

First of all, download and install WSO2 API Manager 3.2. Extract the zip file and move to <APIM_HOME>/repository/conf/deployment.toml add the following configuration. Before that make sure to take the latest U2 update if updates are available

[apim.key_manager]
service_url = "https://localhost:9444/services/"
username = "$ref{super_admin.username}"
password = "$ref{super_admin.password}"
type = "WSO2-IS"

Make sure to keep the type alias same as in the ‘header.X-WSO2-KEY-MANAGER’ property under the [event_listener.properties] tag in the <IS_HOME>/repository/conf/deployment.toml. 

And also comment below configurations in the  <APIM_HOME>/repository/conf/deployment.toml,

#[[event_listener]]
#id = "token_revocation"
#type ="org.wso2.carbon.identity.core.handler.AbstractIdentityHandler"
#name = "org.wso2.is.notification.ApimOauthEventInterceptor"
#order = 1

#[event_listener.properties]
#notification_endpoint = "https://localhost:${mgt.transport.https.port}/internal/data/v1/notif"
#username = "${admin.username}"
#password = "${admin.password}"
#'header.X-WSO2-KEY-MANAGER' = "default"

1.3 –  Configuring Databases to PostgreSQL

By default in both WSO2 API-M and WSO2-IS there’s H2 databases for storing user management and registry data. Follow the instructions below to change the database to postgreSQL.

We chose Postgres as the database because commonly developers used MySQL as the database. As a result, we tried the migration using Postgres to clarify the issues that arise and how to resolve them . So as in the below configurations we can set up the databases , in the migration process we have faced some problems with in the scripts , those will be discussed from the below sections;

1. Log in to the postgreSQL using a client (eg : psql), enter the following command on the terminal,
   psql -h localhost -u postgres -w
   Enter the default password as the PostgreSQL to access the PostgreSQL server. 
   
2. Create the databases as in the following,
   CREATE DATABASE apim_db;
   CREATE DATABASE shared_db;
   
3. Change the ownership of the created databases to newly created user as in below,
   ALTER DATABASE apim_db OWNER TO PostgreSQL;
   Repeat the same on shared_db also.
   
4. Give authorization to the user you use to access the databases as follows,
   rant all privileges on database apim_db to postgres;
   Repeat the same on shared_db also.
   
5. Basic setup is okay, so now exit from the postgres console. To exist use below command,
   Quit;
   
6. Download the PostgreSQL JDBC driver,then copy it to both the <API-M_HOME>/repository/components/lib/ and <IS-HOME>/repository/components/lib/
   
7. Execute below commands to create the tables on the created databases,
   psql -U postgres -d shared_db -f <API-M_HOME>/dbscripts/postgresql.sql -W
   By executing the above command it will create the tables related to shared_db.
   psql -U postgres -d apim_db -f <API-M_HOME>/dbscripts/apimgt/postgresql.sql  -W
   By executing the above command it will create the tables related to apim_db.
   
8. Now add the below configuration in the <APIM_HOME>/repository/conf/deployment.toml establish the connection to the database.

[database.shared_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/shared_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

[database.apim_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/apim_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

After that add the below configuration in the  <IS_HOME>/repository/conf/deployment.toml establish the connection to the database.

[database.shared_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/shared_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

[database.identity_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/apim_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

1.4 –  Exporting Key Store

Generate keystores, follow the below steps to export key stores,

1. Move to the  <IS_HOME>/repository/resources/security directory and copy 
   • client-truststore.jks
   • wso2carbon.jks
2. Replace those copied files into <API-M_HOME>/repository/resources/security
3. Open the terminal in the <API-M_HOME>/repository/resources/security directory and execute the following commands,
   • keytool -export -alias wso2carbon -keystore wso2carbon.jks -file sample.pem
   • keytool -import -alias wso2carbon1 -file sample.pem -keystore client-truststore.jks -storepass wso2carbon

Now start the Identity server and the API manager, to effect the changes done. Move to <IS_HOME>/bin/ and  <API-M_HOME>/bin/ directories and run the below command,

• Windows

wso2server.bat --run

• Linux / Mac OS

sh wso2server.sh

1.5 –  Scenarios Created on API-M 3.2

WSO2 API Manager is a powerful open-source platform that enables users to design, publish, and manage APIs in a secure and scalable manner. With WSO2 API Manager 3.2, creating APIs and managing them has become even easier. In this section, we will discuss how to create an API, how to invoke it, and how to create an application and subscribe to it using the Publisher Portal and Dev Portal. 

Below are the scenarios followed before the migration, API-M 3.2 to API-M 4.2 ;

Scenario 1 :

Created and published an API , let’s assume the API name as “A” , After “A” API subscribed to the default application with all the default configurations (token expiration time etc). Then generate the access token and invoke the “A” API and tested.

Scenario 2 :

Created and published an API , let’s assume the API name as “B” and also created an application called “MyApplication” , After “B” API subscribed to the “MyApplication” with all the default configurations (token expiration time etc). Then generate the access token and invoke the “B” API and tested.

Scenario 3 :

Created two users as “UserA” and “UserB” , in orderly “UserA” assigned Internal/publisher and Internal/creator roles. After “UserB” assigned the internal/subscriber role. Then logged to the publisher portal from “UserA” and created an API, let’s assume the API name as “C” , as next step logged to the developer portal from “UserB” and subscribed the “C” API to an application and tested.

Scenario 4 :

Created and published an API , let’s assume the API name as “D” and also created an application called “TestApplication” ,  then changed the “TestApplication” token validity period and subscribed an API to “TestApplication” and tested.

2. Migration Configurations of WSO2 APIM 4.2 with PostgreSQL Database with Identity Server as Key Manager Pack

2.1 –  Configure WSO2 Identity Server as the Key Manager

Make sure to take the latest U2 update if updates are available. Extract the downloaded folder and move to,  <IS_HOME>/repository/conf/deployment.toml and add the following configuration to it,

[server]
Offset = 1

By adding the above line to the deployment.toml it will increase the default port of the product by one.

To configure the WSO2 Identity server with WSO2 API manager , you should download the WSO2-IS connector which is compatible with the WSO2 API-M version. Mostly it’s better to move forward with WSO2 IS Connector.

Extract the downloaded zip file and move to <wso2is-extensions-1.6.8>/dropins folder and copy the,

• wso2is.key.manager.core-1.6.8.jar
• wso2is.notification.event.handlers_1.6.8.jar

Into the  <IS_HOME>/repository/components/dropins directory. And also move to <wso2is-extensions-1.6.8>/webapps directory and copy the keymanager-operations.war into   the <IS_HOME>/repository/deployment/server/webapps directory.

After that add the below configurations into the <IS_HOME>/repository/conf/deployment.toml to configure the traffic manager endpoints as below,

[[event_listener]]
id = "token_revocation"
type = "org.wso2.carbon.identity.core.handler.AbstractIdentityHandler"
name = "org.wso2.is.notification.ApimOauthEventInterceptor"
order = 1

[[resource.access_control]]
context = "(.)/keymanager-operations/user-info/claims(.)"
secure = true
http_method = "GET"
permissions = "/permission/admin/manage/identity/usermgt/list"
scopes = "internal_user_mgt_list"

[[resource.access_control]]
context = "(.*)/keymanager-operations/user-info/claims/generate"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/usermgt/list"
scopes = "internal_user_mgt_list"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/applicationmgt/create"
scopes = "internal_application_mgt_create"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "GET"
permissions = "/permission/admin/manage/identity/applicationmgt/view"
scopes = "internal_application_mgt_view"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "DELETE"
permissions = "/permission/admin/manage/identity/applicationmgt/delete"
scopes = "internal_application_mgt_delete"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "PUT"
permissions = "/permission/admin/manage/identity/applicationmgt/update"
scopes = "internal_application_mgt_update"

[[resource.access_control]]
context = "(.*)/keymanager-operations/dcr/register(.*)"
secure = true
http_method = "POST"
permissions = "/permission/admin/manage/identity/applicationmgt/update"
scopes = "internal_application_mgt_update"

[tenant_context.rewrite]
custom_webapps = ["/keymanager-operations/"]

Then configure the event listener endpoint to publish controller events to the traffic manager.

[event_listener.properties]
notification_endpoint = "https://<cp.wso2.com>:9443/internal/data/v1/notify"
username = "${admin.username}"
password = "${admin.password}"
'header.X-WSO2-KEY-MANAGER' = "WSO2-IS"

Make sure to remove the  keystore.primary configuration from the <IS_HOME>/repository/conf/deployment.toml file and add the following configurations to the deployment.toml to change the default encryption type.

[keystore]
userstore_password_encryption = "InternalKeyStore"

[system.parameter]
"org.wso2.CipherTransformation" = "RSA/ECB/OAEPwithSHA1andMGF1Padding"

[encryption]
internal_crypto_provider = "org.wso2.carbon.crypto.provider.KeyStoreBasedInternalCryptoProvider"

And copy the same database configurations as in the WSO2-IS  5.10, and download the PostgreSQL JDBC driver,then copy it to the <IS-HOME>/repository/components/lib/ directory.

[database.identity_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/apim_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

[database.shared_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/shared_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

Make sure to copy client-truststore.jks and wso2carbon.jks files in the <IS_HOME>/repository/resources/security directory on WSO2-IS 5.10 to WSO2-IS 6.0.

Now start the Identity server , to effect the changes done. Move to <IS_HOME>/bin/ directory and run the below command,

• Windows

wso2server.bat –run

• Linux / Mac OS

sh wso2server.sh

2.2 –  Migrate the API Manager Configurations

First of all, download and install WSO2 API Manager 4.2. Extract the zip file and move to <APIM_HOME>/repository/conf/deployment.toml add the following configuration. Before that make sure to take the latest U2 update if updates are available

Do not copy all the configurations into the API-M 3.2 to API-M 4.2 , because there are many changes with upgrade. First of all just move the database configurations same as in the API-M 3 ; as in the below implementation,

[database.shared_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/shared_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

[database.apim_db]
type = "postgre"
url = "jdbc:postgresql://localhost:5432/apim_db?useSSL=false"
username = "postgres"
password = "postgres"
driver = "org.postgresql.Driver"
validationQuery = "SELECT 1"

Then modify the name tag of the  [apim.gateway.environment] to “Production and Sandbox” in the <APIM_HOME>/repository/conf/deployment.toml by default it’s as “Default”. Check the below configuration,

[apim.gateway.environment]
name = “Production and Sandbox”

After adding this modification should add the gateway_lables in the  [apim.sync_runtime_artifacts.gateway] as in the below configuration,

[apim.sync_runtime_artifacts.gateway]
gateway_lables = “Production and Sandbox”

Via these gateway labels it’s capturing the gateway environments specified. 

Also add the below configuration on the <API-M_HOME>/repository/conf/deployment.toml file.

[apim.key_manager]
service_url = "https://localhost:9444/services/"
username = "$ref{super_admin.username}"
password = "$ref{super_admin.password}"
type = "WSO2-IS"

Make sure to keep the type alias same as in the ‘header.X-WSO2-KEY-MANAGER’ property under the [event_listener.properties] tag in the <IS_HOME>/repository/conf/deployment.toml.

After adding those configurations download the PostgreSQL JDBC driver,and add it to the <API-M_HOME>/repository/components/lib/ directory. 

If there’s added dropins files on the  <API-M_3.2.0_HOME>/repository/components/dropins directory, copy those to the <API-M_4.2.0_HOME>/repository/components/dropins directory.

Copy the keystores in the <API-M_3.2.0_HOME>/repository/resources/security directory to the <API-M_4.2.0_HOME>/repository/resources/security directory.

2.3 – Challenges Face on Migration

Make sure to have a backup of all the databases which were created on API-M 3.2.0 . Before running the migration contact the WSO2 support team to get the required documents and attachments needed to process the migration.  

The latest version of WSO2 APIM, version 4.2, comes with a number of improvements and new features that make it even more powerful and flexible than its predecessor. However, upgrading from an older version of WSO2 APIM, such as version 3.2.0, can present some challenges, especially when using Postgres Database with Identity Server as Key Manager Pack. In this section, we will discuss some of the key challenges that may face when migrating to WSO2 APIM 4.2 and how to overcome them.

Challenge 1: Compatibility issues with Postgres Database

One of the major challenges we face when upgrading to WSO2 APIM 4.2 is compatibility issues with Postgres Database. This is because WSO2 APIM 4.2 requires Postgres Database version 10 or higher, while older versions of WSO2 APIM may be using earlier versions of Postgres Database.

To overcome this challenge, we must first ensure that they are using Postgres Database version 10 or higher. They can then migrate their existing data to the new version of Postgres Database before upgrading to WSO2 APIM 4.2. Alternatively, they can perform a fresh installation of WSO2 APIM 4.2 and configure it to use the existing Postgres Database instance.  And also there may cause some script issues because older versions of Postgres with keywords, to resolve them should find-tune the keywords compatible for the new version.

Challenge 2: Configuration changes in Identity Server as Key Manager Pack

Another challenge that organizations may face when upgrading to WSO2 APIM 4.2 is configuration changes in Identity Server as Key Manager Pack. WSO2 APIM 4.2 uses a new version of Identity Server, which comes with a number of changes to its configuration parameters.

To overcome this challenge, we must first identify the configuration changes that need to be made in Identity Server as Key Manager Pack. They can then update the configuration parameters accordingly and test the new configuration to ensure that it works as expected.

Challenge 3: Copying the Userstores

Make sure to copy the Userstores if you created on WSO2 API-M to the WSO2-IS basically it’s in <APIM_HOME>/repository/deployment/server/userstores directory so should copy those userstores to <IS_HOME>/repository/deployment/server/userstores directory. If not there’s no connectivity in between the created userstores.

In conclusion, upgrading from WSO2 APIM 3.2.0 with Postgres Database with Identity Server as Key Manager Pack to WSO2 APIM 4.2 with Postgres Database with Identity Server as Key Manager Pack can present some challenges. However, by identifying these challenges and taking appropriate measures to overcome them, we can successfully migrate to the latest version of WSO2 APIM and take advantage of its new features and improvements.

2.4 – Benefits and Expectations of Migration

After successfully migrating from WSO2 APIM 3.2.0 with Postgres Database with Identity Server as Key Manager Pack to WSO2 APIM 4.2 with Postgres Database with Identity Server as Key Manager Pack, we can take advantage of the new features and improvements offered by the latest version of WSO2 APIM. In this section, we will discuss some of the benefits of upgrading to WSO2 APIM 4.2 and what organizations can expect after the migration process.

Benefits of upgrading to WSO2 APIM 4.2

WSO2 APIM 4.2 comes with a number of new features and improvements that make it even more powerful and flexible than its predecessor. Some of the key benefits of upgrading to WSO2 APIM 4.2 include:

1. Improved performance and scalability: WSO2 APIM 4.2 is designed to handle high traffic loads and can scale easily to meet the needs of growing organizations.

2. Enhanced security: WSO2 APIM 4.2 comes with improved security features, including OAuth 2.0 support and enhanced encryption capabilities.

3. New developer-friendly features: WSO2 APIM 4.2 includes new developer-friendly features such as API auto-discovery and automated API testing.

4. Improved user experience: WSO2 APIM 4.2 includes a number of improvements to its user interface, making it easier for users to manage and monitor their APIs.

What to expect after the migration process

After successfully migrating to WSO2 APIM 4.2, organizations can expect to see a number of changes and improvements in their API management system. Some of the key changes that organizations can expect include:

1. New user interface: WSO2 APIM 4.2 comes with a new user interface that is designed to be more intuitive and user-friendly.

2. Updated API gateway: The API gateway in WSO2 APIM 4.2 has been updated to provide better performance and scalability.

3. New security features: WSO2 APIM 4.2 comes with new security features, including OAuth 2.0 support and enhanced encryption capabilities.

4. Improved API testing: WSO2 APIM 4.2 includes automated API testing capabilities, making it easier for developers to test their APIs.

5. Better performance and scalability: WSO2 APIM 4.2 is designed to handle high traffic loads and can scale easily to meet the needs of growing organizations.

6. Improved analytics: WSO2 APIM 4.2 includes improved analytics capabilities, making it easier for organizations to monitor and analyze their API traffic.

In conclusion, migrating from WSO2 APIM 3.2.0 with Postgres Database with Identity Server as Key Manager Pack to WSO2 APIM 4.2 with Postgres Database with Identity Server as Key Manager Pack can provide organizations with a number of benefits, including improved performance, enhanced security, and new developer-friendly features. After the migration process, organizations can expect to see changes and improvements in their API management system, including a new user interface, updated API gateway, and improved analytics capabilities.