Upgrade Path 4: From a Classic Configuration in a Datacenter to a Cloud-Native Configuration in a Datacenter

Overview

In this upgrade path:

Your source CloudBlue Commerce 20.5 installation is:
- System components installed on physical or virtual servers (OSS MN, BSS application server, and so on)
- System databases placed on physical or virtual servers
- DNS services installed on physical or virtual servers
- Application endpoints installed on physical or virtual servers
Your destination CloudBlue Commerce 21.0 installation is:
- System components installed in a Kubernetes cluster
- System databases placed on the same physical or virtual servers
- DNS services installed on the same physical or virtual servers
- Application endpoints installed on the same physical or virtual servers

To upgrade to CloudBlue Commerce 21.0, perform these steps:

Perform upgrade prechecks.
Prepare for the upgrade.
Deploy CloudBlue Commerce 21.0.
Perform post-deployment actions.

Step 1. Perform Upgrade Prechecks

Perform the instructions in Appendix A: Performing Upgrade Prechecks. If there are any errors, resolve them.

Important: If you use DNS services such as BIND or PowerDNS on your installation, prepare these services for the upgrade as described in this article.

Step 2. Prepare for the Upgrade

In your datacenter, prepare a Kubernetes cluster that has enough resources to run your CloudBlue Commerce 21.0 installation. To plan the resources of that cluster, see the CloudBlue Commerce 21.0 Deployment Guide.

When creating a new Kubernetes cluster, follow the recommendations and requirements in the CloudBlue Commerce 21.0 Deployment Guide.

Important: Pod Security Policies are not supported in the current CloudBlue Commerce version. Do not enable them in your cluster.
In your Kubernetes cluster, ensure that dynamic provisioning of persistent volumes is configured as described in the CloudBlue Commerce 21.0 Deployment Guide.
Ensure that your Kubernetes cluster and the servers of your CloudBlue Commerce 20.5 installation have network access to each other.

Ensure that your firewall configuration allows these connections:

Source	Destination	Ports (TCP)
Your OSS MN	Your Kubernetes cluster	8440, 5224, and 6308
Your APS endpoint servers	Your Kubernetes cluster	8440, 5224, and 6308
Your Kubernetes cluster	Your APS endpoint servers	443 or any other ports used by application endpoints To obtain all ports used by application endpoints, see the endpoint_uri column in the aps_application table of the oss database.
Your Kubernetes cluster	Your DNS servers	22 (for BIND) and 443 (for PowerDNS)
Your Kubernetes cluster	Your database servers	5432
Your internal systems integrated with CloudBlue Commerce (if any)	Your Kubernetes cluster	8440, 5224, and 6308
Your Kubernetes cluster	Your internal systems integrated with CloudBlue Commerce	Ports used by your internal systems (if any)

If you use custom domain registration, fraud-screening and payment plugins on your CloudBlue Commerce 20.5 installation, contact the developers of those plugins and obtain versions that are compatible with CloudBlue Commerce 21.0. You can install compatible versions of those plugins before or after the upgrade. To obtain the instructions on how to install domain registration, fraud-screening, and payment plugins, please refer to the CloudBlue Commerce 20.5 or CloudBlue Commerce 21.0 documentation.

Note: If you install them before the upgrade, the files of those plugins will be automatically copied by the upgrade scripts.
During the upgrade, the IP addresses of the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be changed to new ones. We recommend that you create DNS records for the current IP addresses and replace these IP addresses with the corresponding DNS names in the configuration of your application endpoints, custom scripts, and internal systems integrated with CloudBlue Commerce. After the upgrade, you must update these DNS records so that they point to the new IP addresses of the APIs.

Notes:

- For application endpoints that are based on the APS PHP runtime, the URLs of the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be updated automatically in the default configuration files by the fix-aps-endpoints.py script during the upgrade.

- After the upgrade, the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be accessible through the oss-proxy and bss-xmlrpc LoadBalancer services.
Configure your SMTP server that is used for sending mail messages from CloudBlue Commerce to the Internet:
- If this is an external server, make sure that it can accept mail from the IP addresses of your Kubernetes cluster nodes.
- If you use your BSS application server as an SMTP server, complete these steps:
  1. Update the /etc/postfix/main.cf configuration file on the BSS application server:
```
inet_interfaces = all
mynetworks = CLUSTER_NODE_NETWORK/NETMASK, 127.0.0.1
```
  2. Restart the Postfix service:
```
systemctl restart postfix
```
  Note: Using the BSS application server as an SMTP server is not recommended. We recommend that you use a separate SMTP server. To learn how to specify an SMTP server for sending messages to the Internet in CloudBlue Commerce 21.0, see the CloudBlue Commerce 21.0 Deployment Guide.
Allow access from your Kubernetes cluster to your database servers:
1. On your OSS database server, add the following lines to the ~postgres/11/data/pg_hba.conf file:
```
# MIGRATION
host    all     all     CLUSTER_NODE_NETWORK/NETMASK       md5
```
2. On your BSS database server, add the following lines to the ~postgres/11/data/pg_hba.conf file:
```
# MIGRATION
host    all     all     CLUSTER_NODE_NETWORK/NETMASK       md5
```
3. On both servers, reload the PostgreSQL service by executing this command:
```
service postgresql-11 reload
```
On your OSS MN, add the admin credentials of your Kubernetes cluster to your kubeconfig file (/root/.kube/config).
On your OSS MN, download the cnc-migration-latest.tgz archive. Then, unpack it and go to the CNC-MIGRATION directory:
```
tar xzvf CNC-MIGRATION.tgz
cd CNC-MIGRATION
```
In the CNC-MIGRATION directory, run the preparation.py script. This script collects the configuration settings of the installed CloudBlue Commerce 20.5 components and prepares the config.yaml file that you will use to install CloudBlue Commerce 21.0 components. To run the script, execute these commands:
```
python preparation.py
```
In the CNC-MIGRATION directory, edit the config.yaml file:
1. Specify the credentials of the postgres admin user in the components.a8n-db.admin_user and components.a8n-db.admin_password settings.
2. Define the format for hostnames of automatically created ingress rules. You can use one of these formats:
  - PREFIX.NAMESPACE_OF_YOUR_CBC_21_INSTALLATION.CLUSTER_DOMAIN
  - PREFIX.CLUSTER_DOMAIN_OVERRIDE
  where:
  - PREFIX is a service-specific auto-generated prefix.
  - NAMESPACE_OF_YOUR_CBC_21_INSTALLATION is the namespace of your CloudBlue Commerce 21.0 installation.
  - CLUSTER_DOMAIN is a domain specified in the cluster_domain setting.
  - CLUSTER_DOMAIN_OVERRIDE is a domain specified in the cluster_domain_override setting.
  If you specify a domain in the cluster_domain setting only, the first format will be used. If you specify a domain in the cluster_domain_override setting, the second format will be used regardless of whether the cluster_domain setting is specified or not.
  
  Examples of hostnames of automatically created ingress rules:
  - aps.cbc.example.com
  - aps.example.com
  Notes:
  
  - During the upgrade, an ingress rule for the incoming APS proxy will be automatically created in your Kubernetes cluster.
  
  - To learn more about the incoming APS proxy, please refer to the APS SDK documentation.
3. Review and adjust the values of the requests and limits settings of the OSS and BSS components.
On your OSS MN, install the necessary tools:
- Helm 3.5.4 or later
  
  Note: In the CloudBlue Commerce documentation, command options are provided for Helm 3. Helm 2 can also be used, but there may be differences in command options. If you already have Helm 2 installed, you can either migrate from it to Helm 3 or take into account differences in command options. For example, if a Helm 3 command cannot be performed due to such differences, you can find suitable Helm 2 command options.
- kubectl
In the CNC-MIGRATION directory, run the generate-secrets.py scripts. This script creates a namespace for CloudBlue Commerce 21.0 and secrets with database access credentials in your Kubernetes cluster. To run the script, execute these commands:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
python generate-secrets.py
```
In the CNC-MIGRATION directory, run the copy-persistent-data.py script. In your Kubernetes cluster, this script creates persistent volumes for the OSS and BSS components, and a pod named temporary-pod. It then mounts those persistent volumes to that pod, and copies APS packages, skins, and keystore files from your OSS MN and BSS application server to those persistent volumes. To run the script, execute these commands:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
python copy-persistent-data.py
```
In the provider control panel, go to Settings > Public API > the Allowed Networks tab and allow access to the OSS XML-RPC API from:
- the network used by pods of your Kubernetes cluster
- the network used by the nodes of your Kubernetes cluster
If you have APS endpoints whose DNS names are resolved through specific DNS servers, for example, internal DNS servers, specify these DNS servers as forward servers for those DNS names in the DNS service of your Kubernetes cluster. To learn how to do this, please refer to the Kubernetes documentation. For example, add customizations similar to this to the configuration of the DNS service of your Kubernetes cluster:
```
my.server: |
  DNS_NAME_OF_APPLICATION_ENDPOINT:53 {
      errors
      cache 30
      forward . IP_ADDRESS_OF_DNS_SERVER
  }
```
Important: The DNS servers that you specify as forward servers must be accessible from your Kubernetes cluster.
If domain registrars used by domain registration plugins of your CloudBlue Commerce 20.5 installation use IP address restrictions for their APIs, add the public IP addresses of your Kubernetes cluster to the whitelists of those registrars.
If you have the Microsoft 365 application installed and its application endpoint server uses IP address restrictions, add the respective allow entries for the node network of your Kubernetes cluster. To do this, use these instructions.
During the upgrade, the old incoming APS proxy will be replaced with a new one. To prepare for this, perform these steps:
1. In your Kubernetes cluster, install ingress controllers as described in the CloudBlue Commerce 21.0 Deployment Guide.
2. Obtain the service IP address of the external ingress controller:
```
kubectl -n NAMESPACE_FOR_INFRASTRUCTURE_COMPONENTS get svc nginx-ingress-ingress-nginx-controller -o jsonpath="{.status.loadBalancer.ingress[].ip}"
```
3. According to the format for hostnames of automatically created ingress rules that you chose previously, create an A DNS record for the new incoming APS proxy:
  - aps.NAMESPACE_OF_YOUR_CBC_21_INSTALLATION.CLUSTER_DOMAIN A SERVICE_IP_OF_INGRESS_CONTROLLER
  - aps.CLUSTER_DOMAIN_OVERRIDE A SERVICE_IP_OF_INGRESS_CONTROLLER
  Important: After you create that DNS record, make sure that the DNS name of the new incoming APS proxy is resolved to the respective IP address on the CloudBlue Connect side.
In your Kubernetes cluster, install logging and monitoring solutions. For details, see the CloudBlue Commerce 21.0 Deployment Guide.
In your Kubernetes cluster, install a backup solution. For details, see the CloudBlue Commerce 21.0 Backup and Recovery Guide.

Step 3. Deploy CloudBlue Commerce 21.0

Perform upgrade prechecks again. For instructions, see Appendix A: Performing Upgrade Prechecks.
Back up the data and configuration of your CloudBlue Commerce 20.5 installation.
Suspend the daily billing process:
1. In the provider control panel, go to System > Settings.
2. In the System group, click Events and switch to the Scheduler tab.
3. Reschedule the DailyBillingProcessing task so that you have enough time to complete the upgrade.
Warning: Remember to resume the daily billing process after you complete the upgrade.

Check whether new APS packages were added to the /usr/local/pem/APS/packages directory on your OSS MN after the date you executed the copy-persistent-data.py script. To check this, use the following commands:

ls -lt /usr/local/pem/APS/packages | head -100

and

kubectl config use-context YOUR_CLUSTER
kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION exec -ti temporary-pod bash
ls -lt /var/oss/APS/packages/ | head 100

If the /usr/local/pem/APS/packages directory contains new APS packages, copy the directories and ZIP files of those APS packages to your Kubernetes cluster with the following commands:

kubectl config use-context YOUR_CLUSTER
kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION cp PACKAGE_DIRECTORY temporary-pod:/var/oss/APS/packages/
kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION cp PACKAGE_ZIP_FILE temporary-pod:/var/oss/APS/packages/

Stop all components of your CloudBlue Commerce 20.5 installation:

On your OSS MN, execute the following commands:

service pau stop
service pa-agent stop
 
chkconfig pau off
chkconfig pa-agent off

On your BSS application server, execute the following commands:

chkconfig pau off
chkconfig pba off
chkconfig pa-agent off
chkconfig ssm off

service pba stop
service pau stop
service pa-agent stop
service ssm stop

mv /usr/local/bm/etc/ssm.conf.d/global.conf /usr/local/bm/etc/ssm.conf.d/global.conf.inactive

In the oss database, execute the following SQL commands:

ALTER TABLE aps_resource_link ALTER COLUMN target_type_info_id SET NOT NULL, ALTER COLUMN source_app_id SET NOT NULL, ALTER COLUMN target_app_id SET NOT NULL;
 
CREATE INDEX IF NOT EXISTS "IX_aps_security_link_resource_type_info_id" ON aps_security_link(resource_type_info_id);

Note: Executing these SQL commands is necessary for large databases and may take a long time to complete. We recommend that you perform them in advance.

In the CNC-MIGRATION directory, run the pre-migration.py script. This script adds the URLs of CloudBlue Commerce 21.0 Helm repositories and disables online stores in the BSS database. To run the script, execute these commands:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
python pre-migration.py
```

Deploy CloudBlue Commerce 21.0 components in your Kubernetes cluster:

Make sure you are in the CNC-MIGRATION directory on your OSS MN.

Execute these commands:

export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
helm install helmfile-deploy a8n/helmfile-deploy \
    -n $NAMESPACE \
     --set configYaml=$(cat config.yaml | base64 -w0) \
     --set namespace=$NAMESPACE \
     --set dockerrepo=odindevops-a8n-docker-21-public.jfrog.io

Step 4. Perform Post-deployment Actions

In the CNC-MIGRATION directory, run the post-migration.py script. This script:
- copies UI configuration files and locales from your OSS MN and BSS application node to the appropriate persistent volumes,
- copies custom domain registration, fraud-screening and payment PHP plugins from your OSS MN and BSS application node to the persistent volumes of the Domain SDK, Payment SDK, and Fraud SDK components, and
- restarts the necessary CloudBlue Commerce components.
To run the script, execute the following commands:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
python post-migration.py
```

Set the namespace of your CloudBlue Commerce 21.0 installation as default in the context of your Kubernetes cluster:

kubectl config use-context YOUR_CLUSTER
kubectl config set-context --current --namespace=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION

In your Kubernetes cluster, check that the status of all APSInstance custom resources is Ready. To do this, execute the following commands:
```
kubectl config use-context YOUR_CLUSTER
kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION get apsinstance
```
If the status of an APSInstance custom resource is not Ready, you must fix this before performing the next step. For troubleshooting, see the logs of the following components: the component whose APSInstance custom resource is not ready, oss, and a8n-operator.
After the upgrade, the APS REST API and OSS XML-RPC API are accessible through the oss-proxy LoadBalancer service, and the BSS XML-RPC API is accessible through the bss-xmlrpc LoadBalancer service. This means that you must update the URLs of those APIs in the configuration files of your application endpoints, custom scripts, and systems integrated with CloudBlue Commerce.

Notes:

- For some applications, you may need to update those API URLs in their application instance settings through the provider control panel.

- If you created DNS records for the APS REST API, OSS XML-RPC API, and BSS XML-RPC API on your internal DNS servers, you must update IP addresses in those DNS records.

To update the API URLs in the default configuration files of the application endpoints that are based on the APS PHP runtime, use the fix-aps-endpoints.py script:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
python fix-aps-endpoints.py
```
Notes:

- The script will ask you to provide administrative credentials to access your APS application endpoint servers through SSH.

- The script automatically updates the /ets/hosts files on your APS application endpoint servers so that the hostnames in the API URLs point to the correct IP addresses.
If you have the Microsoft 365 application installed, perform these steps:
1. Obtain the load balancer IP address of the oss-proxy LoadBalancer service:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
kubectl get svc -n $NAMESPACE oss-proxy -o jsonpath={.status.loadBalancer.ingress[].ip}
```
2. Obtain the load balancer IP address of the bss-xmlrpc LoadBalancer service:
```
export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
kubectl get svc -n $NAMESPACE bss-xmlrpc -o jsonpath={.status.loadBalancer.ingress[].ip}
```
3. On your Microsoft 365 application endpoint server, add the following line to C:/Windows/System32/drivers/etc/hosts:
```
LOAD_BALANCER_IP_ADDRESS_OF_OSS_PROXY_SERVICE oss-node-0.oss-dns
```
  Note: The hostname of the first OSS pod is oss-node-0.oss-dns. The APS controller uses that hostname in HTTP headers when communicating with application endpoints. For the Microsoft 365 application endpoints to resolve that hostname, you must manually update C:/Windows/System32/drivers/etc/hosts. For application endpoints that are based on APS PHP runtime, the fix-aps-endpoints.py script automatically updates the /ets/hosts files on your APS application endpoint servers.
4. In settings of your Microsoft 365 application instances, specify the new IP addresses of the OSS and BSS APIs. For details, see the Microsoft 365 Provider's Guide.
Ensure that your BSS pods and your BSS application server use the same time zone:
- To check the time zone of your BSS pods, execute these commands:
```
kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION exec -ti BSS_POD_NAME bash
date
```
- To check the time zone of your BSS application server, execute the date command.
In the provider control panel, in the task manager, run the UX1 Marketplace ... Plan indexer periodic task.
In the provider control panel, ensure that the daily billing process has resumed.
During the upgrade, IP addresses used by brand domains will be replaced with the load balancer IP address of the branding-ui-cluster-ui LoadBalancer service created in your Kubernetes cluster. If you have brand domains whose DNS zones are not managed by CloudBlue Commerce, ensure that DNS records of those brand domains point to the correct IP address.
Important: Please note that upcoming End of Life has been announced for the XML RPC in CloudBlue Commerce. More details can be found here.

To run scripts that use the OSS XML-RPC API, BSS XML-RPC API, and APS REST API on your OSS MN, update the configuration of poaupdater:
```
python generate_configs.py --namespace NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
```
As a result, this script updates the following configuration files:
- /usr/local/pem/etc/Kernel.conf
- /usr/local/pem/etc/pleskd.props
- /usr/local/pem/etc/odbc.ini
These configuration files contain the database and API credentials necessary for poaupdater.

Also, the script updates the /usr/local/pem/APS/certificates/poa.pem certificate file. You or your scripts can use it to access the APS controller.
If you use a custom PDF conversion tool on your CloudBlue Commerce installation, perform the instructions in this article.
If your OSS MN and BSS application server are no longer used by your CloudBlue Commerce 21.0 installation, consider removing them or decreasing their capacity.

Note: You can decrease the capacity of your OSS MN, keep it, and then use it for running various scripts that use the OSS XML-RPC API, BSS XML-RPC API, and APS REST API. You can also create a separate physical or virtual server for running such scripts as described in Appendix B: Creating a Cloud Management Node.