Upgrade Path 2: From Azure VMs to a Cloud-Native Configuration in Azure

Overview

In this upgrade path:

  • Your source CloudBlue Commerce 20.5 installation is:

    • System components installed in Azure VMs (OSS MN, BSS application node, and so on)

    • System databases placed in Azure VMs or in the Azure Database for PostgreSQL service

    • DNS services installed in Azure VMs

    • Application endpoints installed in Azure VMs

  • Your destination CloudBlue Commerce 21.0 installation is:

    • System components installed in an AKS cluster

    • System databases placed in the same Azure VMs or in the Azure Database for PostgreSQL service

    • DNS services installed in the same Azure VMs

    • Application endpoints installed in the same Azure VMs

To upgrade to CloudBlue Commerce 21.0, perform these steps:

  1. Perform upgrade prechecks.

  2. Prepare for the upgrade.

  3. Deploy CloudBlue Commerce 21.0.

  4. Perform post-deployment actions.

Step 1. Perform Upgrade Prechecks

Perform the instructions in Appendix A: Performing Upgrade Prechecks. If there are any errors, resolve them.

Important: If you use DNS services such as BIND or PowerDNS on your installation, prepare these services for the upgrade as described in this article.

Step 2. Prepare for the Upgrade

  1. In Azure, prepare an AKS cluster that has enough resources to run your CloudBlue Commerce 21.0 installation. To plan the resources of that cluster, see the CloudBlue Commerce 21.0 Deployment Guide.

    When creating a new AKS cluster, use the following values for its settings:

    • Availability zones: Select only one zone.

      Notes:

      - Communication between cluster nodes is faster when they are in the same availability zone.

      - Persistent volumes that are based on Azure disks can be accessed by pods only within the same availability zone.

    • Kubernetes version: Any Kubernetes version supported by CloudBlue Commerce

    • Node size: Standard D8as v4

    • Scale method: Manual

    • Node count: 4

    To learn how to create AKS clusters, please refer to the Azure documentation.

    Important: Pod Security Policies are not supported in the current CloudBlue Commerce version. Do not enable them in your cluster.

  2. Ensure that your AKS cluster has access to the subnets of your VMs. To do this, you can:

    • Create a separate subnet for your AKS cluster in the virtual network of your VMs.

      Important: For your AKS cluster, you must not use an existing subnet already used by your VMs.

    • Create a peering between your AKS cluster's virtual network and the virtual network of your VMs.

  3. Ensure that the service principal of your AKS cluster has sufficient permissions to create internal load balancers in its subnet. You can learn more about this in the Azure documentation.

  4. Ensure that your firewall configuration allows these connections:

    Source Destination Ports (TCP)
    Your OSS MN Your AKS cluster 8440, 5224, and 6308
    Your APS endpoint VMs Your AKS cluster 8440, 5224, and 6308
    Your AKS cluster Your APS endpoint VMs

    443 or any other ports used by application endpoints

    To obtain all ports used by application endpoints, see the endpoint_uri column in the aps_application table of the oss database.

    Your AKS cluster Your DNS nodes 22 (for BIND) and 443 (for PowerDNS)
    Your AKS cluster Your database servers 5432
    Your internal systems integrated with CloudBlue Commerce (if any) Your AKS cluster 8440, 5224, and 6308
    Your AKS cluster Your internal systems integrated with CloudBlue Commerce Ports used by your internal systems (if any)
  5. Ensure that your AKS cluster and your application endpoints, nodes running custom scripts, and internal systems integrated with CloudBlue Commerce have access to each other.

  6. If you use custom domain registration, fraud-screening and payment plugins on your CloudBlue Commerce 20.5 installation, contact the developers of those plugins and obtain versions that are compatible with CloudBlue Commerce 21.0. You can install compatible versions of those plugins before or after the upgrade. To obtain the instructions on how to install domain registration, fraud-screening, and payment plugins, please refer to the CloudBlue Commerce 20.5 or CloudBlue Commerce 21.0 documentation.

    Note: If you install them before the upgrade, the files of those plugins will be automatically copied by the upgrade scripts.

  7. During the upgrade, the IP addresses of the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be changed to new ones. We recommend that you create DNS records for the current IP addresses and replace these IP addresses with the corresponding DNS names in the configuration of your application endpoints, custom scripts, and internal systems integrated with CloudBlue Commerce. After the upgrade, you must update these DNS records so that they point to the new IP addresses of the APIs.

    Notes:

    - For application endpoints that are based on the APS PHP runtime, the URLs of the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be updated automatically in the default configuration files by the fix-aps-endpoints.py script during the upgrade.

    - After the upgrade, the OSS XML-RPC API, BSS XML-RPC API, and APS REST API will be accessible through the oss-proxy and bss-xmlrpc LoadBalancer services. By default, the load balancers of those services use internal IP addresses.

  8. Configure your SMTP server that is used for sending mail messages from CloudBlue Commerce to the Internet:

    • If this is an external server, make sure that it can accept mail from the public IP address of your AKS cluster.

      Note: To find that IP address, log in to the Azure portal, select your AKS cluster, click Properties in the Settings group, click on the name of the infrastructure resource group, and then click on the public IP address whose name is a GUID.

    • If you use your BSS application node as an SMTP server, complete these steps:

      1. Update the /etc/postfix/main.cf configuration file on the BSS application node:

        inet_interfaces = all
        mynetworks = YOUR_AKS_CLUSTER_SUBNET/NETMASK, 127.0.0.1
      2. Restart the Postfix service:

        systemctl restart postfix

      Note: Using the BSS application node as an SMTP server is not recommended. We recommend that you use a separate SMTP server. To learn how to specify an SMTP server for sending messages to the Internet in CloudBlue Commerce 21.0, see the CloudBlue Commerce 21.0 Deployment Guide.

  9. Allow access from your AKS cluster to your database servers:

    1. On your OSS database server, add the following lines to the ~postgres/11/data/pg_hba.conf file:

      # MIGRATION AKS
      host    all     all     YOUR_AKS_CLUSTER_SUBNET/NETMASK       md5
    2. On your BSS database server, add the following lines to the ~postgres/11/data/pg_hba.conf file:

      # MIGRATION AKS
      host    all     all     YOUR_AKS_CLUSTER_SUBNET/NETMASK       md5
    3. On both servers, reload the PostgreSQL service by executing this command:

      service postgresql-11 reload
  10. On your OSS MN, install and configure the necessary tools. To do this, perform these steps:

    • Install the Azure CLI by following these instructions.

    • Get access credentials for your cluster by following these instructions.

    • Install kubectl by following these instructions.

    • Install Helm 3.5.4 or later.

      Note: In the CloudBlue Commerce documentation, command options are provided for Helm 3. Helm 2 can also be used, but there may be differences in command options. If you already have Helm 2 installed, you can either migrate from it to Helm 3 or take into account differences in command options. For example, if a Helm 3 command cannot be performed due to such differences, you can find suitable Helm 2 command options.

  11. On your OSS MN, download the cnc-migration-latest.tgz archive. Then, unpack it and go to the CNC-MIGRATION directory:

    tar xzvf CNC-MIGRATION.tgz
    cd CNC-MIGRATION
  12. In the CNC-MIGRATION directory, run the preparation.py script. This script collects the configuration settings of the installed CloudBlue Commerce 20.5 components and prepares the config.yaml file that you will use to install CloudBlue Commerce 21.0 components. To run the script, execute these commands:

    python preparation.py
  13. In the CNC-MIGRATION directory, edit the config.yaml file:

    1. Specify the credentials of the postgres admin user in the components.a8n-db.admin_user and components.a8n-db.admin_password settings.

    2. Define the format for hostnames of automatically created ingress rules. You can use one of these formats:

      • PREFIX.NAMESPACE_OF_YOUR_CBC_21_INSTALLATION.CLUSTER_DOMAIN

      • PREFIX.CLUSTER_DOMAIN_OVERRIDE

      where:

      • PREFIX is a service-specific auto-generated prefix.

      • NAMESPACE_OF_YOUR_CBC_21_INSTALLATION is the namespace of your CloudBlue Commerce 21.0 installation.

      • CLUSTER_DOMAIN is a domain specified in the cluster_domain setting.

      • CLUSTER_DOMAIN_OVERRIDE is a domain specified in the cluster_domain_override setting.

      If you specify a domain in the cluster_domain setting only, the first format will be used. If you specify a domain in the cluster_domain_override setting, the second format will be used regardless of whether the cluster_domain setting is specified or not.

      Examples of hostnames of automatically created ingress rules:

      • aps.cbc.example.com

      • aps.example.com

      Notes:

      - During the upgrade, an ingress rule for the incoming APS proxy will be automatically created in your AKS cluster.

      - To learn more about the incoming APS proxy, please refer to the APS SDK documentation.

    3. Review and adjust the values of the requests and limits settings of the OSS and BSS components.

  14. In the CNC-MIGRATION directory, run the generate-secrets.py scripts. This script creates a namespace for CloudBlue Commerce 21.0 and secrets with database access credentials in your AKS cluster. To run the script, execute these commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
    python generate-secrets.py
  15. In the CNC-MIGRATION directory, run the copy-persistent-data.py script. In your AKS cluster, this script creates persistent volumes for the OSS and BSS components, and a pod named temporary-pod. It then mounts those persistent volumes to that pod, and copies APS packages, skins, and keystore files from your OSS MN and BSS application node to those persistent volumes. To run the script, execute these commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
    python copy-persistent-data.py
  16. In the provider control panel, go to Settings > Public API > the Allowed Networks tab and allow access to the OSS XML-RPC API from:

    • the network used by pods of your AKS cluster

    • the network used by the nodes of your AKS cluster

  17. If you have APS endpoints whose DNS names are resolved through specific DNS servers, for example, internal DNS servers, specify these DNS servers as forward servers for those DNS names in the DNS service of your AKS cluster. To do this, create a ConfigMap named coredns-custom in the kube-system namespace of your AKS cluster:

    my.server: |
      DNS_NAME_OF_APPLICATION_ENDPOINT:53 {
          errors
          cache 30
          forward . IP_ADDRESS_OF_DNS_SERVER
      }

    To learn more about customizing the DNS service of an AKS cluster, please refer to the Azure documentation.

    Important: The DNS servers that you specify as forward servers must be accessible from your new AKS cluster.

  18. If domain registrars used by domain registration plugins of your CloudBlue Commerce 20.5 installation use IP address restrictions for their APIs, add the public IP address of your AKS cluster to the whitelists of those registrars.

    Note: To find that IP address, log in to the Azure portal, select your AKS cluster, click Properties in the Settings group, click on the name of the infrastructure resource group, and then click on the public IP address whose name is a GUID.

  19. If you have the Microsoft 365 application installed and its application endpoint node uses IP address restrictions, add the respective allow entries for the subnet of your AKS cluster. To do this, use these instructions.

  20. During the upgrade, the old incoming APS proxy will be replaced with a new one. To prepare for this, perform these steps:

    1. In your AKS cluster, install ingress controllers as described in the CloudBlue Commerce 21.0 Deployment Guide.

    2. Obtain the service IP address of the external ingress controller:

      kubectl config use-context YOUR_AKS_CLUSTER
      kubectl -n NAMESPACE_FOR_INFRASTRUCTURE_COMPONENTS get svc nginx-ingress-ingress-nginx-controller -o jsonpath="{.status.loadBalancer.ingress[].ip}"
    3. According to the format for hostnames of automatically created ingress rules that you chose previously, create an A DNS record for the new incoming APS proxy:

      • aps.NAMESPACE_OF_YOUR_CBC_21_INSTALLATION.CLUSTER_DOMAIN A SERVICE_IP_OF_EXTERNAL_INGRESS_CONTROLLER

      • aps.CLUSTER_DOMAIN_OVERRIDE A SERVICE_IP_OF_EXTERNAL_INGRESS_CONTROLLER

      Important: After you create that DNS record, make sure that the DNS name of the new incoming APS proxy is resolved to the respective IP address on the CloudBlue Connect side.

  21. In your AKS cluster, install logging and monitoring solutions. For details, see the CloudBlue Commerce 21.0 Deployment Guide.

  22. In your AKS cluster, install a backup solution. For details, see the CloudBlue Commerce 21.0 Backup and Recovery Guide.

Step 3. Deploy CloudBlue Commerce 21.0

  1. Perform upgrade prechecks again. For instructions, see Appendix A: Performing Upgrade Prechecks.

  2. In Azure, create snapshots of your VMs.

  3. Suspend the daily billing process:

    1. In the provider control panel, go to System > Settings.

    2. In the System group, click Events and switch to the Scheduler tab.

    3. Reschedule the DailyBillingProcessing task so that you have enough time to complete the upgrade.

    Warning: Remember to resume the daily billing process after you complete the upgrade.

  4. Check whether new APS packages were added to the /usr/local/pem/APS/packages directory on your OSS MN after the date you executed the copy-persistent-data.py script. To check this, use the following commands:

    ls -lt /usr/local/pem/APS/packages | head -100

    and

    kubectl config use-context YOUR_AKS_CLUSTER
    kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION exec -ti temporary-pod bash
    ls -lt /var/oss/APS/packages/ | head 100

    If the /usr/local/pem/APS/packages directory contains new APS packages, copy the directories and ZIP files of those APS packages to your AKS cluster with the following commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION cp PACKAGE_DIRECTORY temporary-pod:/var/oss/APS/packages/
    kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION cp PACKAGE_ZIP_FILE temporary-pod:/var/oss/APS/packages/
  5. Stop all components of your CloudBlue Commerce 20.5 installation:

    1. On your OSS MN, execute the following commands:

      service pau stop
      service pa-agent stop
       
      chkconfig pau off
      chkconfig pa-agent off
    2. On your BSS application node, execute the following commands:

      chkconfig pau off
      chkconfig pba off
      chkconfig pa-agent off
      chkconfig ssm off
      
      service pba stop
      service pau stop
      service pa-agent stop
      service ssm stop
      
      mv /usr/local/bm/etc/ssm.conf.d/global.conf /usr/local/bm/etc/ssm.conf.d/global.conf.inactive
  6. In the oss database, execute the following SQL commands:

    ALTER TABLE aps_resource_link ALTER COLUMN target_type_info_id SET NOT NULL, ALTER COLUMN source_app_id SET NOT NULL, ALTER COLUMN target_app_id SET NOT NULL;
     
    CREATE INDEX IF NOT EXISTS "IX_aps_security_link_resource_type_info_id" ON aps_security_link(resource_type_info_id);

    Note: Executing these SQL commands is necessary for large databases and may take a long time to complete. We recommend that you perform them in advance.

  7. In the CNC-MIGRATION directory, run the pre-migration.py script. This script adds the URLs of CloudBlue Commerce 21.0 Helm repositories and disables online stores in the BSS database. To run the script, execute these commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
    python pre-migration.py
  8. Deploy CloudBlue Commerce 21.0 components in your AKS cluster:

    1. Make sure you are in the CNC-MIGRATION directory on your OSS MN.

    2. Execute these commands:

      kubectl config use-context YOUR_AKS_CLUSTER
      export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
      helm install helmfile-deploy a8n/helmfile-deploy \
        -n $NAMESPACE \
        --set configYaml=$(cat config.yaml | base64 -w0) \
        --set namespace=$NAMESPACE \
        --set dockerrepo=odindevops-a8n-docker-21-public.jfrog.io

Step 4. Perform Post-deployment Actions

  1. In the CNC-MIGRATION directory, run the post-migration.py script. This script:

    • copies UI configuration files and locales from your OSS MN and BSS application node to the appropriate persistent volumes,

    • copies custom domain registration, fraud-screening and payment PHP plugins from your OSS MN and BSS application node to the persistent volumes of the Domain SDK, Payment SDK, and Fraud SDK components, and

    • restarts the necessary CloudBlue Commerce components.

    To run the script, execute the following commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
    python post-migration.py
  2. Set the namespace of your CloudBlue Commerce 21.0 installation as default in the context of your AKS cluster:

    kubectl config use-context YOUR_AKS_CLUSTER
    kubectl config set-context --current --namespace=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
  3. In your AKS cluster, check that the status of all APSInstance custom resources is Ready. To do this, execute the following commands:

    kubectl config use-context YOUR_AKS_CLUSTER
    kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION get apsinstance

    If the status of an APSInstance custom resource is not Ready, you must fix this before performing the next step. For troubleshooting, see the logs of the following components: the component whose APSInstance custom resource is not ready, oss, and a8n-operator.

  4. After the upgrade, the APS REST API and OSS XML-RPC API are accessible through the oss-proxy LoadBalancer service, and the BSS XML-RPC API is accessible through the bss-xmlrpc LoadBalancer service. This means that you must update the URLs of those APIs in the configuration files of your application endpoints, custom scripts, and systems integrated with CloudBlue Commerce.

    Notes:

    - For some applications, you may need to update those API URLs in their application instance settings through the provider control panel.

    - If you created DNS records for the APS REST API, OSS XML-RPC API, and BSS XML-RPC API on your internal DNS servers, you must update IP addresses in those DNS records.

    To update the API URLs in the default configuration files of the application endpoints that are based on the APS PHP runtime, use the fix-aps-endpoints.py script:

    export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
    kubectl config use-context YOUR_AKS_CLUSTER
    python fix-aps-endpoints.py

    Notes:

    - The script will ask you to provide administrative credentials to access your VMs through SSH.

    - The script automatically updates the /ets/hosts files in your VMs so that the hostnames in the API URLs point to the correct IP addresses.

  5. If you have the Microsoft 365 application installed, perform these steps:

    1. Obtain the load balancer IP address of the oss-proxy LoadBalancer service:

      export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
      kubectl config use-context YOUR_AKS_CLUSTER
      kubectl get svc -n $NAMESPACE oss-proxy -o jsonpath={.status.loadBalancer.ingress[].ip}
    2. Obtain the load balancer IP address of the bss-xmlrpc LoadBalancer service:

      export NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION
      kubectl config use-context YOUR_AKS_CLUSTER
      kubectl get svc -n $NAMESPACE bss-xmlrpc -o jsonpath={.status.loadBalancer.ingress[].ip}
    3. On your Microsoft 365 application endpoint node, add the following line to C:/Windows/System32/drivers/etc/hosts:

      LOAD_BALANCER_IP_ADDRESS_OF_OSS_PROXY_SERVICE oss-node-0.oss-dns

      Note: The hostname of the first OSS pod is oss-node-0.oss-dns. The APS controller uses that hostname in HTTP headers when communicating with application endpoints. For the Microsoft 365 application endpoints to resolve that hostname, you must manually update C:/Windows/System32/drivers/etc/hosts. For application endpoints that are based on APS PHP runtime, the fix-aps-endpoints.py script automatically updates the /ets/hosts files in your VMs.

    4. In settings of your Microsoft 365 application instances, specify the new IP addresses of the OSS and BSS APIs. For details, see the Microsoft 365 Provider's Guide.

  6. Ensure that your BSS pods and your BSS application node use the same time zone:

    • To check the time zone of your BSS pods, execute these commands:

      kubectl config use-context YOUR_AKS_CLUSTER
      kubectl -n NAMESPACE_OF_YOUR_CBC_21_INSTALLATION exec -ti BSS_POD_NAME bash
      date
    • To check the time zone of your BSS application node, execute the date command.
  7. In the provider control panel, in the task manager, run the UX1 Marketplace ... Plan indexer periodic task.
  8. In the provider control panel, ensure that the daily billing process has resumed.
  9. During the upgrade, IP addresses used by brand domains will be replaced with the load balancer IP address of the branding-ui-cluster-ui LoadBalancer service created in your AKS cluster. If you have brand domains whose DNS zones are not managed by CloudBlue Commerce, ensure that DNS records of those brand domains point to the load balancer IP address of the branding-ui-cluster-ui LoadBalancer service.

  10. Important: Please note that upcoming End of Life has been announced for the XML RPC in CloudBlue Commerce. More details can be found here.

    To run scripts that use the OSS XML-RPC API, BSS XML-RPC API, and APS REST API on your OSS MN, update the configuration of poaupdater:

    kubectl config use-context YOUR_AKS_CLUSTER
    python generate_configs.py --namespace NAMESPACE=NAMESPACE_OF_YOUR_CBC_21_INSTALLATION

    As a result, this script updates the following configuration files:

    • /usr/local/pem/etc/Kernel.conf

    • /usr/local/pem/etc/pleskd.props

    • /usr/local/pem/etc/odbc.ini

    These configuration files contain the database and API credentials necessary for poaupdater.

    Also, the script updates the /usr/local/pem/APS/certificates/poa.pem certificate file. You or your scripts can use it to access the APS controller.

  11. If you use a custom PDF conversion tool on your CloudBlue Commerce installation, perform the instructions in this article.

  12. If your OSS MN and BSS application node are no longer used by your CloudBlue Commerce 21.0 installation, consider removing them or decreasing their capacity.

    Note: You can decrease the capacity of your OSS MN, keep it, and then use it for running various scripts that use the OSS XML-RPC API, BSS XML-RPC API, and APS REST API. You can also create a separate VM for running such scripts as described in Appendix B: Creating a Cloud Management Node.

CloudBlue, an Ingram Micro business, uses cookies to improve the usability of our site. By continuing to use this site and/or logging in you are accepting the use of these cookies. For more information, visit our Privacy Policy.