Usage Collector Overview

Audience of This Guide

This guide is primarily for:

Product and sales managers
CloudBlue Commerce configuration engineers
Maintenance and support engineers
Service connector developers

About Usage Collector

CloudBlue Commerce is an e-commerce solution that consists of a variable set of components based on micro-service architecture.

Business Purpose

To be a fully functional solution for usage-based services, your CloudBlue Commerce installation must include the following components:

Usage Collector is a component which:

Automates data collection and validation for PAYG services.
Ensures auditability by storing the images of the received reports.
Provides the user interface in UX1 for Providers:
- to manage sales margins and markups.
- to manage usage report import including manual upload.

Technical Purpose

Usage Collector is a microservice with its own release cycle which:

Collects and validates usage reports from service vendors.
Stores source usage report files to a file storage.
Provides the UI in UX1 for Providers to manage sales margins and markups.
Provides the UI in UX1 for Providers to manage usage report import including manual upload.
Guides uploaded usage reports (discovers subscriptions and resources in CloudBlue Commerce by Reconciliation IDs and MPNs from the imported usage records).
Provides API to import usage to CloudBlue Commerce.

Distribution Chain Structure

Usage Collector manages usage data for Rating Engine to calculate charges for the following three interrelated distribution parties:

Provider: the owner of the distribution contract with the vendor (distribution contract owner)
- Reseller (can be several reseller levels)
  - End Customer

These roles can be played by different parties depending on the distribution chain structure. However, the relationship between them is always the same:

Distribution through regional operating units ("OPCOs"):
- L₀ (Provider): CloudBlue Commerce installation owner
  - L₁ (Provider's OPCO): Provider (distribution contract owner)
    - L₂ (Reseller): Reseller
      - ...
        L_i (Reseller): Reseller
        L_i+1(Reseller's OPCO): Reseller
        ...
        L_n (Customer): End Customer
Distribution through third-party resellers:
- L₀ (Provider): Provider (distribution contract owner)
  - L₁ (Reseller): Reseller
    - ...
      - L_n-1 (Reseller): Reseller
        L_n (Customer): End Customer
Direct distribution:
- L₀ (Provider or Reseller): Provider (distribution contract owner)
  - L₂ (Customer): End Customer

How It Works

Usage Data Flow Diagram

The functional diagram below illustrates a high level usage data flow for CloudBlue Commerce installations that support usage-based services, including the Usage Collector function:

Integration with Vendor Systems

Usage Collector can integrate with vendor systems by using either Batch Import API or File Import API.

One readily available integration option for service vendors is CloudBlue Connect, which uses file import API to send usage data to Usage Collector.

Another integration capability for service vendors is APS connectors, integration middleware between service vendors and CloudBlue Commerce. For example, AWS services can be integrated with the help of an AWS APS connector. APS connectors can be implemented to deliver usage data to Usage Collector through VRD Collector, a built-in OSS module that imports usage data from APS connectors using batch import API.

To deliver usage data to VRD Collector, APS connectors must support usage data import and implement the RatedDataSupport APS type. An example of how to implement an APS connector that supports usage data collection can be found in the APS documentation. To understand whether a particular APS connector supports usage data collection, please refer to the connector documentation.

Data Processing Prerequisites

To be able to validate and guide usage data, Usage Collector requires that all reported usage items and corresponding CloudBlue Commerce resources have:

MPNs
Reconciliation IDs
Vendor Contract IDs: must be set at the level of the vendor contract owner or vendor contract owner’s operating unit that owns the service plan.

How Usage Collector Receives and Guides Usage Data

Usage Collector collects from service vendors and manages usage data for a distribution chain of three main parties:

Provider: a distribution contract owner who distributes through regional operating units (OPCOs) or third-party resellers
Reseller: an OPCO or a third-party reseller
End Customer: a service consumer

Note: The supported distribution network compositions are listed in Distribution Chain Structure.

Usage Collector supports the following rating models (or schemas) for usage data exported by vendors:

Usage data rated by the vendor for all parties ('Tier Rated' or 'TR'): The vendor calculates the costs and prices for consumed services for the Provider, Reseller, and End Customer based on sales margins configured in the vendor's system. No additional calculations are necessary.
Usage data rated by the vendor for the End Customer ('Price Rated' or 'PR'): The vendor calculates the charges for the End Customer. The costs for the Provider and Reseller must be calculated on the Provider's side based on the End Customer prices and sales margins configured in CloudBlue Commerce.
Usage data rated by the vendor for the Provider ('Cost Rated' or 'CR'): The vendor calculates the charges for the Provider. The prices for the Reseller and End Customer must be calculated on the Provider's side based on the Provider costs and sales markups configured in CloudBlue Commerce.
Non-rated quantity-only usage data ('Quantity' or 'QT'): The vendor reports only the consumed resource quantity. The extended price is calculated based on the reported quantity and resource prices configured in CloudBlue Commerce.

Important: The QT schema works only with the Product Lifecycle Management component.

Usage Collector receives usage data in one of these ways:

Usage data is imported automatically through an API endpoint (file import or batch import API).
Usage data is uploaded manually in the UX1 for Providers UI (file import API).

Warning: APS applications must submit usage data using only one method: either APS counters or report files. Submitting usage data alternating the two methods is not supported and will lead to malfunction.

How Usage Reports Are Processed

Usage Collector receives usage reports from service vendors through API:
- Batch Import API, which is used by VRD Collector and can be used by service vendor systems.
- File Import API , which is used by CloudBlue Connect and UX1 for Providers UI and can also be used by service vendor systems.
Usage Collector stores usage report source files to a file storage. Providers can download them for reconciliation purposes from the UX1 for Providers UI.
Usage Collector validates the uploaded usage report.
The periodic OSS task Usage Collector ... guiding schedule guides usage reports: discovers CloudBlue Commerce Resource IDs and Subscription IDs by the reported Reconciliation IDs and MPNs.
If guiding of all records is successful, the usage report gets the Validated status and is now ready to be rated.
If guiding of at least one record fails, the usage report gets the Data Validation Failed status and cannot proceed to the rating stage. To continue the processing, the provider must download the error report, troubleshoot the errors, and re-upload the usage report in the UX1 for Providers UI.
The periodic OSS task Rating Orchestrator ... schedule (brought by Rating Orchestrator) processes usage reports with the Validated status (the default interval is 30 minutes):
1. Rating Orchestrator requests unprocessed usage reports from Usage Collector.
2. Rating Orchestrator sends records from usage report to Rating Engine to rate.
3. Rating Engine rates the delivered usage records and stores the calculated charges in the Rating Engine database.
4. If Rating Engine completes the rating of all usage records of a usage report, the report receives the Rated status.
5. If Rating Engine cannot rate at least one record from a usage report, the report receives the Rating Failed status and cannot proceed to further processing. To continue processing, the provider must download the error report, troubleshoot the errors, and resubmit the usage report for processing in the UX1 for Providers UI.
When the rating process for all usage records is completed:
1. For the CR, PR, and TR rating models: With the help of Order Management, Rating Orchestrator immediately places billing orders for each subscription from the usage report. When all the necessary billing orders are placed, the usage report receives the Processed status.
2. For the QT rating model: If the processing is successful, the usage report receives the Processed status. Then, for each subscription from the usage report, the Daily Billing Process creates a billing order on the next subscription billing date.
After all the necessary billing orders are placed:
1. For each billing order, the BSS Order Flow requests Rating Engine to bill subscriptions from the billing order.
2. Rating Engine returns order details (including the usage data from the usage reports).
The applicable BSS Order Flow continues: The order details are added to the billing order, taxes are applied and so on.

Usage Report Lifecycle

Status	Description
Validating	A usage report receives this status when either manual uploading of the report file in the UX1 for Providers UI starts or a charge import API session to import the usage report is opened.
Validated	A usage report receives this status when either the usage report file is fully manually uploaded, parsed, and guided without errors or the charge import API session to import the usage report is successfully closed.
File Validation Failed	A usage report receives this status when the usage report file has an invalid format. This status applies only to the usage reported through the file import API.
Data Validation Failed	A usage report receives this status when Usage Collector cannot guide external IDs from the usage report file to CloudBlue Commerce IDs or another data validation error occurs. This status applies only to the usage reported through the file import API.
Rating in Progress	A usage report receives this status when Rating Orchestrator starts rating the usage report.
Rated	A usage report receives this status when Rating Engine rates all the usage report records successfully.
Rating Failed	A usage report receives this status when Rating Engine cannot rate at least one record from the usage report.
Billing in Progress	A usage report receives this status when Rating Orchestrator starts creating a Billing Order.
Processed	A usage report receives this status when Rating Orchestrator creates Billing Orders for all records from the usage report successfully.
Billing Failed	A usage report receives this status when Rating Orchestrator cannot create a Billing Order for at least one record from the usage report.
Canceled	A usage report receives this status when another version of the same usage report is uploaded in the UX1 for Providers UI or if all records in the report are in the Closed status. This status can occur only after one of these statuses: File Validation Failed or Data Validation Failed.

Version Compatibility with Other CloudBlue Commerce 21.0 Components

Usage Collector is compatible with other CloudBlue Commerce 21.0 components in the following way:

	CloudBlue Commerce 21.0
BSS	21.1
OSS	21.1
Branding UI Cluster	21.1
Order Management	1.5
Discount Manager	1.1