Revenue Assurance: Iterative discovery and testing

Summary

Revenue assurance confirms the correct operation of billing processes according to documented business rules. Revenue assurance can discover correctable billing errors or revalidate that billing is working as expected. Revenue assurance is a process /approach that iteratively removes correctly processed transactions and database entries from analysis to reveal the underlying errors.

A business and its customers are more likely to retain confidence in the billing process when errors are quickly discovered and corrected. An ongoing revenue assurance program will proactively identify when system code and reference data changes introduce new billing errors.

What is Revenue Assurance (RA)?

Revenue assurance is a confirmation process that ensures billing is accurately performing the correct actions on its customers and transaction dataflows. This processing is performed to ensure the biller sends customers bills containing only the charges they are entitled to bill with the prices calculated correctly. Over or under billing can impact a biller's financials and the customer's perception of the billing process.

Revenue assurance achieves this by comparing a recalculated set of expected results for a billing process against what 'actually' happened in the billing system. For example, expected results might describe which phone calls should have been directed to the correct billing platform and what their correct price should be. Revenue assurance uses historical data to perform its testing, though that data may only be days or weeks old. The intent is not to run alongside an operational system in real-time, but to validate processing and discover any errors so that they can be fixed.

Revenue assurance does this by treating the billing process as a black-box where the testing is not concerned with 'how' the billing system performs its work, but rather 'what' the end results are. For example, business documentation describing record selection, and details of pricing plans can be used to confirm (by reselection and rerating) that the rated transaction stream is correct.

Revenue assurance seeks to address questions including:

Have 'all' billable transactions been included in processing?
Do all transactions make it through the end-to-end billing process? e.g. network-to-bill
Are all transactions associated with an appropriate processing outcome? i.e. billed, omitted with reason, or errored for correction / review
Are transactions omitted from billing only where 'appropriate'? e.g. emergency calls to 000 / 911
Are errored transactions treated appropriately, including resubmission after correction?
Are customers billed for 'all' that they ordered?
Are customers billed 'only' for what they ordered?
Is the customer's network provisioning aligned with their billing?
Is the correct price applied to each one-time, recurring and transaction charge?

Billing processes where revenue assurance testing might be applied include:

Mediation: Are the correct network records selected and passed to billing? Are partial records (e.g. from tollway meter points, mobile phone towers) aggregated appropriately? Are network field values mapped correctly when passed to billing? Are transactions correctly split between wholesale (e.g. interconnect) and retail billing? Are the appropriate transactions (e.g. network information-only records) omitted from downstream processing?
Rating: Are transactions correctly identified? Is the correct rating plan used? Are rate periods changes addressed appropriately? Is the final price calculated accurately? Are any taxes applied correctly?
Billing: Are all records selected for billing? Is the rated price carried to the final bill? Is discounting performed accurately? Do the financial postings match the billed charges? Are financial postings classified appropriately?

Test Preparation

When preparing to perform revenue assurance testing, a scoping and discovery process must be performed to determine:

Testing Scope: How long an elapsed period will be tested (e.g. week, month)? Which specific dates will be tested? Which products will be included / excluded from testing?
Testing Focus: Which steps of the end-to-end billing system are being tested for the in-scope products? e.g. correct record selection, database alignment, pricing, derived field calculations
Data Availability: Is sufficient data from the appropriate sources available? Are the data sources equivalent / aligned and hence comparable? Some operational data may be deleted quickly after use necessitating specific intervention and retention for revenue assurance. Transactions performed in real-time may not leave residual details that are easily collected for use in revenue assurance comparisons.
Tool Selection: Is the data extracted from multiple environments? Can the toolsets used process both ASCII from mid-range UNIX boxes and EBCDIC from mainframes? If the comparison files are from different coding regimes, they may require pre-processing before comparison. Different tools may be selected for the high-volume comparison processing and the low-volume analysis phases of work.
Data Volumes: Can the tools and processing process the data volumes involved in a 'reasonable' timeframe? Can the processing platform store the original data and multiple result sets together for subsequent review? As configuration errors and additional business rules are determined, multiple iterations will be required to recalculate the test results.
Resources: An unlimited budget and timeframe can test a billing stream until all processing aligns, but a more realistic (and cost effective) target is achieving a 'low' 'error rate' for the tested scope. A 0.01% error rate will generate 100 reviewable errors per million records compared. This error number may be acceptable for a one million record test, but if one billion transactions are compared this can result in 100,000 records.

Documentation and data discovery

Initial revenue assurance testing may encounter difficulties in sourcing:

Accurate and complete documentation: The older a billing systems is, the less accurate its documentation is likely to be. The technical and business staff who originally implemented the billing system are likely to have moved on, and the documentation may only be partially updated by subsequent development. The knowledge of 'why' processing occurs is reduced until the context of existing processing is no longer understood.
Technical details: Data flows and databases related to the testing scope must be determined. Once testing is underway, additional (missing) details may be required. The testing scope may have been imperfectly mapped to the operational environment when dataflows were identified.
Data file sourcing: Test data must be 'physically' moved from where it was generated / is stored to the test environment. The time to retrieve and move large data volumes must be factored into the schedule of larger projects.
Business rules: Business rules must be translated from technical details (e.g. record selection by field values), or sourced from public documentation (e.g. pricing plans). Over time, the meaning of field values used in selection criteria may be lost reducing the understanding of why specific processing is performed. Without understanding why rules are applied, they may be retained in ignorance after their specific products are retired / withdrawn.

Once initial testing has been performed, sourcing the necessary documentation and data will be easier and some aspects may be candidates for automation. Inadequate documentation may trigger updates to collate the missing details for subsequent retesting (and to assist day-to-day operations).

Iterative testing

Iteration is present in many areas of revenue assurance testing:

Data alignment: Available technical documentation is used to access the testing data. Accurate documentation allows the data to be understood and accessed immediately. Iterations may be required until any extra fields, field values or records not described in the documents are fully understood.
Business rule implementation: Business rules' initial implementations are likely to highlight multiple errors. These errors will be due to configuration defects, missing business rules and/or unexpected field values. Once the initial error categories are identified and corrected, testing will be rerun until these errors are fixed and any underlying processing errors are revealed.
Business analysis: Unattributed errors will need analysis to characterise and understand their source. Errors may be aligned with certain products, dates, locations or business units. Discovering what the errors' characteristics are allows the underlying problems to be discovered and corrected. A single error type is likely to result in many error records.
Test refinement: Once transaction flows are validated as correct, derived field validations may be checked, and calculated values confirmed. Each confirmation that the underlying processes are correct allows testing to be escalated to the next level of value-added testing.
Test development: Initial testing solutions are likely to require re-engineering if implemented for ongoing use in production. Approaches used in the initial solution may be changed to remove manual steps, address delayed transactions and increase the revenue assurance test changes performed through data configuration rather than development.
Ongoing operations: Revenue assurance may be performed once to find errors in the current billing systems and their customer / reference data. These systems will change over time and new billing errors may be introduced. Without a program of repeated testing (e.g. each quarter), the billing processes can accumulate undiscovered processing problems.

The 80 / 20 rule in revenue assurance analysis

In a stable billing environment, most databases are aligned and transactions process correctly. Revenue assurance is therefore about identifying the less obvious errors that only apply in certain circumstances, or that occur globally with a low occurrence per customer.

If this were not the case, customer complaints and a biller's staff would quickly highlight errors that could then be corrected. Errors in the biller's favour will be highlighted by detail oriented customers. Errors in the customer's favour are likely to remain unannounced and left to the biller to discover.

Revenue assurance testing must quickly reach a point where the volume of correctly processed records are 'removed' (eliminated) from the test process leaving only the discrepancies behind. As each error type is identified its characteristics can be used to filter out its specific errors leaving a smaller error pool for review. When the revenue assurance process is rerun, additional (possibly temporary) record filters may be introduced to whittle the error records down to the smallest possible pool of unexplained errors.

Ongoing RA testing of a specific scope must be considered based on a cost/benefit equation. The cost of equipment and staff must be balanced against the benefits (value) of the remaining errored records. A low error rate in infrequent transactions (e.g. local calls to a time service) may justify an end to further testing, but the same error rate in high value, high volume content (e.g. on-demand movies) may necessitate additional testing iterations.

Telecom Revenue Assurance

"The Telco Revenue Assurance Handbook" - This book introduces the basic concepts of revenue assurance and how it can be performed in a telecom environment. This book has a companion website at telco-revenue-assurance.com maintained by the author Rob Mattison. [ISBN: 1-41162-801-2 / Lulu Press / Rob Mattison / 2005]