Note 33: Data Mediation

Posted: 18 May 2007

Raw transactions collected from the network may omit information required by billing (and other downstream systems) since those details are not required to deliver network services. Data mediation takes what information is collected from the network and provides a standard process to enhance and manipulate it. Data mediation thus serves to isolate the idiosyncrasies of the network from those of the billing process.

Mediation processing is general in nature and is performed in the same way for all transactions (of a certain type). No customer specific processing is performed in mediation. Only where transactions are sent to a different downstream billing systems based on their customer segmentation (e.g. consumer, corporate, wholesale) do different formats and processing become important.

Once collected from the network, the following broad steps may be performed:

Identification of the transaction's type: This allows transaction specific validation and manipulation to be performed. The vendor equipment from which the transaction is sourced can also influence subsequent processing due to vendor-specific formatting and field population.
Intra-transaction validation: This checks the transaction's format and data field population. This is performed on the transaction in isolation without reference to other transactions. Subsequent processing against other transactions may find errors and discrepancies.
Recycling: Partial transactions may need to be held until all the other 'pieces' have been received from the network. Mediation can temporarily hold information until all the expected pieces arrive. e.g. fragments of a mobile phone call. Transactions that are held for an excessive period may be escalated for examination.
Determine any manipulation steps: Validation and manipulation steps may be interwoven since an error may only be found when a complex manipulation is attempted.
Determine downstream outputs: Collection and mediation are transitory functions with the data from the network intended for downstream systems, including billing. The downstream systems will require their own specific data layouts possibly requiring further data manipulations.
Replicate transactions as required: A single transaction can be sent to multiple downstream systems. For example, a phone call to another biller's network could result in records being sent to the retail billing platform (for the customer), fraud (to monitor the processing for all customers), and interconnect processing (to allocate (share) revenue between the two network operators).
Divide a processing stream by customer segment: With all network transactions passing through mediation, mediation is ideally placed to divide a common source into different processing streams. The basis for making such a division must be available to mediation based on either the transaction's content or an externally maintained customer database. The division can then be made with little additional effort. If a customer database is used, issues of data currency and maintenance will need to be addressed.
Batch or cache outputs: Downstream systems will differ in how they receive transactions. Some need each transaction as it is performed (e.g. prepaid processing). Others will require transactions be batched and dispatched every hour, or may only want or be able to receive transactions in bulk once a day (e.g. for overnight processing). Mediation must delay its outputs as required to address the unique requirements of each downstream system.

Active versus Passive Mediation

Passive mediation occurs after the network has completed its processing. Details of phone calls, meter readings or tollway journeys are processed but do not affect how the network operates. Passive mediation can be used for all networks and is likely to be performed in batch for processing efficiency.

Active mediation is different and involves itself in the transaction's processing as it is taking place. Active mediation is likely to be used in situations where the credit amount must be actively managed (e.g. prepaid accounts), where the biller seeks the customer's agreement before processing high value transactions, or other customer / biller databases are referenced to confirm a service can be provided. Examples of active mediation can be found in prepaid phone calls, mobile phone data products, and mobile internet access (e.g. short-term access to a coffee shop's WiFi hotspot).

Active mediation sits across the transaction's network path and can intervene in real-time. Such interventions can include:

Redirection: Sitting on the transaction's network path, a customer transaction can be temporarily redirected to another destination. Destinations can seek customer permissions, receive payments or seek additional customer information.
Recharge: Real-time payment processing allows customers to top-up account balances. Payments methods could include credit cards and vouchers.
Permission: Customers requesting a product or service are asked explicitly for payment agreement. If their agreement is not given they are denied access. Permission may be requested to confirm access to adult content even if payment is not an issue.
Credit checking: Once a customer has chosen their transaction, the mediation system checks that there is sufficient balance in the customer's account (i.e. prepaid) or credit limit (e.g. post-paid). If there is not enough money available, the customer may be redirected to a location where additional (real-time) payments can be taken.
Termination: Mediation can actively monitor the customer's session and determine when they have exceeded their permitted resource quota (e.g. data downloads) or balance. Mediation can then terminate access, possibly redirecting the customer to an appropriate destination for further funds or additional permissions.

Active mediation must be supported by infrastructure with a much higher level of performance (i.e. minimal processing delay) and availability (i.e. always there). If active mediation is broken then no processing can be performed due to its location across the transaction's processing path. For phone networks, active mediation must complete all its processing in under a second to conform with the network's expected response time. Internet and data networks have a slower requirement for session checking.

If active mediation references the customer's account balance stored in billing, then that portion of the billing system must have similar performance and availability levels as the mediation system to avoid being the 'weakest link' that fails. If rating must be performed to determine a call's cost, then that processing must also be provided with elevated performance and availability. The remainder of the billing system(s), where processing is performed after the fact, can be provided at a cheaper level of availability and performance.

Tags: Billing, Data Mediation, Transaction