Reduce migration impacts by merging outbound interfaces

Summary

When moving from an old application to its replacement, a careful interface design will allow the purposeful isolation of downstream interface recipients from the application's migration process. Careful design will reduce the impact of the downstream systems and support the removal of the migration software once the old application is fully decommissioned.

Actions

When the downstream interface's format or field codeset will change, consider changing these as an initial step performed before the core migration commences
Evaluate the desirability or appropriateness of maintaining a single downstream interface
If required, design a merge to be easily removable once the migration is complete
Avoid distorting the long-term interface design due to the needs of the short-term migration
Take the opportunity to update (or create) the documentation for each interface as it is reviewed

Downstream Interfaces

Applications interact with external areas via upstream and downstream interfaces. Upstream interfaces convey new information to an application, whilst downstream interfaces receive the application's information and processing results. This column will focus on downstream interfaces.

Examples of downstream interfaces include financial files passed to the corporate general ledger (GL), generated reports, and bill details conveyed to external vendors for printing and posting to customers.

When an application is migrated from its existing instance to a new implementation / vendor, an impact assessment must be performed on all of its upstream and downstream interfaces.

A migration is an opportunity to reconsider each interface and judge whether it is required in the new implementation. Most interfaces will be easily evaluated and retained, but the opportunity should be taken to remove those no longer required. This avoids dragging historical decisions into the future. Taking this approach, some interfaces may be decommissioned rather than migrated.

A long-term perspective needs to be taken when performing the migration impact assessment. This perspective helps to avoid using short-term 'easy' solutions that, if they become entrenched, can be difficult or expensive to remove, and later constrain the new application's processing.

In some circumstances this long-term approach may not be possible due to interface complexity. If an interim solution is required, it should be consciously performed including a decision to realign with the long-term solution once the migrations is complete. To avoid constraining the long-term solution, care must be taken that the realignment is not canceled due to short-term cost or development needs. A history of such short-term 'savings' can lead to higher long-term costs as each interim solution must be worked around to deploy future changes.

To merge or not...

It is not appropriate to merge all downstream interfaces. In many cases a new interface will be created when the new application is commissioned, and the old interface will be decommissioned once the migration from the old application is completed.

Some downstream interfaces are kept separate to allow correlation of an interface's content to a specific (originating) application. An example of this is a GL file which, to support reconciliation and investigation, aligns an interface's content to the events in a specific application. A new application will utilise a new interface that tracks events that occur only in the new application. During migration, the separate financial transactions in these two interfaces will support the migration reconciliation of transferred financial postings.

Interfaces may be quite different based on new capabilities of the new application. For example, a new billing system may provide sophisticated bill formats unavailable in the existing billing system. In this case, there may be little benefit from merging its new bill image stream sent to the printer with that of the existing billing application. This would be an example of a complex interface not worth the high cost and integration effort.

Where downstream interfaces (e.g. reports) are passed to external parties (e.g. regulators, business users, customers), interface merging is likely to be the most appropriate solution. Many of these external groups are interested in the result of an application's processing but not how it was achieved. When the application's functions are migrated, the continuing accuracy of their (say) reports is of more interest. Indeed, there may be some effort expended to hide the existence and status of any migration (i.e. business as usual) to avoid alarm.

External parties (e.g. customers) may be unable to accept more than the (say) one file they normally receive. Their systems will have been built to a defined interface specification and are unlikely to easily address the temporary needs of a migration. This will drive a requirement to merge eliminating the customer's impact.

Changes of format and/or content

Taking a long-term perspective, a new application may introduce new code values or interface formats rather than be forced to fit within the existing application's definitions. In this circumstance, when a merge takes place there will be a need to manipulate one or both of the interface formats to provide a consolidated downstream interface.

To do this, either the old application's interface format or values will need to be transformed into the new application's interface format / values, or the new interface will need to be transformed to comply with the old format / values. Since the new interface will exist in the long-term, it is better to transform the old interface to comply with the new interface's definitions. The deployment of this transformation step would need to align with the corresponding changes in the downstream recipient.

If the old interface's format / codesets cannot be mapped to the new format values in a deterministic manner, then the appropriateness of merging these interfaces should be reconsidered (since the interfaces do not appear to be equivalent).

Merge Sophistication

Merging an interface can be more than just a mechanical addition of two (say) files. Merging an interface can include a format change (outlined above), recalculation of totals, and/or a record resorting. To perform this more sophisticated processing, the files may need to be loaded into memory and processed as a whole, or be processed externally through a sequence of steps to achieve a merged output in the desired format.

The performance impact of a sophisticated merge needs to be assessed ensuring that the additional processing can be completed in a timely manner given the data volumes and the environment's processing capabilities. This evaluation should consider the processing performed when no data has been migrated, midway through the migration process, and when all data has been migrated to the new application, If the performance assessment indicates problems, then the merge's approach or processing capabilities (e.g. available hardware) should be reassessed.

Outline of the migration approach

Prior to Migration: At this stage all processing is performed on the old application. No data has been migrated across to the new application. The 'merge' function can be deployed, including any reformat processing. If there is a interface change due to the new application, the downstream recipient would also be updated. The performance and accuracy of the merge process can be assessed in isolation - if there are problems, updates can be backed-out (including the downstream changes) with a reduced impact. The downstream application can confirm their processing of the new format is accurate and stable.
Pilot: A small segment of the old application's 'customer' data and processing is migrated to the new application. Testing is performed to confirm both the new application's processing and that downstream interfaces are processed as planned. Any problems found are limited to the Pilot 'customers' - the remaining customers continue to be processed on the old application. A 'back-out' at this stage can be performed, but will require more complexity, cost and processing impact due to the difficulty in moving the data back to its original (old application) location.
Migration: Having confirmed processing using the Pilot phase, the remainder of the old application's data and processing is moved to the new application. Throughout this phase, the merge processing must be monitored to proactively identify and address any performance problems.
Decommissioning: Once the old application's data and processing has been moved across to the new application, there should be no interface data provided by the old application (e.g. an empty file). The format and content of the new application's interface (file) should be the same as the post-merge file allowing the removal of the merge process and its direct connection to the downstream application.

Tags: Migration, Interfaces