Skip to main content

Consolidations

Automated Consolidations (Main Menu / Admin / Consolidations) offer the possibility to enhance data quality in a variety of ways, e.g. by merging duplicates or by connecting assets with links based on their properties.

Introduction​

Txture is a great place to integrate data from various sources. However, joining different Data Sources together can lead to issues, such as duplicated assets or missing links. Consolidations solve this problem in an automated fashion. Use the Merge Assets Consolidation to get rid of duplicated assets, and the Generate Links Consolidation to connect assets. The assets to be linked / merged are determined via match criteria.

Example: Merging Servers with equal IP Addresses​

When importing Servers from several Data Sources, and the Data Source contents overlap, a single Server may be imported as two separate assets. Merging them together in Txture is possible with a Merge Consolidation based on the IP Address of the server:

Note that the match condition is IP address match which is a smart matcher specifically for IPv4 Addresses. For example, 10.3.0.4 will be treated as equal to 010.003.000.004. Saving and executing this consolidation will merge together all Virtual Servers with the same IP Address.

Assuming that Application assets and Virtual Server assets are coming from different Data Sources, we still might want to connect them via a runsOn link. In this example, we establish the connection via the IP Address property which is present both on the Applications as well as on the Virtual Servers:

Merge Assets​

The Merge Assets Consolidation merges assets based on their properties. In order to configure a Merge Assets Consolidation, you need to specify the Type of the assets you want to merge, as well as one or more Merge conditions. Please note that the conditions are implicitly and-connected: any two assets of the specified type will be merged if and only if they match all merge conditions.

Each Match Condition consists of the property which should be matched (only properties linked to the selected type will be available for selection), and a match mode. The available match modes are:

  • Exact Match: The Property values of the two assets must match exactly for the selected property.
  • IP Address Match: Special match mode for IPv4 Addresses. This match mode will ignore leading zeroes in all four parts of the address.
  • Case Insensitive Match: Behaves the same as Exact Match, except that differences in casing will be ignored.

When two or more assets are being merged, their property values and links will also be merged:

  • If one merge candidate has property P set to p, and the others do not have it assigned at all, the merged result will have P set to p.
  • If several merge candidates have property P assigned, and P is a multi-valued property, the merged result will have P set to the union of all values.
  • If several merge candidates have property P assigned, and P is a single-valued property, the merged result will have P set to any of those values (this is intentionally unspecified).
  • links behave in the same way, except that links with the same source and target asset in the merged result will be merged as well.

The Generate Links Consolidation creates new links between existing assets, based on their properties. The configuration of a Generate Links Consolidation involves selecting a link type (which also fixes the source and target asset types) and specifying the match conditions.

Unlike Merge Consolidations, for Generate Links Consolidations you need to select the Source Property and Target Property separately. The source property is the property in the asset which will become the source of the link, and the target property is a property on the potential link target asset. Please note that property selection will be restricted to the properties linked to the selected source and target asset types, respectively. The final part of the match criterion is the match mode. The available match modes are the same as for Merge Assets Consolidations.

When working with assets and links in Txture, many different cases may arise, most of which are valid and meaningful. However, there is one particular setup which provides no semantic value and may lead to technical issues:

Two (or more) links between the same assets with the same link type provide no additional information and may become rather confusing to work with. Txture refers to such cases as Duplicate Links and attempts to avoid them whenever possible (e.g. when merging assets), but in some cases this scenario may still occur (e.g. two independent users may manually create this link at the same time). The Fix Duplicate Links Consolidation can be used to eliminate duplicate links after they've been created.

The Fix Duplicate Links Consolidation will merge links if all of the following conditions apply:

  • The links have the same source asset
  • The links have the same target asset
  • The links have the same link type

This consolidation can be configured with a query to restrict the link source assets and a query to restrict the link target assets. Both queries are optional; if no query is given, all assets are considered.

Caution when merging links with properties

If you've assigned properties to links, it is not guaranteed which property values will be stored in the resulting link after you merge them.

This action is permanent

Once you run the Fix Duplicate Links Consolidation, the changes made cannot be undone. Make sure you select the correct sources and targets.

Default Consolidations in your Txture instance​

If you're using Txture Transformation Cockpit or Business Case Builder, your Txture instance already has three consolidations configured. Those three consolidations are required and must remain enabled in the instance:

  • Target Architecture Asset Update
  • Target Architecture Property Update
  • Target Architecture Cleanup

Target Architecture Asset Update​

The Target Architecture Asset Update Consolidation manages the contents of your Target Architecture Landscape Scenario. It is responsible for fetching up-to-date cloud proposals from the Txture service network for the applications in your AS-IS scenario, and saving them as assets in your Target Architecture Landscape Scenario. The selected proposal depends on the cloud migration goals, the application and its deployment stack, and the chosen migration strategy.

If the contents of your AS-IS branch change after a proposal has been stored in the Target Architecture, this task will update the proposal in the Target Architecture accordingly. Manually selected migration proposals will (by default) not be touched by this task.

This consolidation is available if:

  • your Txture license has the cloud module enabled AND
  • your Information Model has the Cloud tag and contains no errors.

The Target Architecture Asset Consolidation executes the process outlined in the figure above:

  • It extracts the assets that represent your applications and their deployment stack from the AS-IS Scenario of your Txture repository.
  • The data is anonymized and sent to the Txture Cloud Knowledge Engine.
  • The resulting cloud migration proposal is returned to your Txture instance and saved in the Target Architecture Landscape Scenario.

The Txture team recommends to schedule an instance of this task on a regular basis (e.g. once per hour) to ensure that your Target Architecture Landscape Scenario is up-to-date with your AS-IS Scenario.

This consolidation has the following parameters:

  • Batch Size: The number of applications to process per run. For example, if you have 100 applications in your AS-IS scenario, the first execution of the consolidation will take 10 applications and perform the task outlined above. When the task is invoked again (e.g. by a scheduling trigger), the next 10 applications will be considered. Once all 100 applications have been processed, the consolidation will start again with the first 10 applications. If the Txture Server is restarted, the cycle will be reset as well and the first 10 applications will be processed again.

  • Scope Query: This query allows to restrict the set of applications which should be updated. If no scope query is specified, all applications will be updated. Your scope query can return arbitrary assets; however only resulting applications will be considered for further processing. If your scope query returns no assets, the consolidation will not have any effect. If you want to learn more about how to use queries, please refer to the documentation about the Query Bar.

  • Force Overwrite: If this setting is enabled, all data in the Target Architecture (even manually selected proposals) will be overwritten by the automatically selected proposal. If this setting is disabled, manual changes in the Target Architecture will never be overwritten.

Use 'force overwrite' with Caution

The 'force overwrite' parameter can be useful to revert back to the automated workflow after a proposal has been manually selected. However, if it isn't used with care, you can unintentionally overwrite manual work performed by you or other Txture users.

If you enable this setting, we highly recommend specifying a scope query as well to avoid unintentional overwrites in your Target Architecture.

Target Architecture Property Update​

The Target Architecture Property Update consolidation will update several properties of all assets in your Target Architecture Landscape Scenario and keep them up-to-date with the latest information from the Txture Cloud Knowledge Base. For example, if you use a cloud product in your Target Architecture, and this cloud product is renamed, this consolidation will update the product name stored in your assets. This consolidation is also useful in cases where the Txture structure has changed and new cloud-related properties have been added, as this consolidation will populate these properties with the appropriate data (where possible).

The process for this consolidation is as follows:

  • A subset of applications (according to the batch size setting) in your Target Architecture Landscape Scenario will be collected.
  • The assets in the deployment stacks of the selected applications will be resolved and collected.
  • A batch request for the latest cloud product information will be sent to the Cloud Knowledge Base.
  • The updated cloud product information will be stored in your Target Architecture Landscape Scenario.

In contrast to the Target Architecture Asset Update, this consolidation will never add or remove assets (or links) in the Target Architecture Landscape Scenario; it will only update their property values.

This consolidation has the following parameters:

  • Batch Size: The number of applications to process per run. For example, if you have 100 applications in your Target Architecture Landscape Scenario, the first execution of the consolidation will take 10 applications and perform the task outlined above. When the task is invoked again (e.g. by a scheduling trigger), the next 10 applications will be considered. Once all 100 applications have been processed, the consolidation will start again with the first 10 applications. If the Txture Server is restarted, the cycle will be reset as well and the first 10 applications will be processed again.

  • Scope Query: This query allows to restrict the set of applications which should be updated. If no scope query is specified, all applications will be updated. Your scope query can return arbitrary assets; however only resulting applications will be considered for further processing. If your scope query returns no assets, the consolidation will not have any effect. This parameter is useful if you need to update a specific subset of applications immediately.

Target Architecture Cleanup​

The Target Architecture Cleanup Consolidation removes all assets from the Target Architecture Landscape Scenario for applications which are not present in the AS-IS Scenario anymore.

FAQ​

Can consolidations be scheduled and repeated?​

Yes. Consolidations can be scheduled in the Job Scheduling View. Please note that consolidations are always the last actions to be executed in a Scheduling Group (i.e. any Importer in a group will always run before any consolidation). In order to ensure consistent results across repeated executions, please make sure that your match conditions are as specific as possible.

What will be the source of a merged asset?​

If you merge two or more assets which were created by Importers, the merged asset will reference all of the importers of the original assets as source. If you delete an Importer, it will be removed from the list of sources. Note that the same logic also applies to the sources of individual properties of the merged asset.