Error Handling - Informatica B2B Data Transformation

Error Handling - Informatica B2B Data Transformation
Error Handling - Informatica B2B

Data Transformation
SUBHASIS DEY (245186)

DWAIPAYAN MUKHERJEE (173240)
ANAND KUMAR SINGH (360812)
INFORMATICA COE
KOLKATA, INDIA
MARCH- 2016
Release ID: QBDW.INFORMATICABP
C3: Protected Page1 of 14
TABLE OF CONTENTS
1.
INTRODUCTION.....................................................................................3
2.
PURPOSE.............................................................................................. 3
3.
TARGET AUDIENCE................................................................................3
4.
OVERVIEW............................................................................................ 3
4.1
4.2
5.
TYPE OF ERRORS IN B2B DATA TRANSFORMATION SOLUTIONS..............................................4

SEVERITY OF ERRORS.................................................................................................... 4
ERROR TRACKING AND HANDLING MECHANISMS IN B2B DT.....................5
5.1
5.2
5.3
BUILT IN ERROR TRACKING MECHANISM IN B2B DT.............................................................5

DEBUGGING TECHNIQUE FOR ERROR TRACKING IN B2B DT CODE..........................................5
5.2.1
Add debug component in the code............................................................5
5.2.2
Schema Modification.................................................................................6
5.2.3
Error Data in a Different File......................................................................6
EXCEPTION HANDLING MECHANISM IN DT CODE................................................................6
5.3.1
Concept of Optional Property....................................................................6
5.3.2
Setting the Optional Property....................................................................7
optional property can be set in the following ways:................................................7
5.3.3
Writing a Failure Message to the User Log................................................7
5.3.4
Setting User-Log Location at Runtime.......................................................9
5.3.5
Creation of customized error report:.........................................................9
5.3.6
Example.................................................................................................. 10
6.
ERROR HANDLING STRATEGY WHEN DATA TRANSFORMATION
INTEGRATED WITH POWERCENTER....................................................................12
6.1
BLOCK DIAGRAM
FOR ERROR HANDLING
:........................................................................13
1. Introduction
This document is primarily providing the basic concepts about error handling in Informatica B2B Data
transformation (DT). This document will provide a basic idea about deferent types of error those are
encounter during processing file in using Informatica B2B data transformation and how to handle those
error in DT.
2. Purpose
This document aims to
Provide guidelines to the development team to handle error in Informatica B2B Data transformation.
Provide steps to debug a DT code.
Detection of errors in the transformation leading to successive refinement of the transformation

logic during an iterative development cycle.
3. Target Audience
This document is written for professional Informatica developers to:
Develop DT code for processing a file.
Create error file that contain error information during processing the file.
4. Overview
The challenge for B2B Data Transformation (B2B DT) based solutions is to create efficient and accurate
processes for transforming data to appropriate intermediate data formats and to subsequently transform
data from those formats to correct output formats. Error handling strategies are a core part of assuring
the accuracy of any transformation process.

An expected error is an error condition that we do anticipate to occur periodically. For example, a printer
running out of paper is an expected error. In a B2B scenario this may correspond to a partner company
sending a file in an incorrect format although it is an error condition it is expected from time to time.
Usually processing of an expected error is part of normal system functionality and does not constitute a
failure of the system to perform as designed.
Unexpected errors typically occur when the designers of a system believe a particular scenario is
handled, but due to logic flaws or some other implementation fault, the scenario is not handled. These
errors might include hardware failures, out of memory situations or unexpected situations due to
software bugs and even coding mistakes.
4.1 Type of Errors in B2B Data Transformation Solutions

There are several types of errors possible in a B2B data transformation. The common types of
errors that should be handled while designing and developing are:
Logic errors
Errors in structural aspects of inbound data (missing syntax etc.)
Value errors
Errors reported by downstream components (i.e., legacy components in data hubs)
Data-type errors for individual fields
Unrealistic values (e.g., impossible dates)
Business rule breaches
4.2 Severity of Errors

Not all the errors in a system are equally important. Some errors may require that the process be halted
until they are corrected (e.g., an incorrect format of source files). These types of errors are termed as
critical/fatal errors. Whereas there could be a scenario where a description field is longer than the field
length specified, but the truncation does not affect the process. These types of errors are termed as
warnings. The severity of a particular error can only be defined with respect to the impact it creates on the
business process it supports. In B2B
DT the severity of errors is classified into the following categories:
Warning: A warning about a possible error. For example, B2B Data Transformation
generates a warning event if an operation overwrites the existing content of a data holder.
The execution continues.
Failure: A component failed. For example, an anchor fails if B2B Data Transformation
cannot find it in the source document. The execution continues.
Optional Failure: An optional component, configured with the optional property, failed.
For example, an optional anchor is missing from the source document. The execution
continues.
Fatal error: A serious error occurred, for example, a parser has an illegal configuration.
B2B Data Transformation halts the execution.
Unknown: The event status cannot be determined.
5. Error Tracking and Handling Mechanisms in B2B DT

A failure is an event that prevents a component from processing data in the expected way. An anchor
might fail if it searches for text that does not exist in the source document. A transformer or action might
fail if its input is empty or has an inappropriate data type . The common techniques that would help a
B2B DT designer in designing an error handling strategy are summarized below.
5.1 Built in Error tracking mechanism in B2B DT
B2B DT event log: This is a B2B DT specific event generation mechanism where each event
corresponds to an action taken by a transformation such as recognizing a particular lexical
sequence. It is useful in the troubleshooting of work in progress, but event files can grow very
large, hence it is not recommended for production systems. It is distinct from the event system
offered by other B2B DT products and from the OS based event system. Custom events can be
generated within transformation scripts. Event based failures are reported as exceptions or other
errors in the calling environment.
B2B DT Trace files: Trace files are controlled by the B2B DT configuration application.
Automated strategies may be applied for the recycling of trace files
Custom error information: At the simplest levels, custom errors can be generated as B2B DT
events (using the Add Event Action). However if the event mechanism is disabled for memory or
performance reasons, these are omitted.
5.2 Debugging technique for Error tracking in B2B DT code

5.2.1
Add debug component in the code
This method of error handling underlines the usage of the built-in capabilities of B2B DT for most
basic errors. The debug of a B2B DT parser/serializer can be done in multiple ways.
Highlight the selection of an element on the example source file.
Use a Write Val component along with disabling automatic output in the project property.
Use of the disable and enable feature for each of the components.
Run the parser/serializer and browse the event log for any failures.
All the debug components should be removed before deploying the service in
production.
5.2.2
Schema Modification
This method of error handling demonstrates one of the ways to communicate the erroneous
record once it is identified. The erroneous data can be captured at different levels (e.g., at field
level or at record level). The XML schema dictates adding XML elements into the schema
structure for the error data and error message holding. This allows the developer to validate
each of the elements with the business rules and if any element or record does not conform to
the rules then that data and a corresponding error message can be stored in the XML structure.
5.2.3
Error Data in a Different File
This approach stresses the point of storing the erroneous records or elements in a separate file
other than the output data stream. This method is useful when a business critical timeline for the
data processing cannot be compromised for a couple of erroneous records. This method allows
the processing for the correct records to be done and the erroneous records to be inspected and
corrected as a different stream function. The business validations are done for each of the
elements with the specified rules and if any of the elements or records fails to conform, they are
directed to a predefined error file. The path to the file is generally passed in the output file for
further investigation or the path of the file is a static path upon which a script is executed to send
those error files to operations for correction.
5.3 Exception Handling Mechanism in DT Code

A failure is an event that prevents a component from processing data in the expected way. An
anchor might fail if it searches for text that does not exist in the source document. A transformer
or action might fail if its input is empty or has an inappropriate data type.
A failure can be a perfectly normal occurrence. For example, a source document might contain
an optional date. A parser contains a Content anchor that processes the date, if it exists. If the
date does not exist in a particular source document, the Content anchor fails. By configuring the
transformation appropriately, we can control the result of a failure. In the above example, we
might configure the parser to ignore the missing data and continue processing. The event log
displays warnings about failures. In addition, we can configure a transformation to write a failure
message in a user log.
5.3.1
Concept of Optional Property
Use optional property of the anchor to control the behavior of a transformation when a failure
occurs. If the optional property of a component is not selected, a failure of the component
causes its parent to fail. If the parent is also non-optional, its own parent fails, and so forth.
For example, suppose that a Parser contains a Group, and the Group contains a Marker. All
the components are non-optional. If the Marker does not exist in the source document, the
Marker fails. This causes the Group to fail, which in turn causes the Parser to fail.
Pictorially, we can represent these relationships in the following way:
Parser
-Failed
GroupFailed
Marker-Failed
If the optional property of a component is selected, a failure of the component does not bubble
up to the parent. In the above example, suppose that the Group is optional. The failed Marker
causes the Group to fail, but the Parser does not fail.
Parser
-Succeeded
GroupFailed
Marker-Failed
5.3.2
Setting the Optional Property
Optional property can be set in the following ways:
Edit the advanced properties of a component in the script.

Right-click the component and then click Make Optional or Make Mandatory.
5.3.3
Writing a Failure Message to the User Log
We can configure a component to output failure events to a user-defined log. For example, if an
anchor fails to find text in the source document, it can write a message in the user log. This can
occur even if the anchor is defined as optional, so that the failure does not terminate the
transformation processing.
The user log can contain information such as:
Failure level: Information, Warning, or Error
Name of the component that failed
Failure description
Location of the failed component in the script
To define the user log output, assign the on_fail property of the appropriate transformation
components. The following components have an on_fail property:
Parsers and anchors
Serializers and serialization anchors
Mappers and mapper anchors
The on_fail property can have the following values:
Log Error: Writes an error message containing the VarLastFailure system variable to the
user log.
Log Warning:. Same as Log Error, but displays the message as a warning rather than an
error.
Log Info: Same as Log Error, but displays the message as information rather than an error.
Custom Log: Runs a serializer that writes a custom message to the user log or another
location. For more information, see Custom Log
Notify Failure: Triggers a notification.
The following example illustrates a Marker anchor with a LogInfo configuration:
If the Marker does not exist in the source document, the system writes the following entry in the
user log:
*** INFO *** : Marker, [MyParser[11].Marker], Can't find Marker<optional>('Height').
The user log is an ASCII text file. On Windows platforms, the default location of the user log is:
c:\Informatica\DataTransformation\UserLogs
On UNIX platforms, the default location is:
<INSTALL_DIR>/UserLogs
By default, each execution of a transformation generates a user log having a unique name:
<service_name>+<unique_string>.log
5.3.4
Setting User-Log Location at Runtime
A transformation can set the user-log location at runtime by using SetValue actions to assign the
following system variables. Set the phase property of SetValue to initial. Ensure that SetValue
runs before any component that writes to the user log.
In the following example, a SetValue action sets the user-log directory to c:\mydirectory.
Note:
VarServiceInfo>StandardError>StandardErrorDir: Directory path of the user log.
VarServiceInfo>StandardError>StandardErrorName: File name of the user log.
5.3.5
Creation of customized error report:
A component can be configured to output failure events to a user-defined log. For example, if an
anchor fails to find text in the source document, it can write a message in the user log. This can
occur even if the anchor is defined as optional so that the failure does not terminate the
transformation processing.
The DT Error handling code will specifically perform the following functions for handling any
error during processing of the file:
Determine that an error occurred while executing a Map.
Determine that error occurred during evaluation of the expression of a Mapping Group
Determines the Mapping Group Name.
Extract Source and Target field names from the Map name.
Extract Source Parent Element Name and Target Parent Element Name
Using Source Parent and Source Field Name, Retrieve value of the Source field.
Retrieves the Key Field values from the Current Source Record
Write a record in the Error File for the data elements to be write:
Key Field name and values of the Current Source Record

Types of error
Source and Target field names and value
The Custom Log component can be used as the value of the on_fail property. In the event of a
failure, the Custom Log component runs a serializer that prepares a log message. The system
writes the message to a specified location.
5.3.6
Example
For capturing failure in any "Calculate Value" and "Map" statement we need to create
Expression failure notification for calculate Value failure and map failure notification for map
failure and call that notify handler from each map or Calculate Value statement.
Create Notification failure for Map and Expression:
Call Expression Failure notification form each expression in code:
Call Map Failure notification from each map statement of the code:
Create Notification handeler handle each error notification create the error masage:
Call sterilizer code to create error report:
6. Error Handling Strategy when Data Transformation

Integrated with Power Center
In a production environment the turnaround time of the processes should be as short as possible and as
automated as possible. Using B2B DT integration with Power Center these requirements should be met
seamlessly without intervention from IT professionals for error reporting, the correction of the data file
and the reprocessing of data. Below given steps to implement Error Handling in B2B DT with Power
Center Integration:
Create DT code with to process source data with customize error handling logic.
Process the source file in Power Center with that DT code.
If there are any error find during file processing B2B DT will create an error file to specific
location
Power Center will email that customize error file to user.
User will rectify the error in source file and reprocess the same.
6.1 Block Diagram for error handling:

Error Handling - Informatica B2B Data Transformation

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Error Handling - Informatica B2B Data Transformation

Uploaded by

Copyright:

Available Formats

Error Handling - Informatica B2B Data Transformation