#oopin

$%/%&/&$$'

Structure in Informatica

Informatica / Shared Services Group Internal IT Service Delivery

Gitika Khanchandani Varun Ajmera itika!khanchandani"tcs!com varun!ajmera"tcs!com

TCS Public

Looping Structure in Informatica

In a normal day to day programming, we might encounter various scenarios where a loop might be required. However, as we know that in Informatica a looping structure is not available (e cept when using a !ava "ransformation# and due to this reason we might resort to a stored procedure. $t times, using a stored procedure could cost more time (due to various reasons# and hence cause delay in the overall system. %e had encountered a similar scenario in our pro&ect, where a typical looping structure was required. "here were multiple offerings corresponding to a vendor and in our reports, all the offerings corresponding to the vendor are supposed to be shown in a single record. "he offerings would be hash separated or in other words to be shown as 'offering( ) offering*'. "he implementation of this requires a loop as the ne t values in the record are to be appended to the previous value. +elow is an e ample of the source data, which is used as the input to the mapping (all the data mentioned below is hypothetical#.

Departmen (ustomer)*ame
( ( ( ( ( * * * 6 6 * ,- .urnitures Ltd. ,- .urnitures Ltd. "5 .urnishings "5 .urnishings ,- .urnitures Ltd. 8wi9 "ech Ltd. -im +ros Ltd 8wi9 "ech Ltd. -< .ittings -< .ittings 8wi9 "ech Ltd.

+fferin )*ame
/esks 2hairs /esks 2hairs Softboard :onitors L2/ :onitors ;etwork $dapter =verhead lamps Switches ,rinter

+fferin )Value)I*,
*01 (34 *64 (74 611 (411 (*111 *4111 011 *6 61111

;ow, the desired output is as below.

"2S Internal

Looping Structure in Informatica

Departme (ustomer)*ame
( ( * * 6 ,- .urnitures Ltd. "5 .urnishings 8wi9 "ech Ltd. -im +ros Ltd. -< .ittings

+fferin )*ame
/esks ) 2hairs ) Softboard /esks ) 2hairs

+fferin )Value)I*,
*01 ) (34 ) 611 *64 ) (74

:onitors ) ;etwork $dapter ) (411 ) *4111 ) 61111 L2/ :onitors =verhead lamps ) Switches (*111 011 ) *6

,lease note that the hash separated values can be in any order but the corresponding values should be consistent, ie., if the first row has =ffering>;ame as '2hairs ) /esks ) Softboard' then the corresponding =ffering>?alue>I;- would be '(34 ) *01 ) 611'. "he reason for this is that the mapping e pects a sorted input data, and the sorting can be as per user's discretion.

Implementation
@@@ "he backbone of the logic is that the variable ports store the data until it is changed. @@@ It is very important that the source data should be sorted (asc A desc# according to the primary key, before it is passed to any transformation in the mapping. "his can also be achieved by introducing a sorter transformation in the mapping. /rawing ( shows the mapping m>Looping>8 ample. Boing forward, each transformation of the mapping will be e plained.

"2S Internal

Looping Structure in Informatica

L = = , > I; , C " ( = ra c le #

S 5 > L = = , > I; , C"

s rt> L o o p

lk p > L o o p > I n p u t

e p>Loop

fil> L o o p

L==,>=C ",C " (= ra c le #

Drawing 1: Mapping m_Looping_Example

Source and Source -ualifier. "here can be multiple source to a particular source qualifier and the &oining condition may be provided in the S5L 5uery. "he data which is fetched can either be sorted (sorter transformation not required then# or unsorted. Sorter Transformation. "he input data is sorted according to the primary A composite keys. "he reason why sorting is required will be e plained later. Here in this case, the values are sorted on the basis of /epartment, 2ustomer>;ame and =ffering ;ame. $lthough the composite key is /epartment and 2ustomer>;ame, we can sort the data according to =ffering>;ame so that the final hash separated values are shown alphabetically. "he output of this transformation would be as belowD

Departme (ustomer)*ame
( ( ( ( ( ,- .urnitures Ltd. ,- .urnitures Ltd. ,- .urnitures Ltd. "5 .urnishings "5 .urnishings

+fferin )*ame
2hairs /esks Softboard 2hairs /esks

+fferin )Value)I*,
(34 *01 611 (74 *64

"2S Internal

Looping Structure in Informatica

* * * * 6 6

8wi9 "ech Ltd. 8wi9 "ech Ltd. 8wi9 "ech Ltd. -im +ros Ltd -< .ittings -< .ittings

:onitors ;etwork $dapter ,rinter L2/ :onitors =verhead lamps Switches

(411 *4111 61111 (*111 011 *6

"he output of this transformation is sent to a look up transformation and an e pression transformation.

#ookup Transformation. "he lookup transformation is used to get the total no. of records for a particular primary A composite key. "he reason why we require count is to counter check that we do not miss any values to be appended. "he values that are passed to the look up are /epartment and 2ustomer>;ame. "he value that we retrieve is the total no. of records for the particular combination. 8g. %hen /epartment * and the 2ustomer>;ame is 8wi9 "ech Ltd. the return value is 6.

/0pression Transformation. "his is the transformation where the looping logic is applied. "he screen shot of the ports of the transformation is shown below, for more clarity.

"2S Internal

Looping Structure in Informatica

$ll the input, variable and output ports are prefi ed with 'in>', 'v>' and 'o>', respectively. .irst four input ports are brought from the sorter transformation used earlier. "he port 'in>"otal>2ount' contains the count value from the lookup transformation, as discussed above. "he following variable ports are used in the mapping to implement the logic. v)+ld)12)Key E "his port holds the previous value of the primary A composite key. In this case, the composite key values are concatenated to avoid individual comparisons of the keys. "he variable port v>;ew>,:><ey will hold the previous value until it is changed, hence to record this value we use v>=ld>,:><ey port. v)+fferin )*ame)3sh)Sep)old E "he idea is same as that of v>=ld>,:><ey. "he difference is that it is used to store the old hash

"2S Internal

Looping Structure in Informatica

separated values for =ffering>;ame. ,lease note, that the data type is string and the precision is increased to a sufficiently large value. v)+fferin )Value)3sh)Sep)old E Similar to 'v>=ffering>;ame>Hsh>Sep>old'. ,lease note that the datatype for this port is string (even though the =ffering>?alue port was of double datatype# and the precision is a sufficiently large value. v)*e4)12)Key E "his port holds the concatenated composite key value. "his is done to avoid subsequent comparisons for all the keys and then deciding if the record is new or old.

v)(ompare E Here we compare the values of v>=ld>,:><ey and v>;ew>,:><ey. If the values match then it means that the =ffering>;ame and =ffering>?alue values have to be appended to the old values stored in 'v>=ffering>;ame>Hsh>Sep>old' and 'v>=ffering>?alue>Hsh>Sep>old', respectively.

"2S Internal

Looping Structure in Informatica

v)+fferin )*ame)3sh)Sep E /epending on the 'v>2ompare' value, this port will either concatenate the new value to the value present in 'v>=ffering>;ame>Hsh>Sep>old' or will hold the new input value only.

v)+fferin )Value)3sh)Sep 'v>=ffering>;ame>Hsh>Sep'.

Similar

to

"2S Internal

Looping Structure in Informatica

v)(ounter E %e count the total no. of values that are concatenated till now for a particular primary A composite key. If 'v>2ompare' value 1 then the counter is reset to (.

"he output values o>2ounter, o>=ffering>;ame and o>=ffering>?alue hold the values from v>2ounter, v>=ffering>;ame>Hsh>Sep and

"2S Internal

Looping Structure in Informatica

v>=ffering>?alue>Hsh>Sep, respectively. "hese values are then passed to a filter transformation. Dry Run: %hen the first record arrives at the e pression transformation the values of the above variable ports will be as follows v)+ld)12)Key 5 66 v)+fferin )*ame)3sh)Sep)old 5 66 v)+fferin )Value)3sh)Sep)old 5 66 v)*e4)12)Key 5 6%1, 7urnitures #td!6 v)(ompare 5 $ v)+fferin )*ame)3sh)Sep 5 6(hairs6 v)+fferin )Value)3sh)Sep 5 6%896 v)(ounter 5 % %hen the ne t record arrives following would be the values of the variable portsD v)+ld)12)Key 5 6%1, 7urnitures #td!6 v)+fferin )*ame)3sh)Sep)old 5 6(hairs6 v)+fferin )Value)3sh)Sep)old 5 6%896 v)*e4)12)Key 5 6%1, 7urnitures #td!6 v)(ompare 5 % v)+fferin )*ame)3sh)Sep 5 6(hairs : Desks6 v)+fferin )Value)3sh)Sep 5 6%89 : &;$6 v)(ounter 5 & $nd hence, the looping structure is implemented. 7ilter Transformation. "he ob&ective of the filter transformation here is to pass only those values where the 'in>"otal>2ount' and 'in>2ounter' ('o>2ounter' from e pression transformation# match. If the values does not match then it means that

"2S Internal

10

Looping Structure in Informatica

more values are yet to concatenated to the '=ffering>;ame' and '=ffering>?alue' ports.

@@@ "he reason why we sort the data earlier is to loop all the data for a particular key at once. =therwise, if the data is not sorted and the mapping is e ecuted, there may not be any records in the target with hash separated values, as the total count (retrieved using the look up# will not match the counter value in the e pression transformation, and finally will be filtered out.

"2S Internal

11

Looping Structure in Informatica

"he values which are passed from the filter are then loaded into the target.

"2S Internal

12