DataStage Best Practices

Datastage: Job parameters

Employee id-149198

TCS-Internal
DataStage Best Practices

Contents:

DATASTAGE: JOB PARAMETERS..........................................................................................................1

INTRUDUCTION:................................................................................................................................................2
STATIC PARAMETERS:.......................................................................................................................................2
SLOWLY CHANGING PARAMETERS:.....................................................................................................................3
ENVIRONMENTAL PARAMETERS:.........................................................................................................................3
DYNAMIC PARAMETERS:....................................................................................................................................3

Intruduction:
ETL job parameters are values you can change when you run the job, as opposed to properties
you can only change by modifying and recompiling the job, don't know if I am going to get to
101 but thought it was an eye catching title.

Static Parameters:

TCS-Internal
DataStage Best Practices

In programmer speak you could also call these constants. They are values you never expect to
change. So why make them a job parameter? Because they might change, or because you don't
want to have to try and remember the value every time you use it in a job.

Assume that you are having a static parameter called MAX_DATE which holds a high end date
value 2999-12-31 23:59:00. I use this on tables that track variant data such as a slowly changing
dimension type 2 tables, it provides a high end date for an open or current item. This
parameter gets copied into all jobs that update dimension tables, the good thing about having
it around is that I don't need to remember the value of the field; I just need enter MAX_DATE
into my transformer.

Another good thing is that come July of 2999 when a project team is assembled to address Y3K
problems they only have to update this date in one place and will not have to recompile jobs

Slowly Changing Parameters:

The most common example is the database password that **should** be changed every month.
At the current project the DBA teams change the password on the database and the DataStage
support teams change the project specific environment variable.

Environmental Parameters:

These are the job parameters that change value as you move from one environment to
another, for example from development into testing and into production. Typical values that
change are directory locations and database login details.

Dynamic Parameters:

These are parameters that get changed with every job run.
- A shell script starts a job with the dsjob command; it can set a dynamic parameter using the
Unix scripting language and set it in the dsjob command.
- A sequence job calls parallel jobs that require a dynamic parameter; the value can be set in a
User Variables stage or in the Job Activity Stage using BASIC commands or a BASIC routine.

E.g.
Currently I have two parameters PROCESS_ID and PROCESS_DATE. The first is a unique number
generated for every execution of a job. It goes into the ODS tables to indicate which job
execution loaded the data so we can rollback loads or identify data volumes. The
PROCESS_DATE is the delta data we are processing as we do daily loads.

The PROCESS_DATE is retrieved from a database table via a Unix script. The script uses a
simple DB2 command line statement to retrieve the date and then uses the dsjob command to
set the parameter and call a sequence job. It populates PROCESS_ID with an eight character
string representation of the PROCESS_DATE.

The Sequence job uses a User Variable stage to append a unique 2 character sequence job code
to the PROCESS_ID to make it unique for this sequence job. This User Variable is then used in

TCS-Internal
DataStage Best Practices

each job activity stage with an additional 2 character code added that is unique for each job
activity stage within the sequence job. Therefore each parallel or server job across the entire
project gets a unique PROCESS_ID each time they run.

TCS-Internal