Introduction to teradata utilities : Multiload

"Teradata MultiLoad" DEFINITION Teradata MultiLoad, also called "MultiLoad," "MLoad" or "ML," is a command-driven parallel load utility for high-volume batch maintenance on multiple tables and views of the Teradata Database. It is specially designed for high-speed batch creation and maintenance of large databases. OVERVIEW Teradata MultiLoad executes a series of MultiLoad commands and Teradata SQL statements written in a batch mode job script or interactively entered. The MultiLoad commands provide the session control and data handling specifications for the data transfer operations, and the Teradata SQL statements perform the actual maintenance functions on the Teradata RDBMS tables and views. A single MultiLoad job performs a number of different import and delete tasks on database tables and views: Each MultiLoad import task can do multiple data insert, update, and delete functions on up to five different tables or views; Each MultiLoad import task can have up to 100 DML steps; Each MultiLoad delete task can remove large numbers of rows from a single table. Teradata MultiLoad is characterized by its serialization feature. In certain scenarios, multiple changes are possible to occur to a single row in the same MultiLoad job. By using its serialization feature, Teradata MultiLoad automatically guarantees that the ordering of such operations is maintained correctly. If a MultiLoad job has only one import task, then Teradata FastLoad has to be chosen for it, since FastLoad will finish it much faster. However, the FastLoad utility works only on empty tables. A FastLoad job can not be used to: Insert additional data rows into an existing table; Update individual rows of existing tables; Delete individual rows from existing tables; Load data into multiple tables.

DATA SOURCE Teradata MultiLoad imports data from: Disk or tape files Input files on a network-attached workstation; Special input module (INMOD) routines that select, validate, and preprocess input data; Access modules; Any device providing properly formatted source data; The table or view in the database receiving the data can be any existing table or view for which the user has access privileges for the required maintenance tasks. MultiLoad command IMPORT has an optional parameter FORMAT, which is used to specify the format of the external data source. The format may be: FASTLOAD; BINARY; TEXT;

 UNFORMAT; VARTEXT.

OPERATING MODES Teradata MultiLoad runs in the following operating modes: Interactive mode; Batch mode. In interactive mode, Teradata MultiLoad uses terminal screen and keyboard as the standard output and input streams. In batch mode, MultiLoad uses > and < to redirect the standard output / input streams. To invoke Teradata MultiLoad in interactive mode, enter the following command (for Microsoft Windows): c:\ncr\bin\MultiLoad To invoke MultiLoad in batch mode, use the command below (for Microsoft Windows): c:\ncr\bin\MultiLoad [options] < infile > outfile Here, the infile is a Teradata MultiLoad job script file which includes all the required MultiLoad commands and Teradata SQL statements, whereas the outfile is the outfile is the output stream file. Character Sets Teradata MultiLoad supports Latin, Chinese, Japanese and Korean character sets, along with ASCII, EBCDIC, UTF-8 and UTF-16. It also supports user-defined session character sets. Task Status Reporting Teradata MultiLoad has three reporting methods to monitor the status of jobs that are still in progress and those that have just completed: MultiLoad utility messages provide job status reports, including: o Options messages that list the settings of MultiLoad task parameters; o Logoff/disconnect messages that list key run statistics. QrySessn (i.e., Query Session Utility) provides real-time, phase-oriented progress reports at selected intervals during the MultiLoad job; Statistics messages present information at the end of the MultiLoad job. Data Conversion Capabilities Teradata MultiLoad can redefine the data type specification of input data to match it with that of the destination column in the MultiLoad table on the target database. MultiLoad data conversion types are: Numeric-to-numeric (for example integer-to-decimal); Character-to-numeric; Character-to-date; Date-to-character. Use the DATADESC specification of the MultiLoad FIELD command to convert input data to a different data type before inserting it into the MultiLoad table on the Teradata Database. Note: Redundant conversions, like integer-to-integer, are legal and necessary to support the zoned decimal format. MULTILOAD COMMANDS

MultiLoad Command Summary The MultiLoad commands perform two types of activities: Support activities Support commands perform support activities to establish the MultiLoad sessions with the Teradata Database and define the operational support environment for Teradata MultiLoad. Established support environment options remain in effect until another support command changes them. Support commands do not specify a MultiLoad task; Task activities Task commands perform task activities to specify the actual processing of each

MultiLoad task. 1. Teradata MultiLoad Commands for Support Activities: COMMAND NAME ACCEPT FUNCTION Allows the value of one or more utility variables to be accepted from either a file or an environment variable Specifies the form of the DATE data type specifications for the MultiLoad job Writes messages to the specified destination Introduces commands and statements that execute when a preceding IF condition is false Delimits the group of MultiLoad commands that were subject to previous IF or ELSE conditions Introduces a conditional expression whose value initiates execution of subsequent commands Disconnects all active sessions and terminates MultiLoad on the client system. Specifies the LOGON command string to be used in connecting all sessions established by MultiLoad. Identifies the table to be used to journal checkpoint information required for safe, automatic restart of MultiLoad when the client or Teradata Database system fails. Identifies the destination of output produced by MultiLoad support environment. Invokes the specified external file as the current source of utility commands and Teradata SQL statements Assigns a data type and a value to a utility variable Suspends operation of MultiLoad and executes any valid local operating system command

DATEFORM

DISPLAY ELSE (see IF, ELSE, and ENDIF)

ENDIF (see IF, ELSE, and ENDIF) IF (see IF, ELSE, and ENDIF) LOGOFF

LOGON LOGTABLE

ROUTE MESSAGES RUN FILE

SET SYSTEM

2. Teradata MultiLoad Commands for Task Activities: COMMAND NAME FUNCTION

BEGIN MLOAD and BEGIN DELETE MLOAD

Specifies:

The kind of MultiLoad task to be executed; The target tables in the Teradata Database; The parameters for executing the task.

DML LABEL

Defines a label and error treatment options for a following group of DML statements. Indicates completion of MultiLoad command entries and initiates execution of the task. Used with the LAYOUT command to define a field of the data source record that is sent to the Teradata Database. Used with the LAYOUT command to define a field in the data source record that is not sent to the Teradata Database. Identifies the data source, the layout used to describe the data record, and optional conditions for performing DML operations. Introduces the record format of the data source to be used in the MultiLoad task. This command is followed by a sequence or combination of FIELD and TABLE commands. Supplies parameters to the LOGMECH command beyond those needed by the logon mechanism, such as userid and password, to successfully authenticate the user. Identifies the appropriate logon mechanism by name. Temporarily suspends the acquisition phase of a MultiLoad job.
Used with the LAYOUT command to identify a table whose column names and data descriptions are used as the field names and data descriptions of the data source records.

END MLOAD FIELD

FILLER

IMPORT

LAYOUT

LOGDATA

LOGMECH PAUSE ACQUISITION

TABLE

Teradata SQL Statements in MultiLoad MultiLoad supports a subset of Teradata SQL statements in order to define and manipulate the data stored in the Teradata Database. Consequently, other utilities do not have to be invoked to perform

routine database maintenance functions before executing MultiLoad utility tasks. The Teradata SQL statements supported by Teradata MultiLoad are listed below: COMMAND NAME ALTER TABLE FUNCTION Changes the column configuration or options of an existing table Adds a checkpoint entry to a journal table Collects statistical data for one or more columns of a table Stores or retrieves comment string associated with a database object Creates a new database, macro, table, or view

CHECKPOINT COLLECT STATISTICS COMMENT

CREATE DATABASE CREATE MACRO CREATE TABLE CREATE VIEW

Specifies a new default database for the current session DATABASE DELETE DELETE DATABASE DROP DATABASE Drops the definition for an empty database from the Data Dictionary DROP TABLE GIVE GRANT INSERT MODIFY DATABASE RELEASE MLOAD Removes a table from the database Transfers ownership of a database to another user Grants access privileges to a database object Insert new rows to a table Changes the options of an existing database Removes the access locks from the target tables in the Teradata Database This inhibits any attempts to restart the MultiLoad job when a MultiLoad task has been suspended or aborted. Note: A RELEASE MLOAD statement must be entered from BTEQ. It cannot be entered from MultiLoad. Removes rows from a table Removes all tables, views, and macros from a database

RENAME REPLACE MACRO REPLACE VIEW REVOKE SET QUERY_BAND

Changes the name of an existing table, view, or macro Redefines an existing macro or view Rescinds access privileges to a database object

Identifies type and source of query to determine prioritization of queries Overrides the collation specification for the current session Turns replication services on or off

SET SESSION COLLATION

SET SESSION OVERRIDE REPLICATION ON/OFF UPDATE Changes the column values of an existing row in a table

Phases Of MultiLoad
Preliminary Phase : -- Checks SQL commands or Multiload commands are valid . -- Establishes multiload sessions with Teradata. -- Creates Error Tables, Work Tables and log table. Error Tables : requires 2 error tables per target table. First contains constraint violations. Second contains UPI violations. Work Tables : holds the DML tasks requested and the input data that is ready to APPLY to AMPS. LOGTABLE : keeps a record of results from each phase of the load so that multiload knows proper point from which to restart. -- Apply utility locks to the target table.

DML Transaction Phase : -- All the SQL DML statements are sent to database.

-- PE parses the DML and generates a step-by-step plan to execute the request. -- This execution plan is then communicated to each AMP and stored in appropriate work tables for each target table. -- Later, during the Acquisition phase the actual input data will also be stored in the worktable so that it may be applied the Application Phase. -- Next, a match tag is assigned to each DML request that will match it with the appropriate rows of input data. The match tags will not actually be used until the data has already been acquired and is about to be applied to the worktable. This is somewhat like a student who receives a letter from the university in the summer that lists his courses, professors names, and classroom locations for the upcoming semester. The letter is a match tag for the student to his school schedule, although it will not be used for several months. This matching tag for SQL and data is the reason that the data is replicated for each SQL statement using the same data record.

Acquisition Phase : -- MultiLoad now acquires the data in large, unsorted 64K blocks from the host and sends it to the AMPs.(round Robin). -- The AMP hashes each row on the primary index and sends it over the BYNET to the proper AMP where it will ultimately be used. But the row does not get inserted into its target table, just yet. -- The AMP puts all of the hashed rows it has received from other AMPs into the worktables where it assembles them into the SQL. -- Now the utility places a load lock on each target table in preparation for the Application Phase.

Application Phase : The purpose of this phase is to write, or APPLY, the specified changes to both the target tables and NUSI subtables.

Cleanup phase : The utility looks at the final Error Code (&SYSRC). MultiLoad believes the adage, All is well that ends well. If the last error code is zero (0), all of the job steps have ended successfully (i.e., all has certainly ended well). This being the case, all empty error tables, worktables

and the log table are dropped. All locks, both Teradata and MultiLoad, are released. The statistics for the job are generated for output (SYSPRINT) and the system count variables are set. After this, each MultiLoad session is logged off.

The Teradata Multiload utility provides the capability to perform batch maintenance on tables (insert, update, delete and upsert). It loads data from external sources and provides the capability to restart jobs interrupted by errors, exceptions and failures. A Multiload job executes in 5 phases: 1. Preliminary Parses and validates all of the MultiLoad commands and Tera-data SQL statements in your MultiLoad job Establishes sessions and process control with the Teradata Database Submits special Teradata SQL requests to the Teradata Data-base Creates and protects temporary work tables and error tables in the Teradata Database 2. DML Transaction Submits the DML statements specifying the insert, update, and delete tasks to the Teradata Database 3. Acquisition Imports data from the specified input data source Evaluates each record according to specified application con-ditions Loads the selected records into the worktables in the Tera-data Database There is no acquisition phase activity for a MultiLoad delete task. 4. Application Acquires locks on the specified target tables and views in the Teradata Database For an import task, inserts the data from the temporary work tables into the target tables or views in the Teradata Data-base For a delete task, deletes the specified rows from the target table in the Teradata Database Updates the error tables associated with each MultiLoad task 5. Cleanup Forces an automatic restart/rebuild if an AMP went offline and came back online during the application phase Releases all locks on the target tables and views

Drops the temporary work tables and all empty error tables from the Teradata Database Reports the transaction statistics associated with the import and delete tasks. Multiload returns the returns codes described previously (0, 4, 8, 12) to the operating system or calling program. Restarts It is possible to restart a Multiload job in any of the phases after an er-ror or failure has interrupted an executing Multiload job. There are differences in the restart procedures in some of the phases that will be described. It is also possible to rerun an interrupted Multiload job in some of the phases which will also be discussed. A Multiload job creates a restart log table that must be present for a job to be restarted and must be dropped before a job can be rerun. It creates a work table for each input source. These tables must be pre-sent for a job to restart and must be dropped to rerun a job. Multiload also creates two error tables for each target table that must be present for a restart and that must be dropped to rerun a Multiload job. A Multiload job can not be restarted or rerun if these tables are not present for an interruption that occurs during the APPLICATION PHASE. Checkpoint frequency can be set for checkpoint to occur after a num-ber of records have been read from an input source. The checkpoints are taken during the acquisition phase. A restart after an interruption in the acquisition phase will start reading the input source at the point following the last checkpoint executed before the interruption. Interruptions that occur during the application phase will be restarted based on internal checkpoints created by the database. The following is a simple Multiload script that will be use to illustrate the Multiload restart capabilities. ? code 1 .logtable test.tranmllog; 2 .LOGON test/testuser,test; 3 4 .begin import 5 6 7 8 .layout tranlayout; mload tables test.tran checkpoint 20000;

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

.FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); .field CUST_NO * varchar (10); .field NAME * varchar (30); .field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); .field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); .field PURCHASE_ORDER_NUM * varchar (20);

31.dml label insdml; 32insert into test.tran.*; 33 34.import infile C:\DataIntegration\Data\TRAN2.DAT 35 36 format vartext '|' layout tranlayout

37

apply insdml;

38.end mload; 39.logoff; This script inserts rows into one table, TRAN. It set a checkpoint every 20000 input records. Restart after database reset The following is an example of an automatic restart that occurs for an interruption and reset that occurred during the acquisition phase. ? code 1 2 3 4 5 6 7 8 9 10 11 12 C:\DataIntegration\Scripts>mload -b < tran.ml **** 22:11:09 UTY2414 BRIEF option is enabled. ======================================================================== = = MultiLoad Utility Release MLOD.13.00.00.000 Platform WIN32 = =

======================================================================== = Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

======================================================================== **** 22:11:09 UTY2411 Processing start date: SUN AUG 09, 2009 ======================================================================== = Logon/Connection =

========================================================================

13 0001 .logtable test.tranmllog; 14 0002 .LOGON test/testuser,; 15 **** 22:11:11 UTY8400 Teradata Database Release: 12.00.01.24 16 **** 22:11:11 UTY8400 Teradata Database Version: 12.00.01.24 17 **** 22:11:11 UTY8400 Default character set: ASCII 18 **** 22:11:11 UTY8400 Current RDBMS has interval support 19 **** 22:11:11 UTY8400 Current RDBMS has UDT support 20 **** 22:11:11 UTY8400 Maximum supported buffer size: 1M

21 **** 22:11:11 UTY8400 Data Encryption supported by RDBMS server 22 **** 22:11:12 UTY6211 A successful connect was made to the RDBMS. 23 **** 22:11:12 UTY6217 Logtable 'test.tranmllog' has been created. 24 25 0003 .begin import 26 27 28 29 30 31 32 0004 .layout tranlayout; 33 0005 34 0006 35 0007 36 0008 37 0009 38 0010 39 0011 40 0012 41 0013 42 0014 43 0015 44 0016 45 0017 46 0018 47 0019 48 0020 .FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); .field CUST_NO * varchar (10); .field NAME * varchar (30); .field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); mload tables test.tran checkpoint 20000; ======================================================================== = Processing MultiLoad Statements =

========================================================================

49 0021 50 0022 51 0023 52 0024 53 0025 54

.field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); .field PURCHASE_ORDER_NUM * varchar (20);

55 0026 .dml label insdml; 56 0027 insert into test.tran.*; 57 58 0028 .import infile C:\DataIntegration\Data\TRAN2.DAT 59 60 61 format vartext '|' layout tranlayout apply insdml;

62 0029 .end mload; 63 64 65 ======================================================================== = MultiLoad Initial Phase =

========================================================================

66 **** 22:11:12 UTY0829 Options in effect for this MultiLoad import task: 67 68 69 70 71 . . . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

72 **** 22:11:13 UTY0815 MLOAD session(s) connected: 2. 73 74 75 ======================================================================== = MultiLoad Acquisition Phase =

========================================================================

76 **** 22:11:16 UTY1812 A checkpoint is being initiated because 20000 output recor

77 ds have been 78 sent to the RDBMS.

79 **** 22:11:17 UTY0827 A checkpoint has been taken, recording that input record 2 80 0000 has been 81 processed for IMPORT 1 of this MultiLoad Import task.

82 **** 22:11:17 UTY1812 A checkpoint is being initiated because 40000 output recor 83 ds have been 84 sent to the RDBMS.

85 **** 22:11:43 UTY0817 MultiLoad submitting the following request: 86 CHECKPOINT LOADING INTERVAL 2;

87 **** 22:11:43 UTY0806 RDBMS error, 2825: NO RECORD OF THE LAST REQUEST WAS FOUND 88 AFTER DBC 89 RESTART

90 **** 22:11:43 UTY1804 Re-starting this MultiLoad task because The RDBMS crashed. 91 92 93 94 ======================================================================== = MultiLoad Initial Phase =

========================================================================

95 **** 22:11:43 UTY0829 Options in effect for this MultiLoad import task: 96 97 98 99 100 . . . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

101**** 22:11:44 UTY0815 MLOAD session(s) connected: 2. 102 103 104 ======================================================================== = MultiLoad Acquisition Phase =

========================================================================

105**** 22:11:46 UTY1806 This MultiLoad Import task is re-starting from checkpoint 106taken at 107 record 20000 of IMPORT 1.

108**** 22:11:47 UTY1812 A checkpoint is being initiated because 40000 output recor 109ds have been 110 sent to the RDBMS.

111**** 22:11:48 UTY0827 A checkpoint has been taken, recording that input record 4 1120000 has been 113 processed for IMPORT 1 of this MultiLoad Import task.

114**** 22:11:49 UTY1812 A checkpoint is being initiated because 60000 output recor 115ds have been 116 sent to the RDBMS.

117**** 22:11:50 UTY0827 A checkpoint has been taken, recording that input record 6 1180000 has been 119 processed for IMPORT 1 of this MultiLoad Import task.

120**** 22:11:51 UTY0826 A checkpoint has been taken, recording that end of file ha 121s been reached 122 for IMPORT 1 of this MultiLoad Import task.

123**** 22:11:51 UTY1803 Import processing statistics 124 125 126 127 128 129 130 131 132 . . IMPORT 1 ========= Total thus far ============== 67817....... 67817....... 0....... 0....... 0 67817 67817 0

Candidate records considered:........ Apply conditions satisfied:.......... Candidate records not applied:....... Candidate records rejected:..........

======================================================================== = MultiLoad Application Phase =

========================================================================

133**** 22:12:10 UTY0818 Statistics for table test.tran: 134 135 136 137 138 139 Inserts: Updates: Deletes: 67817 0 0

======================================================================== = MultiLoad Task Cleanup =

========================================================================

140**** 22:12:10 UTY0821 Error table test.ET_tran is EMPTY, dropping table. 141**** 22:12:11 UTY0821 Error table test.UV_tran is EMPTY, dropping table. 142**** 22:12:11 UTY0825 Error table statistics for: 143 144 145 146 147 148 149 150**** 22:12:12 UTY0822 MultiLoad processing complete for this MultiLoad import ta 151sk. 152 153 154 155 1560030 .logoff; 157 158 159 ======================================================================== = Logoff/Disconnect = ======================================================================== = MultiLoad Task Complete = Target table 1: test.tran Number of Rows Error Table Name ============== ======================================================== 0 test.ET_tran 0 test.UV_tran

========================================================================

========================================================================

160**** 22:12:12 UTY6216 The restart log table has been dropped.

161**** 22:12:12 UTY6212 A successful disconnect was made from the RDBMS. 162**** 22:12:12 UTY2410 Total processor time used = '1.625 Seconds' 163 164 165 . . . Start : 22:11:09 - SUN AUG 09, 2009 End : 22:12:12 - SUN AUG 09, 2009 Highest return code encountered = '0'.

An automatic restart may also occur if an interruption occurs during the application phase. The following output illustrates this type of re-start. ? code 1 2 3 4 5 6 7 8 9 10 11 12 C:\DataIntegration\Scripts>mload -b < tran.ml **** 22:22:57 UTY2414 BRIEF option is enabled. ======================================================================== = = MultiLoad Utility Release MLOD.13.00.00.000 Platform WIN32 = =

======================================================================== = Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

======================================================================== **** 22:22:57 UTY2411 Processing start date: SUN AUG 09, 2009 ======================================================================== = Logon/Connection =

========================================================================

13 0001 .logtable test.tranmllog; 14 0002 .LOGON test/testuser,; 15 **** 22:23:00 UTY8400 Teradata Database Release: 12.00.01.24 16 **** 22:23:00 UTY8400 Teradata Database Version: 12.00.01.24 17 **** 22:23:00 UTY8400 Default character set: ASCII 18 **** 22:23:00 UTY8400 Current RDBMS has interval support 19 **** 22:23:00 UTY8400 Current RDBMS has UDT support 20 **** 22:23:00 UTY8400 Maximum supported buffer size: 1M

21 **** 22:23:00 UTY8400 Data Encryption supported by RDBMS server 22 **** 22:23:00 UTY6211 A successful connect was made to the RDBMS. 23 **** 22:23:00 UTY6217 Logtable 'test.tranmllog' has been created. 24 25 0003 .begin import 26 27 28 29 30 31 32 0004 .layout tranlayout; 33 0005 34 0006 35 0007 36 0008 37 0009 38 0010 39 0011 40 0012 41 0013 42 0014 43 0015 44 0016 45 0017 46 0018 47 0019 48 0020 .FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); .field CUST_NO * varchar (10); .field NAME * varchar (30); .field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); mload tables test.tran checkpoint 20000; ======================================================================== = Processing MultiLoad Statements =

========================================================================

49 0021 50 0022 51 0023 52 0024 53 0025 54

.field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); .field PURCHASE_ORDER_NUM * varchar (20);

55 0026 .dml label insdml; 56 0027 insert into test.tran.*; 57 58 0028 .import infile C:\DataIntegration\Data\TRAN2.DAT 59 60 61 format vartext '|' layout tranlayout apply insdml;

62 0029 .end mload; 63 64 65 ======================================================================== = MultiLoad Initial Phase =

========================================================================

66 **** 22:23:00 UTY0829 Options in effect for this MultiLoad import task: 67 68 69 70 71 . . . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

72 **** 22:23:01 UTY0815 MLOAD session(s) connected: 2. 73 74 75 ======================================================================== = MultiLoad Acquisition Phase =

========================================================================

76 **** 22:23:04 UTY1812 A checkpoint is being initiated because 20000 output recor

77 ds have been 78 sent to the RDBMS.

79 **** 22:23:05 UTY0827 A checkpoint has been taken, recording that input record 2 80 0000 has been 81 processed for IMPORT 1 of this MultiLoad Import task.

82 **** 22:23:06 UTY1812 A checkpoint is being initiated because 40000 output recor 83 ds have been 84 sent to the RDBMS.

85 **** 22:23:07 UTY0827 A checkpoint has been taken, recording that input record 4 86 0000 has been 87 processed for IMPORT 1 of this MultiLoad Import task.

88 **** 22:23:07 UTY1812 A checkpoint is being initiated because 60000 output recor 89 ds have been 90 sent to the RDBMS.

91 **** 22:23:08 UTY0827 A checkpoint has been taken, recording that input record 6 92 0000 has been 93 processed for IMPORT 1 of this MultiLoad Import task.

94 **** 22:23:10 UTY0826 A checkpoint has been taken, recording that end of file ha 95 s been reached 96 for IMPORT 1 of this MultiLoad Import task.

97 **** 22:23:10 UTY1803 Import processing statistics 98 99 100 101 102 103 104 . . IMPORT 1 ========= Total thus far ============== 67817....... 67817....... 0....... 0....... 0 67817 67817 0

Candidate records considered:........ Apply conditions satisfied:.......... Candidate records not applied:....... Candidate records rejected:..........

========================================================================

105 106

MultiLoad Application Phase

========================================================================

107**** 22:24:19 UTY0817 MultiLoad submitting the following request: 108 EXEC MLOAD test.tran;

109**** 22:24:19 UTY0805 RDBMS failure, 2828: Request was rolled back during system 110 recovery. 111**** 22:24:19 UTY1804 Re-starting this MultiLoad task because The RDBMS crashed. 112 113 114 115 ======================================================================== = MultiLoad Initial Phase =

========================================================================

116**** 22:24:19 UTY0829 Options in effect for this MultiLoad import task: 117 118 119 120 121 . . . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

122**** 22:24:20 UTY0815 MLOAD session(s) connected: 2. 123 124 125 ======================================================================== = MultiLoad Application Phase =

========================================================================

126**** 22:24:27 UTY0818 Statistics for table test.tran: 127 128 129 130 131 132 Inserts: Updates: Deletes: 67817 0 0

======================================================================== = MultiLoad Task Cleanup =

========================================================================

133**** 22:24:27 UTY0821 Error table test.ET_tran is EMPTY, dropping table. 134**** 22:24:27 UTY0821 Error table test.UV_tran is EMPTY, dropping table. 135**** 22:24:27 UTY0825 Error table statistics for: 136 137 138 139 140 141 142 143**** 22:24:28 UTY0822 MultiLoad processing complete for this MultiLoad import ta 144sk. 145 146 147 148 1490030 .logoff; 150 151 152 ======================================================================== = Logoff/Disconnect = ======================================================================== = MultiLoad Task Complete = Target table 1: test.tran Number of Rows Error Table Name ============== ======================================================== 0 test.ET_tran 0 test.UV_tran

========================================================================

========================================================================

153**** 22:24:29 UTY6216 The restart log table has been dropped. 154**** 22:24:29 UTY6212 A successful disconnect was made from the RDBMS. 155**** 22:24:29 UTY2410 Total processor time used = '1.70313 Seconds' 156 157 158 . . . Start : 22:22:57 - SUN AUG 09, 2009 End : 22:24:29 - SUN AUG 09, 2009 Highest return code encountered = '0'.

Restarts after client failure A Multiload job may be restarted after it is interrupted by a failure of the Multiload process or the client processing platform. This type of er-ror may be a system error or can be a programming error or data er-

ror. The system can be restarted by correcting the problem and re-running the Multiload job. The following output is an example of a re-start for a failure of a Multiload process during the acquisition phase. ? code 1 C:\DataIntegration\Scripts>mload -b < tran.ml 2 **** 22:35:51 UTY2414 BRIEF option is enabled. 3 4 5 6 7 8 ======================================================================== = = MultiLoad Utility Release MLOD.13.00.00.000 Platform WIN32 = =

======================================================================== = Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

========================================================================

9 **** 22:35:51 UTY2411 Processing start date: SUN AUG 09, 2009 10 11 12 ======================================================================== = Logon/Connection =

========================================================================

130001 .logtable test.tranmllog; 140002 .LOGON test/testuser,; 15**** 22:35:53 UTY8400 Teradata Database Release: 12.00.01.24 16**** 22:35:53 UTY8400 Teradata Database Version: 12.00.01.24 17**** 22:35:53 UTY8400 Default character set: ASCII 18**** 22:35:53 UTY8400 Current RDBMS has interval support 19**** 22:35:53 UTY8400 Current RDBMS has UDT support 20**** 22:35:53 UTY8400 Maximum supported buffer size: 1M 21**** 22:35:53 UTY8400 Data Encryption supported by RDBMS server 22**** 22:35:54 UTY6211 A successful connect was made to the RDBMS. 23**** 22:35:54 UTY6217 Logtable 'test.tranmllog' has been created. 24

250003 .begin import 26 27 28 29 30 31 320004 .layout tranlayout; 330005 340006 350007 360008 370009 380010 390011 400012 410013 420014 430015 440016 450017 460018 470019 480020 490021 500022 510023 520024 .FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); .field CUST_NO * varchar (10); .field NAME * varchar (30); .field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); .field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); mload tables test.tran checkpoint 20000; ======================================================================== = Processing MultiLoad Statements =

========================================================================

530025 54

.field PURCHASE_ORDER_NUM * varchar (20);

550026 .dml label insdml; 560027 insert into test.tran.*; 57 580028 .import infile C:\DataIntegration\Data\TRAN2.DAT 59 60 61 format vartext '|' layout tranlayout apply insdml;

620029 .end mload; 63 64 65 ======================================================================== = MultiLoad Initial Phase =

========================================================================

66**** 22:35:54 UTY0829 Options in effect for this MultiLoad import task: 67 68 69 70 71 . . . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

72**** 22:35:54 UTY0815 MLOAD session(s) connected: 2. 73 74 75 ======================================================================== = MultiLoad Acquisition Phase =

========================================================================

76**** 22:35:57 UTY1812 A checkpoint is being initiated because 20000 output reco 77ds have been 78 sent to the RDBMS.

79**** 22:35:58 UTY0827 A checkpoint has been taken, recording that input record 800000 has been

81

processed for IMPORT 1 of this MultiLoad Import task.

82**** 22:35:59 UTY1812 A checkpoint is being initiated because 40000 output reco 83ds have been 84 sent to the RDBMS.

85**** 22:36:00 UTY0827 A checkpoint has been taken, recording that input record 860000 has been 87 processed for IMPORT 1 of this MultiLoad Import task.

88**** 22:36:01 UTY1812 A checkpoint is being initiated because 60000 output reco 89ds have been 90 sent to the RDBMS.

The gateway sessions associated with the Multiload job will remain ac-tive for the gateway timeout time (20 minutes default). These sessions can be killed using the gtwglobal utility in the same manner that was described for the Fastload utility. The interrupted Multiload job can be restarted by re-executing the job. The following output is from a re-execution of the example Multiload job. ? code 1 2 3 4 5 6 7 8 9 10 11 12 C:\DataIntegration\Scripts>mload -b < tran.ml **** 22:42:35 UTY2414 BRIEF option is enabled. ======================================================================== = = MultiLoad Utility Release MLOD.13.00.00.000 Platform WIN32 = =

======================================================================== = Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

======================================================================== **** 22:42:35 UTY2411 Processing start date: SUN AUG 09, 2009 ======================================================================== = Logon/Connection =

========================================================================

13 0001 .logtable test.tranmllog; 14 0002 .LOGON test/testuser,; 15 **** 22:42:38 UTY8400 Teradata Database Release: 12.00.01.24 16 **** 22:42:38 UTY8400 Teradata Database Version: 12.00.01.24 17 **** 22:42:38 UTY8400 Default character set: ASCII 18 **** 22:42:38 UTY8400 Current RDBMS has interval support 19 **** 22:42:38 UTY8400 Current RDBMS has UDT support 20 **** 22:42:38 UTY8400 Maximum supported buffer size: 1M 21 **** 22:42:38 UTY8400 Data Encryption supported by RDBMS server 22 **** 22:42:38 UTY6211 A successful connect was made to the RDBMS. 23 **** 22:42:38 UTY6210 Logtable 'test.tranmllog' indicates that a restart is in p 24 rogress. 25 26 0003 .begin import 27 28 29 30 31 32 33 0004 .layout tranlayout; 34 0005 35 0006 36 0007 37 0008 38 0009 39 0010 40 0011 .FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); mload tables test.tran checkpoint 20000; ======================================================================== = Processing MultiLoad Statements =

========================================================================

41 0012 42 0013 43 0014 44 0015 45 0016 46 0017 47 0018 48 0019 49 0020 50 0021 51 0022 52 0023 53 0024 54 0025 55

.field CUST_NO * varchar (10); .field NAME * varchar (30); .field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); .field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); .field PURCHASE_ORDER_NUM * varchar (20);

56 0026 .dml label insdml; 57 0027 insert into test.tran.*; 58 59 0028 .import infile C:\DataIntegration\Data\TRAN2.DAT 60 61 62 format vartext '|' layout tranlayout apply insdml;

63 0029 .end mload; 64 65 66 ======================================================================== = MultiLoad Initial Phase =

========================================================================

67 **** 22:42:38 UTY0829 Options in effect for this MultiLoad import task: 68 . Sessions: One session per available amp.

69 70 71 72

. . . .

Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions. Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

73 **** 22:42:39 UTY0815 MLOAD session(s) connected: 2. 74 75 76 ======================================================================== = MultiLoad Acquisition Phase =

========================================================================

77 **** 22:42:40 UTY1806 This MultiLoad Import task is re-starting from checkpoint 78 taken at 79 record 40000 of IMPORT 1.

80 **** 22:42:40 UTY1812 A checkpoint is being initiated because 60000 output recor 81 ds have been 82 sent to the RDBMS.

83 **** 22:42:41 UTY0827 A checkpoint has been taken, recording that input record 6 84 0000 has been 85 processed for IMPORT 1 of this MultiLoad Import task.

86 **** 22:42:43 UTY0826 A checkpoint has been taken, recording that end of file ha 87 s been reached 88 for IMPORT 1 of this MultiLoad Import task.

89 **** 22:42:43 UTY1803 Import processing statistics 90 91 92 93 94 95 96 . . IMPORT 1 ========= Total thus far ============== 67817....... 67817....... 0....... 0....... 0 67817 67817 0

Candidate records considered:........ Apply conditions satisfied:.......... Candidate records not applied:....... Candidate records rejected:..........

========================================================================

97 98

MultiLoad Application Phase

========================================================================

99 **** 22:42:57 UTY0818 Statistics for table test.tran: 100 101 102 103 104 105 Inserts: Updates: Deletes: 67817 0 0

======================================================================== = MultiLoad Task Cleanup =

========================================================================

106**** 22:42:57 UTY0821 Error table test.ET_tran is EMPTY, dropping table. 107**** 22:42:57 UTY0821 Error table test.UV_tran is EMPTY, dropping table. 108**** 22:42:57 UTY0825 Error table statistics for: 109 110 111 112 113 114 115 116**** 22:42:58 UTY0822 MultiLoad processing complete for this MultiLoad import ta 117sk. 118 119 120 121 1220030 .logoff; 123 124 ======================================================================== = Logoff/Disconnect = ======================================================================== = MultiLoad Task Complete = Target table 1: test.tran Number of Rows Error Table Name ============== ======================================================== 0 test.ET_tran 0 test.UV_tran

========================================================================

125

========================================================================

126**** 22:42:59 UTY6216 The restart log table has been dropped. 127**** 22:42:59 UTY6212 A successful disconnect was made from the RDBMS. 128**** 22:42:59 UTY2410 Total processor time used = '0.8125 Seconds' 129 130 131 . . . Start : 22:42:35 - SUN AUG 09, 2009 End : 22:42:59 - SUN AUG 09, 2009 Highest return code encountered = '0'.

A Multiload job may also be restarted if interrupted during the application phase. It is important to note that the restart log and error tables must be available. If they are not the Multiload job can not be restarted or rerun. In this case the table must be reloaded from backup or source because they are unusable. The following output is an example of a Multiload job restarted after an interruption during the application phase. ? code 1 2 3 4 5 6 7 8 9 10 11 12 C:\DataIntegration\Scripts>mload -b < tran.ml **** 22:53:17 UTY2414 BRIEF option is enabled. ======================================================================== = = MultiLoad Utility Release MLOD.13.00.00.000 Platform WIN32 = =

======================================================================== = Copyright 1990-2009, Teradata Corporation. ALL RIGHTS RESERVED. =

======================================================================== **** 22:53:17 UTY2411 Processing start date: SUN AUG 09, 2009 ======================================================================== = Logon/Connection =

========================================================================

13 0001 .logtable test.tranmllog; 14 0002 .LOGON test/testuser,;

15 **** 22:53:19 UTY8400 Teradata Database Release: 12.00.01.24 16 **** 22:53:19 UTY8400 Teradata Database Version: 12.00.01.24 17 **** 22:53:19 UTY8400 Default character set: ASCII 18 **** 22:53:19 UTY8400 Current RDBMS has interval support 19 **** 22:53:19 UTY8400 Current RDBMS has UDT support 20 **** 22:53:19 UTY8400 Maximum supported buffer size: 1M 21 **** 22:53:19 UTY8400 Data Encryption supported by RDBMS server 22 **** 22:53:19 UTY6211 A successful connect was made to the RDBMS. 23 **** 22:53:19 UTY6210 Logtable 'test.tranmllog' indicates that a restart is in p 24 rogress. 25 26 0003 .begin import 27 28 29 30 31 32 33 0004 .layout tranlayout; 34 0005 35 0006 36 0007 37 0008 38 0009 39 0010 40 0011 41 0012 42 0013 .FIELD STORE * varchar(10); .field TRAN_NO * varchar (10); .field TRAN_DATE * varchar (10); .field TRAN_TIME * varchar (10); .field TRAN_TYPE * varchar (1); .field TRAN_TERMINAL * varchar (3); .field CLERK * varchar (10); .field CUST_NO * varchar (10); .field NAME * varchar (30); mload tables test.tran checkpoint 20000; ======================================================================== = Processing MultiLoad Statements =

========================================================================

43 0014 44 0015 45 0016 46 0017 47 0018 48 0019 49 0020 50 0021 51 0022 52 0023 53 0024 54 0025 55

.field TOTAL_SALE * varchar (14); .field SALES_TAX * varchar (14); .field DISCOUNT * varchar (14); .field DISC_PCT * varchar (10); .field DISC_TAKEN_FLAG * varchar (1); .field CASH_TENDERED * varchar (14); .field CHANGE_GIVEN * varchar (14); .field CHARGE_TENDERED* varchar (14); .field CHECK_NUMBER * varchar (20); .field CHECK_TENDERED * varchar (14); .field BANKCARD_TENDERED * varchar (14); .field PURCHASE_ORDER_NUM * varchar (20);

56 0026 .dml label insdml; 57 0027 insert into test.tran.*; 58 59 0028 .import infile C:\DataIntegration\Data\TRAN2.DAT 60 61 62 format vartext '|' layout tranlayout apply insdml;

63 0029 .end mload; 64 65 66 ======================================================================== = MultiLoad Initial Phase =

========================================================================

67 **** 22:53:19 UTY0829 Options in effect for this MultiLoad import task: 68 69 70 . . . Sessions: One session per available amp. Checkpoint: 20000 records. Tenacity: 4 hour limit to successfully connect load sessions.

71 72

. .

Errlimit: No limit in effect. AmpCheck: In effect for apply phase transitions.

73 **** 22:53:20 UTY0815 MLOAD session(s) connected: 2. 74 75 76 ======================================================================== = MultiLoad Application Phase =

========================================================================

77 **** 22:53:25 UTY0818 Statistics for table test.tran: 78 79 80 81 82 83 Inserts: Updates: Deletes: 67817 0 0

======================================================================== = MultiLoad Task Cleanup =

========================================================================

84 **** 22:53:25 UTY0821 Error table test.ET_tran is EMPTY, dropping table. 85 **** 22:53:26 UTY0821 Error table test.UV_tran is EMPTY, dropping table. 86 **** 22:53:26 UTY0825 Error table statistics for: 87 88 89 90 91 92 93 94 **** 22:53:27 UTY0822 MultiLoad processing complete for this MultiLoad import ta 95 sk. 96 97 98 ======================================================================== = MultiLoad Task Complete = Target table 1: test.tran Number of Rows Error Table Name ============== ======================================================== 0 test.ET_tran 0 test.UV_tran

========================================================================

99 1000030 .logoff; 101 102 103 ======================================================================== = Logoff/Disconnect =

========================================================================

104**** 22:53:27 UTY6216 The restart log table has been dropped. 105**** 22:53:27 UTY6212 A successful disconnect was made from the RDBMS. 106**** 22:53:27 UTY2410 Total processor time used = '0.671875 Seconds' 107 108 109 Rerun Multiload jobs that are interrupted before the application phase starts or after the application phase ends may be rerun. Jobs that are inter-rupted during the application phase must be restarted. To rerun an entire Multiload job the following actions must be taken before the job is rerun: Use BTEQ to execute RELEASE MLOAD command for each target table in Multiload script Drop work table for each target table Drop acquisition error table for each target table Drop application error table for each target table Drop restart log table These actions can be executed by a BTEQ SQL script that is run as part of a Multiload script or run by an external job scheduling system. The following is an example of a SQL script that will allow this example to be rerun. ? code 1 .logon test/testuser,test 2 3 database test; . . . Start : 22:53:17 - SUN AUG 09, 2009 End : 22:53:27 - SUN AUG 09, 2009 Highest return code encountered = '0'.

4 5 release mload tran; 6 drop table uv_tran; 7 drop table et_tran; 8 drop table wt_tran; 9 drop table log_tran; 10 11.logoff 12.quit This script can be executed as part of a Multiload script that executes this script with BTEQ and then executes the Multiload job as illustrated in the following Multiload script. ? code 1.system 'bteq < c:\DataIntegration\scripts\tranreset.sql'; 2.run file c:\DataIntegration\scripts\tranload.ml; 3 4.logoff; This script can always be run when the intent is to run the entire Multi-load job. In the next article in this series we will look at the Tpump Utility.