Professional Documents
Culture Documents
No part of these materials may be reproduced, stored in a retrieval system, or transmitted, in any form or
by any means (photocopying, electronic, mechanical, recording, or otherwise), without permission in
writing from the publisher, except by a reviewer who may quote brief passages in a review.
ACL 9.0
August 2006
Printed in Canada
ACL, the ACL logo, the ACL logo with the text, "ACL Data you can trust. Results you can see." and Audit
Command Language are trademarks or registered trademarks of ACL Services Ltd.
Microsoft, Windows and Windows Server are trademarks or registered trademarks of Microsoft Corpo-
ration. AIX, OS/390, OS/400 and z/OS are registered trademarks of IBM Corporation. Linux is a registered
trademark of Linus Torvalds. SAP, R/2 and R/3 are trademarks or registered trademarks of SAP AG.
Crystal Reports is a trademark or registered trademark of Business Objects SA. All other trademarks are
the property of their respective owners.
iv
C HAPTER 1
I NTRODUCTION 1
Use this guide to help you access different types of In this chapter...
data with ACL software. Data analysts must gain Finding data
access to diverse data sources from both inside and Acquiring data
outside of their organizations. They must also Reading data
understand the related technical issues. This guide
addresses both concerns.
Much of the material for this publication was drawn
from the collective knowledge of the ACL Help Desk
team. The expertise of these individuals derives from
their own specialized training and from their daily
contact with people who use ACL in the workplace.
Because of this contact, technical support staff also
know what questions people ask about data access.
This guide attempts to answer the most common of To find definitions of unfamiliar
those questions. And it attempts to provide a terms, select Help Index and look up
terminology in ACL Help.
reference for some of the more obscure challenges of
data access.
1
Chapter 1: Introduction
Find the data that you need and become acquainted with it
3
Chapter 2: Locate the data
ODBC
Working with an ODBC-compliant structure requires little effort Find a structure that works for
from you. ACL recognizes and creates a table from both you and IS staff.
ODBC-compliant data. In ODBC, you can also choose specific
tables and set a filter to obtain only those records that you need.
But because of security and performance concerns, IS staff
sometimes hesitate to grant ODBC access. One concern is that
you can sometimes circumvent the built-in security features of
the software that reads the database. Also, unregulated access to
a systems data tables can slow down daily operations, especially
if requests come at peak hours. Requests for large amounts of
data create considerable demands on the systems resources.
However, if you follow regulated procedures, you can help to
alleviate such concerns and obtain the access that you need.
dBASE
When you can, request data in fixed-length, dBASE output.
Fixed-length files contain records of equal length where the same
field appears in the same location of every record. ACL
recognizes dBASE files and easily creates tables from them.
Flat files
Often, IS staff can extract data to a flat file that contains only the ? To learn more, select Help Index
data and a CR/LF character at the end of each record. Flat files and look up flat file.
store information in simple records and fields, rather than in
complex database structures. When ACL encounters a flat file, it
automatically detects that the CR/LF character defines the end of
a record. To create a table, you can use the Data Definition
Wizard to help you define the fields manually with the aid of a
record layout.
Print-image files
Sometimes, the only practical way to get data from a legacy
system is to generate electronic report files, known as
print-image files. ACL can read electronic report files. It is
recommended you use the Data Definition Wizard to define a
print-image file. The wizard creates an IMPORT statement that
you can reuse for other print-image files with the same format.
Contact information
Data particulars
Storage medium CD
Additional requirements
Record layout that includes field names, field start positions, field lengths, N/A
data types, formats of numeric and date fields, and field descriptions
Transfer time
The time required to transfer data varies according to the
method that you use. Transfer speeds range from 6 kilobytes to
10,000 kilobytes per minute for PC3270 emulation to 10
megabytes per second for a LAN connection.
A client/server configuration transfers data at a comparable
speed to FTP. In-house benchmark tests indicate that a terminal
emulation transfer utility can take up to thirty times longer than
either a client/server configuration or FTP.
Download precautions
With emulation software, you can choose to transfer data in
ASCII or EBCDIC format. Most mainframes store data in
EBCDIC format, and most PCs store it in ASCII format. But ACL
can process data on the PC in either format. If you have a choice,
always process data in its native format. Any form of data
manipulation, including conversion, introduces the risk of data
Drive mapping
IS staff can map a drive between a mainframe and a PC, but must
use external software, such as Samba, to do so. Drive mapping
from one PC to another provides an easy way for members of a
workgroup to share files. However, all processing goes through a
network, which limits the rate at which data can transfer.
From PC to PC
To work with data from a personal computer, you can:
Transfer data files to the personal computer on which
ACL is installed. Copy files to any disk media such as
floppy disks, CDs, or Zip disks.
Install ACL on the personal computer that contains the Also see Drive mapping on
data files. page 12.
Set up a shared folder or drive. See Help in Windows
Explorer for information about shared folders.
ACL connects to ACL Server Edition across a network to operate ? To learn more, select Help Index
in a client/server architecture. You can process server-based data and look up accessing data with
on the server or on the desktop. ACL Server Edition.
System components
To set up a client/server environment for ACL, you need:
One or more clients that run ACL with the client/server
option enabled.
One or more servers that run ACL Server Edition.
A TCP/IP network connection to link the clients to the
servers. The connection can be direct, on a LAN, or by
dial-up connection on a WAN.
ACL reads data from systems that range from legacy In this chapter...
mainframe systems to modern relational databases. The process of data access
As a read-only application, ACL never alters the The function of a table layout
source data. How ACL reads data
15
Chapter 3: Access the data
Create a table
When you access a data source, ACL presents the data in the
form of a table. For this reason, you can also describe the process
of accessing data as the process of creating a table.
Table layouts
The ACL table contains information about the structure and
content of the source data. Such information includes location,
path, record length, field names, field lengths, field positions,
and data type. ACL stores information about the data contents
and the way that the data is displayed in a table layout.
To examine the layout of any table, select Edit Table Layout.
Methods of access
To access data, ACL either imports the data as a flat file and ? Also see Obtain a record layout on
copies it to a local drive or reads the data directly in its native page 5.
format.
Manual layout
Data source
Import data
Direct access
ACL can access some data sources directly, without the need to
import or copy the data. ACL creates a table with varying degrees
of automation, depending on how much layout information the
source contains.
Import layout
Record-layout or
data-dictionary
document
Manual layout
Import layout
23
Chapter 4: Sources of data
ACCPAC
ACCPAC is a PC-based application that stores data in
transaction files that are associated with master files. When
acquiring ACCPAC data, request both the master and the
transaction files from the data provider. Transaction data is
stored in proprietary fixed-length packed fields.
Method of access
Import ACCPAC data by one of these methods.
ODBC Link
You can use the third-party software, ODBC Link, to connect to ? For information on ODBC Link, visit
ACCPAC data. the ACCPAC website.
AIX-based data
The AIX operating system is a version of UNIX that is produced
by IBM. AIX runs on a variety of hardware platforms, including
servers and workstations.
Method of access
You can access files on a network-enabled computer that runs
AIX just as you access files on a network-enabled PC, provided
you have the appropriate access rights.
You can also access AIX-based data and process it on the server
on which it resides by using ACL Server Edition for AIX. Any flat
files and print-image files that are located on the server can be
processed directly on the server using the offline scripting
functionality.
To access DB2 databases by using ACL Server Edition software, ? For other ways to access files on an
see DB2 on page 31. AIX server, see UNIX-based data on
page 88.
To access Oracle databases by using ACL Server Edition
software, see Oracle on page 69.
Method of access
Access COBOL data by one of these methods.
External Definition
Access the external definition file using the Data Definition
Wizard.
On the Select Data Source screen of the wizard, select External
Definition. On the next screen, select COBOL. Then browse to
locate the file. ACL creates a table layout using the external
definition, then prompts you to save it. If prompted, browse to
locate the data file. ACL creates the table.
Manual layout
If you cannot obtain a copybook file, obtain a printed or
electronic copy of the layout information, and create the table
layout manually.
Reference information
Each line in a copybook represents a field definition. All COBOL ? To learn more, select Help Index
field definitions contain a level, a field name, and a PIC clause and look up OFFSET( ).
that specifies the data type. The data dictionary often contains
more detailed information.
PIC clauses
The PIC clause identifies the data type. The table below lists and
describes common COBOL data types, and provides the
equivalent ACL data type, where applicable.
V A V signifies a virtual
decimal; an implied decimal
place at that location of the
field. This saves space because
the decimal is not actually
stored in the file.
Redefines
REDEFINES is the COBOL equivalent of the ACL overlapping ? To learn more, select Help Index
field definition. ACL creates overlapping fields, but you must and look up data filters.
manually apply the necessary filter.
Occurs
OCCURS specifies the number of times that a field occurs. Only ? To learn more, select Help Index
the first occurrence of the field appears in ACL. If the field occurs and look up bucketed files, LOOP, or
a fixed number of times in each record, manually set up the OFFSET( ).
remaining occurrences, or use the OFFSET( ) function.
For example:
05 CHANGE-DATE PIC 9(6) OCCURS 5 TIMES.
If the number of OCCURS is based on a field, rather than a Contact ACL Help Desk at
record, as in a bucketed file, use the LOOP command and the support@acl.com to obtain a standard
OFFSET( ) function in an ACL script. script to remove or process segments of
bucketed files.
DB2
A DB2 system consists of a catalog and databases.The catalogue
keeps track of all the data in the system and of the access rights to
that data.
DB2 databases are collections of related tables. The rows in a ? With ACL Server Edition, you do not
table are not stored in any particular order, so the data remains need to download the source data.
independent of the applications that access it. Unless you request Instead, you link to it through the
the data in a particular sequence, DB2 determines the order in server.
which it returns the row.
DB2 provides views of data to increase data security and
independence. A view is a logical structure, not a physical one. A
view represents the data that is defined in the DB2 catalog and
can comprise a subset of columns from one table or columns
taken from different tables. You do not need to know whether the
data comes from one table or from several tables. Also, you can
use views to restrict access to sensitive data, without limiting
access to other data held in the same table.
You can access DB2 databases through ODBC or through an ACL
Server Edition product with a DB2 interface.
Method of access
Access DB2 data by one of these methods.
z/OS server
1 On the Select ACL Server Data Source screen, select the
DB2 option.
2 On the Select DB2 Subsystem screen, select the subsystem
that you want.
3 On the Choose DB2 Tables screen, select the tables that you
want from the Available Tables list. If you select more than
one table, you must relate the tables by using the Identify
Relationship dialog box.
Other servers
1 On the Select ACL Server Data Source screen, select the ? To learn more, select Help Index
Database option, then from the drop-down list, select a and look up database profile.
database profile.
2 On the Select DB2 Schema screen, select the schema that
you want.
3 On the Choose Database Tables screen, select the tables that
you want from the Available Tables list. If you select more
than one table, you must relate the tables by using the
Identify Relationship dialog box.
All platforms
4 If you selected more than one table, you can edit the
conditions of the join on the Condition Clause screen. To
edit a condition, select the Edit the Condition Clause
option.
5 Select the fields or columns that you want from the Select
DB2 Columns screen.
6 To obtain a subset of items or a particular sequence of order,
enter WHERE and ORDER clauses in SQL syntax on the
Where and Order Clauses screen.
7 Once you define all of the fields, the wizard prompts you to
name and save the table.
To edit the query after you close the table, right-click the table in
the Overview. Select Properties, then click the Edit Query tab.
In a script, you can use the variables DB2WHERE, DB2ORDER, ? To learn more, select Help Index
and DB2CONDITION to edit WHERE, ORDER, and and look up defining server data files.
CONDITION clauses. The values that you assign to these
ODBC
Use ODBC to import DB2 databases. See ODBC-compliant
databases on page 61.
Reference information
Here are a few tips on processing DB2 data.
Processing subsets
To process the same subset of a DB2 file several times, extract the
required records and fields to a flat file. ACL can process smaller
files more efficiently. This particularly applies when you need the
data in a specific order or when ACL has to return to the top of
the file several times to process the commands.
Ordering data
If you need the data in a particular sort order, use an ORDER
clause to retrieve it in the desired order. This may slow the
transfer of data from the database, but is usually preferred to
sorting large data files on a local drive.
dBASE-compatible
The dBASE format for storing data has become an industry
standard, and most database and spreadsheet applications can
import and export data in this format. dBASE files have a dbf
extension.
ACL recognizes dBASE files and easily creates tables from the
record-layout information that they contain. The resulting tables
will contain a field called Record_deleted. You can delete this
field from the table layout.
Method of access
Import the file through the Data Definition Wizard. Once you ? To learn more, see Direct access,
select the file that contains your data, the wizard recognizes the automatic layout on page 19.
file as dBASE and creates a table for you.
Reference information
Many applications can export data in a dBASE format. However,
the conversion process may not support all the features of the
source application. In addition, conversion errors can occur that
prevent you from reading the file with ACL.
Delimited
In delimited files, a variety of characters can be used to separate
or delimit a field: commas, semicolons, tabs, pipes (|), or spaces.
Various characters can also represent text qualifiers, although
quotation marks are most commonly used. Text qualifiers
identify character data and they distinguish punctuation that
serves as text from punctuation that serves as field separators.
For example, in a comma-delimited file, the quotation marks
around "Last_name, First_name" indicate that the
comma represents text rather than a field separator. A carriage
return and a line feed (CR/LF) terminate each record in the file.
Delimited files usually contain no trailing blanks. As a result,
field lengths often differ from one source data file to another,
depending on the contents of the fields. For example, if a field
contains a three-byte value and an eleven-byte value, the length
of the field differs from 3 bytes in one record to 11 bytes in the
other record. One field can contain values of many different
lengths.
When ACL imports a delimited file, it bases the length of a field
on either the longest value that it detects in that field, or on the
field name if that is longer than the longest value. For that reason,
if you import the same file with updated information each
month, the lengths of corresponding fields in the resulting ACL
tables can vary from one month to the next. Differing field
lengths mean that you cannot link a table to new delimited
source data unless you use functions to create a table layout of
computed fields. See Creating a reusable table layout on
page 40.
However, you can change the width of a field on the Delimited
File Properties screen of the wizard to standardize field widths
for such ACL operations as joins or extracts. You can also
increase field width to avoid truncation of wide fields that may
not display until the view is refreshed with new data.
Alternatively, you can set a width of one byte for fields that
contain no data to reduce the space that these fields occupy on
the screen.
After you create a table, you can update it to include changes in
the source file. You can click the Refresh from Source File
button on the task bar or on the Table Layout Options tab of the
Method of access
You usually use the Data Definition Wizard to convert delimited
files. But if ACL cannot recognize a file as delimited, you can use
the Table Layout window to convert the file manually.
If you need to specify the value of a field separator or a text
qualifier, and the character does not appear on your keyboard,
enter an X followed by the hexadecimal value. For example, X09
represents a tab. To specify that no text qualifier exists, use X00,
the hexadecimal equivalent of null. If your delimited data is an
ASCII text file, you can use an ASCII table to find hexadecimal
values.
Reference information
You can use the IMPORT command to work with delimited files
in a script.
If you analyze different files of like structure, you can use
functions to parse the fields, then link the resulting table to new
source data.
Example
IMPORT DELIMITED TO Employees_retail
"Employees_retail.fil" FROM
"Employees_retail.csv" SERVER 1 0
SEPARATOR "," QUALIFIER '"' CONSECUTIVE
STARTLINE 0 KEEPTITLE FIELD "First_name"
C AT 1 DEC 0 WID 20 PIC "" AS "" FIELD
"Last_name" C AT 21 DEC 0 WID 20 PIC ""
AS "" FIELD "Employee_number" C AT 41 DEC
0 WID 16 PIC "" AS "" FIELD
"Employee_number" C AT 57 DEC 0 WID 6 PIC
"" AS "" FIELD "Hire_date" D AT 63 DEC 0
WID 10 PIC "MM/DD/YYYY" AS "" FIELD
"Salary" N AT 73 DEC 0 WID 10 PIC "" AS
"" FIELD "Bonus_2004" N AT 83 DEC 2 WID
11 PIC "" AS ""
When you rerun the IMPORT command, ACL creates a new
table. When you refresh the table, ACL updates the original table
with new data.
To import a different source data file with fields that match in
number, order, and data type, modify the italicized items. In this
example, Employees_retail specifies the table name,
Employees_retail.fil specifies the saved data file, and
Employees_retail.csv specifies the source data file.
Using ODBC
You can refresh from the source to obtain updated data or edit
the IMPORT statement to reuse a table layout when you import
delimited data through ODBC.
See ODBC-compliant databases on page 61.
For a data source driver, use the Microsoft Text Driver that ? For more information on ODBC
comes with Microsoft Jet. This driver works with all types of drivers, see ODBC on page 71.
delimited files, including comma, tab, and custom-delimited.
HTML
HTML (HyperText Markup Language) defines the structure and
layout of World Wide Web documents. Markup tags, embedded
in the text, specify structural elements.
In your browser, a document displays similarly to the image on
the left. But the data that you import consists only of HTML
markup, and looks similar to the image on the right.
Once you import the document into ACL, you might find that
the text no longer lines up properly. The way that your document
looks depends upon the markup tags that were used when the
document was created.
Method of access
Import HTML files by one of these methods.
IBM variable-length
To save space, many IBM mainframe computers use
variable-length records to store records of different types and
different lengths in one file. The file contains a field that stores
the length of every record.
Every IBM variable-length record begins with a four-byte field.
These four bytes consist of a two-byte header field that stores the
record length in binary format and another two-byte field that
you can ignore for ACL purposes. ACL automatically calculates
the record length from the two-byte header field, which is called
a Record Descriptor Word (RDW).
IBM variable-length files also contain a four-byte
block-description field, called a Block Descriptor Word (BDW),
at the beginning of the file and at regular intervals throughout
the file. If the block-size value is 48000, for example, another
Block Descriptor Word occurs 48000 bytes later. ACL hides this
field so you never see it. However, this field must be present for
ACL to recognize the file as IBM variable-length.
This sample IBM variable-length file contains two records with
lengths of 5800 and 5100 bytes respectively:
Sample IBM variable-length file
Name Company Position
Although you never actually see the four-byte BDW, the hidden
structure of the sample file looks like this:
BDW
RDW Record 1, length 5800
RDW Record 2, length 5100
RDW
BDW
RDW
Method of access
Import IBM variable-length data by using either the Data Request your files in EBCDIC
Definition Wizard or the Table Layout window. format to reduce the risk of damage to
the record-length field through
ACL automatically bypasses the first four bytes of the file. conversion. For information on
download formats, see Download
When you manually create the table, ignore the first four bytes of
precautions on page 11.
each record, which contain only record-length information.
Begin defining data in position 5.
The wizard attempts to identify all fields. If the wizard tries to
define the record-length field, select the Ignore this field option
on the Edit Field Properties screen.
In all other respects, you process an IBM variable-length file the
same as you process any fixed-length file.
Reference information
Sometimes the download software strips off both the RDW that
contains the record length and the four-byte BDW. When this
occurs, you must reacquire the data or use an ACL expression to
calculate the record length.
If you lose only the four-byte BDW, you can define the first two ? Also see Avoid record-length
bytes of the record as a binary field called Record_length. Once problems on page 12.
you close and reopen the file, the fields line up correctly.
IMS
The IMS transaction management system from IBM stores data
in a hierarchical format with levels or segments of data, each
dependent on the higher level. To access data at a lower level in
the hierarchy, you follow a hierarchical path from the root to the
segment that you need. The database description (DBD) defines
the order of access.
Method of access
Access IMS data by one of these methods.
ODBC
You can also access IMS data through ODBC. See
ODBC-compliant databases on page 61.
Linux-based data
The open-source Linux operating system is closely related to
UNIX. Linux runs on a variety of hardware platforms, including
PCs, workstations, and mainframes.
Method of access
You can access files on a network-enabled computer that runs
Linux just as you access files on a network-enabled PC, provided
you have the appropriate access rights.
You can also access Linux-based data and process it on the server
on which it resides by using ACL Server Edition for Linux. Any
flat files and print-image files that are located on the server can
be processed directly on the server using the offline scripting
functionality.
For information on other ways to access files on a Linux server,
see UNIX-based data on page 88.
To access DB2 databases by using ACL Server Edition software,
see DB2 on page 31.
To access Oracle databases by using ACL Server Edition
software, see Oracle on page 69.
Microsoft Access
Import tables from a Microsoft Access relational database system
by one of these methods.
ODBC
When you use ODBC, ACL imports the database as a new ACL
flat file. See ODBC-compliant databases on page 61.
Other methods
You can also save the data in dBASE, text, or print-image format ? For more information, see
in the source application. You can then access the data by the dBASE-compatible on page 35.
method applicable to that particular file type. Of these formats,
ACL reads dBASE most easily.
Microsoft Excel
ACL reads data in any version of Excel, starting from Excel 3.0.
You do not need Microsoft Excel installed on your PC to import
Excel files into ACL.
Unlike true database applications, where individual fields
contain single data types, individual columns in Excel can
contain various types of data. For example, in a single column,
one cell might contain a date, another a number, and another a
text string, or even a picture. Such inconsistencies can cause
problems when you transfer the data through ODBC or
Microsoft ADO, the technology that ACL uses to read Excel files.
If you do not format columns before you enter data in an Excel
worksheet, a column that contains a mixture of alpha and
numeric characters may fail to import successfully into ACL.
When you do not apply a format before you enter data, data type
is determined by the contents of the first data cell in the column.
For example, if the first cell contains numbers, the column is
considered a numeric column. Then, if subsequent cells in the
same column contain non-numeric data, these cells display zeros
in ACL. Similarly, if the first cell of a previously unformatted
column contains text, subsequent cells that contain numbers
appear empty in ACL.
To avoid loss of data, ensure that each column in the Excel file
contains only numbers or text. Otherwise, you must save the
Excel file in a format that can preserve the heterogeneous
contents of the file when you import it into ACL.
For more information, see Save the Excel file as delimited on
page 53 and Save the Excel file as dBASE on page 53.
Method of access
Import Excel data by one of these methods.
ODBC
When you use ODBC, ACL imports the spreadsheet or named
range as a flat file. See ODBC-compliant databases on page 61.
Other methods
You can also save the data in dBASE, text, or print-image format ? For more information, see
in the source application. You can then access the data by the dBASE-compatible on page 35.
method applicable to that particular file type. Of these formats,
ACL reads dBASE most easily.
Reference information
To avoid loss of data in columns that were not formatted before
data was entered and that contain both alpha and numeric
characters:
only one of the identically named fields. To retain all of the data,
make sure that the first ten characters of each field name is
unique.
If the first data cell of a previously unformatted column contains
only numbers, Excel saves the column as numeric data and
non-numeric cell values display as zeroes in ACL. To avoid this
problem, select the first data cell of the worksheet in Excel and
insert an apostrophe (') in front of the cell's existing contents.
Then save the worksheet in DBF4 format. Now, when you access
the dBASE file with ACL, the data type becomes ASCII and all
numbers display correctly.
Microsoft Word
Microsoft Word documents are saved under a proprietary
format with a doc extension.
Method of access
You cannot import a Word document directly into ACL. Instead,
you must save the document as a text file or print it to a text file,
then import it through the Data Definition Wizard.
For a description of both procedures, see Save as a text file and
Print to a text file in Rich Text Format (RTF) on page 81.
Multiple-record-type data
Multiple-record-type files contain more than one record type,
and can consist of fixed-length or variable-length records.
You first identify individual record types, then define the fields in
ACL. You then have several options for analyzing the data:
One record type at a time by creating a filter in the view.
For example, if your file contains records of type A and B,
use a filter to display only records of one type or the other.
More than one record type at a time by using a group in
a script or by using conditional or static-conditional
fields. This is a more complex but versatile approach.
Method of access
When multiple-record-type files contain a record-length field,
you can use the wizard to create an ACL table. When
multiple-record-type files do not contain record lengths, you
must use the Table Layout window to manually define the fields.
Print-image files
Electronic report files, also known as print-image files, are
multiple-record-type files, and commonly include header, detail,
and trailer records. See Print-image on page 78.
CR/LF files
In CR/LF files, a carriage return (CR) and a line feed (LF)
character, or sometimes just one or the other, indicate the end of
a line. ACL automatically identifies a CR/LF file and processes it
in the same way that it processes a fixed-length file.
Segmented files
Segmented files, also called bucketed files, are a type of
variable-record-length file in which a set of header fields is
followed by one or more sets of data fields.
Segmented files store data efficiently because each record is only
as long as it needs to be and contains no empty or repeating
fields. COBOL files often contain fields or tables that can occur a
varying number of times, depending on the value in the header
field, as defined by an OCCURS clause.
Here is an example of a segmented file.
Header fields Segment Segment Segment Segment
Segments Emp_ID Emp_Name Year Sales Year Sales Year Sales Year Sales
3 101 Andersen 2002 58500 2001 60400 2000 63000
1 102 Guterriez 2002 60700
2 105 Schmidt 2002 62500 2001 58300
4 107 Lapierre 2002 63300 2001 59500 2000 67500 1999 63500
1 108 Ko 2002 56000
The header fields might contain a field that indicates the number
of segments that follow, as in the example. If this information is
not present, you might be able to calculate the number of
segments in a script by using other information such as the
segment length.
Define the header fields and the first segment. Then you can use
a script to process the file or to extract the fields to a flat file. To
define the fields you might need to refer to a record layout that
provides you with the following kinds of information:
Field name Data type Length
Segments
Emp_ID
Emp_Name
Year
Sales
ODBC-compliant databases
ODBC (open database connectivity) is a Microsoft interface that
you can use to access data from any database application.
ODBC-compliant data sources include Access, Excel, Oracle,
DB2, FoxPro, Peoplesoft, dBASE, Informix, IMS, Sybase, SQL,
SQL Server, ADABAS, SAS, and SPSS, among others.
Developers can create ODBC-enabled applications, also known
as ODBC-compliant or ODBC-client applications. Developers
can also write ODBC drivers, DLLs (dynamic link libraries), for
specific database management systems or data sources.
Method of access
To access an ODBC-compliant data source in ACL, you need an
ODBC driver specific to ACL, and an ODBC driver specific to the
data source. Drivers, developed by data-source and third-party
manufacturers, form the core of ODBC, and act as the translator
between ACL and the data source.
The rules and methods of access do not differ, regardless of the
data source. ACL uses the ACL-ODBC driver to place a call to the
ODBC Administrator for the data. The Administrator, in turn,
uses the data-source ODBC driver to read the data from the data
source. This data is then translated into a standard ODBC format
and returned to ACL, where ACL copies the data to an ACL data
file.
Reference information
This section contains information about the commands that are
associated with ODBC-compliant data and about the SQL
WHERE clause.
Supply a password
To import or refresh password-protected ODBC data sources,
use the PASSWORD command to create a dialog box that
prompts for the password.
Or use SET PASSWORD to store a password in a script.
The PASSWORD method of prompting for a password provides
greater security than the SET PASSWORD method, which
provides a password that is visible in the text of your script.
PASSWORD command syntax
PASSWORD 1 "Enter Payroll Password:"
REFRESH Payroll PASSWORD 1
SET PASSWORD command syntax
SET PASSWORD 1 "passwordXYZ"
REFRESH Payroll PASSWORD 1
Then, ODBC returns only those records that meet the test
condition. For example, you can use the WHERE clause to limit
data to accounts receivable over a specified amount or to
shipping dates that fall within a certain range.
The WHERE clause requires two operands and one operator. Do
not type the word where in the WHERE text box. Syntax can
vary from one database to another, so if you experience
problems, contact your systems administrator.
Oracle
When you access Oracle data through ODBC, ACL imports the
data and copies it to a local drive. When you access Oracle data
through ACL Server Edition for AIX or ACL Server Edition for
Linux, ACL reads the data directly from the Oracle database.
This means that the ACL table reflects changes to source data as
such changes occur.
Method of access
Access Oracle data by one of these methods.
6 Select the fields or columns that you want from the Select
Oracle Columns screen.
7 If you want to use WHERE and ORDER clauses, enter them
in SQL syntax on the Where and Order Clauses screen.
8 Once you define all of the fields, the wizard prompts you to
name and save the table.
To edit the query after you close the table, right-click the table in
the Overview. Select Properties, then click the Edit Query tab.
In a script, you can use the variables DBWHERE, DBORDER, ? To learn more, select Help Index
and DBCONDITION to edit WHERE, ORDER, and and look up defining server data files.
CONDITION clauses. The values that you assign to these
replacement variables remain in effect until you delete the
variables.
ODBC
You can also access Oracle data through ODBC. See
ODBC-compliant databases on page 61.
Method of access
Access OS/400 data by one of these methods.
External definition
Access the external definition file using the Data Definition
Wizard.
On the Select Data Source screen of the wizard, select External
Definition. On the next screen, select OS400 FDF, then browse
to locate the file. ACL creates a table layout using the external
definition, then prompts you to save it. If prompted, browse to
locate the data file. ACL creates the table.
ODBC
When you use ODBC, ACL imports the database as a flat,
fixed-length, sequential file and automatically creates a table.
See ODBC-compliant databases on page 61.
Other methods
If you cannot obtain an FDF file, obtain a printed or electronic
copy of the layout information, and create the table layout
manually.
You can also save the data in dBASE, text, delimited, or
print-image format in the source application. Then access the
data by the method applicable to that particular file type. ACL
Reference information
The structure of your OS/400 FDF must resemble the structure
of the sample FDF below. With the FDF in ASCII format, you can
compare the ACL table to the FDF, and make changes to the table
layout, if necessary.
1 ASCII text
2 DOS random
3 BASIC sequential
4 BASIC random
6 No-conversion file
1 ASCII
2 NUMERIC
3 ASCII
4 BINARY
5 ZONED
6 PACKED
7 MICRO
8 FLOAT
9 FLOAT
10 EBCDIC
11 ZONED
12 PACKED
PDF
You can capture Portable Document Format (PDF) text
electronically provided PDF security features do not prevent
copying or printing. Also, some PDF files store their content as a
raster image that you cannot edit.
ACL cannot read PDF files directly. To access data in a PDF file,
you must open it with an Adobe Acrobat product, such as
Acrobat Reader, from which you can select, copy, and paste
content into a text editor. Alternatively, you can print the
document to a text file.
To print to a text file
1 Use the Windows Add Printer Wizard to add a
Generic/Text Only driver. For assistance on how to add a
printer, contact your systems administrator
2 In the Print dialog box, select Generic/Text Only from the
Name drop-down list and select the Print to file check box.
3 Click OK, and at the prompt, save the file.
You can open the file in a text editor to look at the record layout.
Sometimes the text no longer lines up properly. This occurs
because the PDF format and the application that created the file
store spacing information differently. Consequently, the spacing
information can become lost when you convert the file to text.
Import the text file through the Data Definition Wizard.
Method of access
Access PL/1 data through the Data Definition Wizard, or by
defining the data manually.
If you have the PL/1 copybook on your local drive, this file can
be converted to a table layout in ACL. You can then link this
copybook to the PL/1 data that is defined by this copybook.
To access PL/1 data through the Data Definition Wizard
1 First, you use ACL to convert the PL/1 copybook to an ACL
table layout. Select File > New > Table to open the Data
Definition Wizard.
2 Click Next and select Local. Click Next.
3 Select External Definition and click Next.
4 Select PL/1 and click Next. The Select External Definition
screen of the wizard appears.
5 In the External Definition box, click Browse and select the
PL/1 copybook file. Ensure that the PL/1 copybook file name
appears in the Selected Definitions pane. Click Next.
6 On the Select Conversion Properties screen, you can select
the Remove leading file indicator from field names and the
IBM Variable Length check boxes if applicable. Click Next.
The wizard displays the contents of the table layout. Click
Finish.
7 When prompted, enter the name of the new table layout.
Click OK. The Select File Location box appears. This box
Manual layout
If you cannot obtain a copybook file, obtain a printed or
electronic copy of the layout information, and create the table
layout manually.
Reference information
Here is a sample PL/1 copybook.
DECLARE
1 ZDCBHDRD UNALIGNED BASED(ZDCBHDR_PTR),
3 HDR_RECORD,
5 DC_CONTROL,
7 RECORD_STATUS CHAR(1),
7 STATUS_DATE CHAR(6),
7 HDR_TYPE CHAR(1),
7 ENTRY_DETAIL,
9 POINT_OF_ENTRY,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 ENTRY_DATE BIN FIXED(15),
9 BATCH_BALANCE,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 ENTRY_START_DATE BIN FIXED(15),
11 ESTIME BIN FIXED(31),
11 ENTRY_STOP_DATE BIN FIXED(15),
11 ENTRY_STOP_TIME BIN FIXED(31),
9 CORRECTION,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 CORR_START_DATE BIN FIXED(15),
11 CORR_START_TIME BIN FIXED(31),
11 CORR_STOP_DATE BIN FIXED(15),
11 CORR_STOP_TIME BIN FIXED(31),
Print-image
Print-image (report) files consist of records that contain different
types of data, usually in the form of a text report that is formatted
for print. Print-image data sources often contain extraneous
information, such as page headers, that you ignore when you
define the data.
Method of access
When you define print-image files through the wizard, they fall ? See Direct access, manual layout
into the import and copy, manual layout category of data on page 20.
access. When you manually define print-image files, they fall
into the direct access, manual layout category.
The Print Image File Definition screen of the wizard contains
features to help you process print-image files.
Other options
Instead of using the Print Image File Definition screen when
you define your data source, you can select the Other File
Format option on the File Format screen of the wizard. Then,
on the File Type screen, select either the Print Image (Report)
File or the Multiple Record Type File option.
You can also select either the Skip to Finish or the Skip Field
Identification option in the wizard to define your file in the
Table Layout window.
Reference information
You can reuse or modify a table layout.
Save as text
To save RTF as text
1 From the File menu, select Save As.
2 From the Save as type drop-down list, select Text Only
(*.txt).
You can open the file in a text editor to look at the record layout.
Then import the data through Data Definition Wizard.
Print to file
As another option, you can use the Windows Add Printer ? For assistance on how to add a
Wizard to add a Generic/Text Only driver. printer, contact your systems
administrator.
To print to a text file
1 In the Print dialog box, select Generic/Text Only from the
Name drop-down list and select the Print to file check box.
2 Click OK, and at the prompt, save the file.
You can open the file in a text editor to look at the record layout.
Then import the data through Data Definition Wizard.
SAP
You can access SAP data through Direct Link. Or you can
download the data from SAP queries, reports, or the data
browser in a format that ACL can read. As a final option, you can
print your report to a file.
Direct Link
Direct Link is a complementary product that provides the ability ? To learn more about Direct Link, visit
to link directly to SAP data from within ACL. the ACL website.
First, you open Direct Link and log onto your SAP system. Then
you build queries to select the data that you want to download.
When you next run the queries, ACL automatically downloads
the source data and creates a table that contains the data that you
queried.
SMF
System Management Facility (SMF) data sets collect and record
system and job-related information. Use this information to:
Invoice for system use
Report reliability
Analyze the configuration
Schedule jobs
Summarize direct access volume activity
Evaluate data set activity
Profile system resource use
Maintain system security
When a particular event occurs, SMF writes a certain type of
record to the data sets. Such events include job starts, job abends,
data set deletions, and initial program loads (IPLs).
By studying trends from historical SMF data, you can evaluate
changes in configuration, workload, or job scheduling
procedures. Similarly, you can analyze SMF data to find
inefficient operational procedures and programming
conventions.
Method of access
Download the SMF data set from the mainframe in binary
format. Obtain the SMF layout file, which describes the SMF data
layout, from the IBM Documentation CD.
In the Data Definition Wizard, import the SMF layout file.
Accept the wizard defaults until the File Properties screen,
OPEN SMFLAYOUT
COMMENT
******** This filters out records based on their record type.
COMMENT
******** This will create a an ACL table layout file with multiple IBM record
types.
ELSE
END
Once you run the script, add the FILE_NAME card to the first row
of the table layout file SMF80.FMT.
Finally, import the format file into ACL with the Import from
Server File option, then link the resulting ACL table layout to the
SMF data file.
Reference information
This section provides examples of SMF record types and shows
what types of reports you can create from SMF data. It also offers
suggestions for ways to use these reports.
Multiple extents
An installation can check the number of extents field in the
UCB section of SMF type 14 and 15 records. From this field, the
installation can identify direct access data sets that have exceeded
their primary allocation and used their secondary allocation.
Secondary allocation, while useful, can affect system
performance and fragment the space on direct access volumes.
Check for multiple extents to avoid an X37 abend in a
production job the next time it runs. While the job may get the
space it needs this time, the next allocation may be on a volume
that only has enough space for the primary allocation. The
attempt at secondary allocation will fail and waste resources in
reruns.
Open/close activity
When EOV closes or processes a data set, SMF writes a type 14 or
15 record. The installation can count type 14 and 15 records to
find how many times EOV closed or processed a given data set.
For this kind of report, exclude SMF records for programs such
as sorts, where you know in advance that the open and close
activity is significant.
Blocking factors
From the block size and logical record length fields in SMF type
14 and 15 records, a system can identify data sets that are
processed with ineffective block factors. For example, a 10,000
record data set is processed, unblocked, with a logical record
length of 80, using a 3380 device. When the system encounters
such a data set, it can increase its block size to 6160 or 77 records.
Increased block size minimizes I/O-processing overhead and
reduces direct-access storage requirements.
Optional services
Although they are useful, optional services can hinder system
performance if they are used inappropriately. For example, the
write-validity check (OPTCD=W) service requires an additional
disk rotation to reread the data that is written for each output
block. Similarly, a data set that overuses the chained scheduling
(OPTCD=C) service might monopolize the available time on a
channel. The system can use the SMF type 14 and 15 records to
ensure that applications that use an optional service are
authorized or justified in using the service.
UNIX-based data
The UNIX family of operating systems was one of the first
portable operating system that could be installed on different
hardware platforms, from PCs to mainframes. Originally free,
many businesses and organizations have developed customized,
proprietary versions of the original operating system. AIX is a
version of UNIX produced by IBM. Linux is an open-source
operating system derived from UNIX.
Like Microsoft Windows, UNIX typically uses the ASCII
character set. This means you do not need to convert data from a
computer that runs UNIX to use the data on a PC. However,
some numeric data types that are used on AIX and Linux
systems may be unreadable on a PC.
Method of access
Access UNIX-based data by one of these methods.
Network Access
You can access files on a network-enabled computer that runs a
UNIX-based operating system just as you access files on a
network-enabled PC, provided you have the appropriate access
rights.
To access a UNIX file system as a mapped drive or network share ? See also Drive mapping on
in Windows, the UNIX system must run a Server Message Block page 12.
(SMB) to provide Windows network share support. Samba
software is commonly used for this purpose.
FTP
You can retrieve files from a UNIX computer using FTP.
ODBC
You can access ODBC-compatible databases that run on a UNIX
system through ODBC. For information see ODBC-compliant
databases on page 61.
Direct Link
You can access SAP data on a UNIX system using Direct Link or
the methods of access described in SAP on page 82.
VSAM
ACL handles Virtual Storage Access Method (VSAM) files
similarly to the way it handles IBM variable-length files. You
specify variable length as a file property when you define VSAM
data.
VSAM Entry Sequence Data Sets (ESDS) are read in the order
that the records were written to the file. VSAM Key Sequence
Data Sets (KSDS) are processed in the sequence defined by the
VSAM index. VSAM Relative Record Data Sets (RRDS) are
processed according to the relative record number sequence. In
VSAM files, the data always starts in position 5.
VSAM consists of two major parts:
Catalog management
VSAM maintains extensive information about data sets
and direct access storage space in a catalog. The catalog
can be either an integrated catalog facility, which is
recommended, or a VSAM catalog. The catalogs
collection of information about a particular data set
defines that data set's characteristics. Every VSAM data
set must be defined in a catalog.
Record management
You can use VSAM to organize records into four types of
data sets: key-sequenced, entry-sequenced, linear, or
relative record. The way in which records are stored and
accessed accounts for the difference between these types
of data sets.
Define VSAM data sets by using either access-method-services
commands or Job Control Language (JCL) dynamic allocation.
Method of access
Follow these steps to access VSAM data through a client/server
environment:
1 Once you set up a profile, activate the Data Definition Wizard
and select the ACL server on which the VSAM data resides.
Enter your password if prompted.
2 On the Select ACL Server Data Source screen, select the Flat
Files or VSAM option, then open the file that you want to
define.
3 As you continue through the wizard, select IBM mainframe
or minicomputer (EBCDIC) as the character set and
Variable Length as the file property. Make other choices
appropriate to your file. Define your fields, then name and
save the file.
XML
XML is short for Extensible Markup Language, an open standard
for a markup language that you use to create documents and
exchange data between applications and between organizations.
XML uses tags to identify the structure and content of data that
can be transmitted in the form of text.
ACL reads XML data using Microsoft ActiveX Data Objects
(ADO) technology.
Method of access
ACL automatically creates a table layout from an XML document
provided your XML structure looks like this:
<RECORDS>
<RECORD>
<INVOICE_AMOUNT>618.30</INVOICE_AMOUNT>
<INVOICE_DATE>11/17/2000</INVOICE_DATE>
<INVOICE_NO>5981807</INVOICE_NO>
</RECORD>
<RECORD>
<INVOICE_AMOUNT>217.80</INVOICE_AMOUNT>
<INVOICE_DATE>11/18/2000</INVOICE_DATE>
<INVOICE_NO>5981808</INVOICE_NO>
</RECORD>
</RECORDS>
I NDEX
A identifying with SMF 87 DB2 32
Block-description field Oracle 70
Access. See Microsoft Access
IBM variable-length data 46 Control totals
Accessing data
Bucketed files. See Segmented files importance of 5
ACL methods 16
Conversion
ACCPAC data
character set 7
accessing 24
ACL file
C numeric to character, dBASE 35
Categories problems, avoiding 11
definition of 17
data access 16 copybook
ACL Server Edition
CDs COBOL, interpreting 27
DB2 31
storage media 3 PL/1, sample 77
Oracle data 69
Character data CR/LF
UNIX-based data 89
dBASE format 35 multiple-record-type file 56
ACL table layout
delimited files 37 CTOD( ) function
about 16
Character set parsing date fields 40
ACL Server Edition
about 12 conversion problems 7, 11
ActiveX Data Objects Client Access
and Microsoft Access 51 download utility 71 D
and Microsoft Excel 52 Client/server environment Data
and XML 92 data transfer option 13 finding sources of 5
Advanced Options DB2 31 integrity
Field Definition dialog box 78 Oracle 69 testing for 9
AIX processing data in 12 losing, undefined 80
support in ACL 13 VSAM data 91 missing
AIX-based data Client/server. See also Microsoft Excel 52
accessing 25 ACL Server Edition preparing for transfer 10
AIX-based dateless also UNIX-based COBOL data Data access
data accessing 26 ACL methods 16
AS/400. See OS/400 category of access 22 impediments to 4
ASCII Column titles stages of 2
COBOL copybook format 26 header records 56 Data browser
data conversion 7, 11 Command syntax SAP 82
FDF format 71 See also Scripts Data conversion problems
Automatic fill dBASE files, OPEN 36 avoiding 7
blank fields 78 IMPORT Data dictionary
Automatic opening ODBC 63 record-layout document 15
Table Layout window 39 print-image 79 Data error
REFRESH 64 and client/server 13
SET PASSWORD 64 risks in downloading 11
B SMF data, reading 84 Data requests
Completeness of data formulating 5
Barriers
checking 9 sample form 8
data access 4
Computed fields Data set activities
Blank fields
COBOL level 88 29 monitoring with SMF 86
filling automatically 78
Condition clauses Data source name. See DSN
Block factors
93
Index
94
Index
95
Index
96
Index
97
Index
98
2006 ACL Services Ltd.