1

Edition

PROFESSIONAL DBA SERIES

Christopher Kempster

SQLServer 2000 fortheOracle DBA

SQL Server for the Oracle DBA

Dedicated to Veronica, a wonderful wife and a dearly loved friend; and my great kids Carl and Cassidy. Thanks to Raf Cammarano for editing and providing ongoing encouragement to complete this so called small e-book.

Copyright 2003 Christopher Kempster Perth, Western Australia

Copying, selling, duplication or reproduction of this work is expressly forbidden without the copyright holders written consent. All scripts and examples are used at your own risk. Spotlight on SQL Server is a registered trademark of Quest Software, Inc. Diagnostic Manager is a registered trademark of NetIQ Corporation. Microsoft Word 2000 is a registered trademark of the Microsoft Corporation. SQL Server and SQL Server 2000 is a registered trademark of the Microsoft Corporation. Oracle is a registrated trademark of the Oracle Corporation. SQL Server Magazine Copyright 2002, Penton Media, all rights reserved. SQL-Server-Performance.com website is copywritten by Brad M.McGehee.

T able of Contents
SQL SERVER ARCHITECTURE 101........ 1 OVERVIEW OF DATABASE INSTANCES ....1 Control Files and Data Dictionary ........4 Datafiles ..........................................5 Tablespaces .....................................6 Redo and Archived Logs .....................7 Rollback Segments ............................7 Init.ora ............................................8 Environment Variables .......................8 MEMORY STRUCTURES .........................9 SGA (SQL Server Address Space)...... 11 Buffer Cache .................................. 12 Redo Log Buffer (Log Cache) ............ 14 Library Cache (Procedure Cache) ...... 15 SQL SERVER CONNECTIONS ..................... 17 EXTENDED STORED PROCEDURES ............... 18 CHECKPOINT ........................................ 18 BACKGROUND PROCESSES........................ 20 STORAGE ENGINE .................................. 22 Tablespaces ................................... 22 Blocks / PctUsed & PctFree / Extents / Segments ...................................... 24 Free Space Management .................. 26 Fragmentation ................................ 30 Reporting on Fragmentation .......... 30 Resolving Fragmentation ............... 31 High Water Mark (HWM) .................. 33 Segments ...................................... 35 RELATIONAL ENGINE .............................. 35 INSTALLATION ................................. 37 EDITIONS AND VERSIONS ........................ 37 LICENSING SCHEMES .............................. 38 Alter licensing mode after install?...... 41 BINARIES & INSTANCE INSTALLATION .......... 42 Service Account .............................. 43 Encrypted File System (EFS) ............ 44 Encrypted Database File IO (avg 1min 10sec to create and populate) 45 RAID Set-up with SQL Server, Install Issues, OFA.................................... 46 Items to consider before installation (summary)..................................... 49 Example Installation Standard, Developer or Enterprise Ed............... 50 MOVING YOUR SYSTEM DATABASES AFTER INSTALLATION ...................................... 55 Moving MASTER and Error Logs ........ 55 Moving MSDB and MODEL ................ 55 Moving TEMPDB .............................. 57 GUI AND COMMAND LINE TOOLS ...... 58 WHAT IS INSTALLED? ............................. 58 ENTERPRISE MANAGER (OVERVIEW) ............ 59 Registering Servers ......................... 59 Interface Overview .......................... 61 Meta Data Repository ...................... 62 Meta Data Repository Snap-In ....... 63 Task Pad View ................................ 64 Inserting NULLs into Table Columns.. 65 DBVERIFY ......................................... 65 SVRMGRL ......................................... 65 SQL LOADER ....................................... 66 JAVA AND SQL SERVER ........................... 66 SQL TRACE ......................................... 67 QUERY ANALYSER (SQL*PLUS) ................. 68 DATABASE CONFIG ASSISTANT / ORACLE ADMIN ASSISTANT ETC .................................... 68 NET8 ASSISTANCE / SQLNET ................... 68 UTLBSTAT, UTLESTAT, STATSPACK ...... 69 SRVCTL ............................................ 69 LOG MINER ......................................... 69 ISQL ................................................ 69 ISQL vs OSQL command line ............ 70 BACKUP AND RECOVERY ................... 71 BACKUP.............................................. 71 Recovery Interval............................ 72 Recovery Models ............................. 73 Backup Devices............................... 74 RMAN ............................................ 75 Database Maintenance Plans ............ 75 Data Dictionary Views (Media Sets, Backup Sets) .................................. 77 Hot / Online (Full) Backups .............. 79 Differential Backups ........................ 81 Archived Log (Transaction Log) Backups .................................................... 82 Tablespace (Filegroup) Backups ........ 83 Moving DB &Transportable Tablespaces .................................................... 84 Some issues with MODEL and MSDB databases ................................... 88 Fixed dbid for system databases .... 88 Scripting Databases......................... 89 Backup control file to trace............... 91 Verifying Backups ........................... 91

RECOVERY........................................... 92 Killing User Connections and Stopping Further Connects ............................ 92 Using the GUI for Recovery .............. 93 Options - Leave database in nonoperational state but able to restore additional logs ............................. 94 Options Using the Force restore over existing database option ........ 94 Restoring a databases backup history from backup files ............................ 95 Restore cannot fit on disk................. 96 Restore uses logical names ............ 96 RESTORATION BY EXAMPLE ....................... 97 Restore Master Database ................. 97 Restore MSDB and Model Databases.. 99 Suspect Database ........................... 99 Database is in Loading Mode ? ..... 100 Restore with file move ................... 100 Restore a specific File Group........... 101 Adding or Removing Data Files (affect on recovery)................................. 101 Emergency Mode .......................... 102 Restore Full Backup....................... 103 Partial (stop at time) PITR Restore on a User Database .............................. 104 Corrupt Indexes (DBMS_REPAIR) .... 104 Reinstall NORTHWIND and PUBS ..... 105 Other Recovery Scenarios .............. 105 Scenario 1 - Lost TEMPDB Database ................................................ 105 Scenario 2 - Rebuildm.exe......... 105 Scenario 4 - Disks lost, must restore all system and user databases from backup to new drive/file locations. 109 TRANSACTIONS AND LOCKING ....... 110 ROLLBACK SEGMENTS & REDO LOGS ORACLE SUMMARY OF UNDO MANAGEMENT / ROLLBACK SEGMENTS.......................................... 110 Oracle Redo Logs ....................... 112 Oracle Locking ........................... 113 SQL Server - Undo & Redo Management Architecture ................................. 114 In Summary .............................. 117 LOCKING & ISOLATION LEVELS ................ 118 TRANSACTIONS ................................... 121 MONITORING LOCKING ISSUES ................ 122 CONTROLLING LOCKS ........................... 124 Detecting and dealing with Deadlocks .................................................. 126 Example Deadlock Trace ............. 128 PERFORMANCE TUNING .................. 130 TRACING (PROFILER) ............................ 130 Saving & Scripting Traces .............. 133 Index Tuning Wizard ..................... 134 EXPLAIN PLAN .................................... 137 BLACK BOX TRACING? .......................... 139 LATCH STATISTICS .............................. 140 PAGE SPLITS...................................... 140 PERFORMANCE MONITOR (PERFMON.EXE).... 142

TRACE FLAGS ..................................... 147 INDEXING ......................................... 150 Index Types .............................. 150 Like Clause ................................ 151 Functions and Indexes ................ 152 Composite Indexes ..................... 152 Indexed Views (Materialised Views) ................................................ 152 Covering Indexes ....................... 154 Index Skewing ........................... 155 Index Statistics and Fragmentation ................................................ 156 Heaps, Clustered Indexes and Fragmentation ........................... 156 Rebuilding Indexes ..................... 158 Performance Counters................. 158 Index Management via Enterprise Manager.................................... 159 Summary .................................. 159 DATABASE FILE IO STATS ...................... 160 WAIT STATISTICS................................ 160 STATSPAK....................................... 161 GENERAL PERFORMANCE TIPS ................. 161 Server Configuration ..................... 161 What can affect performance (not an exhaustive list) ............................. 162 Collecting SQL Server Statistics ...... 163 General DB Parameters, Settings and other DBA items ........................... 163 Physical Database Design............... 164 SQL and Developer Tips ................. 166 Backup and Recovery .................... 168 CPU is a Bottleneck?...................... 168 I/O is a Bottleneck?....................... 168 TASK MANAGER .................................. 169 COM+ ............................................ 169 Isolation level of COM+ components 171 Proxying your COM+ components ... 172 STORED PROCEDURES ........................... 172 Using SET NOCOUNT ..................... 172 SUMMARY - TOP PERFORMANCE PROBLEMS .. 173 HIGH AVAILABILITY ....................... 175 CLUSTERING ...................................... 176 Oracle RAC and Application Clustering .................................................. 176 SQL Server Clustering ................... 178 Ongoing Cluster Administration .... 183 NLBS (Web Farm Cluster) .............. 183 CLB (Component Clustering) .......... 185 Microsoft Clustering - The Big Picture .................................................. 185 FEDERATED DATABASES ........................ 187 LOG SHIPPING .................................... 189 Manual Log Shipping - Basic Example .................................................. 191 ONLINE OBJECT REDEFINITION ................ 195 GLOBALISATION SERVICES (NLS) .. 196 ORACLE ARCHITECTURE ......................... 197 SQL SERVER ARCHITECTURE .................. 198

Collation Installation (Server Level) .................................................. 202 Collation Database ..................... 203 Collation Table Column ............... 204 Collation Expressions and Variables .................................................. 205 INTERNATIONALLY AWARE APPLICATIONS .... 206 OLAP............................................... 207 TERMINOLOGY .................................... 207 ADMINISTRATION ................................ 208 Installation................................... 208 Administering from your client PC ... 210 Migrating the repository ................. 211 Storage Models and Connectivity..... 213 OLAP Databases......................... 213 Cubes and Cube Storage Models .. 214 OLAP Connectivity ...................... 214 Security .................................... 214 Example Cell Level Security ...... 215 ADO MD & DSO ................................ 217 BACKUP............................................ 219 SCRIPTS & SCHEDULED JOBS ......... 220 T-SQL ROUTINES ............................... 221 XP_CMDSHELL Example ................. 221 DTS ............................................... 221 ISQL / OSQL.................................... 222 VB SCRIPT ........................................ 222 VB / COM EXAMPLE ............................ 223 MANAGING DATABASES.................. 226 DATABASE CHECKING ROUTINES .............. 226 LOG FILES (UDUMP, BDUMP) ............... 227 LINKED SERVERS (DATABASE LINKS) ........ 229 Example - Databases in the same Instance ...................................... 231 Example - Connecting to an SQL Server Instance ...................................... 231 Example - Connecting to an Oracle Instance ...................................... 232 OPENQUERY, OPENROWSET, OPENDATASOURCE ....................... 235 Remote Servers ............................ 236 DTS PACKAGES .................................. 237 DIAGRAMS (DATA MODEL DIAGRAMS) ....... 243 Transferring Diagrams ................... 244 ROW CHAINING & MIGRATION ................. 245 TABLESPACE (FILE-GROUP) MANAGEMENT ... 245 PINNING TABLES ................................. 247 TRIGGERS ......................................... 247 CONSTRAINTS .................................... 248 LONG RUNNING TRANSACTIONS ............... 248 SQL TIMEOUTS .................................. 249 RENAME DATABASE .............................. 249 USER MANAGEMENT ............................. 250 Login vs User................................ 250 Orphaned Logins ........................... 251 Change DB Owner ......................... 252 User Quotas & Account Expiry ........ 252 OPS$ Logins (OS Authentication) .... 252 Change Object Owner .................... 253

Check object ownership ................. 253 User Lock Timeouts ....................... 253 Transfer Logins between Servers .... 253 Change a Users Password .............. 254 Rename a Login ............................ 254 KILLING SESSIONS .............................. 254 Orphaned Sessions ..................... 256 TEMPDB.......................................... 256 INSTANCE UPTIME.............................. 257 RE-CREATING & AUTO-STARTING SERVICES 257 RAW PARTITIONS ................................ 258 SQL*MAIL ........................................ 258 AUTO-GROWTH AND SPACE MANAGEMENT TRACKING ......................................... 258 Automatic data file growth ............. 258 Space Tracking ............................. 258 STATISTICS AND HISTOGRAMS ................ 259 Building and Maintaining Histograms 260 Import and Export Statistics ........... 261 Re-collecting Statistics & Monitoring Auto-Statistics Collection ............... 261 Updating Statistics ..................... 261 Monitoring Statistics ................... 261 Controlling Stats Collection at the Object Level ................................. 262 Analysing Indexes & Histograms (collections) ................................. 262 SYNONYMS ........................................ 264 SHRINKING DATABASE FILES .................. 264 SQL DMO ........................................ 267 VB Example.................................. 267 VB Script Example (.vbs) ............... 267 SQL & T-SQL ................................... 268 PL/SQL (T-SQL) ............................... 268 Comments and Statement End ....... 268 Understanding GO......................... 269 Block Structure ............................. 270 Declarations / Local Variables / Constants .................................... 270 Assigning values to variables & Create Table as....................................... 271 Exception Handling (error handling) 272 RAISEERROR and Error Messages ... 273 Non-SQL Output ........................... 274 Cursors........................................ 275 STATIC Cursors.......................... 277 UTL_FILE (File IO)......................... 277 Procedures/Functions/Packages ...... 278 User Defined Functions (UDF) ...... 278 User defined Stored Procedures ... 279 Versioning Stored Procedures ...... 280 Security & Ownership Issues........ 280 Calling External Applications (command shell)........................................... 281 DBMS_SQL................................... 282 DBMS_RANDOM............................ 282 DBMS_OUTPUT ............................. 283 DBMS_MAIL, UTL_SMTP................. 283 DBMS_AQ .................................... 283 PSP, PL/SQL Web Toolkit................ 283 DBMS_METADATA ......................... 283

DBMS_JOB ................................... 284 DBMS_LOB................................... 287 BFILE .......................................... 287 Tables ......................................... 287 User Defined (Inline) Functions....... 288 Indexes ....................................... 289 SEQUENCES ....................................... 289 What about UNIQUE IDENTIFIER? ... 292 BUILT IN FUNCTIONS ............................ 293 Numeric Functions ........................ 293 String Functions............................ 294 Date Functions.............................. 295 SQL EXAMPLES WITH ORACLE COMPARISONS ..................................................... 297 PARTITIONED TABLES ........................... 301 PARALLEL QUERY ................................. 301 PARALLEL IO ..................................... 302 SQL HINTS ....................................... 302 Example Hint ............................. 305 DATA TYPES COMPARED ........................ 306 DYNAMIC SQL.................................... 309 T-SQL (PL/SQL) OVERVIEW ................... 311 NETWORKING SQL SERVER............. 313 LISTENER AND ODS ............................. 313 Multi-Protocol ............................... 315 Registry Entries ............................ 315 Client Network Utility..................... 316 ODS ............................................ 317 Tracing TCP/IP port issues.............. 317 List of Running Instances ............... 317 SSL OF SQL SERVER CONNECTIONS ......... 317 SQLOLEDB ...................................... 319 NETWORK MONITORING ........................ 320 SQLDiag ...................................... 322 MTS / ORACLE SHARED SERVER .............. 322 SECURITY ....................................... 323 SECURED INSTALLATION ........................ 323 LOGINS AND USERS ............................. 325 Authentication Modes .................... 325 BUILTIN/Administrator Login .......... 326 Using the SA login......................... 327 Duplexing a Login to Multiple Users . 327 Finding NULL Passwords................. 328 DBO Access .................................. 328 User Quota and Resource Restrictions .................................................. 328 Password File................................ 328 Development Server Privileges ....... 328 Application Security Connection Strings & Authentication ................ 329 Authentication Example............... 330 Obtaining the DB Connection string and Connecting to the Database... 331 Views and Stored Procedures.......... 333 Virtual Private Databases ............... 335 Label Security............................... 335 LOCKING DOWN EXTENDED AND SYSTEM PROCEDURES ..................................... 335 Securing xp_cmdshell (OS Shell) .... 335 Securing xp_cmdshell (OS Shell) .... 336

OLE Automation ......................... 336 Registry Access .......................... 336 Other routines ........................... 336 DATA TRANSFORMATION SERVICES ........... 337 SYSTEM, OBJECT PRIVILEGES AND ROLES ... 337 Fixed Server Roles (Instance) ...... 339 Fixed Server Roles (Database) ..... 339 User Defined Roles ..................... 340 Object Privileges ........................ 340 AUDITING ......................................... 341 Application Auditing....................... 341 Profiler (trace) .............................. 344 Instance level auditing................... 344 C2 Level Auditing .......................... 345 ENCRYPTION / OBFUSCATION OPTIONS ...... 345 Networking Options ....................... 345 Encrypted File System ................... 345 WITH ENCRYPTION option.............. 346 T-SQL .......................................... 347 INDEX............................................. 348 REFERENCES ................................... 352

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

SQL Server Architecture 101

hroughout this chapter we will introduce you to the SQL Server Instance, the database and core memory structures. The overview is reasonably thorough and where possible we will drill through to the DBMS architecture, with some early examples of SQL and database checking and consistency commands (DBCC).

In this and subsequent chapters, we will explore SQL Server 2000 and revisit some of the core Oracle concepts to re-enforce the SQL Server equivalent technology (or lack of in some cases). Please remember that we assume good working knowledge of the Oracle DBMS and the daily tasks and challenges of the DBA; therefore, I dont beat around the bush when explaining core concepts or discuss at length syntax and semantics. Please note that this ebook is not designed to be read from front to back but rather as a reference guide to getting quick answers and where to look next.

OVERVIEW OF DATABASE INSTANCES

Before we drill into SQL Server, it is important that we step back for one moment and take a highlevel look at the Oracle Instance vs. the SQL Server Instance. First of all, the word instance and database tend to be somewhat blurred in the SQL Server realm. When describing Oracle, database is used to describe the physical files and instance as the memory structures and processes to interact with the database (3). In SQL Server 2k an instance describes a complete, near independent installation of SQL Server that includes: exit. 1. Binaries and associated options (mandatory and optionally installed) 2. Windows Services and associated registry entries 3. System databases, their physical database files, log files and other example databases associated with the instance. In general, think of a SQL Server instance as a new installation of SQL Server. To support multiple installations of SQL Server on the same server SQL Server 2k introduced named instances. This is a great feature that effectively allows you to have, say, a SQL Server 2k instance running Service

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Pack 2 installed and other running Service Pack 1 on the same server. In context to Oracle, its like having v8.1.5 and v8.1.6 running on the same server. When we run multiple named instances, we call them near independent installations because there are common files used by all. Some of which (at a high level) include: COM (DLLs) Replication Full text search Directory service integration Import OLEDB provider Proxy message data source provider Virtual device interface for backups Distribution agent for replication (exe) Log reader agent for replication (exe) Tools Enterprise Manager Command line and GUI query analyser tools BCP, Bulk Insert etc DTS Run command line Rebuild Master DB utility Wizards Books online (BOL) SQL Templates Each instance maintains what is left over, namely their data files and complete set of executable files. Installation of named instances is covered in more depth later in the book. It is important to remember that the MDAC (Microsoft Data Access Components, see http://www.microsoft.com/data/whatcom.htm) version is not instance dependent, it is a server wide API for all data access (ADO, OLE-DB, ODBC). Therefore, care must be taken between service packs and even other SQL Server installations as they may inadvertently break existing products.

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Here is a summary of what we discussed.

Instance Binaries

Common files for all SQL Server 2k instances (default and other named instances).

Default instance binaries. Named instances will be something like MSSQL$MyInstance

Instance Service

Instance Databases (views within Enterprise Manager)

Default databases for all instances. Master, Model, MSDB, Tempdb are system databases, the Northwind and Pubs databases are sample databases.

The above can also be shown as:

Instance & Address Space (oracle.exe) Instance & Address Space (sqlservr.exe)

Oracle DB

SQL Server Instance

SYSTEM Tablespace

MASTER Database

Undo Tablespace

MSDB Database

Redo Logs

Model Database

TEMP Seg tablespace

TEMPDB Database

User tablespaces..

User Database and trans logs

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

NOTE When a new user database is created within an instance, its structure is modelled from the MODEL system database. The DBA may customise selected properties of MODEL to reflect these within any new database created. Some named instance issues to be aware of: 1. 16 per server 2. One instantiation of an instance may only run in a multi-server failover cluster. 3. Dont necessarily need to install the default (non-named) instance, can start with a named instance right from the start. 4. Instance can have different service pack levels and languages and sort orders 5. One Enterprise Manager (EM) manages all SS2k instances, with also includes the server, client network utilities and network data providers. 6. Each instance has its own sqlserver NT service and associated binaries and registry entries 7. Named instance services are titled MSSQL$InstanceName If you were wondering, I use the product Process Explorer available at sysinternals.com to drill down into NT processes running on the server and associated file IO activity. This and other internals tools are excellent when exploring the inner workings of the NT/Windows operating system and associated application processes. NOTE When a SQL Server DBA talks about SQL, some are actually referring to the product SQL Server rather than the SQL language.

Control Files and Data Dictionary

The control file is critical to an Oracle instance, it stores a variety of information such as: Database name Timestamp of DB creation Names and locations of database files and redo logs Tablespace information Archive Log history Checkpoint information and more..

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Within a SQL Server instance this information is a divided over a range of databases. The control file is basically the master database. Without it the instance will not be able to start as it consists of all the information about other system and user databases. The master database can also be viewed the system tablespace to some degree, but it distributes its data dictionary objects with other system and user databases (such as MSDB). The data dictionary within SQL Server is a series of system tables with the sys prefix owned by the internal user dbo (which, like the sys account, can never be removed). These reside in all user and system databases within the primary file-group of the database. Because the dictionary is divided over the databases, it is important that backups include all system databases (master, model, msdb) at a minimum, just as you would backup the system tablespace, control files, rollback segments within an Oracle instance. In terms of users databases and dictionary objects, each user database includes a standard set of sys tables using to track space allocation, objects created (views, tables, constraints etc) and basic SCN (called LSN) and database file properties. To work with the data dictionary, SQL Server provides the DBA with: stored procedure calls (prefixed with sp_ in the master and msdb databases) information schema views, special pre-summarised views to simplify direct querying of the data dictionary sys tables. DBCC routines (database consistency checking), special database checking, validation repair routines called from query analyser or other command line utility Extended stored procedure calls for externally developed routines for DB management SQL-DMO, SQL programming interface and associated API Search for master database, information stored in system tables, for an excellent overview of all sys dictionary tables within the system and user databases within the BOL (SQL Server help books online). As there is only one master database, there is no concept of multiplexed control files.

Datafiles
An Oracle tablespace can have one of more data files. Within SQL Server, a file-group can have one or more datafiles. Objects can span multiple data files in its file-group as they can do within Oracle tablespace data files. If there is more than one data file per file-group, then SQL Server will stripe (store) data evenly across each of the datafiles (based on the % free space within each data file). A database file (data file) also consists of a logical-filename, this includes all database files (data and log files). The logical filename is referred to during alter database commands and recover to make life easier when referring to database files rather than their physical name or when referring

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

to the files file-group is not appropriate. The logical name of a database file must be unique for the database but other databases can use the same name if need be. As with Oracle tablespace datafiles, SQL Server supports dynamic datafile growth by percentage or by megabytes to a maximum or unlimited size. An example is shown below: ALTER DATABASE DATAFILE myfile.ora AUTOEXTEND ON NEXT 20M MAXSIZE 2000M; use [mydb] ALTER DATABASE [mydb] MODIFY FILE (NAME = N'mylogicalfilename, FILENAME = myfile.mdf', MAXSIZE = 2000, FILEGROWTH = 20) use [mydb] ALTER DATABASE [mydb] MODIFY FILE (NAME = N'mylogicalfilename', SIZE = 200)

ALTER DATABASE DATAFILE myfile.ora RESIZE 200M;

Tablespaces
Are known in SQL Server as File-Groups, these are logically named structures that hold zero or more physical database files (zero because you can actually create a group then add files later). All databases have a default PRIMARY group that cannot be removed or renamed although the files within it can be manipulated. The primary group will consist of one data file containing the databases data dictionary objects (sys tables). Adding more file-groups is synonymous to adding more tablespaces to an existing database in which tables and indexes can be stored within. At the database level you can specify the default file group that will be used when a users object is created without specifying the destination group. A database user cannot have a default file-group as a property of being a database user. It is important to remember that a databases transaction log data file for a database cannot be allocated to file-group, only non-log files can. The SYSTEM tablespace is the same as the MASTER database with a SQL Server instance. As with Oracle, it must be online for the instance (and other user and system databases within the instance) to successfully start. This database contains a variety of critical information about other databases, logins, extended stored procedures etc. An UNDO tablespace in 9i has replaced rollback segments in previous versions of Oracle. The content or meaning of has not changed within Oracle, but architecturally the undo tablespace is very different to rollback segment management when using automatic undo management. Rollback and Redo is discussed in detail in the Transactions and Locking Chapter. The equivalent SQL Server structure closest to the undo tablespace is the databases transaction log; one transaction log database file exists at a minimum for all system and user databases.

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The TEMPORARY tablespace is equivalent to the tempdb system database that is automatically recreated on instance start-up. The DBA cannot alter the user/login properties to specify the temporary tablespace as with Oracle, it is automatically the tempdb and cannot be removed or altered. The read-only tablespaces are supported in SQL Server as read only file-groups. All user-defined file-groups can be made read-only; the primary file group cannot be made read only. To make the primary file-group read-only you, must make the entire database read only. The DBA cannot Offline/Online a file-group like you can an Oracle tablespace. There is no concept similar to dictionary or locally managed tablespaces within SQL Server. Each database maintains its own storage properties via the sys tables in the primary filegroup of the database, apart from that, all extent and block (pages) are uniform in size and can not be altered. There is no equivalent to the transportable tablespace in SQL Server.

Redo and Archived Logs

As an Oracle database has two or more redo logs, so each database within a SQL Server instance has a transaction log file that is further divided into smaller logical log files. The transaction log file, depending on the databases recovery model, will continue to grow in size until the database issues a backup transaction log statement for the database, or truncates the log (point in time recovery will not be possible though). The backing up of the log is the same as an archived redo log file. When we backup the log we can send it direct to tape, a file share on another server (UNC path), to disk or via a named pipe; even so, SQL Server does not provide any magic duplexing commands with the backup statement, so the DBA needs to copy the backed up file to cover all bases. A transaction log exists for each database within the instance and will hold: modifications associated with incomplete transactions (pending a commit or rollback, they are rolled back on database recovery) modifications not yet flushed to disk (to be rolled forward on database recovery) NOTE Where possible, do NOT create more than one transaction log data file per database. The log is sequential and there is little benefit from duplexing files over multiple spindles. Multiple files can be a pain to administer and deal with in terms of re-attaching databases (see Backup and Recovery) and managing log files sizes/shrinking.

Rollback Segments
There are no rollback segments in a SQL Server database. This concept is described in detail later in this book. Because of this, writers can block readers and SQL Servers locking mechanism is much more complex that Oracles with a higher probability of locking problems with long running,

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

non-committed transactions. These issues can be resolved to some degree with using different isolation levels on connection to the instance or per transactions, or using locking hints to enhance database concurrency. The transaction log acts as a data store for incomplete database transactions pending a commit or rollback. Because of this it can be a type of rollback segment but from an Oracle definition, it is very much limited in function.

Init.ora
There is no equivalent file in SQL Server, the closest we have is the internal settings within the master system database via the sp_configure stored procedure command or set via Enterprise Manager, the registry settings or the service startup parameters (sqlservr.exe).

Environment Variables
There is none, the registry controls all aspects of the SQL Server instance at this level.

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

MEMORY STRUCTURES
It is difficult to compare the two databases at a memory level. In some respects, both use the same, proven, fundamental data structures and techniques for managing their memory structures. The key difference here is that Oracle provides fine grain control over virtually all aspects of the memory (and storage) engine, whereas SQL Server provides very little control. This is not necessarily a bad thing in a majority of cases. NOTE Oracle has many complex memory structures that are configurable by the DBA, some of which are not shown in this diagram on purpose. Read the summaries later for each memory area for more information in context to Oracle.
System Global Area (SGA) Buffer Pool (Cache) Block Buffers Thread Stack Space Instance Address Space

Dirty Buffer Write Queue (LRU)

System Data Structures

Redo Log Buffer Buffer Cache Shared Pool Library Cache Procedure Cache

Dictionary Cache Log Caches

Large Shared Pool

Connection Context

JAVA Shared Pool SQL Server Net-LIB DLLs ODS Services Code SQL Server Code
Distributed Query OLE-DB providers

Server Processes S000..S999 (PGA)

Memory Pool Reserved Pool

OLE Automation Objects Extended Stored Procs

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The following provides a further drill down into the SQL Server architecture in terms of its threads of execution and associated subsidiary engines. The key point here is that SQL Server runs with a similar architecture to that of its underlying operating system, via a pool of threads that asynchronously perform a variety of tasks on behalf of the DBMS and its related engine components (storage, relational etc). Remembering this, we can see where DBWR, LGWR and other Oracle processes are represented in the SQL Server framework. Each arrow/line typically represents one or more of these worker threads (or fibres when DBA selects the option). NOTE The thread architecture for Win95 and 98 clients is not represented below.
SQL Server Instance {thread pool priority 7 or 13 threads} sqlservr.exe Procedure Cache Plan caching and reuse Buffer Cache Divided amongst the worker threads to async. scan, read and write, a reading thread wont affect a subsequent scheduled write Log Caches Parse SQL Relational Optimise Plans Engine Exec Logical ops Process DDL / other Result formatting Manage files Build and read physical pages Data buffer and IO Logging and recovery Utility functions

asynchronous worker threads

asynchronous logwriter threads

Free buffer list

checkpoint process (buffers not added to free list)

Storage Engine

scan & add to free list schedule writers check free

lazywriter thread

{flush}

{flush}

1 or more threads per net-lib & separate login thread allocated via UMS

Data Files

T.Log Files

Net-Lib Providers

DBCC SQLPERF (UMSSTATS) Gets statistics of SQL Servers thread scheduler (UMS) running in native thread or fibre mode, one scheduler per CPU. The num runable is like the Processor Queue Length performance counter. Num workers equates to the number of worker threads in the pool.

10

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DBCC SQLPERF (THREADS) Maps a system thread to a SQL Server spid.

There is one UMS (user mode schedular) for each CPU. On a user connecting, a worker thread and UMS schedular is allocated for the lifetime of the connection. Idle threads are allocated from the pool of waiting threads, if all are busy the work requests are queued and allocated in a round-robin scheme to threads. Use the command DBCC PSS to get UMS and thread details for a SPID:
DBCC PSS (0, 1) -- Where 1 is the spid

ExecutionContext Summary @0x1938A3A0 -----------------------------------ec_pss->pspid = 1 ecid = 0 ec_stat = 0x0 ec_stat2 = 0x0 ec_atomic = 0x0 pcurdb = 0 ec_lasterror = 0 ec_preverror = 0 ec_cpucur = 0 ec_cmderrs = 0 ec_timeslice = 100 ec_dbtable = 0x00000000 ec_reswait = 0xa0df60 ec_dbindex = -1 ec_umsContext->m_pSched->m_id (SchedulerId) = 0x1 ec_umsContext->m_workercntxt->m_id (threadId) = 0x47c

SGA (SQL Server Address Space)

The maximum address space for SQL Server is 2Gb, this can be extended via the /3GB switch in the operating systems boot.ini file. Each instance of course has its own address space and associated memory pool. Under Windows, you will only see the single, sqlservr.exe process running under which a range of worker threads and other memory structures run. It is difficult to get a good view of the underlying threads, but consider: DBCC SQLPERF(UMSSTATS) DBCC SQLPERF(THREADS) Process Monitor and File Monitor found at www.sysinternals.com The virtual address space that is made up of a global memory pool and a reserved working space for the loading of extended stored procedures (in master database and start with xp_, they are external DLLs), OLE-DB data providers and automation objects instantiated via sp_OACreate. This memory allocation is controlled via the g parameter on instance startup and defaults to 128Mb. The only real control the DBA has over memory is the g parameter on instance start-up, the min and max server memory option and the set working size option (both set via sp_configure when the instance has started). These options are best managed via EM as shown in the dialog box below:

11

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Dynamic memory control mode for SQL Server, min and max memory. Working set parameter, only set if the use a fixed memory size parameter is being used. This locks in memory as a working set for the win32 process. Min memory per query. Minimum memory allocated per SQL statement by the query processor to assist with joins, sorts etc. Additional space can be acquired and possibly allocated via the tempdb. Range of 512Kb to 2Gb.

Consider setting the maximum RAM by 10 to 20% less that the total server memory for dedicated database servers. NOTE It is best practice to always control memory, setting a maximum that is appropriate to the server configuration and the software running against it. See system table master..syscurconfigs for current configuration options set.

Buffer Cache
SUMMARY

The buffer cache is a dynamic pool of buffers (8k pages) read from the physical data files and is equivalent to the Oracle buffer cache. All subsequent reads/writes to physical data files occur via the buffer cache. The cache itself consists of: Linked list - to all free buffer pages Each buffer page has a header that, among other things, contains a reference pointer that is incremented when SQL statements reference the page. The increment may differ internally. A value of zero on the reference counter marks the page as dirty. A write is then scheduled. A global pool of worker threads via the user connection will manage writes and reads from the cache. The buffer will include data and index pages to complete all SQL from all databases hosted by the instance. The Oracle buffer cache goes many steps further than SQL Server in terms of configureability and control, for example: Default, Keep or Recycle pools (multiple buffer pools)

12

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DBA controlled sizing Multiple different block sized buffers The SQL Server DBA has no control over the sizing or the reservation or division of the cache. To locate pages not in the buffer cache hash bucket data structures are used. To view them, run the following:
DBCC TRACEON(3604) GO DBCC BUFCOUNT GO

SQL Server can also employ a read-ahead (RA, or parallel data scan) operation concurrently via multiple threads for a single request. The optimiser will determine the requirement for the RA operation and schedule the RA. This may be based on a count of pages missing in the database buffer cache at the time and the sorted nature of the data itself in terms on proximity (especially clustered indexes). The types of operations include table scans, index leaf-level scans, DBCC statements and the updating of statistics. The RA options have been removed in SQL Server 2k and the DBA can no longer control the RA operation.
BUFFER CACHE MANAGEMENT

To drop all clean buffers from the pool, run the following database consistency checking command via query analyser: DBCC DROPCLEANBUFFERS this requires the sysadmin instance permission. The DBA can PIN objects into the cache as in Oracle via the DBCC PINTABLE command.
MONITORING PERFORMANCE

To monitor the hit ratio of the buffer cache, the best tool is Performance Monitor (perfmon.exe). The counter is SQL Server : Buffer Manager, when there are multiple named instances the counter will be something like MSSQL$<instance-name>: Buffer Manager. Other counters related to the buffer manager include Free pages, Free List requests / sec and Free list empty/sec.

13

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Redo Log Buffer (Log Cache)

The redo log buffer in Oracle is a circular buffer of redo entries (which are any changes made by any insert, delete, update, create, alter, drop statement) stored sequentially in the SGA and controlled via the log buffer parameter. The background process lgwr writes the redo log buffer entries to the current active online redo log file. It is important to understand that any block change requires the creation of a redo log buffer record, lgwr will free space as need be in the cache. A series of latches are used to manage this process: a) redo copy b) redo allocation c) redo writing The Oracle DBA subsequently needs to monitor the log buffer size and associated space contention; then to a lesser degree latch contention. In a SQL Server instance we have a single log cache memory structure for all databases within the instance; there is a series of cache buffers for each database within the instance. For every change made to a buffer cache page a log cache entry is made recording the change, this must be written to the databases transaction log before the dirty buffer cache record is written (flushed) to disk. This process is essential to ensure effective rollback during database or user transaction recovery. Due to this write-ahead strategy of log-cache transaction log buffer cache flush the transaction log is at times called the write-ahead log. The SQL Server managed log-writer threads manage the flushing of data from the cache to disk. NOTE - Non-logged operations are primarily controlled via the recovery model used for each database within the instance.
MANAGEMENT OF

To view the contents of the transaction log (similar to that of log miner tools), drill into the command: DBCC LOG(3, TYPE=-1) where 3 is the database ID The key record operations include: LOP_MODIFY_ROWS LOP_INSERT_ROWS LOP_DELETE_ROWS

14

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Library Cache (Procedure Cache)

The procedure cache under SQL Server v7 was part of the global pool of memory that was known as the buffer cache. For the Oracle DBA this is confusing and somewhat strange, but with SQL Server 2k the memory structure has been split away from the buffer cache and dynamically grows and shrinks, as the DBMS deems necessary. The procedure cache has many similar elements to that of the library and dictionary cache combined within Oracle. The query processor (part of the relational engine that parses, compiles, optimises and manages the submitted statement) will develop the necessary plan and return it for execution; the plan and associated data structures are submitted to the procedure cache to complete the call. The different types of cache object include: Compiled plan Executable plan Parse Tree Cursor Parse Tree Extended Stored Procedure This can be further broken down to: Adhoc SQL Plans Prepared SQL Plans Stored procedure Replication procedure plans Trigger Views Default User table System table Check constraint Rules

15

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

A worker thread is allocated to manage concurrent procedure cache operations. Where possible, only one copy of a plan is stored at any one time. This enforces serialisation (not at all times) of the compilation process and is managed via compilation locks. The Microsoft support document ID Q263889 provides a good explanation of such locks and compile blocking, primarily dealing with stored procedures. NOTE See the section on Dynamic SQL for more information about caching plans.
MONITORING

To get some general buffer cache statistics, use the following command. DBCC PROCCACHE

The DBA may also consider the undocumented commands: and DBCC MEMORYSTATUS DBCC MEMUSAGE

MANAGEMENT OF

To flush the procedure cache globally for all databases in the instance, run the DBCC command: DBCC FREEPROCCACHE To flush the procedure cache for a specific database in the instance (where 3 is the database ID from sysdatabases in the master database): DBCC FLUSHPROCINDB(3) Do a before and after check in the master database dictionary table syscacheobjects, to verify the affect of the routine. Be careful with these commands especially on your long running production server. You will effectively force the once warm SQL Server procedure cache to be cleared and all this will undoubtedly result in a degradation of performance for some time whilst plans are recached. You can force the recompilation of stored procedures the next time its run via the sp_recompile system stored procedure. The create procedure command also includes the option RECOMPILE to prevent the caching of the execution plan.
MONITORING PERFORMANCE

Utilise the NT performance monitor and the counter SQL Server : Cache Manager Object and the items Procedure Cache [Hit Ratio (cache hits vs. lookups), Pages (#8k pages in use), Object Counts (#cache objects) . The counter data is also available in master..sysperfinfo.

16

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The 20 largest procedures in the cache can be monitored via the command below and can be used to predict future space requirements: DBCC MEMUSAGE To view the contents of the procedure cache, query the table master..syscacheobjects table in the master database. Here we find a breakdown of cache objects by type (eg. Executable Plan, Extended Proc, Parse Tree, Compiled Plan for example). Some of the core columns are: pagesused, usecounts, refcounts and of course sql (being cached). Third party tools, like Diagnostic Manager from NetIQ, provide cache hit statistics with a hit ratio breakdown per cache object type over time.

SQL Server Connections

Each database connection is managed via a worker thread no matter the underlying DB-Library provider. In SQL Server, each connection is through a database instance login, which defines instance wide privileges, language and other basic properties for the associated process (SPID) managed via a worker thread. The connection itself is controlled via the UMS (user mode schedular) as described in the Memory Structures section. The properties of each SPID can be monitored via the sp_who and sp_who2 stored procedures or by querying the master..sysprocesses table. The DBA determines which of the databases within the instance the login can access, these are called database users; as the login to user mapping exists separately for each instance database with their own database level privileges (i.e. 1 login to many database users within the instance). The sysprocesses table has a variety of information that the DBA may be interested in, namely the ECID (execution context ID in which threads are running under for the SPID, KPID (current NT thread managing the SPID), CMD (command being executed) and much more. Such information may be present in the server error logs on occasions. In terms of memory usage, SQL Server documentation is reasonable scarce in detailing the sequence of events (although you could probably guesstimate it) and associated memory structures. Generally, each connection will take a minimum of 12Kb + (3 x TDS Network Packet Size), the TDS packet being 4Kb by default (see master..sysconfigures where config = 505 for runtime value). The packet size can be set via the network packet size database option or via EM via Tools Options Advanced. The number simultaneous connections accepted can be controlled via the maximum concurrent user connections property. Select properties for the instance and select the connections tab. The DBA can check the value via select @@MAX_CONNECTIONS. With SQL Server 2k, we can programmatically add up to 128 bytes of binary data to the current session (connection). The context data remains accessible to all SQL for the session/connection. The context_info column can be queried in the master..sysprocesses table to retrieve the data for the SPID. To set the binary data in T-SQL code, here is an example:

17

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DECLARE @contextdata varbinary(128) SET @contextdata = CAST( 0x10 AS varbinary(128) ) SET CONTEXT_INFO @contextdata select context_info from master..sysprocesses where spid = @@SPID
-- Value set is.. 0x10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000

Extended Stored Procedures

Extended stored procedures are DLLs written via the ODS (open data services) API (typically via C++). The associated stored procedure in the master database is a wrapper to the underlying DLL that is loaded and run in the same address space of SQL Server (the calling process). The wrapper T-SQL stored procedure is like any other stored procedure in terms of calling it. The object is referred to in the master..sysobjects table and they can be managed via: sp_helpextendedproc sp_dropextendedproc sp_addextendedproc sp_addextendedproperty select * from ::fn_listextendedproperty See Microsoft Support Doc# Q190987 for a great summary of extended procedures albeit with a lack of example code.

Checkpoint
Within the Oracle DBMS, a checkpoint is a database event that synchronises altered data blocks in the buffer cache with the physical data files. The DBWR process on certain conditions, such as every 3 seconds, on a redo log switch or how full the buffer cache is with committed dirty buffers, manages the write event. The CKPT process will update the control file and data files to indicate the last completed checkpoint with the SCN (system change number). In older versions of Oracle the LWGR would perform job of the CKPT process unless the DBA overrides the option with the checkpoint_process start-up parameter. The Oracle DBA has a lot of control of the checkpoint process; its timing and monitoring performance. Even so, many of these parameters are rarely understood and at times can severely impact performance.

18

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

It is not too surprising that fundamentally, a SQL Server checkpoint (in theory) is the same as Oracle in that both are events that synchronise altered database blocks and log files in the buffer and log caches with the physical data files. In saying that, remember the key differences in redo logs and server processes between the architectures. The checkpoint operation itself results in a flush of dirty data and log pages, the databases within each instance checkpoint separately, maintaining their own last SCN (LSN in SQL Server) for recovery. The checkpoint will: write to the transaction log file (redo and rollback segment in Oracle) marking the start of the checkpoint Write LSN to the database boot page, begin recording information about the checkpoint in a chain of log entries. Records the minimum LSN (MinLSN), being the LSN of the first log record to complete a database recovery Record all outstanding and active transactions Remove all virtual log records in the transaction log before the MinLSN (simply recovery model, backup log command will have the same affect, also called archived redo logs) Writes dirty data buffers to disk Writes log entries marking end of checkpoint Automatic checkpoints occur: based on the recovery interval parameter and the number of associated log entries to fulfil the parameters setting. Database transaction log becomes >=70% full Result of a manual CHECKPOINT Selected DBCC commands are run Backup command issued Instance shutdown, offline database, read-only the database. NT service stopped The DBA can control the recovery interval (in seconds) parameter for each database within the instance. The DBA can also force a checkpoint for a specific database via: use mydb
CHECKPOINT

19

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

This command is available to the syadmin, db_owner and db_backupoperator privileged server and database roles.

Background Processes
An Oracle instance is implemented as a single process, namely oracle.exe, under which the background processes, namely pmon, dbwN, lgwr, ckpt, smon, reco run as threads. The oracle.exe process corresponds to an NT service that is typically named OracleServiceSID. NOTE Get the Oracle thread details by querying v$process, v$session, v$bgprocess over the addr and paddr columns. The threads themselves can vary depending on the installation, i.e. if clustering or replication services are used. Some of the common ones include:
Thread (background process) Pmon Description Process monitor SQL Server Equivalent individual worker threads (UMS managed) individual worker threads Lazywriter thread and individual worker threads individual worker threads. individual worker threads. individual worker threads.

DbwN Lgwr Ckpt Smon Reco

Database Writer Log Writer Checkpoint System Monitor Recovery

In the SQL Server world, a SQL Server instance also runs under a single process sqlservr.exe with a number of worker threads. These threads manage (not a definitive list): each database connection pooled if need be under heavy load scanning the buffer cache, reading pages into cache, writing dirty buffers, populating free buffer list scheduled asynchronously flushing log cache entries to disk parsing, compiling and executing plans Selecting rows from master..sysprocesses, you will see a variety of system processes running against the instance, some of which are managers of the individual worker threads. Some examples are:

20

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

LAZY WRITER LOG WRITER1 LOCK MONITOR thread level lock monitoring exists in SQL Server 2k SIGNAL HANDLER TASK MANAGER CHECKPOINT SLEEP These threads utilise a common memory area (stack) within the SQL Server instance address space. The worker thread is controlled by the configuration parameter max worker threads. The default is 255. Each thread is assigned a schedular (see DBCC SQLPERF (UMSSTATS) ). The wait state of worker threads assigned to a UMS schedular is logged in the SQL Server error log and you get a thread starvation error message if all UMS threads are in a wait state. Remember that there is one UMS schedular for each CPU, the value of max worker threads parameter is evenly divided among the UMS schedular(s). Each connection (SPID) is allocated to a thread linked to a UMS that never changes throughout the life of the connection. The idle worker threads are allocated to incoming tasks as need be, queuing requests appropriately whilst worker threads are made available. See Q262973 for a good overview. REMEMBER Each individual database within the instance does not have a separate global pool of worker threads.

21

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Storage Engine
This section covers a variety of areas related to the Storage Engine, namely: tablespaces (file-groups) Redo logs, archived logs and rollback segments Blocks, extents, pctused and pctfree Segments Space management

Tablespaces
Each database within SQL Server has a minimum requirement of: one primary filegroup and its associate data file one transaction log file (logs are not associated with file groups) A filegroup is the logical name for a tablespace. One file-group can consist of zero or more physical database files. The transaction log file (redo log) is dedicated to the database and does not have the concept of filegroups and therefore logical names, it is simply a redo/rollback log with a logical and physical file name that is used for transaction management. The general extension naming conventions for database files are: .mdf .ndf .ldf master data file next data file (for any other files created for the database) transaction log file

The logical name is a unique name that will identify the file within the file-group. I like to use it to identify the general contents of the file within the file-group to simplify recovery, for example MYDB_DATA or MYDB_AUDIT. It is important to remember that a file-group (tablespace) with two or more database files causes SQL Server to stripe writes over the files based on the percentage of free space available within each. Depending on your application this may provide a performance enhancement as SQL Server creates a new IO thread for each file. This is not the case though with transaction log files which are sequential. Best practice states that where possible have one and only one database file per file-group.

22

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Here is an example create database statement: CREATE DATABASE [MyDB] ON PRIMARY ( NAME = N'MyDB_System', FILENAME = N'e:\dbdata\MyDB\MyDB_System_Data.MDF', SIZE = 1, FILEGROWTH = 10%), FILEGROUP DATA01 ( NAME = N'MyDB_Data01', FILENAME = N' e:\dbdata\MyDB\MyDB_Data01.NDF' , SIZE = 1, FILEGROWTH = 10%), FILEGROUP INDEX01 ( NAME = N'MyDB_Index01', FILENAME = N' e:\dbdata\MyDB\MyDB_Index01.NDF' , SIZE = 1, FILEGROWTH = 10%), LOG ON ( NAME = N'MyDB_TransLog01', FILENAME = N' f:\dblog\MyDB\MyDB_TransLog.LDF', SIZE = 1, FILEGROWTH = 10%)

Part of the SYSTEM tablespace for the user database.

User defined tablespace for our table objects

User defined tablespace for our index objects

Redo / Rollback Segment tablespace

NOTE our temporary tablespace is the tempdb database and is shared by all databases. Database creation is discussed in detail later in this book. Take careful note of the syntax, if you use Enterprise Manager it will use the alter database statement to add any files other than the primary file-group and the transaction log, rather than doing it all in one statement. The DBA can specify the file-group in which database objects are created. By default they will be created in the primary filegroup. Only tables, materialised views (indexed views) and indexes can be stored to a specific file-group via the ON clause, eg:
CREATE TABLE mytable (mycol1 integer) ON DATA01

It is important to understand that file groups are logical structures, they can be created independently with no datafiles in them (see alter database command) and physical files can be added or removed later. A file-group cannot be removed until all of its data files are removed. Also, removing a single file, even if its the only file in the filegroup, does not mean the file-group is dropped (this is not the case via EM). Use the alter database <mydb> remove filegroup <logicalname> command. The primary file-group (default) cannot be removed. SQL Server will automatically remove the physical data files when a database file is dropped. So long as there is no user objects in the data file or in the case where the file is a transaction log file, there are no transactions, the DBA can remove the file. See the stored procedure commands sp_dbremove (deletes a database), sp_detach_db (removes database from instance but retains files, preferred method), alter database <mydb> remove file <logical-name>.

23

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

A file-group can be made the default group for user objects. Best practice typically states that the DBA creates additional file-groups (data, index) and does not use the default PRIMARY filegroup that holds all the sys tables (mini data dictionary) for the database. Use the command alter database <mydb> modify filegroup <logicalname> default. We can specify the default operating system location of filegroup files as we can do with Oracle and the db_create_file_dest parameter in 9i. The parameters in SQL Server are very basic and covers data and log files for new user databases at an instance level. These entries are actually stored in the registry for the instance and can be altered via Enterprise Manager (right click properties for the instance and choose the Database Settings tab) or via the extended stored procedure (xp_routine reside in the master database only) command:
exec xp_instance_regwrite 'HKEY_LOCAL_MACHINE',N'Software\Microsoft\MSSQLServer\MSSQLServer',N'DefaultData',REG_SZ,N' E:\SQLServerData\'

SQL Server will not name the files for you or let you overwrite existing data files on a create database statement. When restoring though, you can force an overwrite of existing files if need be. For a new database (includes 1 data file for the primary file group and 1 transaction log), SQL Server will call them:
<db-name>_Data <db-name>_Log eg. mydb_Data, file extension is .MDF to default data dir eg. mydb_Log, file extension is .LDF to default log dir

I tend to call these:

<instance-name>_<db-name>_SYSTEM <instance-name>_<db-name>_LOG01

If its the default instance then leave <instance-name> blank. I use 01 to 99 to ID the file-number in the group. Good naming standards can assist the DBA in emergency recovery scenarios. There is no reason why you cannot use the OFM (Oracle File Management) and associated OFA (Oracle Flexible Architecture) naming conventions for files with SQL Server (it wont pre-empt you anyhow so its good to stick with the conventions). NOTE SQL Server has no concept of Partitioning as in Oracle or transportable tablespaces. There is not such thing as dictionary managed or locally managed tablespaces (file-groups) as such, the space allocation pages are stored in the headers and throughout the physical database files themselves.

Blocks / PctUsed & PctFree / Extents / Segments

In Oracle, we have the logical allocation of space based on blocks extents segments, with blocks being the finest level of granularity for storage. Oracle allows the DBA to specify different

24

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

block sizes for each tablespace (up to 5 standard sizes in 9i) to cater for different storage characteristics of objects. Earlier versions of Oracle allow the block size to be set once on the databases creation, even so, it is completely user defined. The set block (page) size for SQL Server is 8k, you cannot alter this, so we have no DB_BLOCK_SIZE parameters of at the database or file-group level. FACT SQL Server does not support multiple block sizes or variable extents.

Oracle 9i - Block
Header 84 to 107k Table Directory Row Directory

SQL Server 2k - Page

Header (96k) Pointer to text data where appropriate

Row data

Row data

fillfactor (pctused)

There are eight different page types in SQL Server : data index text/image

Row offset pointers to physical data rows

global and secondary allocation map what extents (64k contiguous) are allocation page free space information about page free space index allocation map extent information about index allocation bulk change map lists extents modified from bulk operations since last log backup differential change map - extents modified since last backup database command NOTE - Row offset table entries start at end of page, 1 entry per row, in table entries are in reverse sequence of rows in the page.

25

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In order to view page structural information:

dbcc traceon( 3604) dbcc page( <dbid>, <page#>, [print opt], [cache], [logical]) dbcc traceoff( 3604)

NOTE - DBCC Page is an undocumented command. The fillfactor option is used in place of pctused in Oracle. You cannot use this option in create table command though, it is only used when indexes (clustered or non-clustered) are created. When DBAs of SQLServer talk about defragmentation, it is usually in context with indexes and rebuilding them to resolve storage issue and page splitting. The default fill factor is zero, this is similar to 100% but with space for one record left at the higher level of the index nodes. As a clustered index defines the storage of the table, the option fillfactor has a key influence over the leaf (actual data) pages and the most affect with subsequent inserts or updates against the segment. On top of fillfactor, the DBA can also specify pad_index. This option can only be used in conjunction with fillfactor and specifies the space to leave available at all non-leaf nodes (called intermediate pages) of the index. The pad_index value is internally set by SQL Server and is based on the value specified by fillfactor. Since SQL Server has fixed page sizes the extent size is also fixed at 64k (8x8k pages) as is contiguous in its storage. This is the smallest IO call SQL Server will make. The idea of the DBA pre-allocating extents to objects or change extent size is non-existent in SQL Server. The DBMS has the concept of uniform and mixed extents, this affects the allocation of space to segments. A uniform extent is one whose pages are allocated to a single segment. A mixed extent means its 8 pages may be shared by a variety of segment pages. SQL Server will allocate space to segments (tables/index etc) at a page level, if, during space allocation, the DBMS determines that an entire segments mixed page allocation can fill an extent, then SQL Server will allocate (reserve) uniform extents for the object rather than individual pages. See free space management next for more information.

Free Space Management

Each Oracle segment includes a free list that is implicitly allocated at the segments creation time. This is made up of a common pool (master) free list of blocks containing free space for the extents allocated to the segment and each subsequent extent allocated to the segment as the segments size increases. On top of this and to reduce master free list contention, a process free list is explicitly created when the DBA specifies the FREELISTS parameter on a create/alter table/index statement. This parameter effectively allows the DBA to partition the free list to improve applications with a large number of concurrent transactions. It is worth mentioning that in Oracles locally managed tablespace, the tablespace manages its own extents via a bitmap structure for each datafile to track free or used blocks. As the uniform extents

26

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

are allocated and deallocated the bitmap entries for the associated blocks are also altered. On top of this, there is no requirement for coalescing free extents to consolidate fragmented free space as all extents are uniform in size (unlike dictionary managed tablespaces).In SQL Server, free space management in terms of a free-list entries consists of two page types: GAM global allocation map page Tracks extent allocation 1 x GAM = 64k extents (4Gb data) One bit for each extent 1 = free, 0 = allocated SGAM shared global allocation map page Tracks those extents that have mixed pages and have at least one unused page 1 x SGAM = 64k extents (4Gb data) 1 = extent is mixed and has free pages 0 = not used as mixed, or, is a mixed extent whose pages are all in use. These pages, typically found after the file-header pages are used by SQL Server for extent management for each data file (not including transaction log files). To complete free space management we also have PFS pages. A PFS (page free space) page records individual page allocations within the data file. The page itself is a bitmap structure similar to the GAM and SGAM but only covers 8000 pages at a time. The value of the bit denotes the percentage full within the pages for an extent, for example: Empty 1 - 50% 51 - 80% 81 - 95% 96 - 100% The PFS bitmap is constructed as extents are allocated. NOTE read IO occurs in 64Kb chunks (extent size). Finally, we need to discuss IAMs (Index Allocation Maps). These are critical as they map the extents in a database file used by the heaps (non-indexes tables), indexes (including clustered indexed tables) and blob objects (ntext, image, text pages). Each object has at least one IAM so long as: the object has data allocated

27

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

the object did have data, but it was deleted rather than truncated. The IAMs are linear linked lists and are scattered throughout the data file as the objects within it acquire space. If the file-group has more than one data file then remember that storage will be allocated evenly over the data files based on the total % free space in the data files, therefore IAM linked lists will span the physical data files within the logical file-group. NOTE IAM, PFS and other pages will be bought into the buffer cache along with other standard page types holding row and index data. The IAM and PFS pages are used to locate pages with enough free space to hold incoming rows. The IAMs are used to locate the extents for the object, whilst the PFS is used to locate page free space. Necessary reallocation of extents (mixed to uniform) and subsequent acquisition of new extents is via the GAM and SGAM pages.
file-group data-file (1 or more files per file-group) File Hdr PFS GAM SGAM BCM DCM IAM IAM

All 8k pages, 64k extents (not shown in diagram)

sysindexes entry for all objects to first IAM BCM (bulk changed map) DCM (differential changed map)

We can view the allocation of space via sp_spaceused or dbcc showcontig. With this the DBA can see how a tables space allocation moves from mixed extents to the reservation of uniform extents to the object. The following example was adapted from the work of Kalen Delaney and her classic articles from SQLMag.com: create table largerow (a int identity,b char(8000) ) As we insert rows (row#), and execute the sp_spaceused procedure we get the following results: exec sp_spaceused largerow, @updateusage = true

28

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Row count 0 1 2 3 4 5 6 7 8 9 10 17

Reversed (Kb) 0 16 24 32 40 48 56 64 72 136 136 200

Data (Kb) 0 8 16 24 32 40 48 56 64 72 80 136

Index (Kb) 0 8 8 8 8 8 8 8 8 8 8 8

Unused (Kb) 0 0 0 0 0 0 0 0 0 56 48 56

Notes

Plus one page for IAM

Uniform extent allocated All are uniform extents

NOTE when you create a table with no indexes, an entry is placed in sysobjects and then sysindexes for the table, the sysindexes entry record consists of an IAM binary entry pointing to the first IAM for the object. With no data the entry is 0x000000000, as data is entered the first IAM is allocated. If you delete from the table, the IAM entry remains and sp_spaceused reports the same space utilisation!, but a truncate will clear the IAM entry altogether and associated other GSAM, GAM pages therefore returning those pages/extents quickly back to the storage engine.

29

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Fragmentation
Fragmentation occurs at a variety of levels with DBMSs and seems to be one of those ongoing struggles for the DBA to resolve. In Oracle, fragmentation of segments is a classic problem, occurring primarily where many segments with different growth characteristics share a tablespace. This as been addressed with locally managed tablespaces and fixed extent sizes for segments. In SQL Server as we have seen in previous examples, the allocation of space occurs in the form of a mixed extent then reserving uniform extents (64k - 8 contiguous pages). All this said, we still have the following issues in both databases: physical file fragmentation (OS fragmentation) shutdown instance and defragment at the OS level. distributed extent fragmentation (honeycombing) page (block) fragmentation page row fragmentation (a non issue) In SQL Server, the only way to resolve honeycombing object extents is via: BCP out data (or copy to another database), script foreign keys, drop foreign keys, truncate tables, reimport data, enable keys Backup and Restore For clustered indexed tables only DBCC REINDEX or DBCC INDEXDEFRAG or drop and recreate the clustered index (can be problematic if a primary key constraint). The DBA can view the fragmentation of its indexes via DBCC SHOWCONTIG, which divides the fragmentation into: logical scan fragmentation extent scan fragmentation Be warned that for SHOWCONTIG to display accurate figures you may need to re-run DBCC UPDATEUSAGE and then possibly try shrinking the database. For some strange reason statistics used by SHOWCONTIG can become out of date, causing no end of confusion for the DBA. For heaps (non-clustered indexes), space is not automatically added at the end of the structure, the PFS will be searched for free space for the record and inserted appropriately. Therefore, a table with a sequenced primary key that is also experiencing heavy deletions can have is rows dispersed throughout its pre-allocated extents.
Reporting on Fragmentation

Use the command DBCC SHOWCONTIG to report on a table or indexes fragmentation. When writing scripts to track fragmentation over time, consider this command:

30

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

INSERT INTO msdb..dba_fraghistory (<column list here>) DBCC SHOWCONTIG (21575115, 2) WITH TABLERESULTS, NO_INFOMSGS

Loop for all user objects (sysobjects) in the current database and run the above into our custom created fragmentation history table msdb..dba_fraghistory. An example of an indexes fragmentation summary is shown below: dbcc showcontig ('enrolment')
DBCC SHOWCONTIG scanning 'enrolment' table... Table: 'enrolment' (1903397900); index ID: 0, database ID: 5 TABLE level scan performed. - Pages Scanned................................: 909 - Extents Scanned..............................: 166 - Extent Switches..............................: 165 - Avg. Pages per Extent........................: 5.5 - Scan Density [Best Count:Actual Count].......: 68.67% [114:166] - Extent Scan Fragmentation ...................: 39.16% - Avg. Bytes Free per Page.....................: 441.3 - Avg. Page Density (full).....................: 94.55% DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Clustered indexes represented by index ID: 0 or 1. The values above mean:

Pages scanned = total pages used (in clustered index, this of course represents the actual data pages as well as the index intermediary nodes) Extents scanned = total number of 64Kb extents, 8 pages per extent Extent switches = movements between 1 extent to another over its extent chain. (pages scanned / 8 should-be less than extent-switches). Avg pages / extent = technically, represents unused pages per extent, 8 is the optimal value (no unused) Scan density = fragmentation % (best count is the ideal value vs the actual extent switch count) Extent scan fragmentation = % of out of order extents in the leaf nodes avg bytes free per page = can be skewed with unused space in extents, is the avg free space left over per page, lower value the better. Dependent on row size. avg page density = how full the average page is, again, is dependent on row size, fill-factor, and inserts/deletes/updates.

Be warned with the values presented above. A small table, for example products in the Northwind database, reported 77 rows, two extents (1 switch), and a scan density (fragmentation) of 50%, with a terrible avg pages / extent of 1. The mixed extent issue is the reason and should not overly worry you.
Resolving Fragmentation

Clustered index tables will benefit the most from SQL Servers defragmentation routines as they are all based around indexes (a defrag on a clustered index will of course compact data and index pages), unlike heaps (tables without clustered indexes, compacts indexes only). The following is recommended and can be further explored in the books online. DBCC DBREINDEX

31

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DBCC INDEXDEFRAG DBCC SHRINKDATABASE Consider dropping and re-creating your clustered indexes if the above seems to be skewed (after re-checking the fragmentation report). Always concentrate on the largest and busiest tables first.

32

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

High Water Mark (HWM)

When space is allocated to a table or index, and you remove it all via a delete from statement (unlike a truncate), SQL Server will retain a set amount of space. The actual amount should be re-confirmed with your particular SQL Server installation in case there is variance with DBMS versions, OS and/or IO sub-system configuration. The example below shows the reservation of space.
Row count 79858 0 Reversed (Kb) 638984 38920 Data Index Unused Notes (Kb) (Kb) (Kb) 638984 8 112 <<all rows removed from the table via delete>> 38880 8 32 Approx 38Mb of space left allocated to table after delete DML 1277728 8 160 <<all rows removed from the table via delete>> 38880 8 32 Re-check space left over after delete with double the number of rows. In this case no matter the number of rows the same space is reserved.(this can vary and the DBA should verify themselves on their particular instance).

159716 0

1277896 38920

IMPORTANT This example was with a heap (non-clustered index) table. See the examples below for space utilisation after a delete with a clustered indexed table. In terms of a HWM and the optimiser full-table scanning to the end of the tables memory allocation, we see the following occur for empty tables: Example 1 Heap with no indexing
-- table with 1 row per page create table largerow (a int identity,b char(8000)) -- populate with 1000 rows declare @aa int set @aa = 1 while @aa <= 1000 begin insert into largerow values('a') set @aa = @aa + 1 end

Row Reversed Data Index Unused Notes count (Kb) (Kb) (Kb) (Kb) 1000 8008 8000 8 0 Inserted 1000 rows. -- select * Table 'largerow3'. Scan count 1, logical reads 1000, physical reads 7, read-ahead reads 264. 0 6152 6112 8 32 Deleted all rows -- select * Table 'largerow3'. Scan count 1, logical reads 764, physical reads 28, read-ahead reads 700.

33

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

0 0 0 0 0 Truncate the table -- truncate Table 'largerow3'. Scan count 1, logical reads 0, physical reads 0, read-ahead reads 0.

Example 2 Clustered Indexed Table

-- table with 1 row per page create table largerow (a int identity,b char(8000)) create clustered index largerow_ix on largerow3 (a) -- populate with 1000 rows declare @aa int set @aa = 1 while @aa <= 1000 begin insert into largerow values('a') set @aa = @aa + 1 end

Row Reversed Data Index Unused Notes count (Kb) (Kb) (Kb) (Kb) 1000 8072 8000 40 32 Inserted 1000 rows. -- select * Table 'largerow3'. Scan count 1, logical reads 1002, physical reads 0, read-ahead reads 592. 0 88 8 24 56 Deleted all rows -- select * Table 'largerow3'. Scan count 1, logical reads 3, physical reads 0, read-ahead reads 0. 0 0 0 0 0 Truncate the table -- truncate Table 'largerow3'. Scan count 1, logical reads 0, physical reads 0, read-ahead reads 0.

Example 3 Non Clustered B*Tree Indexed Table

-- table with 1 row per page create table largerow (a int identity,b char(8000)) create index largerow_ix on largerow3 (a) -- populate with 1000 rows declare @aa int set @aa = 1 while @aa <= 1000 begin insert into largerow values('a') set @aa = @aa + 1 end

Row Reversed Data Index Unused Notes count (Kb) (Kb) (Kb) (Kb) 1000 8048 8000 48 0 Inserted 1000 rows. -- select * (index not used by optimizer) Table 'largerow3'. Scan count 1, logical reads 1000, physical reads 0, read-ahead reads 648. -- forced index lookup via hint, select.. with (index (largerow_ix)) Table 'largerow3'. Scan count 1, logical reads 1004, physical reads 0, read-ahead reads 627. 0 88 8 24 56 Deleted all rows -- select * Table 'largerow3'. Scan count 1, logical reads 3, physical reads 0, read-ahead reads 0. 0 0 0 0 0 Truncate the table

34

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- truncate Table 'largerow3'. Scan count 1, logical reads 0, physical reads 0, read-ahead reads 0.

Segments
In early versions of SQL Server (namely 6.x), we had the concept of segments and their preallocation to house set objects (index, tables) typically to separate disks to increase IO speed. This concept is no longer valid in SQL Server 7 and 2000. A SQL Server page can be of segment types: data all data type entries, text/image/ntext data can be stored inline with the data row via the text in row option is used on table creation. index index entries text/image storage of data type text, image, other blob data out of line with the rest of the data row. The data row page has a 16-byte text pointer to this entry. GAM / SGAM Free space BCM DCM Most of these segments have already been discussed. The key fact to remember here is that segment space is not pre-allocated by the DBA, SQL Server manages the space allocation in all cases. The DBA can only control fillfactor (default zero or 100%) and padindex (default 1 space for a single row) options for indexes.

Relational Engine
The relational and storage engines are completely separate within SQL Server and communicate via the OLE-DB API. Its not too difficult to work out what the relational engine does, but I must say that compared to Oracle, the documentation is very scarce and overly broad: parse locate existing cached plans build query tree (depending on b)) optimise (cost based optimisation)

35

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

execute communicate with storage engine to retrieve data process DDL and other statements return result to client The SQL (T-SQL is also regarded as SQL) statements cover all forms and instantiations submitted to the engine, namely: single (adhoc) statements, including views batch statements stored procedures and user defined functions The optimiser relies on histograms (statistics) collected by the instance automatically (auto create statistics setting for each database in the instance), indexes and manually created statistic collections. These are used to establish a variety of figures related to the cost of the plan and choosing the one with the lowest estimated cost. To speed plan estimation a variety of complex algorithms are used, which, in some cases will result in only partial optimisation. The source data structure for the optimiser is the successfully parsed sequence (query) tree. On execution of the optimised query plan, a series of requests are sent to the storage engine in which rowsets are returned. The data is processed by the relational engine that formats and returns the results to the client via a UMS managed worker thread. The relational engine may build temporary data structures (work-tables) in the tempdb system database to complete the SQL statement. All execution plans are stored in the procedure cache. Each plan has two parts: query plan, read-only cached plan, up to two can exist for a batch, one for serial and one for parallel operations execution context, bind variables and other parameter values for the SQL Like Oracle, the use of parameters (bind variables) increases the possibility of plan reuse. Unless altered by the DBA, the parallelisation of the statement will be determined during the optimisation phase.

36

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

Installation
hroughout this chapter will we discuss the installation of a SQL Server instance. Like most DBMSs, the DBA can do a quick and dirty install, or be thorough in terms of security and really lock down the instance and access to it. We will discuss some of the issues to consider in both methods and where appropriate, drill into the finer points of the installation process.

Editions and Versions

Before you install (or buy) SQL Server 2k, here is a short overview of the editions of SQL Server, this is very important as the costs and feature differences may bite you later on. The editions consist of: Standard Edition Enterprise Edition Developer Edition (CD two of Standard and Enterprise) Personal Edition SQL Server CE (small footprint for Windows CE OS) SQL Server MSDE (Microsoft Data Engine, no EM GUI with max 5 concurrent users) Evaluation Ed (typically 128 day evaluation) Some of the key items Standard Ed does not include over Enterprise Ed: Distributed partitioned views Log shipping (can be manually scripted though) Parallel index creation, Parallel scan, Parallel DBCC Fail over Clustering

37

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Indexed views Multi-language capabilities OLAP linked cubes OLAP cube access over HTTP OLAP ROLAP storage Direct SAN support 32 CPUs and 64Gb RAM

Licensing Schemes
When installing Standard or Enterprise Editions, the DBA is prompted for the SQL Server licensing scheme to be used for the instance. The schemes are: Per Processor (typically for web-based applications) for each CPU, unlimited users. Per Seat (known as a CAL or Client Access Licence) this is a per device connection licence, and not per user, if I as a user connect 10 times with mobiles/PDA/laptop etc all simultaneously then all require a CAL. NOTE - SQL Server does not use the pricing model of per power-unit as Oracle does (or used did, this seems to change on a regular basis as RDBMS competition heats up). IMPORTANT No licence for your operating system? then all SQL Server licences are automatically treated as invalid for the server. IMPORTANT - Connection pooling is not an excuse for buying less CALs if using this licensing scheme. If 20 user connections are pooled to 5 SQL Server sessions, you still need 20 CALs. There are many important issues to be considered when dealing with licensing, some of these are: Standard Edition Must have a separate licence for each new instance Enterprise Edition Can create as many instances of the same server as you like under a single license. Active / Active Cluster

38

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

All Active machines required separate licences CAL or Per Processor schemes are fine. Active / Passive Cluster A single licence scheme can be used for both servers. Attempts to use the passive SQL Server for connections will result in further licensing issues. Log Shipping See Active / Passive Cluster Developer Edition Licensed per developer Personal Edition So long as the OS is licensed, Personal Ed is free of further licensing issues. SQL Server CE Edition Requires CAL as part of the Developer Edition If the CE SQL Server connects to another SQL Server (i.e. replication, linked server etc), then that remote SQL Server requires a per-processor licence. It is worthwhile spending the time to read these: http://www.microsoft.com/sql/howtobuy/sqlserverlicensing.asp http://www.microsoft.com/sql/howtobuy/sqlserverce.asp http://www.microsoft.com/sql/howtobuy/msde.asp http://www.microsoft.com/sql/howtobuy/personal.asp Some approximated pricing is shown below (believe me, with the market as it is, dont trust me on this one): Type Of Per Processor Per Processor Per Seat CAL Per Seat CAL Per Seat CAL Per Seat CAL Per Seat CAL Per Seat CAL Edition Enterprise Ed Standard Ed Enterprise Ed Enterprise Ed Enterprise Ed Standard Ed Standard Ed Standard Ed Quantity N/A N/A 5 10 25 5 10 25 Est.Price$ (US) $20,000 $5,000 N/A N/A $11,000 $1,500 $2,200 N/A

39

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To locate information in regards to the existing licensing scheme, run the following within Query Analyser: select SERVERPROPERTY('LicenseType') select SERVERPROPERTY('NumLicenses') where 'NumLicenses' is not applicable for the per CPU scheme. If you are running the Developer Edition, 'LicenseType' returns disabled and 'NumLicenses' is NULL. The License Manager program (control-panel -> administrative tools) cannot display information related to per-processor licensing for SQL Server (Q306247). To get around this issue the installation will create a new administrative tool program to track such information as shown in the screen shots below.

SQL Server 2k licence applet, this is very different under v7.

Version 2000

Version 7

The Settings Control panel Licensing shows only server licensing for SL Server 2000 installations (unlike v7), the applet above is used instead. Notice that you cannot switch to a perprocessor licence; you must re-install the instance to do this. NOTE Unless specified in your licence contract or SQL Server version on installation, the installed instance will not automatically expire or end.

40

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Alter licensing mode after install?

Once the licensing mode is set on instance installation, thats it, you cannot change it back. In control panel we have the icon:

This will allow you administer your licences, but you will see one of the options is greyed out, such as:

To get around this go to the following registry key entry:

Change the mode to 1 (default is zero) and you get both options:

Microsoft may not support you if this method is used.

41

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Binaries & Instance Installation

Upon the installation of the Oracle binaries, the DBA would either create the databases they require via command line script svrmgr or sql*plus or use the Oracle GUI tools. Under NT, the Oracle DBA would perhaps utilise: oradim.exe svrmgrl / SQL*Plus Create instance of the NT service To run our hand written DB script and run cat%.sql files to build dictionary. Configure network connectivity and listener

SQLNet Config or Database Configuration Assistant (GUI)

Throughout the different versions of Oracle we have seen a shift to more GUI tools (driven by Java) that cover most basics and allow the DBA to quickly get databases up and running and add to the database any other tablespaces as need be. In SQL Server, the setup disk not only covers the binary installation, but instance creation, maintenance tasks (registry rebuild, system database fix etc) as well. The SQL Server Setup CD will (not necessarily in order): a) Run through installation options for default or named instance and install the instance with instance and non-instance dependent binaries a. Named Instance? b. Location of binaries and the instances system database files c. Setup the account in which the services will run i. Use the same account to run the instance service and the instances SQL Agent account? ii. Run under the Local system account? iii. Run under another domain user account? iv. Auto-start service? d. Instance Collation (language) and sort order e. Install binaries based on selected options f. Authentication mode for the instance i. Mixed mode (windows and SQL Server authentication) ii. Windows authentication g. Setup NT services h. Setup default networking properties You may need to install the prerequisite options first, SQL Server setup will soon tell you about this and also notify you when trying to install over an OS that doesnt support the edition. You have the flexibility to install options only (like, only the books online which is a very good idea) and not to install or create a new SQL Server instance. These are all self explanatory with the set-up. Additional options include: a) rebuilding the registry entries (will also reinstall MDAC from the CD)

42

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

b) installation of missed or skipped components c) can fix broken installations where server name has changed d) can restore default system database data and log files in case of an emergency recovery e) can assist with installing instances over a configured MSCS (MS Cluster) for failover clustering. f) Install analysis services (Microsoft OLAP and Data Mining)

Service Account
As with Oracle installation under NT, the NT account you are using on installation is called the service account in which associated DBMS services and further installation and administrative tasks will occur under. In many cases (the quick and dirty method), the DBA will use the administrator account as it provides complete access to the server and its resources, reducing the possibility of any flow on issues related to the system privileges. If you need to install using a separate NT account that will not be a member of the administrators group, the consider the following: Create a new NT user Grant login as service rights Grant administrator group privileges to this user Create DB data and log file directories for database files to be created in Install SQL Server as this user Once installed, shutdown the MSSQLServer service Login as Administrator of the server and alter the users rights as follows Full control of SQL Server binaries (install directory) Full control of all directories to store database files (mdf,ndf,ldf) For the above directories, remove the "everyone" group privilege Full control of the registry keys. HKEY_LOCAL_MACHINE\Software\Microsoft\MSSQLServer HKEY_LOCAL_MACHINE \System\CurrentControlset\Services\MSSQLServer or MSSQL$<INSTANCE> for named instances HKEY_LOCAL_MACHINE\Software\Microsoft\WindowsNT\CurrentVersion\Perflib and associated service registry keys for SQLAgent$InstanceName, MSSearch, and MSDTC Revoke administrative rights to the SQL Server user account

43

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Login as the SQL Server user and attempt to re-start the database, debug as required. Access to xp_cmdshell for non-sysadmin users must be via a proxy account (select properties of SQL Agent). NOTE When using replication, create a DOMAIN user account and not a local server account. This may not be definitive and will require some testing and checking of the SQL Server error logs and the NT event logs to locate the reasons why specific operations are not happening with the account being used. A classic problem is the SQL Agent service account, which does require administrative privilege to run successfully.

Encrypted File System (EFS)

Those running Windows 2000 can take advantage of the EFS (encrypted file system) for database and/or backup files. To use this option: shutdown your database instance login with the account the SQL Server instance is using select properties of the folder(s) in which the database files reside select advanced option and follow prompts to encrypt files/folders re-start the instance service verify successful start-up of instance and databases affected via the encryption. Attempting to start the service as any user other than the user that encrypted the database data files, results in the instance not starting, or a suspect database. Example error log entries:
udopen: Operating system error 5(error not found) during the creation/opening of physical device E:\cktemp\efs\efs_Data.MDF.

FCB::Open failed: Could not open device E:\cktemp\efs\efs_Data.MDF for virtual device number (VDN) 1. udopen: Operating system error 5(error not found) during the creation/opening of physical device E:\cktemp\efs\efs_Log.LDF. FCB::Open failed: Could not open device E:\cktemp\efs\efs_Log.LDF for virtual device number (VDN) 2.

In terms of speed, this is a tough one to measure, there are so many factors to consider its not funny, for example: hardware vendor IO drives and OS version and patch level Disk vendor, speed, cache RAID levels, channels, disk interconnect

44

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

RAID controller type, cache, vendor After some testing of the same 5000 row table on an EFS database verses a non-EFS database, we find some interesting results. The following table was created and queried:
drop table efs create table efs (col1 int, col2 varchar(200)) declare @aa int set @aa = 1 while @aa <= 5000 begin insert into efs (col1, col2) values (@aa, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa') set @aa = @aa + 1 end

NOTE - the following statistics are averaged figures of five different attempts at the same SQL. Personally, I do not like to write up such figures, and I pre-warn that careful testing must be done at your specific site with more complex scenarios.
Encrypted Database File IO (avg - 1min 10sec to create and populate)
update efs set col2 = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' where col1 = 3345 checkpoint

-- Averaged 10% slower Table 'efs'. Scan count 1, logical reads 1720, physical reads 0, read-ahead reads 0. SQL Server Execution Times:
CPU time = 40 ms, elapsed time = 40 ms. -- Averaged 6% slower update efs

set col2 = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' Table 'efs'. Scan count 1, logical reads 5875, physical reads 0, read-ahead reads 0. SQL Server Execution Times: CPU time = 1472 ms, elapsed time = 11761 ms. -- Averaged 42% slower (varied substantially based on query complexity) dbcc dropcleanbuffers select count(*) from efs Table 'efs'. Scan count 1, logical reads 1720, physical reads 1, read-ahead reads 48. SQL Server Execution Times: CPU time = 50 ms, elapsed time = 89 ms.

45

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Non-Encrypted Database File IO (avg - 42sec to create and populate)

update efs set col2 = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' where col1 = 3345 checkpoint

Table 'efs'. Scan count 1, logical reads 1721, physical reads 0, read-ahead reads 0. SQL Server Execution Times: CPU time = 30 ms, elapsed time = 36 ms. update efs set col2 = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' Table 'efs'. Scan count 1, logical reads 5875, physical reads 0, read-ahead reads 0. SQL Server Execution Times: CPU time = 1482 ms, elapsed time = 11109 ms. dbcc dropcleanbuffers select count(*) from efs Table 'efs'. Scan count 1, logical reads 1720, physical reads 1, read-ahead reads 48. SQL Server Execution Times: CPU time = 40 ms, elapsed time = 205 ms.

Be very careful, the EFS is an advanced option and I having read a variety of news-group posts regarding recovery of servers and associated tape backups from EFS files, the process is far from simple. Be aware of the dependency on the OS installation, server and user that will all influence the encryption.

RAID Set-up with SQL Server, Install Issues, OFA

The same principals apply between Oracle and SQL Server is terms on RAID, including the hardware selection and channel configuration, on-board RAM and compliance with the Microsoft Windows hardware compatibility list. For the installation directory and its binaries, system database data files and users database data files, we need to consider: reated an installation map? Some of the issues include OS versions and service packs, service accounts to be used, placement of files, disk sub-system configuration, how many databases will be hosted and how will their file place affect other databases?, number of users and connectivity properties, instance names, requirement to install previous SQL Server versions, backup locations, default passwords, who will access to the DBMS, remote administration issues, file shares and their security, virus checking software (and is it auto-updating).

46

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

OLTP or DSS nature of the system, affect on network bandwidth, buffer cache and procedure cache hit ratios Will multiple instances will be sharing the same disk sub-system? Will other IO intensive software will be installed? Will other file-system like activity through the application such as batch document creation, auditing log files, bulk extracts for reporting, OLAP cube builds, affect performance significantly or pose issues with the disk subsystem? Possibility of moving to a clustered environment? Capacity of disk array in terms of free drive slots? Backup sub-system and associated disk space (will disk backups share the drive with other database files?) Do we really understand the expected database growth patterns? and how do you plan to track it? Positioning of file-groups and have we thought of using multiple file-groups over disk RAID arrays to increase IO performance and ease administration? Adhoc queries or reporting databases (IO distribution or bias). These issues are particularly important in large OLTP or DSS based applications sharing the same disk sub-system with other applications or user databases. The question of the RAID level is a tricky one as RAID-1, RAID-5 and RAID-10 arrays are typically determined by the money available and the total databases to be supported. I have yet to come across any mid-sized organisation where money wasnt as issue and dont be fooled, high-end server hardware is expensive (we are not talking desktop PCs here). In terms of performance, RAID-5 should not be overlooked. Be aware that each write results in two IOs plus a single read IO. Even so, I have successfully run a variety of applications over such an array (with hardware caching enabled though at a small risk of database corruption) with very low read/write queue lengths. Like all things, the DBA should be aware of the issue and pre-test where possible. As a guide, we tend to see very little IO activity around the master, msdb and model system databases. As such they and the binaries tend to be placed in the same location, typically a RAID-1 or RAID-10 array. The tempdb database, used for temporary sort, group-by and other operations like the temp tablespace in Oracle, may need to be placed on its own array. The issue with the tempdb is its size and trying to justify its own disks (the minimum is typically 18Gb or 36Gb). A server with large amounts of RAM and very short transactions manipulating small amounts of data will rarely use the tempdb. As an example, my 2Gb production database with over 900 online users, 4Gb of RAM, sees the maximum tempdb of 800Mb.

47

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The DBA needs to consider the total server RAM, index rebuilds, DBCC executions and their timings, the need to large data set processing, and simply monitor tempdb usage, weighing up IO performance via performance monitor. User database files are broken down (at their basic configuration) into two data files: Data File (primary file-group) Transaction Log File (file-groups not applicable) Ideally, the log files should be own their own array due to their serial writes. In summary, as an instance includes numerous databases all with a minimum of two data files, a so called perfect disk placement for all files is a dream for 99% of companies. From the points discussed, I would recommend something like:
RAID Contents Type RAID-1 Windows OS and installed apps CD-ROM RAID-1 or All system database files RAID-10 F: RAID-1 Filegroup DATA DB files G: RAID-1 Filegroup INDEX DB files H: RAID-1 Transaction log files * opt for the RAID-10 for the optimal read/write performance. Drive Letter C: D: E:

As an example, my 800 user database server with heavy OLTP activity has the following drive configuration with close to zero problems in terms of IO contention. The disks are SCSI160 15k 18Gb disks hooked to a HP 4M raid controller with 40Mb cache, read/write ahead caching is on for the RAID-5 array.
Drive Letter C: D: E: F: G: RAID Type RAID-1 CD-ROM RAID-1 RAID-1 RAID-5 (4 disks) Contents Windows OS and installed apps All system database files Transaction log files Filegroup AUDIT DB files Backup file dumps Filegroup DATA DB files Filegroup INDEX DB files

In Oracle, many DBAs prescribe to some form of the OFA (Optimal Flexible Architecture) for file placement over the array. This basically describes: Establish a documented and orderly directory structure for installation binaries and database files that is compatible with any disk resource i.e. c:\SQLServer2kBin\<instance-name>\{binaries} c:\SQLServer2kData\<instance-name>\<db-name>\{data files} c:\SQLServer2kLog\<instance-name>\<db-name>\{log files} c:\SQLServer2kErr\<instance-name>\{error log files}

48

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Separate segments of different behaviour into different tablespaces (file-groups) Consider creating, for users databases, a separate file-group or DATA, INDEXES, AUDITING, and not using the PRIMARY group for user objects. Consider separation based on disk resource contention Consider file-group separation based on their historical data content Separate database components across different disk resources for reliability and performance NOTE Be aware of SQL Server data striping. If you have two or more database files for a single file-group, SQL Server will automatically stipe data across these based on the percentage of free space with each data file. Be aware of this is you are also planning disk striping by hand during installation/restore.

Items to consider before installation (summary)

The following is a summary the DBA should be considering carefully before installation, some of which were covered in previously on RAID arrays: a) Clustered installation? b) Disk sub-system set-up and ready for the installation? c) Upgrading and existing version? a. v7 to 2000 b. v6.5 to 2000 c. mixed environment? (v7 and 2k or other combination) d) Check OS requirements and RAID issues e) Location of binaries and default location for installed system databases (OFA) f) Editions and Versions g) MDAC version support for other applications (will installing SQL Server be an issue, as a general rule it will not). h) Will you need to run SQL Server v7 a. Install named instances of SQL Server 2k b. v7 cannot support multiple instances, therefore will be used as the default instance. i) File and instance naming standards defined? j) Collation and Sort Order determined for instance? k) Security Installing under System User or another domain user account? l) Security Integrated or Mixed Mode? m) Do you want to move the tempdb system database? n) Log shipping support required? Does your edition support it? The SQL Agent service must run under a domain user account with rights to other domain servers. o) Full Text Search and English Query required? p) SA Password Management and the sysadmin privilege user q) Instance memory configuration (how much will be allocated?)

49

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

r) Backup/Recovery plan considered, will be duplex files to another server?, retention period and recovery models used? Have you tested them and can you meet defined SLAs? s) Server and Database ownership in terms of ongoing support and maintenance

Example Installation Standard, Developer or Enterprise Ed

Installing Oracle is a two-part process, one being the installation of the binaries in a specific ORACLE_HOME, and then creating one or more databases via the command line or GUI tools. In SQL Server these two steps are completed in a single running of the set-up. NOTE In general, the setup is self-explanatory and we will only cover some of the wizard dialogs presented to you.

The setup process as described, is multi-functional, to install a new database instance, we select the Server and Client Tools option. The client tools are generic for all instances, therefore, once installed we need to filter out these additional options. This is discussed later. The next question presented to the DBA is the instance name, the default option will create the default unnamed instance (if one doesnt already exist). Based on this early decision the following will eventually be created:
Shared files for all instances Default Instance Other Named instances

and the following example services, the first being the instance and the active directory helper service for all instances, and the second set being SQL Agent (batch job processing service) for each instance.

50

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

When prompted, always select the custom option. It is good practice to select advanced options.

Binaries (keep all instances in the same location). Sets the instances default directory for all instance databases. The directory will be : ..\<instance-name>\Data

You cannot specify the ORACLE_HOME and there is no environment variable; the DBA can specify the installation directory for the binaries for an instance the default being c:\Program Files\Microsoft SQL Server\.

51

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The options for Management Tool (Enterprise Manager and other command line and GUI tools), Client Connectivity, BOL (Books online), Development tools and Samples are shared components for all instances; when installed for one instance, dont re-select them for subsequent instances. This is especially the case when patching a system with service packs or upgrading BOL, as you dont want to revert back to the default installation settings accidentally. Full-text query allows the DBMS to utilise the Microsoft Search Engine to build catalogs on table columns that allows English like queries to be performance at high speed. The next step is selecting the NT account in which the instance services will run. It is important that you read the Security Chapter on Installation Issues regarding security during installation. Many DBAs leave this as default, running the two services for the instance under the system user account. Although fine for most installations, in a production environment this should be altered to another NT domain user account with restricted OS privileges if required. If changing the account, do not forget to check the auto-start service check box.

For the security conscious, run the instance under a new user account. How to configure it is described in the Security Chapter.

The above can be altered by editing the services within NT, and also within SQL Server Enterprise Manager by selecting properties at the top level of the instance tree and for the SQL Server agent. See below.

Select security tab when selecting properties for the instance once connected.

52

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Select properties for the SQL Server Agent option with enterprise manager.

Once the service accounts have been specified, we tell SQL Server about the authentication method it should use. See the Chapter on Security. The Windows Authentication mode is the same as OPS$ (external Oracle authentication) only, and Mixed is both Database and Windows Authentication. Mixed mode will require you to enter the sa account password to complete mixed authentication, guard this login/password religiously, it is equivalent to the sys login with Oracle. The service account in which SQL Server is installed will be the Windows Authentication user, and you will be able to connect to the instance with no password (OS managed). The BUILTIN/Administrators SQL Server login will provide this seamless authentication via the equivalent OS group that can be removed to provide better security.

I recommend Mixed Mode where possible with a complex SA password that is rarely used. This ensures that a DBA inadvertently locking themselves out when altering windows authentication privileges or server configuration is not a completely lost cause. The final BIG question asked of the DBA is the collation and sort order. This single question results in numerous postings in newsgroups and forums, and rightly so. The language and sort order is covered in the Chapter on NLS and Sort Orders. The installation screen is shown below.

53

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The default collation is Latin1_General, and the sort order is Accent sensitive. Example collations are: French Greek Romanian Polish etc

Previous version sort orders can be selected if need be for backward compatibility. To complete the installation, the setup will finally prompt you for the default server network (listener) properties and the supported network libraries that can be used. The standard libraries include named pipes and TCP/IP (a port of 0 equates to port 1433 or SQL Server will pick a random port on first ever connection and retain it). The named pipe will be \\. \pipe\MSSQL[$]<instancename>. The server will not be hidden from the network. See the Chapter on Networking SQL Server. Leave the displayed options as default in a majority of cases. Please read the Chapter on Networking regarding listener set-up and configuration. After the network screen SQL Server will: Install MDAC (check documentation on version being installed) Install binaries and registry entries Install and prepare services Register active-x components Update your system (whatever that means) And this completes the installation. You will need to apply additional services packs as need be to the installation (highly recommended). WARNING Applying SQL Server patches or installing SQL Server will alter your MDAC level. Take are in multi-server environments where this can cause differences in MDAC between servers.

54

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Moving your system databases after installation

The first thing Oracle DBAs will ask, is so how do I move my system database files after a standard installation?. The setup prompts the DBA for a default data directory, in which the setup will then dump ALL default installation data files from the CD to this directory. In most cases, this is fine, as the DBA will have set aside a RAID set for all system files, the only system file that that DBA may want to split away from the master, msdb, model database files is the tempdb database. The default database files installed from the CD-ROM are: MASTER MSDB TEMPDB MODEL master.mdf, mastlog.ldf msdbdata.mdf, msdblog.ldf tempdb.mdf, templog.ldf model.mdf, modellog.ldf

Moving MASTER and Error Logs

The instance start-up parameters in the registry can be altered, effectively allowing you to move the database files to another location.

Alter the registry entries, stop the instance service, including SQL Agent, move the master database files to the new location and re-start the instance. The DBA can alter these within Enterprise Manager by selecting properties of the instance and altering the start-up parameters. This will call an extended stored procedure to alter the registry settings above.

Moving MSDB and MODEL

The DBA can mode the MSDB and MODEL databases via: Full backup then restore to a new destination Use the sp_attach and sp_deattach commands

55

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The backup and restore option is the most simplistic and is covered in the Backup and Recovery chapter. The instance will require less downtime and is a safer option overall. To move these database via the sp_attach and sp_deattach commands, we require a little more work to be done. 1. Open the service control window for the instance, and add the start parameter -T3608

2. Start the instance

3. Run query analyser, login in with sysadmin privileges 4. Run the following command, writing down the name, fileid and filename displayed
use msdb go sp_helpfile

go

5. De-attach the database from the instance with the command

use master go sp_detach_db 'msdb'

56

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

go

6. Refresh enterprise manager, you will notice that MSDB has now disappeared. 7. Copy or Move the database files listed in 4 to their new destination 8. Run the following command to re-attach the MSDB database files
use master go sp_attach_db 'msdb','C:\msdbdata.mdf','c:\msdblog.ldf' go

9. Go back to the service control window, shutdown service, clear the trace flag and re-start the service. 10. Refresh Enterprise Manager or re-start. 11. Re-run step 4 to verify the new file location Repeat the process above for the MODEL database. WARNING - if doing both databases at the same time attach the MODEL database before MSDB.

Moving TEMPDB
The TEMPDB database is the same as the temporary tablespace option within Oracle. This database is used for sorting and temporary tables. It is common practice, where possible, to move this database to its own RAID 1 or RAID 10 set. This needs to be carefully evaluated to determine if the use of the database and the set of disks it is residing on is a bottleneck. To move the database files for this system database: 1. Run query analyser, check existing location of the tempdb database files
use tempdb go sp_helpfile go

2. Get the destination directory ready and document the results above, re-check the location.
3. Issue the alter statement to logically move the files alter database tempdb modify file (name=tempdev, filename=c:\dbdata\sys\tempdb.dbf) alter database tempdb modify file (name=templog, filename=c:\dbdata\sys\templog.ldf)

Restart the instance for the files to be re-created at the new location. Consider trace flag 3608 or 3609 (skips tempdb creation) if you have issues with the new destination or with the model database (from which its created). You can also resize the tempdb database via the SIZE option in the alter database statement.

57

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

GUI and Command Line Tools

hroughout this chapter we will focus on the GUI and command line utilities installed with SQL Server. The tools are simple to use and all they require is time trying out their numerous options. We will not explore each and every option, but provide a high level summary of the tool and its use.

What is installed?
On installation, SQL Server 2k comes with a variety of tools sets. Some of the tools are:
Tool Enterprise Manager (EM) Profiler GUI? Y Y Category Management Trace & Performance SQL Import/ Export Import DTS Summary The core GUI for all database administration of 1 or more instances on remote or local servers. SQL Tracing utility, captures numerous statistics of all DML/DDL running against the instance. Very powerful tuning and debugging tool for the DBA. Similar to SQL*Plus, includes command line isql.exe and GUI wsql.exe versions. Bulk copy process command line, includes import and export functionality. Equiv to sqlloader but is not as powerful and is reasonably limited. Command line utility that can execute a data transformation package create in a SQL Server instance. Command line tool that produces detailed diagnostic type information for a SQL Server instance. Highly recommended. perfmon.exe is a built-in with the OS, Oracle also includes perfmon counters on installation of the Oracle binaries. SQL Server adds a large range of counters for each instance installed for detailed performance monitoring of instance and database internals. Linked to the MSDB system database and exists for each database instance, is the job scheduler for the instance.

Query Analyser BCP

Y N

Textcopy DTSRUN

N N

SQLDIAG

Diagnostic

Performance Monitor (perfmon.exe counters)

Performance

SQLAGENT

N (Y via EM)

Scheduling

58

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Service Manager Server Network Utility

Y Y

Management Network

Client Network Utility

Network

Books Online Itwiz Dtswiz

Y Y Y

Help Performance DTS

Manage SQL Server NT services Similar to SQL*Net listener, acts as the listener for the database server and defines the providers that may communicate to the database instance(s). Defines client connectivity properties to database instances over 1 or more defined providers (protocols) Product Help and Documentation Index Tuning Wizard. Data transformation service import/export wizard. Does have command line options, use dtswiz /? to get a list of them.

Distrib Logread Replmerg Snapshot Regxmls Odbccmpt Sqlftwiz Sqlmaint Vswitch Sqlagent

Y N N N

Replication Replication Replication Replication XML ODBC Full Text Maintenance Switch Job Scheduling and Processing Tape Services

Full text indexing wizard. Maintenance operations on one or more databases such as updating stats, rebuilding indexes etc. Switch between v6, v6.5 and v8 (SS2k) SQL Server installations. Starts SQL Agent service. Mainly used at the command line for diagnostics.

console Scm

N Y

Displays diagnostics information on tape backup and restore operations. Service control manager

Enterprise Manager (overview)

The core DBA tool for managing SQL Server instances is Enterprise Manager. The GUI tool covers virtually all of the key functions the DBA will perform in their day-to-day tasks, even so, as your skills increase you may find yourself tracing the activity of Enterprise Manager via profiler and rewriting your own scripts/stored-procedures to meet your specific requirements in which EM falls short. The EM tool itself is very easy to use and requires very little discussion; it is a simple a matter of trying its features and learning from there. But all that said, we will highlight some issues with the interface and how EM can make life of the DBA a lot easier.

Registering Servers
Registering database instances within EM is a fundamental task for managing multiple instances from the single console. Right click properties on the Microsoft SQL Server group and select New SQL Server Group:

59

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The name must of course be unique within its hierarchy; groups are sorted by name. Select the new group, right click properties and choose new SQL Server registration:

Going through the wizard, it will show a screen of available servers. If yours is not listed ( which is likely) then simply type in: a) server IP address or hostname b) or use the alias created for the instance via the Client Network Utility. Option b) is the standard selection, especially when connecting to named instances, or instances on non-standard ports and/or protocols. The alias method also allows you to register the same instance multiple times.

60

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Interface Overview

Database folder drill through to all databases within the instance, if you cant see the system databases then check the properties of the registration as there is an option to hide them from EM. The DBA can manage all properties and settings of each database from here.

Create/edit DTS packages which are automatically versioned for you. Packages and their content can generate meta-data which the DBA can view/search over, this allows us to track transformations typically used in warehousing applications. Management folder Config and manage SQL Agent jobs and their schedules, manage backup devices, view current session activity and locking issues, create and maintain maintenance plans and view error logs. Can manage alerts, operators.

Replication folder manage publications and the subscriptions to these publications.

Security folder Manage logins, server role permissions, linked server and remote servers.

Support Services folder managing other utilities such as SQL Mail, full text search engine and the DTC.

Meta data services repository (stored in MSDB database).

61

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Meta Data Repository

The Meta Data Services repository folder in EM seems eternally blank. To get it populated with properties from an instances database and working for you in terms of meta data for your data models via extended properties, then follow these screen shots below, remembering to refresh EM after importing the meta-data.

62

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Meta Data Repository Snap-In

The meta data repository is not a small topic by any means. Its API and associated open interfaces are designed to hook into case and designer tools, and follow this onto the implementation with documenting the physical data model, tracking DTS data loads and providing some great documentation on top of the standard E-R model. Unfortunately, we are not covering this topic in detail but will look at the snap-in. The meta data services snap in is the key to administering meta data services. To use the snap in (which is not available in EM) you need to: a) Run mmc.exe b) File --> Add/Remove Snapin, dialog is shown, click on the Add button c) Select the Meta Data Services snap-in and others as need be, close when finished d) Ok e) Save the console, setting yourself up with a custom admin management console To register a instance and set administrative properties, you must have a database with the repository tables. This database is the msdb system database. a) Right click on Meta Data Services, select register database b) Filling in the standard registration database property using the SQL Server OLE-DB provider (which you cannot change). c) The 3 user modes are: a. End User read only access to contents folder (as in EM) b. Power User read/write but cannot edit object properties c. Administrator read only to information models, read/write all other properties d) Based on the user access, the DBA can remove, edit and refresh the repository and properties, this is not possible in EM.

63

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

e) The DBA can also view the information models available for tracking and managing within the repository.

Task Pad View

Some DBAs talk about task pad in SQL Server, this is simply a type of view you can have within EM when looking at an instance or individual database. An example is shown below.

64

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Inserting NULLs into Table Columns

To insert a NULL into a database table column within EM, use CTRL-0 (zero) and tab to the next column. Also take care having the table view open, EM is a classic for taking out escalated table locks. A CTRL-N will close the EM window and you will have to re-start EM.

DBVERIFY
There is no database checking utility that operates whilst the SQL Server database instance is down. Once up, the SQL Server DBA can use the DBCC (database consistency checking) routines, namely DBCC CHECKDB, DBCC CHECKALLOC within query analyser or consider creating a database maintenance plan and selecting the integrity checking options.

SVRMGRL
The closest SQL Server has to the server manager executable (svrmgr23 in 7.3, svrmgr30 in 8.0, svrmgrl in 8.1 and replaced with extended functionality in SQL*Plus for 9i) is sqlservr.exe. This is the core process running the instance and its associated databases and worker threads. In most cases, the DBA will utilise the Service Control Manager (also see the NT services and open the properties of the service to interact with startup parameters) and Enterprise Manager. The DBA may also consider using Query Analyser (SQL*Plus).

65

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL Loader
There is no equivalent to SQL Loader command line. Running Enterprise Manager, the DBA can use the DTS Import / Export wizard and its variety of options to script databases and move data, but it is still limited, i.e. cannot script linked servers etc). The GUI is reasonably powerful, but the DBA may feel inclined to use other 3rd party products for better coverage of all database elements. The DBA should not script the system databases as a method of restoring. The DBA should also consider the BCP command line tool and the bulk insert DML command. The bulk copy process allows the DBA to dump data to text files using defined column and line separators. Here is an example of using bulk insert in a stored procedure to load a temporary table:
CREATE TABLE [#lodgement_enrolment] ([lodgeenrol_id] [int] NOT NULL , [lodgeenrol_match_trainprod_id] [int] NOT NULL , [lodgeenrol_match_tc_id] [varchar] (20) NULL , [[lodgeenrol_match_student_id] [int] NULL , [lodgeenrol_moduleassoc_id] [varchar] (20) Not NULL ) DECLARE @v_sql VARCHAR(1000) set @v_sql = 'BULK INSERT #lodgement_enrolment FROM ''' + @filename + ''' WITH (FORMATFILE = ''c:\lodgement.fmt'')' exec(@v_sql) Example format file:
8.0 18 1 2 3 4 5

SQLINT SQLCHAR SQLCHAR SQLCHAR SQLCHAR

0 0 0 0 0

50 50 50 50 50

"\t" "\t" "\t" "\t" "\t"

2 4 5 6 7

lodgeenrol_cps_no lodgeenrol_intake_no lodgeenrol_module_code lodgeenrol_module_name lodgeenrol_module_hours

SQL_Latin1_General_CP1_CI_AS SQL_Latin1_General_CP1_CI_AS SQL_Latin1_General_CP1_CI_AS SQL_Latin1_General_CP1_CI_AS SQL_Latin1_General_CP1_CI_AS

The bulk insert command requires the user to be a member of the bulk admin fixed server role. The user must also be the owner of the object they are inserting the data into. The command is very fast and is well worth exploring. If you still have issues running the command, try and making the user a dbowner of the tempdb database.

Java and SQL Server

In Oracle, we can store enterprise beans within the SGA via JServer (load and execute java applications with the DBMS ), we can also write Java stored procedures and utilise JSQL. There are not equivalent utilities or DBMS options in SQL Server for such native support. This may change with the release of SQL Server .Net (Yukon).

66

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL Trace
The ORACLE DBA can utilise SQL Trace to write out performance statistics for statements being executed for a session or all connection database sessions. This may typically involve the setup of the trace parameters in the init.ora (SQL_TRACE, TIMED_STATISTICS, USER_DUMP_DEST etc), and invoked from SQL*Forms, SQL*Plus, PL/SQL sub-routine and other methods, using commands such as:
alter session set sql_trace true or DMBS_SESSION.SET_SQL_TRACE(TRUE);

The files produced are formatted via TKPROF. In SQL Server, the DBA can utilise: a) SQL Profiler GUI for tracing and the subsequent reading of trace data files. The GUI may skip trace records depending on the load running against the database. This was apparently was by design. b) Use the T-SQL stored procedures to run the trace, these are stored in the master system database. a. sp_trace_create b. sp_trace_setevent c. sp_trace_setfiler d. sp_trace_setstatus

67

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Query analyser (SQL*Plus)

The Query Analyser is the GUI version of isql.exe, and is initiated with isqlw.exe from the command line. It is the equivalent to SQL*Plus in Oracle, even so, it does not include SQL*Pluss extensive formatting extensions such as REPFOOTER, BREAK, BTITLE, ATTRIBUTE, SPOOL, TTITLE and its ability to parameterise and prompt for input. Some of these options are available within the GUI but are not specific SQL or T-SQL statements. The experienced Oracle developer or DBA may find this lack of equivalent functionality very frustrating.

Database Config Assistant / Oracle Admin Assistant etc

Use the SQL Server Enterprise Manager (EM).

Net8 Assistance / SQLNet

The listener in SQL Server is called the Server Network Utility. When instances are installed via the setup disk, default listener entries are also created for named pipes and the TCP/IP (default port auto-selected on first every connection via the protocol unless specifically set) protocols. Here, the DBA can control the DB-Libraries used for establishing connectivity to the SQL Server instances installed on the server.

Compared to Oracle, the listener configuration in SQL Server is simple and is generally an error free process. The options available are somewhat restrictive though in terms on failure alternatives and encryption etc.

68

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

UTLBSTAT UTLESTAT STATSPACK , ,

Oracle DBAs extensively uses these PL/SQL routines and associated packages for performance tuning and reporting. The SQL Server DBA will utilise: Profiler for statement tracing NT Performance Monitor and the numerous SQL Server counters 3rd party products from Quest, NetIQ and others.

SRVCTL
There is no equivalent to srvctl (manage ORAC environment via the command line) in Oracle. This may be closer to the Cluster Manager utility in Windows and to a lesser degree SQL Server Enterprise Manager.

Log Miner
There is no equivalent to log miner utility in Oracle. Consider 3rd party tools.

ISQL
ISQL is invoked on the command line with isql.exe, the GUI version is called query analyser and is discussed later in this chapter. The command line version help is called by isql.exe -? or isql.exe /?, the screen shown for a SQL Server 2k installation is:
isql: unknown option ? usage: isql [-U login id] [-P password] [-S server] [-H hostname] [-E trusted connection] [-d use database name] [-l login timeout] [-t query timeout] [-h headers] [-s colseparator] [-w columnwidth] [-a packetsize] [-e echo input] [-x max text size] [-L list servers] [-c cmdend] [-q "cmdline query"] [-Q "cmdline query" and exit] [-n remove numbering] [-m errorlevel] [-r msgs to stderr] [-i inputfile] [-o outputfile] [-p print statistics] [-b On error batch abort] [-O use Old ISQL behavior disables the following] <EOF> batch processing Auto console width scaling Wide messages default errorlevel is -1 vs 1

69

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

[-? show syntax summary (this screen)]

ISQL vs OSQL command line

Feature Invocation Connectivity Unicode output files supported OS Environment variables NT Authentication Max command characters per line Call OS commands? Can read in files ISQL isql.exe or isqlw.exe (query analyser) DB-Library No isqluser=.. isqlserver=.. Mixed or NT 1000 Yes, use !! when running isql interactively Yes, in interactive mode with :r command. OSQL osql.exe ODBC (SQL Server ODBC driver via SQL-92 connect options) Yes osqluser=.. osqlserver=.. Mixed or NT 1000 Yes, use !! when running isql interactively

70

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

Backup and Recovery

The key difference between Oracle and SQL Server is not so much the commands and recovery models affecting the type of backup, but the number of backups required to completely recover a database instance verses an individual database. Of course, architecturally, backup and recovery is very different under both DBMS. We will explore backup and recovery in SQL Server with a variety of scenarios.

s with any modern DBMS, SQL Server supports complete or full backups, differential backups and transaction log backups (equivalent to archived log backups)

Backup
As mentioned in the architecture section, a SQL Server instance consists of a number of system databases, these, just like the system tablespace in Oracle, is essential for recovery. For the instance to start, the system databases must be available. It is critical that full backups are performed on a regular basis for your master, msdb, model and distribution (if replication enabled), then of course, the users databases are just as important. We will cover backup and recovery in some detail. To make this section a little easier to put in context with Oracle: Hot Backup all full backups in SQL Server are HOT backups Full Backup (shutdown instance and backup database files) although possible, it is not recommended, the full backup is typically used in all cases RMAN no equivalent in SQL Server. Closest option is Database Maintenance Plans but these are simply wizards that generate scheduled jobs to make life much easier. Import/Export (full) no equivalent utility that is even close to Oracles. The best SQL Server has database scripting (manually run) and BCP (bulk copy data). Oracle is way ahead in this area. Differential Incremental Backups (RMAN) same in SQL Server

71

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Incremental or Archived Log Backups transaction log backup in SQL Server

REMEMBER Treat the master, msdb, model and distribution databases as your system tablespace data files and control files all mixed into one. They are critical in Oracle recovery and are subsequently critical in SQL Server database instance recovery. The tempdb is rebuilt from the model database on instance startup.

Recovery Interval
The recovery interval is set at an instance level and affects the checkpoint timeout period for all databases in the SQL Server instance. This of course has flow on effects for instance recovery in terms of the number of possible transactions SQL Server must rollback (uncommitted transactions) and roll forward (committed but not written to the physical database data files) during its recovery cycle on instance startup. The default value is zero (SQL Server managed) or any value greater than zero in minutes and when altered its value takes affected immediately. In a majority of circumstances leave the setting at the default. The value can be set via Enterprise Manager, or via a SQL statement:

exec sp_configure N'recovery interval (min)', 2 reconfigure with override

Trying to pre-empt the actual goings-on with the DBMS architecture in terms of this value is difficult to predict and the SQL Server documentation is some vague. Use performance monitor counters to monitor checkpoints and alter the recovery interval to review the impact to the instance, this may

72

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

take some time to be reflected. It is important to remember that performance monitor wont measure instance recovery time which, in some circumstances, can effect your SLA if you are not careful.

Number of pages flushed by checkpoint or other operations that require all dirty pages to be flushed. In this case we are monitoring the default instance, other instances will have their own counters.

Recovery Models
Setting a recovery model is like the Oracle DBA tuning on or off archived log mode for the database instance. In SQL Server, each database has a recovery model which determines what statements are logged or not and if point in time recovery is possible. The models are: a) Simple transaction log (redo log) entries are truncated on completion of a checkpoint. Point in time recovery is not possible. Equivalent to non archived log mode in Oracle. b) Full transaction log entries are retained and the log file will grow until the DBA backs up the transaction log and flushed the committed log data to disk (archived log mode in Oracle). c) Bulk Logged as per full but selected commands are not fully logged (therefore not recoverable), these include select into, bcp and bulk insert, create index and indexed view creation, text and image operations (write and update text). The default system database recovery models are: MASTER MSDB MODEL TEMPDB Simple (only full / complete backups can be performance on master) Simple Simple Simple (recovery properties cannot be set for this DB)

Do not alter recovery model properties for any system database. Again the DBA can alter the recovery model properties for user databases via Enterprise Manager or a SQL statement:

73

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

ALTER DATABASE [BUYSMART] SET RECOVERY SIMPLE

The alteration will take immediate affect. The DBA should issue a full backup and if using full or bulk-logged options, continue with planned transaction log backups as need be.

Backup Devices
A backup device in SQL Server 2k is simply a logical name (alias) for a physical file that may be a disk location (physical or UNC) or tape device. The device is visible to all databases within the instance. The device is not necessarily required, but is there for convenience and does allow backup scripts to separate themselves from the physical location of data files. Altering a script to backup elsewhere can be done by changing the destination of the back device.

74

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

exec sp_addumpdevice N'disk', N'mydevice', N'e:\dbbackups\mydevice.BAK'

The above dialog will run exec xp_fileexist "e:\dbbackups\mydevice.BAK" to verify the location and warn the DBA accordingly. The device has some flow on affects within the Enterprise Manager in terms of viewing their content and selecting the device as a drop down item when backing up databases via the EM.

IMPORTANT The Database Maintenance Plan wizards do not utilise these backup devices for some strange reason.

RMAN
There is no equivalent utility to RMAN in SQL Server. Recovery must be either manually instigated by the DBA within Query Analyser or its equivalent command line utility, or via Enterprise Manager and its GUI screens. Backups are performed via Database Maintenance Plans or via t-sql scripts that DBA has custom written.

Database Maintenance Plans

If you are just starting out with SQL Server and want to get backups up and running quickly, along with some integrity checking, then consider database maintenance plans. NOTE Maintenance Plans are found under the Management folder within Enterprise Manager. The maintenance plan is simply a wizard that generates a series of MSDB jobs that are scheduled and run by SQL*Agent. These jobs may include the following against one or more databases: Database backups (full and log backups only) a. Can specify auto-removal of media sets that are N days/weeks/months/seconds/minutes old b. Can specify destination directory and the option to auto-create sub-directories for each database to be backed-up b) Database re-organisation

75

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

c) Update database statistics d) Shrink databases (remove un-used space) e) Database integrity checks For backups, SQL Server will create one media set per backup set. This means one physical disk file (media set) backup and inside it, a single log or full backup (backup set). It will NOT append backup sets to existing media. NOTE Many of these items can be scheduled individually, away from the backup job times. Note that SQL Server will not warn you of overlapping jobs or the fact that another maintenance job already exists of that type. When it comes to backups, the maintenance plan can be a little restrictive though, consider some of these: no support for differential backups many of the backup options are not available, namely the password parameter can not duplex backup files (copy to another disk or server as part of the backup) does not support appended backups to an existing backup device (media set)

NOTE Natively, SQL Server does no built in mechanism for compressing or zipping backup files. Consider writing your own backup t-sql stored procedure and using the xp_cmdshell extended stored procedure. The maintenance plan backup screens are shown below. Remember that this point we have already selected the databases to be included in this overall maintenance plan (first screen of the wizard).

This screen is for FULL backups of the selected databases. We can optionally verify the backup.

Disk or pre-defined tape backups Full backup schedule

76

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Always use a different directory to that recommended. The SQL Server path is too deep in the directory structure. Nice feature, will remove media sets that are N days (and other options) old and auto-create sub-directories for each database.

The next screen is related to transaction log backups. Be warned here that not all databases you have selected in the first screen may be privy to log backups and can result in failure of the scheduled maintenance plan.

This screen and those following it are basically identical to the FULL backup screens. The default file extension is TRN rather than BAK.

The DBA can review and alter the maintenance plan at any time by simply selecting properties for the generated plan and editing it as need be within Enterprise Manager.

Data Dictionary Views (Media Sets, Backup Sets)

77

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

It is important to understand the difference between a media set and a backup set. These concepts are used throughout the following sections and within the online help for SQL Server. A physical backup device is the media set, within the media we can store one or more logical backup sets of one or more databases (typically its all the same database). This is shown below with their properties:
MEDIA SET (NAME, DESCRIPTION, PASSWORD, MEDIA-SEQUENCE) BACKUP SET (NAME, SERVER, DATABASE, BACKUP TYPE, DATE, EXPIRES, SIZE, DESC) BACKUP SET (NAME, SERVER, DATABASE, BACKUP TYPE, DATE, EXPIRES, SIZE, DESC) Backup set

BACKUP DATABASE [mydb] TO DISK = e:\dbbackups\mydb\mydb_20020624_full.bak WITH INIT, NAME = Full Backup of MYDB on 24/06/2002

Media set

78

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The backup information is recorded in the MSDB database, here is a snapshot of the physical data model:
Backup media set media_ set_id Backup set backup _set_id Restore history Restore_ history_id Restore file

media_ set_id

backup _set_id

Restore_ history_id Restore file group

Backup media family

Backup file

Log mark history

The DBA should purge this information on a regular basis. I have personally found that recovering a database via the GUI with a large number of backup records can result in a huge delay (4+ minutes at times) as you wait for the GUI to return control back to you. set dateformat dmy exec msdb..sp_delete_backuphistory 15/06/2002 -- remove records older than date NOTE rather than using msdb.. (which tells SQL Server that it will find the stored procedure in the msdb system database and use the dbo owner), we could have entered use [msdb] before we ran the procedure that also sets the current database. If you append backups to a media set then we refer to each appended backup via the FILE option as you will see in the examples presented throughout the chapter.

Hot / Online (Full) Backups

A full backup in SQL Server is equivalent to a hot backup in Oracle. The database does not come offline or unavailable to the end-user during a full backup. In terms of the entire SQL Server instance though, full backups should encompass all system backups in order to successfully recovery the entire instance. There is no single backup or recovery statement that will cover all databases within the instance. At a bare minimum the DBA should consider: MASTER MSDB MODEL <User DB> Full backups, nightly Full backups, nightly Full backups, nightly Full backups, nightly

The tempdb system database is rebuilt automatically to the destination defined in the sysdatabases system table in the master database on instance start-up.

79

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The GUI is very simple to understand and is not worth discussing. In most cases the DBA will create a Database Maintenance Plan to schedule and manage the full database backup. An example full backup statement is: BACKUP DATABASE [mydb] TO DISK = e:\dbbackups\mydb\mydb_20020624_full.bak WITH INIT, PASSWORD = my db password, NAME = Full Backup of MYDB on 24/06/2002
Processed 112 pages for database 'mydb', file 'mydb_Data' on file 1. Processed 1 pages for database 'mydb', file 'mydb_Log' on file 1. BACKUP DATABASE successfully processed 113 pages in 0.534 seconds (1.720 MB/sec).

BACKUP DATABASE [ANOTHERDB] TO DISK = 'e:\anotherdb_20020603_full.bak' WITH INIT, NAME = 'Full Backup of ANOTHERDB on 03/06/2002', EXPIREDATE = '03/06/2002' If we tried to run the above command again we get the error below due to the expiration date we have set. To get over this and still use the INIT option then we need to use the SKIP option as well.
Server: Msg 4030, Level 16, State 1, Line 1 The medium on device 'e:\aa_20020624_full.bak' expires on Jun 3 2002 12:00:00:000AM and cannot be overwritten. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP DATABASE is terminating abnormally.

NOTE take a close look at the WITH clause syntax, the books online cover this command very well and should be reviewed thoroughly. The DBA should have a good understanding of all backup and recovery options, but some of the key items are: TO [DISK, TAPE] = <backup device name or physical location> o Logical of physical location for the database backup to be place WITH INIT o Force overwrite of the backup file if its exists WITH NOINIT o Will append the backup to the existing backup set within the media. MEDIA[name, password, description] o These options set the name, description and password for the entire media, a backup media (disk or tape) can contain one or more backup sets FORMAT, NOFORMAT o Format renders the entire media set un-usable and ready for new backup sets. Does NOT preserve the media header. o No-format tells SQL Server to retain the existing media header. It will not overwrite the media device unless INIT is also used.

80

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

EXPIREDATE = <date>, RETAINDAYS = <number of days> o Prevents the overwriting of a backup based on the expire date and retain days parameters. BLOCKSIZE = <bytes> o If media requires backups to be of a specific block size in order for a restore from that media to be successfully read.

The Enterprise Manager GUI is a little restrictive when it comes to restoring database backups when the PASSWORD option has been used. It does not give you the option to specify it and comes up with the error:

Differential Backups
The differential backup will backup any 64Kb extent that contains an altered page within the database. Remember this when viewing the backup size of the media set as you may be surprised. The tracking is managed by the SQL Server storage engine using the DCM (differential change map) page present in each non-log data file. In Oracle, differential backups are known as differential incremental backups that are supported within RMAN and it is also possible to do cumulative database exports. A differential backup is not supported in Database Maintenance Plans (should change in the next version of SQL Server), therefore the DBA needs to resort to writing their own scripts that can be a right pain. In many cases, full and log backups will suffice, but can slow the recovery process when applying large numbers of archived log files. This is where differentials can be used to speed the recovery process. The DBA will need to do their own performance tuning to determine if differentials are required to meet their recovery SLA. Here is an example differential backup: BACKUP DATABASE [mydb] TO DISK = e:\dbbackups\mydb\mydb_20020624_full.bak WITH DIFFERENTIAL, INIT, NAME = 'Differential Backup of MYDB on 24/06/2002' The differential backup will backup all extents modified since the last full backup, and NOT the last differential. This is very important to understand, especially during recovery. The last differential backup done must be used on recovery; they are not cumulative as log backups are.

81

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To get these backups up and running quickly, write a T-SQL stored procedure and an associated DTS to call it with an email notification for error tracking. Then simply schedule the package to run as required. Important You cannot use differential backups to do point-in-time recovery (i.e. the STOP AT clause is not valid to recover to a point in time for the period the differential backup covers).

Archived Log (Transaction Log) Backups

Transaction log backups are the same as archived redo log backups in Oracle. Remember that a transaction log exists for each database within the SQL Server instance and is a mandatory requirement for the database to exist. The log backup is supported via Maintenance Plans, making it very simple for the DBA to quickly set up full backups with scheduled log backups. The database must be in full or bulk-logged recovery modes before attempting a transaction log backup. If not you will receive the error:
Server: Msg 4208, Level 16, State 1, Line 1 The statement BACKUP LOG is not allowed while the recovery model is SIMPLE. Use BACKUP DATABASE or change the recovery model using ALTER DATABASE. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP LOG is terminating abnormally.

Attempting to backup the log file for the master database will result in the error:
Server: Msg 4212, Level 16, State 1, Line 1 Cannot back up the log of the master database. Use BACKUP DATABASE instead. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP LOG is terminating abnormally.

You can alter the recovery modes of the MSDB and MODEL databases if you like and do transaction log backups, but it is not recommended unless there is an important requirement to do so. Attempting to backup the log file for the tempdb database will result in the error:
Server: Msg 3147, Level 16, State 1, Line 1 Backup and restore operations are not allowed on database tempdb. Server: Msg 3013, Level 16, State 1, Line 1 BACKUP LOG is terminating abnormally.

Do not attempt a full or differential backup whilst a transaction log backup is running as it may result in error. Even so, Microsoft documentation states that concurrent full and log backups are compatible. There are several important parameters the DBA should understand when using log backups. Note that many of these parameters are NOT available when creating maintenance plans, therefore, for the advanced DBA this may be too restrictive. BACKUP LOG [mydb] TO DISK = 'c:\master.bak' WITH <see books online for comprehensive list of parameters>

82

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Parameter NO_TRUNCATE

Notes Special parameter used when database is in a damaged state, allows us to attempt a log backup without truncating the virtual log files. This is important if we are still trying to recover the instance whilst we attempt to build another (i.e. standby database).

NOTE Remember that the databases transaction log will continue to fill as committed and non-committed transactions execute against the database. The backup transaction log will write (archive in Oracle) all committed transactions to your selected transaction log backup file (an archived log). The DBA can truncate the transaction log via the WITH NO_LOG or WITH TRUNCATE_ONLY option. This is used in a variety of situations, the classic being when you accidentally used the full or bulklogged recovery model when you didnt want transactions permanently logged for point in time recovery and now the log has grown and typically results in full transaction log errors. This command will remove all non-active transactions from the log, from there, the DBA can think shrink the log files and change the recovery model as need be. BACKUP LOG [mydb] WITH TRUNCATE_ONLY Remember - you cannot selectively truncate transactions in the log file, its all or nothing. The DBA must do a full backup as you cannot recovery through a truncate (as you would except).

Tablespace (Filegroup) Backups

The DBA can also do tablespace (file-group) backups, although fine I rarely use them as it typically complicates recovery. Even so, for very large databases this may be the only option. Here is an example:
BACKUP DATABASE [mydb] FILE = N'myprimarydatafile', TO DISK = N'C:\mydb_fg_myprimarydatafile.bak' WITH INIT , NOUNLOAD , NOSKIP , STATS = 10, NOFORMAT -- logical filename of physical file -- backup destination

NOTE Your database must be using the full or bulk-logged recovery model

83

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Moving DB &Transportable Tablespaces

The DBA can move or copy databases via the sp_attach and sp_deattach commands. This works on all database files, not selected file-groups. At present, SQL Server has no equivalent option to Oracles transportable tablespaces, the closest option we have here is to backup and restore filegroups. Even so, we have a variety of other options: a) b) c) d) e) Shutdown instance, copy database files, and re-attach at destination server Offline the database, copy files, and re-attach at destination server. De-attach the database, copy files, and re-attach at destination server. Run a split mirror, offline or read-only the database, break the mirror and get the live database back up and running whist you work with the split mirrored files in the background.

Some of these methods are described below. Remember - copying a database will not take the logins with it, as this information is stored in the master database. Remember If you have not database backups, but still have all the database files, the reattaching the database will be your last remaining hope to recovering your database.
Shutdown instance method

Simply shutdown the SQL Server instance, taking care when running multiple instances on the same server. When down, copy the database files to the other server (or copy/rename/move if it will be attached to the same server). As the database was cleanly shutdown there will be no issues with re-attaching so long as the copy and destination did not fail unexpectedly. If the instance did fail unexpectedly and you have no backups, re-attaching may still be possible (with the added risk of data corruption). When using this method, the database will of course remain on the source server with no change what-so-ever to the source database. To shutdown the instance use one of the following: a) use NET STOP service command from the operating system b) use Enterprise Manager and their GUI option c) issue the SHUTDOWN transact-SQL command d)
Offline a Database method

Once the database is offline, you can copy its database files to a new server and re-attach. Use this method when shutting down the SQL Server instance is not desirable and you want to retain the database on the source server. Reminder User sessions will not be disconnected; this is applicable for sp_dboption and the ALTER database command.

84

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To take the instance offline:

exec sp_dboption N'mydb', N'offline', N'true' or alter database [mydb] set offline with rollback after 60 seconds or alter database [mydb] set offline with rollback immediate or DBCC DBCONTROL (mydb,offline)

Using the alter database statement (SQL Server 2k and beyond) is the preferred method. The rollback after statement will force currently executing statements to rollback after N seconds. The default is to wait for all currently running transactions to complete and for the sessions to be terminated. Use the rollback immediate clause to rollback transactions immediately. When running the command with users connected you will get something like:
sp_dboption (does not wait like the alter database command, see below) Server: Msg 5070, Level 16, State 2, Line 1 Database state cannot be changed while other users are using the database 'mydb' Server: Msg 5069, Level 16, State 1, Line 1 ALTER DATABASE statement failed. sp_dboption command failed. alter database [aa] set offline [any parameter combination] This command will run forever, waiting for sessions to disconnect. When it completes you will get something like: Nonqualified transactions are being rolled back. Estimated rollback completion: 100%.

See the script http://www.sqlservercentral.com/scripts/scriptdetails.asp?scriptid=271 to kill off all connections for a database. To confirm the offline status:
SELECT DATABASEPROPERTY('pubs','IsOffline')

or

-- 1 if yes

SELECT DATABASEPROPERTYEX('mydb', 'Status')

Attempting to connect to the database will give you: Server: Msg 942, Level 14, State 4, Line 1 Database 'mydb' cannot be opened because it is offline.
De-Attaching the database

If you want to completely remove the database from the master database and the SQL Server instance, use the deattach command rather than offlining the database. When attempting to de-attach with Enterprise manager it will warn you when: a) there are users connected to the database b) replication is active

85

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

All user sessions must be disconnected and replication disabled before attempting the deattachment. The command is:
exec sp_detach_db N'mydb', N'false'

The second parameter denotes wether to include a statistics collection before de-attaching the database. You must be a member of the sysadmin system role to issue this command. Also note the error:
Server: Msg 7940, Level 16, State 1, Line 1 System databases master, model, msdb, and tempdb cannot be detached.

Funny enough, statistics are still updated before receiving this error. The de-attachment will remove the database from the sysdatabases table in the master database. The sysxlogins table will retain references to the de-attached database, therefore, you will need to either remove the login(s) or alter their default database connections, this is shown below.
exec sp_defaultdb N'myuser', N'master' change default db from myuser to the master database. exec sp_droplogin N'mytest'

Dropping logins is not straight forward, you need to either orphan the login from its associated database user or drop the user, otherwise you will get this message:
Server: Msg 15175, Level 16, State 1, Procedure sp_droplogin, Line 93 Login 'myuser' is aliased or mapped to a user in one or more database(s). Drop the user or alias before dropping the login.

You cannot remove users that own database objects, the standard drop user command is:
use [mydb] exec sp_dropuser Nmyuser Checking Files before Attaching

You should note that you cannot re-attach more than 16 files for a single database. Before attaching the database, issue the following commands over the primary file-group data file to get a listing of files that make up the database structure:
--Is the file a primary file-group MDF file? dbcc checkprimaryfile (N'E:\SQLServerData\MSSQL\Data\mydb_Data.MDF', 0) --Get me the database name, version and collation dbcc checkprimaryfile (N'E:\SQLServerData\MSSQL\Data\mydb_Data.MDF', 2) --Get a list of all files associated with the database. (original name) dbcc checkprimaryfile (N'E:\SQLServerData\MSSQL\Data\mydb_Data.MDF', 3)

Attaching the database

86

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The sp_attach_db command allows you to re-attach your database onto the SQL Server instance. For example:
exec sp_attach_db N'mydb' , N'E:\SQLServerData\MSSQL\Data\new_aa_Data.MDF', N'E:\SQLServerData\MSSQL\Data\new_aa_Log.LDF'

The syntax is simple enough, the first being the name of the database to attach and its associated database files you have checked via the methods outlined previously. The database being attached must not already exist, you can also attach databases not previously de-attached so long as the database was closed successfully.
Server: Msg 1801, Level 16, State 3, Line 1 Database 'mydb' already exists.

After re-attaching, especially if its on different server, you will need to fix orphaned logins via the command:
exec sp_change_users_login <see SQL Server BOL for parameter list> Attaching a single file

The sp_attach_single_file_db command is quite powerful. It allows you to re-attach a database my specifying only its initial master data file, if your database had other data files (even in the primary file-group) they will be automatically re-attached (only to their previous destination though) for you by reading the sysfiles. This is all fine if you want the data files restored to the same location from which the database once existed along with the physical file name, but apart from that you have no control and will need to opt for sp_attach. When re-attaching with this command, you have the ability for SQL Server to automatically recreate your log file so long as its not available for SQL Server to automatically re-attach when it looks up sysfiles. This method is handy when you want have massive log file and want to shrink it back to a manageable size. For example:
exec sp_attach_single_file_db N'MyxxDb' , N'E:\SQLServerData\MSSQL\Data\xx_Data.MDF' <..shows the message below..> Device activation error. The physical file name 'e:\sqlserverdata\MSSQL\data\xx_Log.LDF' may be incorrect. New log file 'E:\SQLServerData\MSSQL\Data\xxxx_log.LDF' was created. The new file size E:\SQLServerData\MSSQL\Data\xxxx_log.LDF will be 512k.

This will not work if you have multiple log files thought, you will get an error like:
Server: Msg 1813, Level 16, State 2, Line 1 Could not open new database 'mytest'. CREATE DATABASE is aborted. Device activation error. The physical file name 'e:\sqlserverdata\MSSQL\data\mt_Log.LDF' may be incorrect. Device activation error. The physical file name 'e:\sqlserverdata\MSSQL\data\mt_2_Log.LDF' may be incorrect.

87

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In this example, the database had a single data file with two log files. When I have only one file the attachment works file and re-creates the log in the destination of the previous file (but has a modified size). Two or more files will result in the error.
Some issues with MODEL and MSDB databases

To detach the model or msdb system databases, you need to set the trace flag T3608 on instance startup. In all cases you must attach the model before the msdb database, remembering that SQL*Agent for the instance must be stopped. As a side note, the attach command executes something like the following:
CREATE DATABASE [mydb] ON (FILENAME = C:\dbdata\mydb\mydb_data.mdf, FILENAME = C:\dbdata\mydb\mydb_log.ldf) FOR ATTACH

The create database command has dependencies on the model database, therefore affecting its reattachment.
Fixed dbid for system databases

The DBA should also be aware of the master..sysdatabases system table and its value for dbid for system databases. In some very rare occasions, it is possible that a restore results in a corruption, or mixup in the dbid for the database, this may occur when restoring databases in the wrong order. The flow on effect is some very strange errors and confusion all round. See reference (57) for a great example of this. The dbid for system databases are: 1 2 3 4 Master Tempdb Model Msdb

88

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Scripting Databases
The only option here is via Enterprise Manager. Right click properties on any database and the following GUI is shown:

The screen is simplistic and requires no explanation but there are a few things to remember: a) You must select objects (tables, views) in order to script indexes/triggers/constraints/permissions. You cannot generically script all indexes for example without selecting all tables/views first. This can be a pain as you need to filter out what you need from the generated script. b) You cannot script many databases at once c) You cannot script logins specific to database (i.e. logins that map to a single user in one database typically the one you are scripting). You cannot script the sa login. d) You cannot script linked or remote servers. Use the preview option to view the generated script in mini-dialog windows (which you can cut into the clipboard from).

89

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The diagrammer is also another handy place to generate scripts. As an example, if you need to make a variety of schema changes, and then create a new (or use an existing) diagram, save, make you changes and then hit the script button (see below). Once copied, you can apply the changes as need be to the database. You can then use the saved script to apply on other identical databases (i.e. test / support / prod databases) to mimic the changes and/or new objects.

You can cut the scripted text from here into notepad.

One of the key problems with the diagrammer is that you cannot allocate object permissions whilst editing tables. This can adversely complicate your script generation ideas. NOTE Be careful with generated scripts from the diagrammer, it can take the long route to a simple change that can result in heartache for the DBA if the takes are very large. Always review the generated script before running. In all cases though, in my experience EM has never generated a script with errors. If you select the design table option and alter the table, the same script option is available to the DBA. Note that this is not the case for design view although the SQL statement is selectable.

90

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Another method for scripting is via EM and its listing pf tables, for example:

Select objects in Enterprise Manager, CTRL-C to copy.

Run Query Analyser, open a new connection, and paste, a script is generated for the selected objects.

Backup control file to trace

There is no control file so it is not applicable. The closest equivalent is doing a full backup of the system databases.

Verifying Backups
To verify a backup, use the command:
restore verifyonly from c:\myfullbackup.bak

The DBA can also load the backup history in the backup file into the MSDB database. This can be handy when analysing the backup before attempting a recovery.

91

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Recovery
Recovery in Oracle is either: a) manual via svrmgrl / sqlplus b) via RMAN c) via 3rd party product d) Enterprise Manager In SQL Server, we use variety of methods: a) rebuildm.exe (from setup CD for rebuilding the system databases) b) Enterprise Manager and its GUI wizards c) Query Analyser (GUI or command line version) d) SQL Server Service Control Manager Recovery is potentially more complex that Oracle at times due to the fact that we are not dealing with one database, but many system databases then associated user databases. This section provides a summary by example in which the DBA can then base further tests to drill down into this very important topic. NOTE Many of the examples use the GUI tools and at times, with reference to the equivalent T-SQL command.

Killing User Connections and Stopping Further Connects

Killing off user connections is simple enough and there are endless scripts on the internet to do the job. An example script by ruba.kumar@aurumtechnology.com is shown below:
CREATE PROC Kill_Connections (@dbName varchar(128)) as DECLARE @ProcessId varchar(4) DECLARE CurrentProcesses SCROLL CURSOR FOR select spid from sysprocesses where dbid = (select dbid from sysdatabases where name = @dbName ) order by spid FOR READ ONLY OPEN CurrentProcesses FETCH NEXT FROM CurrentProcesses INTO @ProcessId WHILE @@FETCH_STATUS <> -1 BEGIN --print 'Kill ' + @processid Exec ('KILL ' + @ProcessId) --Kill @ProcessId FETCH NEXT FROM CurrentProcesses INTO @ProcessId END CLOSE CurrentProcesses DeAllocate CurrentProcesses

GO

Also consider the command to more elegantly terminate users and closing off the connection:

92

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

ALTER DATABASE mydb SET SINGLE_USER WITH [<termination clause>] eg: ALTER DATABASE mydb SET SINGLE_USER WITH ROLLBACK IMMEDIATE

To stop further connections, consider altering the database to dbo access only, or disable the database logins via sp_denylogin (NT logins only). Remember you cannot restore a database whilst users are connected.

Using the GUI for Recovery

Unless you have major system database problems (which require additional steps before running EM), the DBA will find that using EM for recovery is the most simplistic approach. The best thing about the GUI when it comes to recovery is the reading of the MSDB sys backup tables and correctly listing out the backup files to be used in a recovery scenario. Later, we will discuss a script I wrote some time back that does a similar thing.

Name of the database, we can enter a new name if required. Click on the options tab if you do to double check the name of the database files and the destination

For the selected database in the drop down above, is listed the date of all full backups. From this full backup selected above, the MSDB is searched and lists, in hierarchical order its proposed restore list to do a complete recovery. We can select the option to restore to a point in time if available. The DBA can uncheck the appropriate backup files as need be. Note that we cannot alter the destination of the backups listed which can be very restrictive.

NOTE When using EM for recovery, run profiler at the same time to trace the T-SQL recovery routines being executed. This is the best way to learn the recovery commands and the context in which they are being used.

93

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

WARNING If you restore a database, say, Northwind, and restore it as a different name (database), then be careful when removing the new database. It will ask if you want to remove all backup history, if you say yes then kiss good-bye to the Northwind databases MSDB backup entries. We will cover some of the GUI options in brief. Remember that virtually ALL restore operations require no users to be connected to the database.
Options - Leave database in non-operational state but able to restore additional logs

This option allows us to restore the instance to any specific point, but leave it in a state where we can apply further backups as need be.

Selecting properties of the restored instance in loading state gives us the error:

If you realise that you have no further backups and want to complete the recovery of the instance, then (note that exec sp_helpdb will not show the database):
SELECT DATABASEPROPERTY('aa', N'IsInRecovery') SELECT DATABASEPROPERTYEX('aa', 'status')

restore database aa with recovery RESTORE DATABASE successfully processed 0 pages in 1.178 seconds (0.000 MB/sec).

A instance re-start will also issue the recovery statement. The non-operational state simply executes the with norecovery option on restore of the last specified backup file.
Options Using the Force restore over existing database option

Using EM can be a tad strange when restoring databases. If you attempt to restore the currently selected database, it will never prompt you that you are trying to overwrite an existing databases

94

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

data files, even though (technically speaking here), you are! If we attempted to restore say the northwind database as the pubs database, we will be promoted with the following dialog:

It seems be to something related to the MSDB backup and restore tables which determines whether or not this dialog is shown. Anyhow, to get around this, we click on the options tab and select the Force restore over existing database option. The command is not different to a standard restore, there is no magical restore option related to the prevention of file overrides. RESTORE DATABASE [bb] FROM DISK = N'c:\northwind_full.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, RECOVERY , MOVE N'Northwind_log' TO N'C:\dblog\bb_log.ldf', MOVE N'Northwind' TO N'C:\dbdata\bb_data.mdf' Be very careful with this option in EM. Personally, I never use it unless I am 100% sure that the files I am writing over are fine and I have already backed them up if possible.

Restoring a databases backup history from backup files

In this example we have the following database, with associated backups: Database: mydb Data and Log files: c:\mydb.mdb, c:\mydb.ldf Backups: Full Diff Log Log c:\mydb_full.bak c:\mydb_diff.bak c:\mydb_log1.bak c:\mydb_log2.bak

On selecting the restore in EM for the database, it magically lists all backups for a successful restoration of my database up to the point of mydb_log2.bak. If we lost this information, then suddenly our nice GUI dialog is not so helpful anymore. To re-populate the MSDB database tables with the backup history I recommend that you do not use the GUI, it is overly time consuming for such a simple task:
RESTORE VERIFYONLY FROM DISK = N'C:\mydb_full.bak' WITH NOUNLOAD , LOADHISTORY RESTORE VERIFYONLY FROM DISK = N'C:\mydb_diff.bak' WITH NOUNLOAD , LOADHISTORY RESTORE VERIFYONLY FROM DISK = N'C:\mydb_log1.bak' WITH NOUNLOAD , LOADHISTORY

95

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

RESTORE VERIFYONLY FROM DISK = N'C:\mydb_log2.bak' WITH NOUNLOAD , LOADHISTORY

NOTE - if you backup media had multiple, appended backups, then you may also need to use the WITH FILE = option. Once done, using the EM restore option, we select the database and work off the restore history to pick the best plan for restoration.

Remember, before restoring, always double check the database name and the options, ensuring paths and names are correct.

Restore cannot fit on disk

This is a classic problem. Basically, your attempt to restore a backup results in an out of space error and asks you to free more space before re-attempting the restore. In this particular scenario, SQL Server wants to restore the database files to the same size as at the time when they were backed up. There is no option to alter the physical file size (i.e. shrink) during the restore. The general recommendation here is the shrink the database before any full backup to reduce the possibility of this error. If that doesnt work, try and restore and move files as best you can to distribute the space amongst many disks, then shrink after the restore, backup and try to restore again with a more appropriate placement of database files.

Restore uses logical names

In the examples presented below, the restore operations work over the logical name for each database file being restored (where this is appropriate of course). If you do not know the logical name of the physical files or the name of the file-group, then you will have some problems successfully restoring. Apart from using the GUI, we can execute the command:
restore filelistonly from disk='c:\mydb.bak'

Also review the commands:

96

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

restore headeronly from disk='c:\mydb.bak' restore labelonly from disk='c:\mydb.bak'

Restoration by Example
Restore Master Database
Restoring the master database is not fun but does occur in rare circumstances. In this scenario we need to restore back to the last full backup of the master database as a variety of logins have disappeared and some configuration changes have been made, so we are sure that the restore will assist in resolving the problem. 1. Backup the existing master database and verify the backup i. Copy the back to another server and also to tape where possible 2. Attempting to restore from EM will give you this:

3. Kick off all users, and shutdown the instance. 4. Alter the service properties to force instance startup in single user mode by entering m in the startup options for the service.

5. Leave the service window open 6. Run EM, connect to the instance and open the restore database dialog for the master database. Here, we have selected the backup to be restored and ensured beforehand that the file is ready and available.

97

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

7. On successful restore, the following dialog is shown. Go back to the service control window and remove the m single user mode option and re-start the service.

8. Close and reopen EM, connecting to the instance. Check the SQL Server error logs on failure to start the instance. This example is simplistic and there are scenarios where this operation can create further problems. The key issue here is that the master database includes a variety of system tables, with the file paths for the model and msdb and tempdb system databases. If you restore the master (which stops your instance immediately), and attempt to re-start, unless those paths are still valid, the instance will not start. Consider the rebuildm.exe command (rebuild master) to assist in restoring back to a state where at least the instance starts and then you can recover each system database thereafter.

98

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Restore MSDB and Model Databases

For a system database, this is simplistic and painless. The DBA must shutdown SQL*Agent before attempting a restore. Once done, double check via exec sp_who2 and connections to the MSDB database, they must be disconnected before attempting the restore. Restoring the MODEL database is like any other user database. The DBA should restore MODEL before MSDB.

Suspect Database
A database may become suspect for a variety of reasons, such as device errors, missing files etc, or another process (like a 3rd party backup program) has a lock on the database files during instance startup etc. Within EM you will see this:

First of all, check the error logs to better gauge the extent of the problem. In this particular case the error is:
Starting up database 'Northwind'. udopen: Operating system error 32(error not found) during the creation/opening of physical device C:\Program Files\Microsoft SQL Server\MSSQL$MY2NDINSTANCE\data\northwnd.mdf. FCB::Open failed: Could not open device C:\Program Files\Microsoft SQL Server\MSSQL$MY2NDINSTANCE\data\northwnd.mdf for virtual device number (VDN) 1.

If the physical device is missing, then a simple restore is required with the move option. This is assuming we cannot quickly resolve the error otherwise. The DBA may need to use a third party utility to determine if another process has the file open, there are many available on the internet for example www.sysinternals.com. If the file is open but the process is orphaned for whatever reason, we can attempt: a) Attempt to kill off the rough process holding the file open and stop/start of the instance b) Attempt to run: exec sp_resetstatus northwind c) Reboot of the server WARNING Before attempting any recovery or change of database status, always shutdown the instance and backup the database files.

99

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

On any change of DB status related to recovery, the DBA should run the following on the database :
dbcc checkdb dbcc newalloc dbcc textall

IMPORTANT - I should iterate that suspect databases must be carefully analysed, in some cases I have found that, for some unexplained reason (i.e. no error log entries) the instance starts and a user database is in suspect mode. If you have verified the existence of all database files, then attempt to re-attach the database via the sp_deattach and sp_attach commands. Always backup the database files before attempting any sort of recovery. The DBA may also consider deattaching the suspect database (via EM is fine). Go to your file system, move the missing files if need be and return back to EM and run the attach database wizard. In the wizard window, you will see red crosses where the file name/path is invalid, alter the path/filenames, set the attach as and set the owner to dbo (very rare that is not dbo) and the database should be successfully re-attached and operational.

Database is in Loading Mode ?

The DBA may see something like this:

This typically occurs when the database has been restored to an inconsistent state in which it is still pending full recovery. Attempting complete recovery may give you something like:
restore database nonefs with recovery Server: Msg 4331, Level 16, State 1, Line 1 The database cannot be recovered because the files have been restored to inconsistent points in time.

Verify your order of restore carefully before attempting the restoration again.

Restore with file move

Here is a simple example:
RESTORE DATABASE [nonefs] FROM DISK = N'C:\aa.bak' WITH FILE = 2, NOUNLOAD ,

100

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

STATS = 10, RECOVERY , MOVE N'nonefs_Log' TO N'f:\nonefs_Log.LDF'

Restore a specific File Group

Database: mydb File-group name mydb_primary mydb_data mydb_index N/A Backups: C:\mydb_full.bak C:\mydb_log1.bak C:\mydb_diff1.bak C:\mydb_log2.bak C:\mydb_log3.bak {failure occured} Physical file-name c:\mydb_system.bak c:\mydb_data.bak c:\mydb_index.bak c:\mydb_log.ldf

Full Log Differential Log Log

If mydb_data file-group failed (the logical name of the filegroup and the logical name of the file are the same in this case), we need to restore:
-- File Group from FULL backup RESTORE DATABASE [mydb] FILE = N'mydb_data', -- logical name of the file in the FG FILEGROUP = N'mydb_data' -- thisis optional if only 1 file in the FG FROM DISK = N'C:\mydb_full.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, NORECOVERY -- Log backup @ time 1, restore logs as normal RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log1.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, NORECOVERY -- Log backup @ time 2 RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log2.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, NORECOVERY -- Log backup @ time 3 RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log3.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, RECOVERY

Adding or Removing Data Files (affect on recovery)

Following on from the above, we consider the situation where a database file has been added to the database between transaction logs. Therefore we have this scenario: Backups:
C:\mydb_full.bak Full backup

101

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

C:\mydb_log1.bak Log backup -- new file added to database ALTER DATABASE mydb ADD FILE ( NAME = mydb_newfile, FILENAME ='c:\mydb_newfile.mdf', SIZE = 1MB, FILEGROWTH = 10% ) GO C:\mydb_log2.bak {failure occured} Log backup

To restore we need to:

RESTORE DATABASE [mydb] FROM DISK = N'C:\mydb_full.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, NORECOVERY RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log1.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, NORECOVERY RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log2.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, RECOVERY

The completed restore will show the newly added file with no further issues. Be aware though, Microsoft Support document Q286280 states otherwise, and there may be a scenario where the above does not work. Revise this support document for assistance.

Emergency Mode
This mode is undocumented and is technically unsupported, but is required on very rare occasions. This mode allows the DBA to access a database without the log file being present.
-- Allow updates to sys tables exec sp_configure N'allow updates', 1 reconfigure with override -- If possible, attempt to set db in DBO only access mode (for safety sake) exec sp_dboption N'Northwind', N'dbo use', N'true' -- Record the existing record entry for the database SELECT * FROM master..sysdatabases WHERE NAME='northwind' -- Set DB into emergency mode UPDATE master..SYSDATABASES SET STATUS=32768 WHERE NAME='northwind' Stop and Re-start MSDTC. -- Refresh Enterprise Manager

102

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Attempting a backup or any other operation that uses transactions will result in the error:

To export out the data and associated objects, create a blank database in the same or another database instance. Once done, run the Export wizard, select the database in emergency mode and follow the prompts. A DTS will be created and will happily export the database, typically, without error so long as there are no underlying permission issues. Drop the source database as need be. This is a very simplistic example but provides some direction towards dealing with the problem. NOTE Setting a database to emergency mode is very handy when suspect databases wont allow you to investigate the problem via DBCC commands etc. Altering the status to emergency mode and then running, say, DBCC CHECKDB will allow you access to the database and a variety of commands to resolve the problem.

Restore Full Backup

A full database restore can be as simple or as complex as you like. I very rarely backup direct to tape or individual file groups as I have always had the free disk space. For user databases, I tend to opt for EM as its simple and quick. Before restoring always check: a) can I backup the database before the restore? (i.e. yes) b) notification of end-users and killing sessions c) database name d) location and name of the files e) remembering to fix orphaned logins if restoring to another server f) re-checking the database recovery model and associated options g) verifying subsequent backups will still operate as per normal h) always write down in a log what you did, why and the files used. No example is required for this scenario.

103

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Partial (stop at time) PITR Restore on a User Database

To restore to a point in time, ending at a specific transaction log backup in your backup sequence, we use the STOPAT command, for example:
RESTORE LOG [mydb] FROM DISK = N'C:\mydb_log2.bak' WITH FILE = 1, NOUNLOAD , STATS = 10, RECOVERY , STOPAT = N'8/08/2002 9:42:02 PM'

Use the GUI, or the commands:

restore headeronly from disk = 'C:\mydb_log1.bak'

restore headeronly from disk = 'C:\mydb_log2.bak' and the backupfinishdate column to determine the most appropriate log file to be used.

Corrupt Indexes (DBMS_REPAIR)

The DBA should be regularly running the following against all databases: DBCC CHECKDB DBCC TEXTALL DBCC CHECKCATALOG DBCC CHECKALLOC These routines will report on allocation inconsistencies with tables and indexes that typically point at data corruption. Even so, dont be too quick to react. Before doing anything always full backup the existing databases and try the following: DBCC CHECKDB(mydatabase, REPAIR_REBUILD) a. b. c. d. Kill off all users or wait till they disconnect exec sp_dboption 'northwind', 'SINGLE_USER', 'on' DBCC CHECKDB('northwind', REPAIR_REBUILD) exec sp_dboption 'northwind', 'SINGLE_USER', 'off'

b) As above, but try DBCC CHECKALLOC IMPORTANT Do not use dbcc dbrepair If you are getting desperate, Microsoft has an undocumented command (typically suggested by Microsoft support) called sp_fixindex. Before running this restart the instance in single user mode, checkpoint, run sp_fixindex, checkpoint again and backup once more. Re-start the instance and rerun the DBCC routines once more. See Microsoft support document Q106122 for more information.

104

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Reinstall NORTHWIND and PUBS

Run the scripts found in the /install directory for the instance: Instnwnd.sql Instpubs.sql

Other Recovery Scenarios

Scenario 1 - Lost TEMPDB Database

If you delete the tempdb and templog databases files, they are simply re-created on instance startup, assuming of course the model database is available and the disk sub-system:

It is created based on the entry in master..sysdatabases

use tempdb go sp_helpfile go

The DBA can move this location via the commands below and re-starting the instance.
use master go alter database tempdb modify file (name = tempdev, filename = 'c:\tempdb.mdf') go alter database tempdb modify file (name = templog, filename = 'c:\templog.ldf') Go File 'tempdev' modified in sysaltfiles. Delete old file after restarting SQL Server. File 'templog' modified in sysaltfiles. Delete old file after restarting SQL Server.

Note that after the alter statement, the entries in master..sysaltfiles, master..sysdatabases and master..sysdevices remains unchanged. On restart, the tempdb files have now moved to their new location but the entry in master..sysdevices remains unchanged, only sysaltfiles and sysdatabases has been altered. If the device is no longer available in which the tempdb datafiles are created, the instance will not start, as there is no other default value SQL Server will magically use. To resolve this problem we need to use the rebuildm.exe (see Scenario 2.)
Scenario 2 - Rebuildm.exe

105

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

There comes a time in every DBAs life where the rebuildm.exe (rebuild master) utility is used, either to change the instances global collation or due to a disaster in which one or more system databases need to be restored and we dont have a valid or any full backup (this should never happen for any reason). The rebuildm.exe is found on the installation CD, cd-rom:\x86\binn. In the following example we will run the command and highlight the subsequent steps to complete the recovery. NOTE If copying the CD to disk, make sure the files in ?:\x86\data\ are not read-only or have their archive bit set. When using disk 2 and running rebuildm.exe, I received the following error:

this was an authenticate CD, to get around this unforseen problem I copied it to disk and renamed the directory c:\x86\binn\res\1033 to c:\x86\binn\Resources\3081. The utility then ran without a problem. REMEMBER DONT restore your master database after running rebuildm.exe if the objective was to alter the server collation. Always backup as much as possible, and consider scripting logins before attempting this process.

106

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

1. Shutdown the instance we plan to rebuild 2. Run the rebuildm.exe from the CD-ROM or do the above and copy to disk (not a bad idea generally during emergency recovery scenarios). The following dialog is shown:

The instance whose system databases will be restored over.

Source of the CDROM default database files. The DBA can also set the new collation.

Default data directory for the installation, this can not be altered.

3. Press the rebuild button and respond yes to the prompt

4. The database files are copied to the new destination and the server configuration progress dialog is shown, this takes around 1-2mins maximum.

Try This Run FileMonitor from www.sysinternals.com to view the file IO and thread calls during this process. 5. Dont be fooled, this process affects ALL system database, not just the master database.

107

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

6. Check data file properties before re-starting to ensure they are not read-only. 7. Start your instance 8. Review the previous and current error log. The previous log has some good information about the tasks undertaken with the system databases rebuild. 9. Optionally re-apply service packs 10. Optionally restore your master, model, msdb databases as need be Before your re-start the instance with the files copied by rebuildm.exe, double check they are not read-only. This is a classic problem when the files are copied off the CD-ROM. If this problem affects the use of rebuildm.exe then copy the files to disk and refer to point two above. Be careful when only restoring one or two of the system databases. All system databases should be current with and single service pack, I have never been in a position where a subsequent restore of the master database that had SP2 applied existed with the MSDB database with no service packs. The DBA should think very carefully about this and apply the service pack as required to ensure minimal amount of error.
Scenario 3 Lost all (or have no) backups, only have database files

To recover from this scenario: 1. Backup all database files (if available) to another server and/or to tape. 2. Check the registry for the MASTER database, and alter and/or ensure the files are in the correct registry entry HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Microsoft SQL Server/<instance name>/MSSQLServer/Parameters/{SQLArg0 and 1} 3. Attempt to re-start the instance i. If there are still errors with system databases, namely MSDB, MASTER or MODEL, check error log carefully and attempt to place database files at these locations. ii. If you have no luck, run rebuildm.exe (see previous scenario) 4. The instance should successfully start 5. For MSDB database i. Shutdown SQL*Agent service ii. Drop MSDB database

108

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

iii. Reattach from your original database files 6. For each user database re-attach database files 7. Fix orphaned logins as need be (if any) 8. Run DBCC checkdb and checkalloc against all databases 9. Check database recovery models 10. Backup databases The DBA should revise trace flags on instance startup to assist in the task.
Scenario 4 - Disks lost, must restore all system and user databases from backup to new drive/file locations

This is a difficult scenario. In order to start the instance, we require a valid master database; this database also defines the subsequent location of the MSDB, MODEL and TEMPDB database data files. If we restore the master database from our full backup (with the move option to another disk), the sysdatabases, sysaltfiles and sysaltdevices system tables will still contain invalid paths for the other system and user databases as we lost those particular disks. This is made even worse, as any time you restore the master database the instance shuts down immediately, therefore, an instance re-start will result in numerous file-missing errors and fail to start. This may bring mixed comments from DBAs, but consider the following: 1. 2. 3. 4. 5. 6. 7. 8. 9. Run rebuildm.exe to restore system databases onto new disk(s) Recover MSDB database from last full backup Recover MODEL database (if very different from the original) Restore master database from full backup and master_old Alter system to allow changes to system tables Transfer contents of syslogins from master_old to the master database Re-start instance Check system error log Recover user databases from full backups NOTE SQL Server has no equivalent command to the RESETLOGS option when SQL Server opens a database.

109

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

Transactions and Locking

ne of the most important concepts to grasp for any DBA, is transactions and transaction management. In SQL Server, one of the biggest problems related to overall system performance and stability is the poor handling of transactions, especially long running ones. Throughout this chapter we will discuss this is in depth. If you walk away with nothing else from this chapter, remember that it is critically important for your OLTP systems under SQL Server to keep transactions as short as possible to ensure blocking and lock escalation is kept to an absolute minimum.

Rollback Segments & Redo Logs

Oracle - Summary of Undo Management / Rollback Segments

Oracle has made some inroads in rollback (undo) management with 9i (due to mis-management issues by DBAs and new 9i options), moving away from rollback segments within a rollback segment tablespace, to an undo tablespace with automatic undo management. Before we look at SQL Servers version of undo management (read-consistency), let us revise the architecture of Oracle. The undo / rollback segment allows the Oracle DBMS to provide read-consistency for all transactions and is therefore essential for a transaction to execute and subsequently complete (commit/rollback). The DBA needs to specify the undo management strategy for the database , that being either: a) Manual Undo Management (rollback segments and tablespace) b) Automatic Undo Management (AUM) NOTE - To distinguish between the two types of undo segments, ROLLBACK segments are called UNDO segments when AUM is enabled. At the start of a transaction, the DBMS assigns it a rollback segment and records the current SCN (last completed commit point). As the transaction continues with any variety of DML, the DBMS uses the rollback segment to: store before image of altered (updated or deleted) data newly inserted data

110

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

assist the DBMS in constructing a read-consistent view of the database objects as at the start of the transaction (defined by SCN). To do this, the DBMS will lookup existing and other rollback segments to construct a read-consistent view. This effectively allows transactions to be completely segregated from one another, to do whatever they like to the data as at the SCN and when ready, commit or rollback. On the commit of the transaction, the DBMS will utilise the buffer cache and rollback segments to complete the transaction. At this point, locking is required to complete the transaction. All transactions in Oracle require an explicit commit or rollback; the developer must issue one of the two statements (or end their session to rollback automatically) to end the transaction, which is started when the first DML statement is issued. All DLL is implicitly committed in Oracle and SQL Server. The user (programmer) may issue save point throughout the transaction to rollback to specific points in time. The type of undo method is used within the DBMS can be verified with:
select from where name,value v$parameter name in ('undo_management','undo_tablespace');

Undo Management Manual (Rollback Segments)

Rollback Segment Tablespace

Transaction table Transaction table

Rollback Segment

Rollback Segment

Active Extent (transaction update, insert etc..) they are either idle, active, passive, in doubt distributed. Transactions cannot cross more than one segment. Programmers/DBA can write code to target a segment for the transaction to run under.

OFFLINE ONLINE INVALID PARTLY AVAILABLE NEEDS RECOVERY

NEXT EXTENT SIZE MINEXTENTS MAXEXTENTS OPTIMIAL SIZE

Non-active extent within segment.

ROLLBACK_SEGMENTS in init.ora specify online segments on instance startup

111

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Undo Management Automatic (AUM)

The concepts are the same in principal, but the DBA has no control over the sizing, online/offlining (altering status) and number of segments within the undo (rollback) tablespace on startup. There are a range of new database startup parameters and subsequent DDL to create and setup the new undo tablespace. This paradigm of self-managed DBMS smarts is collectively known as Oracle Managed Files.

Undo Tablespace
Undo segments managed by the DBMS

undo_management undo_tablespace undo_retention, undo_pool

On setting the startup parameter undo_managagement = auto, undo segments are managed automatically within an undo tablespace. The segments are removed when the tablespace is removed, online/offlined as the undo tablespace changes, and segments completely managed by the DBMS. The DBA does not explicitly create or control undo segments (rollback segments), the DBA only specifies: one or more undo tablespaces (undo_tablespaces=), and the associated create undo tablespace command. Undo tablespaces may only hold undo segments. only one undo tablespace can be used at any one time. undo_retention parameter, clock time to retain of committed undo data in the tablespace. Required for the flashback option in 9i. undo_pool multiple pools with restrict undo properties can be created and allocated to users to restrict usage. v$undostat to monitor undo transactions, space consumed etc v$rollstat monitor automatic segment statistics NOTE The Oracle 9i flashback option requires AUM.

Oracle Redo Logs

For redo management Oracle utilises two or more redo log groups, consisting of one or more physical files (members) per redo group. As we know, the redo logs store all changes (there are some variations to this) made to blocks in the buffer cache and are key for recovering committed

112

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

data that has yet to be written to the actual database files. The DBA has full control over the positioning, sizing, onlining / offlining, switching, limiting number of and the associated archive log process. The DBA must archive logs to prevent the database locking up and is the key to PITR (point in time recovery). Oracle supplies the log miner tool to analyse database activity via the reading of redo and archived redo log files. This tool is implemented as a series of PL/SQL packages, views, utilities to the read associated redo log dictionary objects, and the associated java based logminer GUI tool.

Oracle Locking
Where possible, both Oracle and SQL Server attempt to allocate locks at the lowest level possible (that being at the row). Also, like both DBMSs, shared and exclusive locks exist, in which one and only one exclusive lock can be taken out on a resource at any one time, whilst multiple shared locks are possible. The only key difference here was described previously, where in Oracle reads are never blocked by exclusive locks, in SQL Server they will be (especially update locks described next). All locks are held for the duration of the transaction as with SQL Server, which is completed via a commit or rollback command (optionally to a save point as well). Oracle will not escalate locks, but will do the appropriate lock conversion to obtain the most appropriate lock at the highest level as possible (that being a row-exclusive lock - RX). The locks themselves are maintain via enqueue processes in which the DBA via enqueue_resources can control this internal locking structure. Oracle includes the following lock types (may not be definitive): a) DDL a. Protect the schema objects and their definition. b) DML a. Ensures only one row can be updated at any one time. b. Objects cant be dropped whilst transactions are pending c. Row or table level i. TX row level lock 1. readers do not wait for writers for the same data 2. writers not wait for readers (for update excluded) 3. writers wait for other writers over the same row 4. table level shared lock is also taken. ii. TM table level locks 1. RS (SS) row share 2. RX (SX) row exclusive 3. SRX share row exclusive lock 4. X exclusive c) Latches d) Distributed (OPS) e) PCM (parallel cache management) used in the shared server configuration.

113

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The compatibility for lock conversion/acquiring matrix is shown below as summarised on Oracle Metalink: Mode Held NL SS SX SSX X NL Ok Ok Ok Ok Ok SS Ok Ok Ok Ok Fail SX Ok Ok Ok Fail Fail S Ok Ok Fail Ok Fail SSX Ok Ok Fail Fail Fail X Ok Fail Fail Fail Fail

The type of SQL statement will also determine the lock modes permitted against the resource (out of scope for this book).

SQL Server - Undo & Redo Management Architecture

The previous two sections very briefly covered rollback and redo within the Oracle RDBMS. This is an important review in order to understand the SQL Server architecture and bring the concepts of Oracle into context. The key to rollback and redo for SQL Server is the transaction log that is present in each database within the instance. The transaction log is a serial record of all transactions (DML and DDL) executed against the database and is used to store: a) b) c) d) e) start of each transaction before and after changes made by the transaction allocation and de-allocation of pages and extents commit and rollback of transactions all DDL and DML

The transaction log itself consists of one or more physical database files, the size of the first must be greater than or equal to 512Kb in size. SQL Server breaks down the physical file into 2 or more logical transaction logs. The size of the file and its auto-growth settings will influence the number of virtual logs and their size. The DBA cannot control the number of or sizing of virtual logs.
Physical transaction log file (min size 512Kb with 2 virtual logs) Virtual Log File (min 256Kb)

The log writer thread manages the writing of records to the transaction log. As pages are requested and read into the buffer cache, changes are sent to the log-cache as a write-ahead operation which log-writer must complete with a write to the transaction log before the committed change is written to the data files as part of a check-point. In terms of an update, the before and after images are written to the log; a delete the before image is written; an insert tracks only the new records (not including many other record entries as mentioned previously as part of the transaction). The log-writer allocates a LSN (log sequence number, similar to the as the SCN in Oracle) to each statement including the transaction ID.

114

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Log Cache

Log flushed, LSN allocated and transaction id stamped with before/after images of the records updated.

Physical transaction log file

Write-ahead log cache, must complete before buffer cache pages written back to physical data files.

Buffer Cache

Dirty buffer pages, log-write requested

Select and update of database record Database Data Files

Checkpoint completes - writes back to buffer cache entries to data files only after log write is compete.

The transaction entries (committed and uncommitted) in the transaction log are doubly linked lists and each linked entry may contain a range of different (sequential) LSNs. The transaction itself remains in the active portion of the transaction log until it is either committed or rolled back and the checkpoint process successfully completes.

Physical transaction log file Doubly linked list log-entries for the transaction.

Free space in virtual logs

Active Portion of Log

Although not shown above, space is also allocated for each log entry record for rollback purposes; therefore actual space utilisation can be significantly more than expected. Microsoft defines the active part of the log to be the portion of the log file from the MinLSN to the last written log record (end of logical log).

115

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Cyclic and serial transaction log

End/start of next virtual log file

Free virtual log space

BEGIN TRAN T1 LSN 112

UPD TRAN T1 LSN 113

BEGIN TRAN T2 LSN 114

COMIT TRAN T1 LSN 115

CHECK POINT

DEL TRAN T2 LSN 117

LSN 116

- active portion MinLSN Last Log record

The MinLSN is the first of possibly many yet uncommitted (or rollbacked) transactions. When the database is using a full or bulk-logged recovery model, the non-active portion of the log will only become free (can be overwritten) when a full backup or transaction log (archived log backup in Oracle) backup is executed. This ensures that recovery is possible if need be via the backup and the DBMS can happily continue and overwrite the now free log space. If the database is using the simple recovery model at a database checkpoint, any committed (and check pointed) or rollback transactions log space will become immediately free for other records to use. Therefore, point in time recovery is impossible. The checkpoint process is key to completing the committed transactions and writing the dirty buffers back to disk. In relation to the transaction log, this will: a) write log entry for the start of the checkpoint b) write the start LSN for the checkpoint chain to the database book record for subsequent recovery on instance failure c) write list of active (outstanding) transactions d) write all dirty log and data pages to disk e) writes a log file record marking the end of the checkpoint This is far from definitive but you get the idea. The checkpoint will occur: a) b) c) d) On issue of the CHECKPOINT or ALTER DATABASE statement On instance shutdown (SHUTDOWN statement) On service shutdown Automatic checkpointing a. DBMS calculates based on the recovery interval setting b. Fullness of the transaction log and number of transactions c. Based on timing set with recovery internal parameter d. Database using simple recovery mode? i. If becomes 70% full ii. Based on recovery interval parameter

116

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Apart from the recovery interval and recovery mode parameters, the DBA has little control of checkpointing. Although, it can be forced via the CHECKPOINT statement if need be. To drill into the virtual logs and the position, status, size, MinLSN within the transaction log files, use the commands: dbcc loginfo dbcc log(<db#>, TYPE=-1) NOTE The transaction log is a write-ahead log. Log writes are synchronous and single threaded Actual DB writes are asynchronous, multi-threaded and as shown in the diagrams, optimistic.
In Summary

Fundamentally, the two DBMS are very. The key differences are primarily highlighted with the rollback segments, which effectively results in writers blocking readers with its default isolation level (read-committed as with Oracle) as a database change cannot be dynamically re-constructed to a point in time. As we will see next, the result is a more complex locking mechanism, more transaction isolation levels and associated hints, and programmers needing to be much more strict in keeping transactions as short as possible. The following sections discuss locking and isolation levels for transaction management. In terms of redo, the transaction log is the equivalent to the redo logs in Oracle. The DBA can only control the number of logs used and cannot manipulate the virtual logs within them. As with Oracle, the DBA needs to archive logs (backup log command) on a regular basis to prevent the database from stalling. Functionally, the redo logs are identical in terms of their requirement for database recovery.

117

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Locking & Isolation Levels

The locking mechanism for SQL Server is complex to cater for the lack of rollback segments. In SQL Server, locks, lock escalation and the use of additional isolation levels to maintain readconsistency (and concurrency) results in the possibility for writers to block readers, and the increased possibility of deadlocking issues or long sustained blocking. Because of this, programming against a SQL Server DBMS and an Oracle DBMS is somewhat different in terms of transaction longevity. The transaction isolation levels are:
SQL Server Read Uncommitted Read Committed (default) Repeatable Read Serializable N/A Oracle N/A Read Committed (default) N/A Serializable Read Only (ANSI)

The isolation is very important in determining transaction concurrency (enforcing atomicity and consistency in relation to other running transactions) and determining the locking required by the DBMS to meet the isolation level. The SQL Server DBA does not explicitly set the isolation level and cannot control a minimum global setting or restricted set of isolation levels that can be set against the DBMS. The programmer can alter the databases default isolation level on connect (ADO, OLEDB, ODBC property), T-SQL or via a SQL hint. The T-SQL command is: SET TRANSACTION ISOLATION LEVEL [ READ COMMITTED, READ UNCOMMITTED REPEATABLE READ, SERIALIZABLE ] You can view the isolation level via the command: DBCC USEROPTIONS NOTE - that isolation level property is not shown if the default is in use. The SQL Server locking schemes are completely automatic in context with the isolation level, with lock escalation as need be to fulfill the request (Oracle never escalates locks). Where possible, SQL Server will start at the lowest granularity of locking and in terms of exclusiveness, then use a variety of other lock types at higher levels to pre-empt future intentions and escalations on the underlying object. The lock types are divided into four modes: INTENT (I) future intention to acquire other locks SHARED (S) allows many concurrent reads, blocks updates until share is released, there is always 1 shared lock on the DB for the connection. UPDATE (U) prior to data modification, is upgraded to exclusive on change EXCLUSIVE (X) complete lock of the resource

118

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SCHEMA (Sch-[M,S]) Schema modifications, DDL, schema stability lock BULK UPDATE (BU) Bulk copy of data (bcp) with TABLOCK hint RANGE[I]_[S,U,X,N] Key range lock, used in serialized transactions, covers index keys and the ranges of the index rows. And of course combinations of the above, such as: IS IX SIX Intent shared Intent exclusive Shared with intent exclusive

The resources in which locks are applied (and escalated in order of) include rows, pages, keys (index keys), range of keys, indexes, tables, and databases. These are of course abbreviated: RID PAGE EXT TAB IDX FIL KEY DB Row Identifier Page (data or index) Extent (8 contiguous pages, data or index) Table Index File Key (row lock with index) Database

The status of the lock request against the resource may be: GRANT WAIT CNVRT Lock obtained Blocking due to another process In processes of being converted (escalation)

Again, the duration of the lock will depend on the isolation level, hints, DML/DDL operation being performed and the range of rows affected by the operation (some internal threshold being exceeded that results in escalation). The DBA can do very little to control the lock manager, the best we have (apart from isolation levels and hints) is the lock option altered via the sp_configure stored procedure. The default option is to let SQL Server manage the maximum number of lock structures (96Kb each), which in 99% of cases is the best option. It is difficult to describe and summarize every example of locking, the DBA really needs to explore the sp_lock procedure with a variety of different isolation levels and DML/DDL to view the locks taken out not only against the database, but also against the master and tempdb databases. Here is a short example:

119

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Table definition:
CREATE TABLE [mytable] ( [col1] [int] IDENTITY (1, 1) NOT NULL , [col2] [varchar] (50) COLLATE Latin1_General_CI_AS NULL , CONSTRAINT [PK_mytable] PRIMARY KEY CLUSTERED ( [col1] ) ON [PRIMARY] ) ON [PRIMARY] GO User SQL: set transaction isolation level serializable

begin transaction select * from mytable exec sp_lock @@spid commit transaction

Example Results:
set transaction isolation level [repeatable read or serializable]

select OBJECT NAME(1173579219)

It should be noted that KEY locks are shown when the table has a clustered index rather than showing rowlocks. For non-clustered indexes the KEY locks exist for the leaf nodes. Locks may be retained until the transaction is complete; this is especially the case with shared locks over a repeatable read isolation level. At the end of the day, the DBA must be very strict on the timing of transactions, the operations being performed (DDL, DML), objects being affected (multiple indexed tables etc), use of triggers, and keep the transactions as short as possible.

120

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

NOTE There is no equivalent to Oracles IDLM (integrated distributed lock manager) as used in the OPS (Oracle Parallel Server) environment too coordinate parallel access to DBMS resources. With this, there are no equivalent LCKn processes used for inter-instance locking in OPS.

Transactions
Drilling now into transactions, the syntax is similar to Oracle in some respects. The key difference here is the in-ability for SQL Server to pause or re-start transactions as we can in Oracle. The syntax is straight forward enough:
BEGIN TRAN[SACTION] <SOME CODE HERE> [COMMIT, ROLLBACK TRANS[ACTION]]

The transactions can be nested, be aware of the rollback transaction command as it does not give you the result most would expect:
begin transaction print @@trancount begin transaction print @@trancount begin transaction print @@trancount commit transaction commit transaction print @@trancount commit transaction begin transaction print @@trancount begin transaction print @@trancount begin transaction print @@trancount rollback transaction commit transaction print @@trancount commit transaction

Transaction Count 1 2 3 1 0

1 2 3 Server: Msg 3902, Level 16, State 1, Line 9 The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION. 0 Server: Msg 3902, Level 16, State 1, Line 11 The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION.

In SQL Server 2k we can also mark transactions and set save points to rollback to. Marking transactions can be very handy when tracking long running jobs and ease the rollback process on transaction failure. The syntax is simple: The transaction log marks are held in the MSDB database table logmarkhistory.

121

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Monitoring Locking Issues

The DBA should review the following commands via the books online: sp_lock get some reasonably detailed information about locks and spids @@SPID get sessions current spid sp_who2 get list of all sessions connected to the instance sp_helpdb get summary of the database in the instance dbcc inputbuffer(<spid>) - what is a session running/doing now or last statement run sysprocesses table - check the block, cpu and physical IO columns profiler your key tool for session tracing enterprise manager general database administration tool dbcc stackdump - will generate a low level dump of processes,.dmp file is generated in the log directory of the instance The sum of the commands above is similar to utllockt.sql and catblock.sql scripts in Oracle (not including EM and other 3rd party tools). In terms of DBMS_LOCK, apart from table level SQL locking hints, there is no equivalent to this package. Third party tools vary greatly in function and actual usefulness and like all things, require a lot of evaluation and pre-thought before handing over your money. As an example in terms of block/locking issues, Diagnostic manager from NetIQ returns to me SMTP emails:
2838: The monitor has determined that process ID 11 has been blocking another process for longer than 30 seconds. The process is being executed by program 'SQLDMO_1', on computer 'MY-SERVER', by user 'sa'. The blocking lock's details are : Database - msdb, Table ID - 213575799, Lock Type Blocking Intent Exclusive. The statement being executed is : dbcc checkdb WITH NO_INFOMSGS 16/08/2002 5:57:12 PM [MY_DB]

Within Enterprise Manager (EM), we have a small number of options. Some example screen shots of EM and viewing blocking issues are shown below:

122

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Take a careful look at the lock modes, namely X and U or ranged exclusive locks.

Like all locking, most are short and point in time events that at times can be hard to trace. This is where profiler comes into play. Be aware that EM can take some time to refresh and at times the DBA thinks its locked itself, only to return back with the results some minutes later. Here is a classic error; right at the time we are trying to sort out the blocking issue:

123

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The blocking locks can be a real problem with SQL Server, even more so than deadlocking. The DBA may really struggle to pin point the actual problem, as both profiler and DBCC input buffer might not return the statement being blocked. As described by Q224453, Blocking is an unavoidable characteristic of any relational database management system (RDBMS) with lockbased concurrency. On SQL Server, blocking occurs when one SPID holds a lock on a specific resource and a second SPID attempts to acquire a conflicting lock type on the same resource.. Here are some general pointers to resolve the problem: Attempt to repeat the problem on a quiet server in which tracing the blocked and blocking SPIDs and locks is much easier. The developer will have an idea as to the code being executed, discuss the code and consider a mini walkthrough (demand it if necessary), are we taking about COM+ with transactions? How are transactions being used? Isolation levels? Are hints being used? Attempt to trace and filter over the key tables involved with the lock. Rather than using profiler, consider command line trace via sp_trace in Query Analyser to ensure events are not missed in very busy databases. Consider dbcc opentran to analyse long running open transactions that may be causing the blocking and allow more effective (filtered) tracing via profiler.

Controlling Locks
Locks can be controlled at a SQL level via hints, I dont like hints and am never comfortable with them, but there are cases where they come in handy (especially bulk data loading and concurrent data loading/selecting). The hints include: HOLDLOCK NOLOCK PAGLOCK READCOMMITED READUNCOMMITED REPEATABLEREAD READPAST ROWLOCK TABLOCK TABLOCKX UPDLOCK XLOCK

124

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

For example: Select * from mytable (tablockx)

select a.lname, b.job_desc from employee a (xlock), jobs b (tablockx) where a.job_id = b.job_id

Hints are described in more detail later in this book. Apart from SQL hints, the DBA can use the sp_indexoption to set locking options at an index level. The options are somewhat limited but include: AllowRowLocks AllowPageLocks DisAllowRowLocks DisAllowPageLocks FALSE FALSE FALSE FALSE or or or or TRUE TRUE TRUE TRUE

The default is to allow row locks where possible. An example is: exec sp_indexoption 'mydb.mytable', disallowpagelocks', TRUE Do not use this option unless there is a very good reason too.

125

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Finally, we can set the isolation level (and thus control locking), via the begin transaction command. For example: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ In terms of lock timeout, the developer may use the LOCK_TIMEOUT command. Where possible the DBA should discourage (i.e. DONT USE) its use and opt for a more elegant solution. Here is an example of its use:
Session 1 begin transaction update authors set au_lname = timeout test' where au_id = '238-95-7766' waitfor delay '000:00:10' rollback transaction Session 2 SET LOCK_TIMEOUT 5000 begin transaction update authors set au_lname = timeout test' where au_id = '238-95-7766' rollback transaction Server: Msg 1222, Level 16, State 50, Line 1 Lock request time out period exceeded.

Without it, session two will wait indefinitely until the session one completes.

Detecting and dealing with Deadlocks

The NT performance monitor is a good start to determine the extent of the problem. We use the counter:
SQLServer:Locks \ Number of Deadlocks\sec

Ideally its value is zero and/or a rare event. There are situations where this is difficult, especially third party applications or your OLTP database that is also being used for reporting and other batch type events out of your control. The DBA should follow up with SQL Profiler to better trace the deadlocks occurring. Profiler is a powerful tracing tool, but it does have some problems when tracing deadlocks as we will see later. On starting a new trace, the DBA should include the events:
Errors and Warnings Exception Locks Lock: Deadlock Lock: Deadlock Chain

If you stayed with this, and waited for your expectant deadlock to occur, you will get very little information of the objects affected or statements executed unless you select the data column Object Id. From here you need to manually use OBJECT_NAME to determine the object affected. Why is this a problem? to get more information you typically include the event T-SQL: SQL: Batch Completed, if you run the trace with this option then you will be tracing ALL completed batches, and in a busy system, this can mean thousands of entries within minutes; making tracing a difficult

126

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

and time consuming task. Even so, if you can deal with this, you will get a thorough list of statements related to the deadlock; stop the trace after the deadlock occurred and use the find dialog to search the columns for deadlock event, then search backwards from the SPIDS involved in the trace to get a summary of the commands before the deadlock. NOTE Running profiler whilst locking is already underway and a problem, will do you no good, and you may only get a small amount of relevant information about the issue (i.e. profiler doesnt magically trace already running processes before continuing on its way with current events). The client application involved in a deadlock will receive the error# 1205, as shown below:
Server: Msg 1205, Level 13, State 50, Line 1 Transaction (Process ID 54) was deadlocked on {lock} resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

To assist in this circumstance, utilise EM or run the following commands:

exec sp_who2 dbcc inputbuffer (52) exec sp_MSget_current_activity 56,4,@spid=52 -- view all sessions -- get SQL buffer for 52 -- get extended locking information

Finally, the DBA can utilise trace flags. This is an effective method for debugging deadlocks and provides some excellent error log data. The flags are: 1204 1205 1206 3605 Get lock type and current command effected by deadlock Get extended information about the command being executed (eg. graph) Complements 1204, get other locks also participating in the deadlock Send trace output to the error log (optional, will go there anyhow).

The screen shots below illustrate the output from a deadlock with the traces enabled. I have no statistics on the adverse effect to DBMS performance, but this is a very effective method for debugging problem systems that are deadlocking frequently but you can never get a decent set of data to debug it.

127

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The actual deadlock is around the customer and employer tables, in which two processes have updated one of the two separately and have yet to commit the transaction; they attempted to select each others locked resources resulting in the deadlock. This is not reflected in the log dump. The ECID is the execution context ID of a thread for the SPID. The value of zero represents the parent thread and other ECID values are sub-threads. Check with http://support.microsoft.com for some excellent scripts to monitor blocking in SQL Server.
Example Deadlock Trace

We have a large COM+ based application that was experiencing deadlocking issues. The key issue here is that COM+ transactions use an isolation level of serialisable, as such, locks of any sort can be a real problem in terms of concurrency. To start resolving the problem we: a) worked with the developers in working out how to repeat the error a. this allows us to identify the code segments possible causing the error and assist of course with re-testing. b) Set instance startup parameters -T1204 -T1205 -T1206, re-start instance c) Run Profiler a. Filter the database we are concerned with b. Include event classes i. Lock:Deadlock

128

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

ii. Lock:Deadlock Chain iii. SQL:StmtCompleted iv. RPC:Completed c. Include standard columns, namely TextData and SPID d) Run the code to cause the deadlock. Search the profiler trace:

Lock:Deadlock identifies that SPID 67 was killed. Go back through the trace, to locate the commands executed in sequence for the two SPIDS in the chain, take some time with this, you need to go back through the chain of transaction begins (in this case they are COM+ transactions) to clearly determine what has happened for each SPIDs transaction block. To assist with further debugging, goto your instance error log and locate the deadlock dump chain:

IX lock wanting to be taken out Last command from buffer. (stored proc or DML statement

select object_name(918450496) will give you the table name to help identity the possible problem statement to start looking for in the batch of SQL being executed.

Current lock being held

129

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

Performance Tuning
he art of performance tuning requires good theoretical and practical knowledge of the underlying operating system, physical hardware, applications components and of course, the DBMS. Many of these skills can only be acquired with years of practical experience with a variety of servers configurations and, most importantly, applications. This chapter attempts to highlight some of the key items of interest to the DBA.

Tracing (profiler)
Tracing database events and users sessions in SQL Server is done via the profiler utility. This is a powerful tool that allows the DBA to: a) trace a range of database events a. including grouping of events and selectively picking data columns to be used b. apply filtering rules to further define the events you are after c. save output to a file (can set max size) or a database table (set max rows) and optionally enable trace stop time. d. b) edit and define trace templates c) replay saved traces (including break-points and stepping through a trace) d) scripting a trace e) running trace file results through the index wizard f) assist with auditing and of course performance tuning g) error tracing and blocking/locking detection Run the GUI and play with it against a development or test instance. The interface is simplistic and like all things, read the screens carefully and you will find it self-explanatory. The utility, by default, will trace all DML and DDL database activity within the instance you are connecting too. There are numerous events you can trace via profiler, some of which are:
Event Trace Type Cursors Database Summary Cursor prepare open, close, recompiles etc Automatic growth and shrink of data and log files

130

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Errors and warnings

Missing column statistics, missing join predicates, oledb errors, tempdb sorts over memory sorts, exceptions etc. Lock escalation, timeout, deadlocked etc Auto stats collection, objects accessed via DML, objects created and objects dropped or deleted. Execution plan, degree of parallism Table or index scan started/stopped Numerous audit events, including password changes, failed logins, adding and removing role members, adding/removing database users, server starts and stops etc. Memory change Recompiles, started and completed, procedure cache miss, removal from the procedure cache, cache hit for stored procedures. DTC transaction, SQL transaction (commits, rollback transactions etc), log entry written to transaction log SQL started, completed, batch started/completed etc.

Locks Objects

Performance Scans Security audit

Server Stored procedures

Transactions

T-SQL

Within profiler, you can apply many filter conditions for a single trace event. In virtually all cases I do not include system objects by checking the box provided:

The DBA may also want to filter on a specific database within the instance. Use the database ID as the name will not work. Get the database ID by querying sysdatabases found in the master system database. Here are some example event filters and traces to consider:
Events RPC:Completed SP:Completed SQL:BatchCompleted Columns DatabaseID Duration Event Class TextData LoginName CPU (ms) Filters Filter on a specific database ID as need be. Exclude or Include selected statements as need be. Select a specific SPID for single user tracing. Select a duration >= ???? (i.e. 1000) milliseconds Our Aim Performance Tune SQL and Monitor Adhoc SQL

131

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

RPC:Starting SQL:BatchStarting Lock:Deadlock Lock:Deadlock Chain

SP:CacheHit SP:CacheInsert SP:CacheMiss SP:Completed SP:Recompile SP:Starting SP:StmtCompleted SP:StmtStarting Audit Add DB User Event Audit Add Member to DB Role Event Audit Addlogin Event Audit Backup/Restore Event Audit Login Audit Change Password Event Audit Login Failed Audit Server Starts and Stops Audit Statement GDR Event Errors and Warnings: errorlog Oledberror Exception Exception warnings

Reads Writes SPID Start Time Object Id Event Sub Class Connection Id SPID Text Server Name Database Id Index Id Application Name SQL Use Name Start Time Integer Data Binary Data DatabaseID Duration Event Class TextData LoginName CPU (ms) Reads Writes SPID Start Time Database ID Event Class NT Domain Name Role Name Target Login Name Target User name Application Name NT User Name Client Process ID SPID Start Time Database ID Event Class TextData NT User Name SPID Start Time

for more selective tracing.

Deadlock Detection

Stored Procedures

Filter on a specific database ID as need be.

Auditing

Consider SP:StmtCompeted and SQL:BatchStarted to get more information about the event before the error.

Error Tracing

132

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Use the predefined trace template called SQLProfilerTuning.

Filter on a specific database ID as need be. The DBA may still selectively apply filters for specific types of SQL, namely:

Use the Index Tuning Wizard

The DBA may add further events to the trace, such as SQLTransaction to capture the begin and subsequent commit/rollback sub-classes. Additionally, the DBA may also consider the RPC events (remote procedure calls). I have to say that profiler is something Oracle is sorely missing in terms of fast and easy to use tracing. NOTE the generic trace is a good start, use the default trace, exclude system processes, and use the database ID filter (see sysdatabases in the master database or exec sp_helpdb, filtering on database name may not work for you and in many cases will return a blank string). This trace is an excellent start as it includes text data of submitted SQL and is a good basis for further exploring the power of the tool. IMPORTANT Be very careful re-running saved traces through profiler, if they include a range of inserts, updates, deletes, they will all be repeated against the database instance. This is very handy in various debugging and tuning workloads, but you do not want to get it wrong if you know what I mean. The DBA should spend a good amount of time on profiler. It is a very powerful tool for not only SQL performance tracing, but for auditing, error detection, cache misses and recompiles and much more. For a good overview with some performance tuning tips, I highly recommend the e-book Start of Finish Guide to SQL Server Performance Monitoring, Brian Kelly 2002. NOTE Take care with the duration column. This may not accurately reflect the execution time of the actual SQL statement. For example, I had a large application where profiler reported that the selected statements were taking 20sec to execute. Running them via query analyser gave a different story and they were running under 1sec. Going back to the client application and drilling into the code segment was the only way to more effectively time and resolve the problem.

Saving & Scripting Traces

The DBA can choose to save a profiler trace to a database. The DBA can specify the name of the database in which to create the table, and the name of the table. The tables columns are created based on the data columns selected for tracing. The table has a single primary key called RowNumber (identity) that is a clustered index. If the DBA selects an existing table, it will ask if it is ok to overwrite it, which involves dropping an re-creating the table. The greatest option in regard to profiler tracing, is that we can script out the trace script in T-SQL code. For the DBA wanting to further customise the trace, this is an excellent feature. Once a

133

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

trace has been set and is running, select File Script Trace For SQL Server 2000 to generate a .sql file of the sp_ stored procedures used to create the trace. NOTE Dont assume the GUI is showing all commands executed. If you are relying on the trace window be aware that it buffers trace data. On very busy systems this can result in a loss of data in the trace window. If you forgot to save the trace to a trace file or trace table, dont fear, stop the trace and select File Save As Trace File and specify your .trc filename. Be careful though, if you start again the trace screen (buffer) is cleared and all will be lost, also, pausing the trace, altering event columns and starting again will also clear the buffer of all previous tracing. One handy feature in SQL Server is the ::fn_trace_gettable system function. After creating a trace file via profiler, run the following and get a table representation of the files contents: SELECT * FROM ::fn_trace_gettable('C:\mytrace.trc', default) GO Other system functions to check are: ::fn_trace_setstatus <id>, <status> ::fn_trace_setevent ::fn_trace_setfilter ::fn_trace_getfilterinfo(<id>) ::fn_trace_getinfo(0) ::fn_trace_geteventinfo(<id)

Index Tuning Wizard

The DBA can access the index-tuning wizard via query analyser. The wizard option is under the query menu option and is only available when a connection has been established to an instance and something has been entered into the query pane (even a space is enough), as the option remains greyed out until you do. The wizard works hand-in-hand with the traced output file from profiler or a manual trace via sp_trace. The first main screen is shown below.

134

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Select the database to tune, this and the server is defaulted from the connection made in query analyser. I tend to opt for not keeping existing indexes to get a better idea of new indexes to support the SQL trace I am tuning, even so, if you know the batch of SQL has been running well, retain the option to see what additional recommendations are made over and on top of existing indexes. Use the thorough analysis method in all cases.

The DBA can select a previously run trace from profiler, remembering that profiler allows the DBA to save the trace as a .trc file or to a database table. If the DBA has a range of SQL statements in the query analyser pane, then it can utilise them to perform its analysis.

ALWAYS select the advanced options. The defaults can be very strange indeed. Remove the limit for the workload queries to sample, bump up the space recommended for indexes to around 100Mb+ (worst case largest index size?). The next option really depends on the DBA, if you leave the default at 16, then SQL Server may decide to recommend a single index with all columns in the table, this is not good practice for many installations.

135

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Select all tables, unless of you know for a fact that only a small number of the total tables in your schema are being used. The cool feature here is the projected column, in the example I have projected a growth of 80000 orders over the existing 830 currently in the table.

The index analysis will start when the next button is pressed. The existing database structure is not altered in any way what so ever. If you traced inserts, deletes and updates they are not actually run, they are only parsed and the tuning wizard will apply the rules previously set. The analysis is reasonably well presented; the analysis screen provides a series of reports that can be saved to disk for further checking. Be very careful though with the overall recommendations. A classic example is shown below, where my three statements in the trace (which only covered 3 of the total tables) result in the recommendation to drop all existing indexes off the non-utilised tables. At the end of the day, the DBA should save all recommended changes to disk and evaluate each one very carefully in context with the big picture.

The index wizard will also recommend statistics (histograms) for tables.

136

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

On clicking Next the DBA is asked if the recommendations should be saved to disk and/or for them to be applied immediately or scheduled as a SQL*Agent job on the instance to be run later. Take the time to re-check the individual table indexes, a classic example is with the sample northwind database. Here in the Orders table we have two indexes over the CustomerID column, why? well the index tuning wizard asked the same question and suggested that we remove the duplicate indexes.

Explain Plan
Within Query Analyser the DBA has a variety of choices when it comes to viewing the optimisers execution plan:

set showplan_text {on, off} or set showplan_all {on, off}

Some of these can be set via tools

options

connection properties.

When using the Query menu options above, the only toggle item is the show execution plan, the display estimated execution plan will be executed immediately based on the SQL selected, or, if nothing is selected then all SQL is parsed and the plan displayed. A different plan of course will be shown from top to bottom if there are multiple SQL statements. Both query menu options show a graphical representation of the generated plan. This, like all plans, should be read from bottom to top. Placing the cursor over each step within the plan provides further statistical information.

137

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Place cursor here to view stats on rows returned

If you set the showplan_text or showplan_all options, the graphical version of the plan is not shown. A text version will shown instead and will continue to be shown until you set the options off. Switch between Results in Text and Results in Grid to get a better view of the plan. Microsoft Technet provides some good documentation related to SHOWPLAN and the optimiser in general. Search in the books online for Graphically Displaying the Execution Plan Using SQL Query Analyzer for a general overview of each displayed icon. See reference (45). When performance-tuning queries, like all things, indexes and update statistics will play a major part in the optimiser selecting the best execution path. The DBA should revise index and the query joins and filter conditions carefully before choosing hints. NOTE You can find the complete books online at http://www.ddart.net/mssql/sql2000/html/, which is equivalent to http://tahiti.oracle.com To obtain further statistics in terms of execution time and IO, we can set the following:
set statistics time on eg. SQL Server Execution Times: CPU time = 270 ms, elapsed time = 330 ms.

set statistics io on eg.

Table Table Table Table 'Order Details'. Scan count 77, logical reads 4467, physical reads 0, read-ahead reads 0. 'Categories'. Scan count 77, logical reads 154, physical reads 0, read-ahead reads 0. 'Products'. Scan count 1, logical reads 2, physical reads 0, read-ahead reads 0. 'Orders'. Scan count 1, logical reads 3, physical reads 0, read-ahead reads 0.

Where:
Scan Count #times that tables referenced in the query where accessed. Logical Reads database buffer cache page reads Physical Reads physical page disk read from disk into the data cache for processing

138

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Read Ahead Reads - read ahead cache manager page reads, is a separate asynchronous process (anticipated page reads)

Switch to results in text to view the timings. Take care with statistics time as other server activity can also skew the result set. The time displayed will also list parse and compile time when appropriate. In all cases, ignore the first portion of the timing of the SQL statement being analysed, this is not related to the command but previously run SQL. SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 5 ms. SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 0 ms. Ignore this.

NOTE Developers can run explain plan and set the time and IO statistics on without any special privileges. Also note that index pages are also read into the buffer cache. Another method of timing is simply declaring datetime variables and printing the time difference between the start and current time in each phase of your SQL. Be aware that you will be restricted to seconds, rather than hundredths of a second.

Black Box Tracing?

A range of 3rd party tools I have used typically recommends the black box trace. The trace itself will create the files blackbox.trc, blackbox_0n.trc (switches every 5Mb), of critical system error message traces. This can be very helpful when working with Microsoft support in diagnosing system errors. The trace is activated via:
-- ########################### -- Start the black box trace -- ########################### declare @traceID int exec sp_trace_create @traceID OUTPUT, 8 exec sp_trace_setstatus @traceID, 1 -- Create the TRACE_PRODUCE_BLACKBOX -- Start the trace

and produces this file:

If you want to start the trace every time SQL Server starts, then place the code into a stored procedure within the master database and set the option: exec sp_procoption mystoredprocnamehere, startup, true To verify the trace: USE master

139

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

GO SELECT * FROM ::fn_trace_getinfo(1) GO

Latch Statistics
The term latch is synonymous to that used in Oracle. To monitor latch based statistics we can use the command: DBCC SQLPERF(WAITSTATS, CLEAR) DBCC SQLPERF(WAITSTATS) -- initialise counters -- get statistics

This will return cumulative data so we need to re-run this command and compare the difference to determine the point-in-time (or an average) statistic. In a majority of cases the DBA will be able to determine the performance problem via profiler and tracing poor performing SQL, deadlocking, page-splitting and more without having to drill into latch statistics. Be aware that SQLPERF returns more than just latch statistics, the command can provide some interesting lock architecture statistics. See section on Wait Statistics for more information.

Page Splits
The page splitting problem primarily occurs when new data insertions (over a non-sequenced key) or updates result in the row not fitting into a page. This results in a page split, as the destination page is split into two and requires the re-linking of the index pages in the b-tree and associated linked leaf page list chain to accommodate the row. As you can imagine, this is a resource intensive tasks that can affects overall system performance, longer latch waits and concurrency issues. NOTE No matter the index (clustered or non-clustered), the leaf nodes are always kept in order of the indexed key. The fillfactor and padindex parameters can be altered to reflect the storage characteristics of indexes only. The fill factor controls the table density, where a high value increases the number of rows stored per page and enhances index-scan and read-ahead performance, but adversely affects page splits on row updates or new insertions. The padindex setting controls free space higher up

140

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

in the b-tree structure, leaving space at the intermediate nodes in the b-tree based on the current fill-factor setting (will always leave space for one row). Monitoring page splits is tough, why? because there is not real way to tracing in real time where the splits are occurring (especially on a very busy database). To begin with we can use the NT performance monitor, select the Access Methods performance object for the instance and choose the page splits/sec counter. There is no specific value as such we should be looking for, I tend to look at an hourly average over a number of days to determine the need for further concern. You may be experiencing bursts of splits based on batch data loads and bulk record manipulation which will ease the process of identifying the tables concerned. To drill down further, I typically scour the internet for a script that will run DBCC SHOWCONTIG over all tables in my database. It is important to note that page splitting not only occurs in your user databases, but can occur frequently in the system tables (especially MSDB). Anyhow, this DBCC command displays fragmentation information for data and indexes. This will assist in determining the tables requiring index rebuilds (re-indexing). Saying that, online index defragmentation should be a regularly scheduled job for virtually all databases (consider maintenance plans or again, writing your own script). This of course is based on the premise that page splitting will increase index and table fragmentation. The DBA should not spend hours attempting to resolve page splitting. I have yet to see any SQL Server database that doesnt experience some page splitting. The DBA will find it very difficult to get any indicative figure on the rate of page splitting vs the overarching system performance. As such, you should refocus on data reorganisation, and setting padindex and fillfactor appropriately for frequently updated tables. An example of re-indexing to alter padding and fill factor:
CREATE UNIQUE INDEX [PK_efs] ON [dbo].[efs] ([col1]) WITH PAD_INDEX, FILLFACTOR = 80, DROP_EXISTING ON [PRIMARY]

On a side note to page splits, defragmentation of clustered indexes provides the best opportunity to resolve fragmentation on the index and the physical (data) pages. The online defragmentation option allows the DBA to alter the fillfactor, therefore, assisting in reducing page splitting for the table. As there is no specific trace for a page split (there is but it is very difficult to use), we need to proactively monitor fragmentation historically to determine the effect of our change. At the a) b) c) d) end of the day, the DBA needs to have a good understanding about: database hot spots indexing currently in place frequency of de-fragmentation jobs types of operations performed against the hardest hit tables

141

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Read the section on Index Skewing for more information on page splitting. Important Page splitting does not occur on clustered indexes only, any page that is near full and a subsequent update cannot be accommodated in the page, resulting in a split. The point is, index structure maintenance adds to total time to complete the split.

Performance Monitor (perfmon.exe)

For each SQL Server instance installed on the server a series of NT performance monitor counters are installed. The DBA or system administrator should not remove them, as it will be very difficult to monitor instance and database performance without them. In Oracle, the counters are also applicable per database instance. The DBA can utilise the command line utility operfcfg.exe as shown below:
C:\Chris Kempster>operfcfg -usystem -pking -dseca Usage: OPerfCfg [-U <username>] [-P <password>] [-D [database name]] C:\Chris Kempster>operfcfg -u system -p king -d seca OPerfCfg: New registry values have been successfully set.

This utility alters selected entries in the registry for Oracle, namely: HKEY_LOCAL_MACHINE\SYSTEM\CURRENTCONTROLSET\SERVICES\ORACLE?\PERFORMANCE

See \Oracle\DBS\operfXX.txt and sqlnet.txt for more error information related to loading the performance counters. See oracleX.pmw performance monitor script file to load the counters.

The SQL Server counteres are automatically installed with the instance. If they are not, then try to re-register sqlctr80.dll and run the file sqlctr.ini, both located in the binn directory for the instance. The DBA should also try the command lodctr.exe sqlctr.ini, and the unlodctr command. It is assumed that the DBA knows how to use performance monitor (perfmon.exe) and will not be discussed in detail. Some important items to remember though: a) each database instance has its own set of counters. b) the system table master..sysperfinfo includes many of the database performance counters. Many of these values are cumulative. See this Microsoft article for example views over this table: http://support.microsoft.com/search/preview.aspx?scid=kb;en-us;Q283886

142

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

c) you can monitor a server remotely, if you are having problems, map a drive or authenticate to the remote domain first then try again, try the server IP then the hostname (\\163.222.12.11 for example). You may need to re-start performance monitor after authenticating as you can continue to have authentication problems. d) use a scheduled collection of statistics over a period of time, attempt to get a typical series of working days as a base line for analysis. Watch out for pay periods in HR systems and key events for financial systems that can see dramatic variances in performance. e) attempt to understand the applications running against the database and associated server architecture well in advance, this can take some time in complex environment but is essential in the overall interpretation of performance monitor statistics. f) dont just rely on performance monitor, real-time analysis for a short period may see dramatic spiking in CPU, IO and other counters that, when averaged out, may not be evident in collected statistics. g) get a list of all software installed and running (especially scheduled applications) on all servers in the domain you are analysing. Relate these back to counters and counter processes where possible. h) take care with setting Y scale properties i) be carefully when selecting counters and test with a small collection (a few minutes) to be 100% sure you are collecting all counter data you require. This sounds obvious, but never take it for granted as it is not worth the trouble.

j) to enable disk counters, execute the following from the command line and re-boot the server. Run diskperf.exe /? for list of all options. diskperf -y To assist with performance analysis via the NT counters, the following are discussed:
Counter Meaning Category

Processor (_Total): % Processor Time (or single process)

Utilization of each CPU of all threads for a specified process (or all processes if selected). Is total elapsed time that all threads of the process used the processor to execute instructions. Anything over 75% over a sustained period needs to be investigated, users may be experiencing response time issues at lower figures though. Filter on the sqlservr process if need be. This can also be used to determine if SQL Server is using all CPUs. (Each processor has an idle thread to consume CPU cycles when no other activity is going on.) Average CPU % utilization of all CPUs are busy executing nonidle threads. See System : Context Switches/Sec if >80% for 510min periods Thread queue length per CPU (ready threads only, not running ones), should be <= 2 per CPU. Reduce max worker threads

CPU

System: % Total Processor Time Processor Queue Length

CPU CPU

143

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Processor (_Total) : % Privileged Time and Processor (_Total) : % User Time System : Context Switches/Sec

if this exceeds max/cpu and CPU utilization is not high. Use this counter and one above to get overall CPU utilization. This is the time spent running Kernal mode (privileged) code. If this counter is >= 20% over a period in relation to %UserTime, this may indicate IO (paging) and disk array problems. Recheck % Disk Time counter (over 55%?) and other disk related counters (such as queue length) to determine is this is not the cause. How often NT switches context from one thread to another. If >15000/sec for over 5-10mins then consider NT fibres. Reevaluate CPU performance by checking queue lengths. A context switch occurs when a running thread is pre-empted by a higher priority thread to hand over its CPU time. Other scenarios include switching user modes. The counter is the sum of all threads running on all processes. To monitor each disk array (physical disk). Monitors read/write wait time for each array. If >2 for 5-10min periods the probably have a IO bottleneck. Use calculation physical-disk / disk-queue-length = avg-queue-length-per-disk. Use diskperf y to enable disk performance statistics on server. For the drive being analysed, you need to divide the value by the number of spindles in the drive, remember this, especially for your RAID arrays. As an example, a RAID-5 array of 4 disks, means you need to divide the value returned by 4, but a RAID0+1 array of 10 disks means a division of 5, so a queue length any greater than 10 (10/5 = 2) may be a problem. How busy the physical array is. Should be <90%, use two counters below to determine where the majority of the work may be (on the read or on the write) % IO Read performance. % IO Write performance A value >0 indicates paging, both hard and soft. Soft paging is applications referring to page file memory pages in RAM rather than within the physical page files. Use for formula : "Memory: Pages Faults/sec" minus "Memory: Pages Input/sec" = Soft Page Fault/sec Monitor Process : Page Faults/sec counter for sqlserver process to determine if the sqlserver.exe process is being affected. Check utilization of pagefile.sys. Is rarely used by SQL*Server (if at all!), resize page file to a value slightly higher than what is reported. Check SQL*Server memory alloc to determine if the usage of pagefile.sys is due to SQL*Server dynamically allocating memory. Anything over 70% is a concern. Pages/sec is the #pages per second that are paged out to disk or paged into memory from disk (to resolve hard page faults). If pages/sec > 0 or page reads/sec > 5, windows may be going to disk to resolve memory references (hard page fault). This is an important counter when evaluating paging on the server as it

CPU

CPU

Physical Disk Object: Avg. Disk [read, write] Queue Length

I/O

Physical Disk: % Disk Time Physical Disk: % Disk Read Physical Disk: % Disk Write Soft Paging Memory : Page Faults / sec

I/O I/O I/O Soft Paging

Paging File: % Usage

Page File Utilisation

Memory: Pages/Sec and Memory Object : Page Reads/Sec

Memory and Paging

144

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Memory: Available MBytes Network Interface : Bytes Total/Sec SQLServer Buffer Manager: Buffer Cache Hit Ratio SQLServer Cache Manager: Cache Hit Ratio SQLServer Buffer Manager: Cache Size (pages) SQLServer General Statistics: User Connections SQLServer Databases: Transactions/sec SQLServer Access Methods: Page Splits / Sec

will adversely affect IO and Kernal CPU time. Should be >=5mb. SQLServer attempts to keep free at least 410Mb. Is a sum of zeroed, free and standby memory lists. #bytes sent back and forth between the server and the network. Includes ALL traffic. Compare statistics with network card supported maximums etc and take into consideration network bandwidth and the number of cards in the server. Ratio of DB buffer cache hits. OLTP >= 95%, influences by free RAM and type of SQL running against server. Anything under 90% is a concern. Procedure cache (execution plan) hit ratio. Should be >= 90% although can be application specific (i.e. large numbers of adhoc queries). How much physical RAM devoted to the DB buffer cache. Value is in pages so multiple by 8192 to determine size in Kilobytes. Should be very high, check if using dynamic memory allocation. #user connections currently made to SQLServer. Single user may have multiple connections and multiple users may be sharing a single connection. Select all or some of the database instances. Count of transactions started on the database. Occurs when an index page becomes full and is split between the current page and newly allocation page on data insertion or key update. This is an intensive process that should be kept to a minimum. Consider an index rebuild, and decreasing the fillfactor from 100 (with padindex option) on indexes before the rebuild. Use DBCC showcontig as required to evaluate user tables. The ratio of recompilations to compilations should be very small. High recompilations means numerous, adhoc dynamic SQL being run against the instance. This reduced procedure cache hit ratios and subsequently, overall performance. Measure of unrestricted full table or index scans per second. Evaluate SQL and consider using the Index Tuning Wizard. Performance may be fine and this is very much application dependent. Remember that full table scans are not necessarily bad (large number of smaller tables?) and this counter can be difficult to interpret without good knowledge of the underlying schema. Consider profiling (tracing) the application to determine Average wait time of a variety of locks (DB, key, page, RID, table). Need to ID transactions that are causing the locks, consider using profiler for in-depth analysis. # deadlocks per second to monitor locking. Use profiler for detailed analysis of deadlocks. Check application isolation levels and database isolation level. Evaluation code segments and long running tasks with ranged operations over numerous data sets. Anything > zero is a concern. These three counters can provide a measure of how busy your instance is in terms of transactions per second, coupled with the total number of SQL batches (all DML activity). The transactions / sec counter can be a tough one to really gauge actual

Memory Network

SQLServer SQLServer SQLServer SQLServer SQLServer SQLServer

SQLServer : SQL Server SQL Compilations/sec and SQL ReCompilations/sec SQLServer Access Methods: Full Scans / sec

SQLServer

SQLServer

SQLServer Access Methods: Average Wait Time (ms) SQLServer Locks: Number of Deadlocks / sec SQLServer Database: (_Total) Transactions/sec and

SQLServer SQLServer

SQLServer

145

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQLServer Database: (_Total) Active Transactions/sec and Batch Requests/ sec

transactions, for example, I may open a transaction, execute 20 SQL statements, and end the transactions, as such, the counters reflect figures I may not expect. Example via Query Analyser (you should check this with COM+ VB code and a simple OLE-DB connection not using COM+): Transactions/sec only measures inserts/deletes/updates Batch Requests/sec -- query analyser, batch requests / sec = 1 select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses -- query analyser, batch requests / sec = 8 select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go select count(*) from sysprocesses go -- query analyser, batch requests / sec = 1 begin transaction select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses select count(*) from sysprocesses commit transaction -- query analyser, batch requests / sec = 1 -- transactions / sec = 3 insert into aa values (1) insert into aa values (1)

146

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

insert into aa values (1) -- query analyser, batch requests / sec = 1 -- transactions / sec = 0 begin transaction insert into aa values (1) insert into aa values (1) insert into aa values (1) commit transaction -- query analyser, batch requests / sec = 3 -- transactions / sec = 1 begin transaction insert into aa values (1) go insert into aa values (1) go insert into aa values (1) go commit transaction SQLServer Backup Device: Device Throughput Bytes/sec How fast backups are performing. If going to disk also review queue length and the RAID array configuration in conjunction with the counter. Backups

I should point out that it is easy to get hooked up with some of them, and one poor performing figure may not correspond directly to the specific problem, only to find another counter is skewing the result. There are a range of websites and good books that cover performance counters in detail to provide further assistance with analysis. The DBA should take a baseline data collection using these counters. The interval may range from every five minutes to as little as every twenty seconds (ideally over a range of days). The collection is essential for ongoing performance monitoring, and performance tuning.

Trace Flags
There are numerous trace flags available to the DBA but most are not formally documented. A great source of information for the documented and undocumented trace events comes from the ebook Transact-SQL Language Reference Guide (47). I highly recommend that you register this ebook as it is an excellent introduction to T-SQL programming commands. The T-SQL language reference lists 20+ trace flags, whilst the online documentation lists four. Some of the flags are shown below. Trace# 260 Summary Print extended stored procedure version information

147

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

326 552 653 1204 1205 1206 1807 3104 3502 3602 3607 3608 3609 4022 7300 8687 8602 8687

Prints information about the cost of non-clustered index sorts Disable read-ahead manager Disable read-ahead manager for current connection The locks participating in the deadlock Detailed information about the command executed at the time of a deadlock. Shows additional formation about locks at the time of a deadlock Allows the use of network based database files. Bypass checking for free space Log start and end of a checkpoint Record all error and warning messages send to the client Skip recovery of all database at instance startup As above except the master database Skip creation of the tempdb on startup. (important for tempdb creation issues related to model database problems). By-pass auto-start stored procedures in the master database. Get extended error information related to the execution of distributed queries. Disable query parallelism Ignore all index hints Disable query parallelism

The trace events can be set on instance start-up, or for a session. On startup, we use the T or t option, this can be set via Enterprise Manager as shown below, or you can go to the registry and alter the startup options. You need a new T[trace#] for each flag being set, do not try separating them via commas or spaces.

148

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

This particular trace will force the logging of checkpoints: checkpoint -- force the checkpoint

Important Trace flags set with the T startup option affect all connections. To view the current enabled session and database wide traces, use the command: DBCC TRACESTATUS(-1) Or rather than using 1, enter the specific trace event#. For the current connection we can use the DBCC TRACEON and DBCC TRACEOFF commands. You can set multiple flags on and off via DBCC TRACEON(8722, 8602[,etc]) Use profiler to trace existing connections, even so, remember that these trace flags are not the same as trace events as used by profiler (sp_trace_setevent). NOTE Use trace flags T1024 and T3605 to enable extended deadlock tracing.

149

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Indexing
It is worth mentioning indexing in terms of their use, performance, reorganisation and statistical collection. It is important to remember that, no matter the DBMS, the indexing mechanics are typically the same (in concept) and are critical for the overall performance of your system.
Index Types

In SQL Server, we are somewhat restricted in terms of the types of indexes available when compared to Oracle, this may change over time but the time being we only have: a) Indexed Views (materialised views) b) Clustered indexes a. Leaf nodes contain the actual row data pages c) Non-clustered indexes a. Leaf nodes are pointers to row data pages, or, if the table also includes a clustered index then it will point to the clustered index leaf nodes (requires a bitmap lookup step in the execution plan). The first question we ask ourselves is when to use a clustered index verses the standard heap. As you can only have one per table, this gets somewhat tricky as you need to understand the types of queries being executed. There is no definitive answer, but consider the following: a) clustered indexes are best suited to range queries rather that highly selective row lookups. A range based query may be over a series of dates, category types etc. b) clustered indexes are also beneficial for single key lookups, primarily when most of the record will be retrieved. Remember that non-clustered indexes use the clustered index to locate the data rows and hold the clustered keys in their leaf nodes (resulting in more IO). c) clustered indexes are ideal around columns used in order by and group by clauses d) a tables data pages can only be defragmented online via its clustered index (row reallocation may occur with the command DBCC shrinkdatabase command, being a form of re-organisation). e) identity column primary keys, clustered indexes can be beneficial (so long as its not being wasted on a more commonly used complex query). With identity keys in general (clustered or not) the possibility of page splits is close to zero (not taking into consideration subsequent row updates etc) as the in increasing key value will be inserted in order. Be aware though, that the adverse affect is the creation of hot spots over the last page that can result in locking/blocking issues.

150

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Although fine, such points are close to useless unless you have a good understanding of the DML running against your DBMS. NOTE - As with oracle, the more indexes on a table, the slower subsequent DML will be. Consider this when bulk loading data especially or systems requiring high insert/update/delete throughout over query speed. Moving to non-clustered (heap) indexes, be aware that if you drop or re-created a clustered index associated with the table, all non-clustered indexes must also be rebuilt. The DBA should consider in non-clustered indexes over all foreign key columns not partaking in the primary key. This is a common practice; even when there is poor selectivity in improving merge join performance. Always remember the following statement from the SQL Server 2k BOL:
If the table does have a clustered index, or the index is on an indexed view, the row locator is the clustered index key for the row. If the clustered index is not a unique index, SQL Server 2000 makes duplicate keys unique by adding an internally generated value. This value is not visible to users; it is used to make the key unique for use in nonclustered indexes. SQL Server retrieves the data row by searching the clustered index using the clustered index key stored in the leaf row of the nonclustered index.

The non-clustered indexed as in Oracle are ideal for highly selective columns. As you expect, a scan over a non-clustered index results in a scan over the clustered index to locate the physical row. You will see this in your explain plans as:

In a bookmark lookup, we utilise the clustered index (over the primary key in this example), to look up the rest of the table data. This can be a misnomer though, as the optimiser will also shown this operation even though a clustered index does not exist, but we still need to locate the rest of the column data to suffice the query. Also note that this lookup will occur if we have a covering index. WARNING Take care with indexing, you can index the same column multiple times in SQL Server. The final index type is not really any different from the two mentioned, but in terms of Oracle, is equivalent to materialised views.
Like Clause

Using a leading % wildcard pattern character at the start of a string will not allow optimizer to use indexes. Take are good think about what the end-user really wants, and consider full text indexing for complex searching over string columns.

151

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Functions and Indexes

Within Oracle, the Function Based Index was created to allow the DBA to index a column with a function applied to it. A classic example is: where UPPER(surname) = KEMPSTER where a function based index is created over the function/column UPPER(surname). In SQL Server this is a problem that will result in indexes not being used. To get around this, take a careful look at the where clauses, and determine if you can remove or reverse the call, here is a classic example: where convert(varchar, sale_date, 103) = 19/1/2001 use this instead: where sale_date >= 19/1/2001 and sale_date < 20/1/2001
Composite Indexes

As with Oracle, a composite key will not be used unless the left most column is used in the where predicate. Within the where, the column must also be first (top most) column to be used by the optimizer. Such indexes are good for clustered indexes, especially where only some of the leading columns are being used.
Indexed Views (Materialised Views)

A view can be indexed, unfortunately there are a whole range of restrictions that can make the option undesirable. Some of which are: a) select statement cannot include a. union b. sub-queries c. self joins d. TOP e. ORDER BY f. DISTINCT g. COUNT(*) h. * or table.* b) Session options set ON a. ANSI_NULLS i. When set ON, this also sets the following SQL-92 options: 1. ANSI_NULLS (set at execute time) 2. ANSI_PADDING (set at execute time) 3. ANSI_WARNINGS (set at execute time) 4. ANSI_NULL_DFLT_ON (set at execute time) 5. CURSOR_CLOSE_ON_COMMIT (set at execute time) 6. IMPLICT_TRANSACTIONS (set at execute time) 7. QUOTED_IDENTIFIER (set at parse time)

152

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

c) d) e) f) g) h)

b. ANSI_PADDING c. ANSI_WARNINGS d. ARITHABORT e. CONCAT_NULL_YEILDS_NULL f. QUOTED_IDENTIFIERS Session options set OFF a. NUMERIC_ABORT All columns must be named (column name with suffice but if using function calls the explicitly name the column). Cant index columns using indeterministic functions (eg. Getdate) a. The view must be deterministic Clustered index must exist before other non-clustered indexes are created. The clustered index must be unique Must refer to tables using the two part syntax a. dbo.mytable ok b. mytable fails Must be bound to the schema (tables) in which the view is sourced from

The DBA can check session properties via the command: Select sessionproperty('ARITHABORT') or DBCC USEROPTIONS A value of zero means the property is not set. The schema binding option is the critical part to creating the materialized view. Once done, we will create a clustered index over the view and optionally a number of non-clustered indexes. For example:
CREATE VIEW viewtest1 WITH SCHEMABINDING AS select title, au_ord, au_lname, price, ytd_sales, pub_id from dbo.authors, dbo.titles, dbo.titleauthor where authors.au_id = titleauthor.au_id and titleauthor.title_id = titles.title_id CREATE UNIQUE CLUSTERED INDEX viewtest1_ix ON viewtest1(title, au_ord)

Its not until you create the index will you get errors related to function calls used within the view. For example, you may get something like:
Server: Msg 8662, Level 16, State 1, Line 1 An index cannot be created on the view 'viewtest' because the view definition includes an unknown value (the sum of a nullable expression).

You can pre-determine this via the command:

select ObjectProperty(object_id('viewtest2'),'IsIndexable')

To force the index over the view, use the hint:

153

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

select * from viewtest1 with (noexpand)

Take care with hints, the optimizer will chose the optimal indexing path where possible based on the expansion of the view, its tables and the tables underlying indexing and histograms. The index on the view may simply be the wrong way to go to suffice the query. In the above view created against the PUBS database, the expansion and nonexpansion gives some interesting results:
set statistics io on -- use the indexed view select * from viewtest1 with (noexpand) Table 'viewtest1'. Scan count 1, logical reads 2, physical reads 0, read-ahead reads 0. -- optimizer chooses, expand view forces the optimizer not to use indexed views select * from viewtest1 option (expand views) or select * from viewtest1 -- even with no hint, the optimizer expanded the view Table 'titles'. Scan count 25, logical reads 50, physical reads 0, read-ahead reads 0. Table 'titleauthor'. Scan count 24, logical reads 48, physical reads 0, read-ahead reads 0. Table 'authors'. Scan count 1, logical reads 1, physical reads 0, read-ahead reads 0.

To change the underlying tables in any way, the schema bound view must be dropped first.
Covering Indexes

A covering index is one in which the index includes all columns required for the index to resolve the query without having to do a further lookup for the rest of the column data. An example is shown below.
drop table mycovering create table mycovering (col1 int, col2 int, col3 int, col4 int) create index mycovering_ix on mycovering (col2, col3, col4) select col2 select from mycovering from where col4 = 2 where and col3 > 1 and

col2 mycovering col4 = 2 col1 > 1

154

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Index Skewing

In Oracle, we have the concept of Reverse Key Indexes, which resolves index skewing in the btree, where one side of the tree becomes a hot spot in terms of new insertions and whose depth also increases (skewed to one side) with a monotonically increasing key values. For example:

101 101 102

201

So in the example above (very rough example mind you), the key value is reversed to produce a more balanced tree structure, especially during deletions resulting in index browning. The only option in SQL Server is the ASC or DESC options when creating the index, altering the sort order of the key columns, this is not the same as reverse key indexes. To monitor index height, use the INDEXPROPERTY command, for example:
SELECT INDEXPROPERTY(OBJECT_ID('MyTable'), 'PK_MyTable', 'IndexDepth')

In the case of identity columns, the depth increases slowly, with more intermediate index keys being added (resulting in page splitting higher at these nodes). In the example below, the index height remains at two, but during the bulk insertion of records over the clustered primary key identity column, we see a large number of page splitting due to page allocations and the increased number of intermediate nodes.

page splits during 115k record insertion. CREATE TABLE [test1] ( [col1] [int] IDENTITY (1, 1) NOT NULL , [col2] [char] (200) COLLATE Latin1_General_CI_AS NULL , CONSTRAINT [PK_test1] PRIMARY KEY CLUSTERED ( [col1] ) ON [PRIMARY] ) ON [PRIMARY] GO With 303104 rows, index depth 2, index size 216Kb, data 63816Kb, reserved 64064Kb

155

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Index Statistics and Fragmentation

The following commands can be used to view table indexes and subsequent statistical information:
exec sp_helpindex products
Notice the duplicate indexes over categoryID and supplierID.

To analyse index effectiveness (products = table, productname = index name):

DBCC SHOW_STATISTICS (products, productname)

To analyse index and table fragmentation:

DBCC UPDATEUSAGE ('northwind', 'products','productname') DBCC SHOWCONTIG (products, productname) use DBCC SHOWCONTIG (products) to view the tables storage characteristics.

To rebuild indexes:
DBCC DBREINDEX ('northwind.dbo.products', productname, 80) -- 80 is new fillfactor DBCC INDEXDEFRAG (northwind, products, productname)

NOTE As with composite keyed indexes in Oracle, the leading column must be used for the index to be utilised by the optimiser. Also note that in non-clustered indexes (heaps), the leaf nodes are not linearly linked listed. You cannot alter the padindex parameter via the DBCC commands.
Heaps, Clustered Indexes and Fragmentation

We know that DBCC DBREINDEX and INDEDDEFRAG can be used to defragment the index, but not necessarily the table itself. We have assumed though, that a clustered index that is defragmented will also defragment the table itself and not only the index. The following example attempts to test this theory: In the Northwind sample database, we have duplicated the products table, retained the identity column used for its primary key and forced some fragmentation to occur. We will defragment with the DBCC DBREINDEX command and review the subsequent storage characteristics of the table and the associated index structure.

156

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- Insert rows into the new table (non-clustered primary key)

name MyProducts rows 19712 reserved 2576 KB data 2216 KB index size 328 KB unused 32 KB

Table Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 0, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 277 - Extents Scanned..............................: 39 - Extent Switches..............................: 38 - Avg. Pages per Extent........................: 7.1 - Scan Density [Best Count:Actual Count].......: 89.74% [35:39] - Extent Scan Fragmentation ...................: 12.82% - Avg. Bytes Free per Page.....................: 616.5 - Avg. Page Density (full).....................: 92.38%

-- Delete random rows (493 rows)

name MyProducts rows 19219 reserved 2576 KB data 2216 KB index size 328 KB unused 32 KB

Table Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 0, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 277 - Extents Scanned..............................: 39 - Extent Switches..............................: 38 - Avg. Pages per Extent........................: 7.1 - Scan Density [Best Count:Actual Count].......: 89.74% [35:39] - Extent Scan Fragmentation ...................: 12.82% - Avg. Bytes Free per Page.....................: 801.8 - Avg. Page Density (full).....................: 90.09%

-- Insert more data (cause fragmentation)

name MyProducts rows 38438 reserved 4944 KB data 4240 KB index size 608 KB unused 96 KB

Table Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 0, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 530 - Extents Scanned..............................: 71 - Extent Switches..............................: 70 - Avg. Pages per Extent........................: 7.5 - Scan Density [Best Count:Actual Count].......: 94.37% [67:71] - Extent Scan Fragmentation ...................: 8.45% - Avg. Bytes Free per Page.....................: 475.1 - Avg. Page Density (full).....................: 94.13%

Primary Key Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 2, database ID: 6 LEAF level scan performed. - Pages Scanned................................: 73 - Extents Scanned..............................: 12 - Extent Switches..............................: 12 - Avg. Pages per Extent........................: 6.1 - Scan Density [Best Count:Actual Count].......: 76.92% [10:13] - Logical Scan Fragmentation ..................: 1.37% - Extent Scan Fragmentation ...................: 41.67% - Avg. Bytes Free per Page.....................: 197.8 - Avg. Page Density (full).....................: 97.56%

-- attempt to fix fragmentation DBCC DBREINDEX ('northwind.dbo.myproducts', PK_MyProducts)

name MyProducts rows 38438 reserved 4944 KB data 4240 KB index size 608 KB unused 96 KB

Table Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 0, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 530 - Extents Scanned..............................: 71 - Extent Switches..............................: 70 - Avg. Pages per Extent........................: 7.5 - Scan Density [Best Count:Actual Count].......: 94.37% [67:71] - Extent Scan Fragmentation ...................: 8.45% - Avg. Bytes Free per Page.....................: 475.1 - Avg. Page Density (full).....................: 94.13%

Primary Key Storage Summary

Table: 'MyProducts' (1877581727); index ID: 2, database ID: 6 LEAF level scan performed. - Pages Scanned................................: 72 - Extents Scanned..............................: 10 - Extent Switches..............................: 9 - Avg. Pages per Extent........................: 7.2 - Scan Density [Best Count:Actual Count].......: 90.00% [9:10] - Logical Scan Fragmentation ..................: 0.00% - Extent Scan Fragmentation ...................: 0.00% - Avg. Bytes Free per Page.....................: 88.1 - Avg. Page Density (full).....................: 98.91%

157

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

We see no change in the physical table storage with a heap index. Altering the index to a cluster, deleting and inserting more data, we see that re-indexing will also defragment the underlying table as well as the immediate nodes for the index. As we would expect.
name MyProducts rows 38438 reserved 4944 KB data 4240 KB index size 608 KB unused 96 KB

Table Storage Summary

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 1, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 757 - Extents Scanned..............................: 100 - Extent Switches..............................: 99 - Avg. Pages per Extent........................: 7.6 - Scan Density [Best Count:Actual Count].......: 95.00% [95:100] - Logical Scan Fragmentation ..................: 0.13% - Extent Scan Fragmentation ...................: 4.00% - Avg. Bytes Free per Page.....................: 2681.3 - Avg. Page Density (full).....................: 66.87%

Primary Key Storage Summary (identical to that of the table)

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 1, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 757 - Extents Scanned..............................: 100 - Extent Switches..............................: 99 - Avg. Pages per Extent........................: 7.6 - Scan Density [Best Count:Actual Count].......: 95.00% [95:100] - Logical Scan Fragmentation ..................: 0.13% - Extent Scan Fragmentation ...................: 4.00% - Avg. Bytes Free per Page.....................: 2681.3 - Avg. Page Density (full).....................: 66.87%

Table Storage Summary after reindex

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 1, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 510 - Extents Scanned..............................: 64 - Extent Switches..............................: 63 - Avg. Pages per Extent........................: 8.0 - Scan Density [Best Count:Actual Count].......: 100.00% [64:64] - Logical Scan Fragmentation ..................: 0.00% - Extent Scan Fragmentation ...................: 0.00% - Avg. Bytes Free per Page.....................: 58.9 - Avg. Page Density (full).....................: 99.27%

Primary Key Storage Summary (identical to that of the table)

DBCC SHOWCONTIG scanning 'MyProducts' table... Table: 'MyProducts' (1877581727); index ID: 1, database ID: 6 TABLE level scan performed. - Pages Scanned................................: 510 - Extents Scanned..............................: 64 - Extent Switches..............................: 63 - Avg. Pages per Extent........................: 8.0 - Scan Density [Best Count:Actual Count].......: 100.00% [64:64] - Logical Scan Fragmentation ..................: 0.00% - Extent Scan Fragmentation ...................: 0.00% - Avg. Bytes Free per Page.....................: 58.9 - Avg. Page Density (full).....................: 99.27%

Rebuilding Indexes

The DBA can use WITH DROP EXISTING clause to rebuild indexes. If a clustered index is rebuilt, so will any other index. . SQL Server support parallel index rebuilds.
Performance Counters

There is no simple way of determining the relevancy of indexing. I tend to run profiler (trace) and collect all DML being executed against the database of interest (or filter out specific tables). I then run this data through the index-tuning wizard and carefully read the summary reports on table access, columns accessed and recommended indexes. For in-house applications where you have some control over indexing (without breaking your support agreement) this method works well on your development and test servers to get a feel of the queries being executed and the most commonly used tables. The performance counters may compliment the tracing approach, but I find them somewhat limited and very easy to misinterpret, as you will see in the comments below. The counters are: SQL Server:Access Methods\Index Searches/sec As an example of this, a 6 row table with a clustered index scan over the primary key, resulted in a avg count of 12 to 20 for this counter. Dropping the table returned a value of 30 and the batch insertion of 6 rows also saw a count of 30. This may be helpful in

158

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

determining the performance impact of too many indexes with heavy insert/delete/update SQL. With no indexes but 6 rows, we see a single full tablescan but also 6 index scans. SQL Server:Access Methods\Full Scans/sec From a lot of testing, this counter provides a accurate figure of full table scans. The counters are for the entire instance and not database specific.
Index Management via Enterprise Manager

The DBA can alter index structures via: a) Database Diagrammer b) Designing a table (right click on table, select design table and select index icon) c) Manage Indexes option (right click on table, All Tasks Manage Indexes)

Option c) is quite functional and has that sort of Oracle Enterprise Manager type feeling to it. Be aware though that this option can be a little restrictive when shifting between clustered and nonclustered index storage. As such, switch to b) to resolve the issue. Trace the output via profiler (or view scripts generated via their menu options) to understand how to manage indexes via raw SQL commands.
Summary

There is no magical wand for indexing. The DBA needs to have an intimate understanding of the application(s) running against the database, and from there, make informed decisions on the indexing paradigm. As a very broad rule, you can not go too wrong with: a) indexing all foreign keys b) altering and monitoring index storage properties (padindex and fill factor) according to the avg% of inserts, updates, deleting going on verses reads. c) Leave auto-statistics collection on d) Utilising profiler to collection SQL, pass through index tuning wizard to assist in gauging the possibility for new indexes and reworking existing indexes. e) Re-indexing (de-fragmenting) on a regular basis. f) Utilising clustered index primary keys if another option is not suitable (see clustered index issues previously).

159

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Database File IO stats

At SQL Server Central.com, the columnist Cathan Kirkwood wrote a great article titled How to Diagnose and Fix Wait Locks. Within this it describes a call to ::fn_virtualfilestats to retrieve file IO statistics. With the appropriate smarts you could write a great real-time IO monitoring tool for your database with such a function. The call is simple enough: -- 6 = DB ID, 1 = file id select * from ::fn_virtualfilestats(6,1) Showing these cumulative results:

Take care with the NumberReads, these are physical IOs, and will not measure buffer cache lookups. The NumberWrites is another tricky one, as its very much dependent on the checkpointing and flushing pages to disk. The IoStall MS represents IO waits in milliseconds between refreshes. Here is an example, without the DBCC and checkpoint commands, you may see little or no change in the figures returned by the file IO function call.
DBCC DROPCLEANBUFFERS select * from ::fn_virtualfilestats(6,1) select count(*) from products delete from EmployeeTerritories checkpoint -- flush the log buffer select * from ::fn_virtualfilestats(6,1)

Wait Statistics
To view wait statistics, use the command: DBCC SQLPERF(WAITSTATS)

160

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Also query sysprocesses where waittime > 0 and spid > 50, looking up waittype, waittime, lastwaittype, wait resources. Many of the wait types listed is related to the specific types of lock objects SQL Server can take out (see Transaction Management chapter). Two key waits to review are related to transaction log management, they are: LOGMGR IO_COMPLETION WRITELOG Waiting on the log writer to start writing Waiting on disk resources to be free Waiting on the write (finish) of transactions to disk

Microsoft poorly documents the figures. Carefully evaluate their use and always use in conjunction with broader statistics from performance monitor.

STATSPAK
There is no equivalent utility to Oracle statspack in SQL Server. The analysis of the databases overarching performance requires a range of tools but we can get similar statistics via performance monitor and its numerous counters.

General Performance Tips

Some readers will not like this section, but I am not converting it to lengthy paragraphs that take forever to complete. Here we present a series of tips and food for thought in terms of performance.

Server Configuration
NTFS o partitions <= 80% capacity o cluster size to 64k o format in NTFS 5.x or above o dont place database files on compressed NTFS partitions Control Panel o Start -> settings -> control panel -> system -> performance, Set Application Performance to None. o Start -> settings -> control panel -> network -> services, set maximize throughput for network applications as the default is maximize throughout for file sharing. Reduces RAM usage. o Remove unused network protocols Networking o Windows 2k Server supports a larger MTU (1.5Kb to 9kb), consider altering it on your network card based on careful analysis of network performance o Check your network protocol list, if using, for example TCPIP, then ensures it is the first protocol to be validated by Windows. o Disable power management features for your network cards General

161

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

o o o o

o o

By as much RAM as possible for the server, include array cache Use a very simple screen saver Avoid multiple terminal service sessions. Try and contact in professional NT administrators where possible to check/recommend/configure your servers. Clearly outline the purpose and application architecture up-front and any other constraints (SW/HW/$/MTTR). Dont use your domain server as a database server, dont install DHCP, DNS, time servers etc, avoid or take care with Virus Scanners and their file scan properties. Verify the tape backup strategy, are backups running throughout the day? where is the tape loader? can a delay cause the file backups to run during your busiest parts of the day?

What can affect performance (not an exhaustive list)

Application code o T-SQL stored procedure code performance, use of stored procs vs inline (embedded) data access code at the business layer o Cursors are notoriously bad performers, utilise single DML statements where possible o Architecture (tiered architecture on separate servers?) o Not using native XML from SQLServer (or code managed/generated?) o Poor selecting of connection provider o Lengthy transactions resulting in blocks and deadlocks o Mis-understanding of COM+ transaction handling o Triggers and user defined function (check affect of DML performance carefully) SQL performance o Use of cursors vs batch-sql statements o Indexing scheme o Length of transactions o Batch processing during the day o Mix of OLTP vs DSS vs Reporting requirements on server verses auditing requirements (trigger driven?) SQL Server Configuration o Disk and file layout o Automatic statistics collection o Memory configuration o Isolation level and transaction processing Physical data model design o Query plan reuse, use of cursors, adhoc reporting o Middleware (COM, COM+, MSMQ), location of data tier code o Database Design o Indexing scheme vs actual usage patterns from programmers o Backup/Recovery model vs database usage and recoverability o Normalisation o Trigger design and usage Client and Server Hardware o Client and Server Operating System Settings and Configuration o Network Hardware and Bandwidth o Verify with network administrators

162

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Link speeds and network card settings/config based on existing network infrastructure. o Other applications or systems sharing the same network links. o How communication flows between all servers (routing, linked servers, firewalls) o How internet users access your webservers The workload expected vs reality and peak periods. o Total Number of Clients, connection method and pooling Varied client usage patterns, including batch jobs throughout the day,month,year Network design, routing, packet timeouts/loss, competing with other applications

Collecting SQL Server Statistics

When running performance monitor, take care when setting the interval. The collection and reporting is based on an average between the current and last collection, therefore, if you are monitoring every 5mins, but your CPU time is spiking throughout this period, the overall average may be somewhat low and will not reflect this in your graphs. A consistent spiking in CPU time is typically related to poor performing application code and/or SQL, if multiple users happen to hit the same code at the same time, this spike will of course run longer but still may not reflect in the graphs. This may result in good server performance statistics but poor application performance (end user experience). If you are using COM+ (component services), look at the properties for each installed package, it will show: o #objects instantiated o #objects activated o #objects pooled o #objects in-call o #objects call-time(ms) this is very handy when tracking possible issues with DLLHOST.EXE processes going wild with the CPU. To locate the PID for a COM+ package, select properties for the COM+ Applications folder within Component Services. Profiler is the key utility for collecting SQL and other SQLServer internal event activities. It is important that the DBA has a good understanding on this trace utility and the numerous parameters and events that can be monitored. Dont run Profiler on same server as you are monitoring. If collecting statistics to a file or a database, dont write to the server you are monitoring. Index tuning wizard

General DB Parameters, Settings and other DBA items

Turn off the auto-close and auto-shrink options for your databases Manually configure database memory settings to control SQL Server memory usage and its potential impact on the server. I rarely recommend any internal database setting changes unless there is a real, testable need for it. Even so, setting such as max-async-io, recovery interval, min memory per query, max degree of parallelism, max worker threads and common areas that DBAs consider altering. The DBA should put in place:

163

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Script (or use maintenance plan) to reindex user databases on a weekly basis Dbcc Dbreindex Dbcc indexdefrag Consider a script to manually update statistics daily Dbcc updateusage Update statistics.. Do not alter locking and other advanced options without direction from Microsoft OR you are completely satisfied that the change IS making a difference and is repeatable on dev/test before moving to production. The DBA must be able to monitor it over-time. Use dbcc traceon and dbcc traceoff as required, some flags are at instance startup only via T parameter.

Physical Database Design

Determine the requirements for fill-factors early. The fillfactor not only applies to indexes but also affects the physical table themselves if clustered indexes used. Consider the following settings: o Very few updates/inserts (DSS/OLAP) 0 or 100% o Minor insert/delete/update activity 90% o OLTP system Reference data 90% to 100% Other 50 to 70% Heavy inserts/updates 25 to 45% Utilise file-groups (FG) with one or more physical data files (file-group striping) to assist with separate data structures from one another, for example: o PRIMARY sys only objects o DATA_FG all tables (made default FG) o INDEX_FG all indexes o AUDIT_FG all audit tables File group considerations o # files per group? o what datafile should be placed on which storage set? o what file groups do objects reside? o FG backups vs other Do not use multiple files for transaction logs. A shorter row length = more data per page (also consider pad and fill-factors) = more data in buffer cache per read Use varchar instead of char unless its: o Always of a set, fixed length o Very short (i.e. Y / N), if so, consider bit data type for 0 and 1 indicators Avoid uni-code data types where possible (eg. Nvarchar etc) Only make columns large enough (namely varchar data type) to support what is really needed. Consider using the TEXT IN ROW feature for S2k. This allows you to control the storage of smaller amounts of text and image data onto the data page. See article http://www.sqlservercentral.com/columnists/bknight/textinrow.asp for a thorough summary. Consider alternative datatypes for numerics based on the following:

164

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

o Bit (0 or 1) o Tinyint (0 to 255) o Smallint (-32768 to 32767) o Integer (-2,147,483,648 to -2,147,483,647) o Bigint (-9,223,372,036,854,775,808 to -9,223,372,036,854,775,807) o Money (-922,337,203,685,477.5808 to +922,337,203,685,477.5807) o Smallmoney (-214,748.3648 to +214,748.3647) o Float (-1.79E + 308 to -2.23E 308, 0, .23E + 308 to 1.79E + 308) o Real (-3.40E + 38 to -1.18E - 38, 0, 1.18E - 38 to 3.40E + 38) Consider smalldatetime if the nearest minute is ok. Avoid GUIID columns, but may be unavoidable in some forms of replication Primary keys are, by default, clustered indexes. In many cases this may not be the ideal candidate and other columns with range based scan characteristics may be a lot more optimal. Index foreign keys not already part of the primary key to speed common table joins. For disconnected/stateless environment, consider the 3 columns: o Last_updated_on o Last_updated_by o Update_count for all tables to manage concurrency issues, but is also handy with applying batch updates where you can use the last_updated_by column to mark the rows altered. Review auditing requirements as early as possible, retention periods, and the use of these audit tables (in reporting and general lookup). Consider covering indexes (non-clustered index for every column in the table) for selected tables with heavy read characteristics. Indexed views o Can take a significant performance hit in write intensive applications o Must create view with schemabinding option, when done, you cant alter table structure until the indexed view dropped. o In a situation where SELECT statements retrieve a result set that is an aggregate, including summation or other calculations from one or more different tables. In this case, a pre-aggregate result set stored as an indexed view speeds up read performance tremendously. Consider using CREATE STATISTICS where you believe the optimizer may benefit from information about data distribution Use non-clustered for highly selective columns (95% or more of results can be eliminated) and clustered for range lookups. Remember that primary keys are, by default, clustered indexes unless altered. This rule is not hard-and-fast, and should be carefully considered early in the project as there is only 1 clustered index per table and it can be difficult to alter this at a later point in time. Use identity columns carefully. Follow standard normalisation practices. Dont us identity because its simply convenience, sure, single column numerics keys are fast, but can have adverse affects with: a) replication b) readability and maintainability In the end, the because its simple and easy attitude should not be you key reason for sequenced columns. Begin to evaluate the need for federated databases early in the design phase. It will be very costly to change to this infrastructure later.

165

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Evaluate as soon as possible database links (local and or remote ones) to the network and what applications will they be sharing the data lines with (also the server performance of the remote connection). Consider indexed views and indexed computed columns, this may be a result of denormalisation to some degree. This will impact insert performance. Consider computed columns over standard de-normalisation practices: o Example: create table mytab (col1 integer, col2 integer) alter table mytab add mycalccol as col1 + col2 creates a new column with the formula property set of data type INT. If one data type was a int and the other float, the calculated col will take on the float datatype. You can now create an index on the computed column (known as a function based index) so long as the calculation is deterministic. Will full-text indexing be used? o Use timestamp datatype for all tables to track incremental updates o Plan well ahead of time with server administrators on CPU and IO impacts, they can be significant.

SQL and Developer Tips

Minimise network round trips where possible. o utilise stored procedures and consider carefully the use of the FOR XML clause to return XML wrappered result sets. o Use SET NOCOUNT ON for all stored procedures once debugged and ready for testing/production. o Use stored procedures for all queries rather than embedding SQL logic in your COM/COM+ VB code at the middle tier. The code will run faster and it will be much easier to debug and perform impact analysis due to database changes. Only select what you need to, i.e. SELECT * FROM is not required in any case. Use the ANSI SQL standard where possible, i.e. INNER/OUTER etc join clause syntax to assist the parser in generating plans. Review the indexing scheme used on tables before writing queries and suggest new indexes to the DBA as required. Search arguments o Optimal exact match, range of values, joins with ANDs o Sub-optimal not, not in, !=, <>, NOT EXISTS, computations prior to data access. o Between equivalent to >= and <= ranged lookup o LIKE if % is used at the start of the search criteria, an index cannot be used. Deadlocking o Describe isolation levels and the SQL Server locking mechanism with senior development staff and document the strategy. o Access tables in the same chronological (name) order and DDL operation order (if possible). o Keep transactions short as possible, take care with COM+ transactions which use a serialisable isolation level. o Reduce large, range DML operations where possible.

166

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Consider locking hints (in liaison with the DBA) where appropriate to reduce locking issues. Describe the following concepts to developers o Selectivity of indexes (# rows returned) o Join types o Index usage and control o Storage (row/page/extents and logical vs physical page access) For concatenated index, the 1st column in the index must be used in order for the index to be included as a valid option for the optimizer. In most cases, NOT EXISTS will out perform inner join and NOT IN alternatives (based on appropriate indexing). Take care with the WITH COMPILE option for stored procedures, unless the stored procedure uses dynamic SQL, there is no need to re-generate the execution plan. Dont name stored procedures with sp_ for its prefix as a master db lookup will occur first, then a dbo owner check, then a check based on the users DB login name. Objects that are called within the same stored procedure should the same owner, preferably dbo, own all. If not, SQL Server may need to do further name resolution to resolve the objects (all objects should be dbo owned as best practice). Developers should have some basic understanding on reading the plan output. Some notes on the SHOWPLAN option: o Use SHOWPLAN_ALL in SS2k, but will not actually run the SQL, therefore IO and TIME set options will return nothing. o Query type (insert, update, delete, select) o Tables listed in order of processing o For each table the access method is indicated (table scan, index lookup etc) o Parallelism is indicated o Worktables may be included temporary table in tempdb o Aggregates may require many steps Vector (group by used in conjunction with aggregate function) Scalar (aggregate function used in select statement) o Exists table exists, in, or,>=, any used o STEPs, where SQLServer required many steps to retrieve or access data Step may involve sorting Getsorted temp work table created for sorting Some notes on the STATISTICS IO option: o Also see STATISTICS TIME option o Logical reads total # of pages accessed o Physical reads - # pages read from disk into buffer cache o Read ahead reads # pages cached by read-ahead manager o Scan count - # times table was accessed When attempting to tune queries, consider dbcc dropcleanbuffers and free proc cache to clear out the data base buffer cache. Sorting o Optimiser will sort when distinct, union or order by clauses are specified o If no order by criteria is specified, records are returned randomly based on buffer cache lookup. o Default is ascending order o

167

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Backup and Recovery

Backup Destination o Is this a dedicated disk or also used for database files? o How many disk spindles are in use? o Is the backup over a network connection that is already very busy o If you are going directly to tape, ensure stream speed is not imposing on your backup window. Restoring Databases o Purge MSDB backup history on a regular basis. If you are using the GUI to restore and you select a DB that has a lot of historical data, the GUI may take some minutes to respond whilst it reads the sys tables. The SQL Server backup command is very slow and produces large files, as it uses very little of the CPU. Consider SQL*Lightspeed (3rd party product) for a fantastic performance increase and much smaller backup files.

CPU is a Bottleneck?

Have you accurately determined where the CPU bottleneck is occurring?, can it be repeated successful for further testing, and is it kernel related or user process related. Get faster CPUs or add additional CPUs (do not forget the licenses). Get CPUs with a larger L2 cache. In most cases, simple opt for faster or more CPUs Re-check application components and try to get an accurate measure between database activity vs business layer activity. For example, is the wrappering of record sets with XML the real problem vs database query performance issues? Move some of the processing load to another SQL Server. Consider turning on Windows NT fibres. Be sure that both OLTP and OLAP queries are not being run on the same server. These different database applications should be performed on separate servers. Re-check your statistic, namely kernel time, if kernel time is high recheck your paging statistics and other IO statistics as this may be a flow-on affect.

I/O is a Bottleneck?
Consider file-group striping, create one file per disk array and re-import data, data will be evenly striped over the file-groups physical file set, for a sequential read a separate thread is created for each file in parallel. Add additional physical RAM, checking paging. RAID level 10 is the fastest RAID level you can choose that supports redundancy. Add more physical drives to the current arrays. This helps to boost both read and write access times. But don't add more drives to the array than your I/O controller can support. Replace your current hard drives with faster drives. Review SCSI channel usage, controller cache and the connection itself (i.e. fibre interconnect?) Add faster or additional I/O controllers. Consider adding more cache (of possible) to your current controllers.

168

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Revisit SQL tuning and index utilisation. Revisit buffer cache hit ratios and memory utilisation. Move transaction log to its own dedicated RAID-1 array. Move tempdb to RAID-1 or RAID-0 (no redundancy though).

Task Manager
This sounds obvious, but many people seem to draw a blank at times and race off to use profiler and performance monitor when simply opening task manager can give you a good snapshot of the state of the system and its running processes.

You cannot kill database threads, only the entire sqlservr.exe process. Also see the kill command at the NT level (may need to source from resource disks).

Selecting different tabs will alter the options available under the View menu option. The key tabs are: a) performance b) processes Select the process tab, the select columns option as shown below provides a good set of columns to include for real-time process monitoring and problem identification. Sorting the columns over, say, CPU or memory usage is a great start.

COM+
It is worthwhile spending a little time investigating how to view COM+ activity. The Component Services (also known as DTC or Distributed Transaction Coordinator) program installed with the OS allows the user to install and subsequently manage COM+ packages, an example screen shot is shown below:

169

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To view runtime statistics, highlight COM+ Applications in the left-hand panel and click the properties icon in the top menu-bar. Shown is the list of COM+ packages installed and their running stat with associated PID. To drill down, open up the Windows Task Manager:

170

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

We can drill further into COM+ and go to a class level within the package. This allows is to monitor the number of instantiated objects, wether they are activated and pooled, in-call and the call time in milliseconds.

Component Services under XP is a little different that Windows 2k Server and Advanced Server. In XP the class strings (as found in the registry) are shown. Also, utilise the details display on components at a variety of levels to view other information in regard to the user which the objects are instantiated, their activation context etc. Going any further requires intimate knowledge of the application, but further profiler tracing may assist you.

Isolation level of COM+ components

Be aware that COM+ components utilise a isolation level of serialisable (Q215520). As with all transactions in SQL Server, keep them small and fast as this sort of isolation level can destroy your performance. In .Net Server and Enterprise Services (COM+ v1.5), this can be configured to serializable, repeatable read, read committed, read uncommitted, or any. The developer should consider altering the isolation level on connection to the instance where possible to read committed. There is a great article by Alan Gordan regarding COM 1.x titled Generation 1.X: A New Era for COM+ that is worth a read for all SQL Server DBAs. Take time to understand the issues with COM+ and the new features for enhanced scalability and manageability.

171

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Proxying your COM+ components

On monitoring your COM+ components, you may find some are hitting the servers harder than others. Depending on your scenario in terms of server availability and your overarching server infrastructure, consider proxying off you COM (known as DCOM) to another server. Here is an example scenario:

Server A (WebServer)

Server C (Active Directory Server)

UserSecurityAccess.dll (proxy very small CPU footprint)

UserSecurityAccess.dll (using 30-40% cpu time was moved from webserver and called via a COM+ proxy, turning the AD box into an application server as well as servicing its standard LDAP requests).

Stored Procedures
The use of stored procedures is the most effective way to: a) reduce network bandwidth between the business layer and the database server b) manage security it is not uncommon in SQL Server applications to only grant stored procedure access rather than access to the underlying tables/views. c) Ease of administration, tuning and maintenance much easier for the DBA to review SQL and for developers to debug SQL problems or alter code without recompiling. d) Higher probability of plan reuse and caching

Using SET NOCOUNT

When you execute a stored procedure, a variety of information is sent back to the client indicating the number of rows affected (and other information). Generally, it is rare that the client (or middle tier COMs) require this information, whose effect increases network traffic. This is especially evident in large stored procedures with looping clauses and DML activity, sending a message for each DML executed. Turning this feature off is always best practice unless there is a specific application requirement for it to be left on. To turn this off in T-SQL we use:
create procedure myprocedurename AS SET NOCOUNT ON <..various code here..> GO

172

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The DBA can turn this off globally via (current logins not affected):
exec sp_configure user options, 512 reconfigure

Or via the instance startup flag 3640, eliminating the DONE_IN_PROC messages globally for all connections.

Summary - Top Performance Problems

Here is a broad summary of what we discussed throughout this chapter in terms of common performance problems: Blocking, locked/waiting and long running transactions, incorrect use of COM+ transactions or mix of OLTP and DSS based transactions, poor selection of isolation level. Non-fixed database instance memory settings chewing all server memory, or very little memory available for a very busy instance. Auto close and Auto shrink database options are enabled Missing indexes, poor understanding of application SQL and indexing requirement, composite keys not being used due to leading columns not be part of the where clause(s), poor selection of clustered index verses heaps, poor key selectivity. Long time between index defragmentation or re-collection of statistics (histograms) Auto-statistics turned off at the database or individual indexes High recompilation and cache misses due to large numbers of adhoc queries or SQL using hard-coded predicate parameters preventing reuse Server shared with other applications resulting in a conflict of interest in terms of resource usage. Disk IO and IO distribution (affects associated RAID level, array cache and disk queue lengths) High page splitting due to a poor understanding of DML activity against user databases and their clustered indexes. Lack of server memory Incorrect use of hints SET NOCOUNT is not being used in your stored procedures, creating additional network bandwidth between server and client (or middle tier).

173

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Not utilising stored procedures for the data tier or returning XML directly from the DBMS and wrappering XML tags manually thereafter.

174

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

High Availability
When we talk about high availability, we are primarily focused on seamless failover of our servers hosting the applications they are running; and the technologies to support the continuation of service with as little interruption to the business as possible. The solution you come up with will be dictated by the realisation of its: a) value-add to the business and customer expectations (latent and blatant) b) cost of system downtime and manual cutover c) issues of business continuity and system reliance The problem you tend to have is that systems typically grow into this realisation rather than being born with it. As such, the DBA and system architects must carefully consider the overarching issues with application state, server configuration, OS and DBMS editions purchased, technologies being used (clusterable?) and to some degree, brail the future environment in which the application will live and breath. Throughout this chapter, we will compare and contrast the high availability options to support your DR plan and assist in realising the possible issues in advance.

ne of the most important issues for many organisations revolves around disaster recovery (DR), which goes hand-in-hand with the topic of high availability.

175

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Clustering
Oracle RAC and Application Clustering
Trying to diagram the architecture is difficult, but the following provides a basic overview:
Node 1 (server 1)
MyDBInstance

Node 2 (server 2)
MyDBInstance

Buffer Cache Redo Log Buffer Variable part of SGA Global Resource Directory LCK LGWR DBWR Global Cache Service Global Enqueue Service

Buffer Cache Redo Log Buffer Variable part of SGA Global Resource Directory

LMSn LMD
LMON

Global Cache Service Global Enqueue Service

LCK LGWR DBWR

Interprocess Communications (IPC)

Redo Log Files SAN (storage area network)

Redo Log Files

The following key points (not definitive) should be noted about this architecture: Known as Oracle RAC (renamed and enhance version of Oracle OPS) Does not utilise the Microsoft Clustering Service The database instances in the cluster are called component instances. The Oracle Net listener provides load-balancing, connection-timeout failover, interlistener connection failover etc of client connections amongst cluster nodes, supporting shared and non-shared connection models. Client network configuration settings can be altered to specify multiple listeners and the failover path the client will take.

176

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Server vendors provide OSD (operating system dependent) cluster-ware to access the underlying OS and for cluster communication regarding the database instance. The OSD is made up of: o Cluster manager (CM) oversee inter-nodes communications o Node monitor polls status of cluster resources, include DB instances o Interconnect IPC controls messaging amongst the nodes Global Daemon Service (GSD) runs on each node, this coordinates with the CM and deals with client administrative tool requests. It is not an Oracle instance process. LMSn (Global Cache Service, GCS) locate, prepare and transmit buffer cache blocks to multiple instances. The same block may exist over multiple caches and can exist in different modes (null, shared, exclusive). The GCS allocates roles and resource modes to control concurrency and the instance will retain read-consistent block versions via RBS entries. LMON (global enqueue service, GES) monitor global enqueues and resources and perform enqueue recovery. LMD (global enqueue service daemon) manages incoming remote resource requests LCK (lock process) manages non-cache fusion resource requests (library, row cache requests) DIAG (diagnostic daemon) diagnostics amount process failure and alert log management. The GCS and GES maintain the Global Resource Directory to record resource information. This is an in memory data structure that is distributed throughout the cluster. Each instance has a part of the directory in the SGA. GCS and GES nominate one instance to manage all information about a particular cluster resource, this instance is regarded as the resource master. Interprocess Communications (IPC) asynchronous queued message model to route messages and other comms traffic to coordinate other node resources. May be either ethernet or fibre-interconnect. Multiple servers are working together against the same data files, offering great scalability and performance. On NT servers, you still require the Quorum (voting) Disk, Oracle stores essential cluster configuration information here. All administrative tools (Enterprise Manager etc) must also be on a shared disk, typically the Quorum disk.

Taking this one step further in terms of a complete Enterprise solution, we have many alternatives based on the environment we are running against. Oracle has made distinct strategic moves to Java where possible, and in many cases, is the core development environment over PL/SQL and/or Oracle Forms and Reports for medium to large scale development. As such, we have seen the expansion of the Oracle Application Server and associated middle tier components (web servers, caching servers etc) to encompass the Java suite of modules. Clustering in this environment is complex and is out of scope of this e-book, even so, see references (54) and (55) as a starting point.

177

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

There are many scenarios as you can imagine in the complete clustering solution in Oracle, but here is an example:

Oracle9i Application Server Clusters Web Cache Node 1 Web Cache Node 2 Web Cache Node 3 OHS/OC4J Node 1 OHS/OC4J Node 2 OHS/OC4J Node 3

Oracle RAC DB Node 1 SAN

DB Node 2

DB Node 3

The iAS Web Cache providers server-side caching for dynamic content over HTTP. This caching takes into consideration cookies, http request headers, parameters and more. Result sets from Java or PL/SQL procedures can also be cached and subsequently clustered. The OC4J represents the possible J2EE cluster (Oracle Containers for J2EE). This includes the clustering capabilities for Servlets, JSP and EJBs. This also includes the Java Object Cache. The OHS component represents the Oracle HTTP Server. This typically comprises of a webserver (apache), Perl execution environment and PL/SQL and J2EE container routing. In terms of Oracle and Microsoft Web Development, although not technically part of this chapter, we typically see the following comparisons: a) PSP, JSP ASP, ASPX T-SQL Stored Procedures COM+ Components, Enterprise Services

b) PL/SQL Stored Procedures

c) Servlets, Enterprise Java Beans d) Oracle RAC e) OHS/OC4J f)

MSCS and SQL Server Virtual Instances Microsoft Application Centre (COM+ clustering) IIS 5 or higher, Proxy Server

Web Cache / HTTP Web Server

SQL Server Clustering

Clustering is a big topic that is an e-book all on its own. Within the section we will cover as best we can a variety of key issues that DBA and systems administrator must consider when utilising clustering as the ultimate high availability option in SQL Server 2k. The Oracle DBA will firs of all recognise that, when compared to Oracle, SQL Server is a long way behind in

178

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

terms of incorporating high availability with a horizontally scalable performance gain (i.e. many servers utilising the same database data files at once). Before we begin, remember that SQL Server Clustering is designed to provide high availability, not a
load-balancing solution.

The Microsoft Windows and SQL Server cluster solution is a share nothing resource model, meaning that one and only one database instance can run on any server in the cluster at any one time. Therefore, if we had 4 nodes (servers) in the cluster, one and only one node can actually accept connections and run against the database files for the instance. This adversely means that the clustering solution does not provide horizontal scalability as Oracle does. An example is shown below we have an Active/Active cluster which we will explain in more detail.
Connect by specifying virtual server (cluster enabled SQLServer with own IP or network-name) and the database instance name

Node 1 InstanceA (virtual instance) InstanceB (failover) InstanceA (failover)

Node 2

failover

failover Heartbeat (private network) Interconnect

InstanceB (virtual instance)

Cluster Service (MSCS)

SAN or SCSI/Fibre connected Disk Array Cluster Quorum Drive (Q:), Raid 1 Transaction Log Files (M:), Raid 1 Data Files (N:), Raid 5 Transaction Log Files (M:), Raid 1 Data Files (N:), Raid 5

179

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

IMPORTANT Remember that when we talk about an instance we dont mean a specific database. See the first chapter for more information if this confuses you. The configuration of a cluster is not overly simple and I would highly recommend visiting Microsofts website and reading up on their comprehensive technical manuals beforehand. Here is a summary of the issues for installation and configuration: a) 2 or more cluster nodes a. node is simply a physical server running Windows 2000 Advanced Server or Data Center Server. The operating system determines the number of nodes per cluster supported, where Advanced Server supports two nodes and data centre server supports 4 failover nodes. i. Cluster IP for all nodes in the cluster, external connections use this ii. Cluster Server Name - WINS name of the cluster for all nodes iii. Cluster Administrator Account - administer and own the MSCS, it is a domain level administrator account for all nodes. iv. Cluster Group(s) collection of cluster-aware applications (i.e. SQL Server virtual instances) or resources. b. all of which run in the same windows domain (cross-domain clustering is not supported) c. Each node has a minimum of two network cards. The public and private networks must appear (or appear to be in) in a non-routed LAN. i. One for the private network that interconnects all nodes in the cluster 1. disable NetBIOS for the private network cards. ii. One for the public network connection 1. nodes may be remote, i.e. over an VPN so long as the ping rate is <= 500ms b) Disk array a. One or more disk controllers installed on each of the nodes b. SCSI or Fibre interconnect c. NAS is not supported by MSCS d. SCSI not supported by Data Center server (fibre channel only) e. Dynamic disks are not supported by MSCS i. This means cluster downtime when adding more disks to the cluster f. File compression not supported by MSCS g. Software RAID not supported by MSCS h. Use RAID 0+1 for ultimate performance, be fully aware of your node configuration and number of instances you want to support as the number of drives can grow very quickly. i. Only one node can control a disk at any one time. j. Remember the disk configuration is based on the SQL Server cluster model chosen (active/active or active/passives). Be very careful with your decision and the capacity of the disk array.

180

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

c) The comclust.exe binary must be run after MSCS has been installed, this is used to create the clustered MS DTC resource. This should occur after one node is installed. d) 1 or more virtual sql-server instances a. a virtual server instance is similar to a standard instance but is cluster aware. The MSS exposes a cluster IP for the entire cluster of nodes, within this we have 1 or more virtual servers which SQL Server installer calls virtual server instances that clients connect too. Both SQL Server and MSCS work in unison to establish user connections and node availability. b. the binaries of the instance are stored on the nodes local disks c. the MSCS cluster must be installed, resource groups allocated and NICs configured before attempting the SQL Server instance install. d. on installation i. select virtual server instance 1. virtual server name virtual server name, this is not technically the name of the actual database instance. This is the virtual sqlserver name, its like a logical server name. 2. virtual server IP enter public IP address in which the node is available for clients to connect, there may be multiple IPs if there are many connecting IP networks. The subnet mask comes from the MSCS. 3. cluster group disk for data files (assumes MSCS cluster group of resource disks setup previously) a. dont use the quorum disk for data files b. all default instance databases (system and sample database) data and log files are installed on the drive you select. 4. pick other nodes in the cluster in which you want to partake in the failover of the virtual instance (by default all nodes are selected). This is also called the cluster management screen of the installer. 5. enter login credentials for the remote nodes, this account must have administrator access to the other nodes. It is not uncommon to use the domain administrator account. This MSCD service account must have login privileges to SQL Server to perform isalive checks. If its the Administrator account this will naturally occur with the BUILTIN\Administrators domain group login installed with SQL Server. 6. enter the database instance name, use a named instance rather than the default to ease maintenance. 7. select destination path for the sql-server instance binaries. 8. enter the domain user name in which the database instance will run. Using a local domain user can be a problem to administer and its highly recommended using the domain administrator account or equivalent. The DBA should keep the Administrator and the SQL Server service account separate where possible. The DBA should

181

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

consider using the same service account for SQL Sever and SQL*Agent services. It should be a valid domain account. 9. select SA account authentication mode 10. select licensing mode ii. use the VIRTUAL_SERVER\Instance-name string to connect to a clustered instance I cannot stress that it is important to understand that the virtual SQL Server, which maps 1:1 to a single database instance, can not share disks with any other node, this is completely different to the Oracle clustering model. The active/active model allows the DBA to utilise the Node2 server with another SQL Server virtual server and associated database. Therefore, both servers are actually working for us and failing over to one another on failure. In this particular model, for example, I can run DatabaseInstanceA on Node1 as my OLTP database, and on Node2 run my DSS/Reporting DatabaseInstanceB simultaneously. Of course, if either server fails I will have a DSS and a OLTP database on the same box, competing for CPU and memory resources which we need to consider when purchasing the hardware solution. NOTE Always set the specific memory maximum the SQL Server database instance can use. Also note that active/active and active/passive are functions of SQL Server clustering and not the clustering service itself. The SQL Server 2000 DBMS is a cluster aware application, in that it utilises the underlying MSCS API. As such, it uses the heartbeat NIC to communicate between nodes to detect node and sql-server instance failure (it cant detect a single database failure in the instance and result in a fail over). The SQL Server instance ping (is-alive check) is a select @@servername query. On five failures the instance is declared dead and a failover occurs. This involves starting the next named instance thats part of the virtual sql-server. NOTE If you need to alter the service account logins for SQL Server, use Enterprise Manager rather than editing the NT services directly. Going back to disk configuration, the quorum disk is critical to MSCS. DONT use your quorum disk for database backups/restores. The MS DTC is usually configured on this resource (500Mb free required). See Q294209 on Microsoft Support for more information relating to MS DTC. As for each database instance, standard database file placement strategies kick in based on the database being installed and its IO characteristics. In general, this tends to mean: a) separate disk for tempdb (if deemed an issue) b) separate disk for database transaction log files c) separate disk for database data files a. this may extend one step further if we use different file-groups, i.e. DATA and INDEX groups which may be place again on their own sub-set of disks. Always remember that we are talking about an instance, not a single database. To finish this section off before we get too carried away, some final notes: a) SQL*Mail is not clusterable as the MAPI protocol is not clusterable

182

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

b) SQL Server Analysis Services (OLAP, Data Mining) is not clusterable c) select * from ::fn_virtualservernodes() to review nodes for the instance d) full text indexing is cluster ready
Ongoing Cluster Administration

To administer the cluster: a) Enterprise Manager and other associated SQL Server utilities a. Especially for database and SQL*Agent service account and password management b. Remember to install on both nodes. c. In SS2k, EM can be safely used for start/stopping instances rather than the cluster manager. b) SQL Server setup a. To install/uninstall virtual servers, add/remove nodes, change or add IP addresses to the failover cluster b. See advanced options after installation to see administration functions c) Widows Cluster Administrator a. Dont use it to add nodes to the SQL Server resource definitions d) CLUSTER command line utility a. As per the above GUI utility. e) comclust.exe

NLBS (Web Farm Cluster)

The DBA should not be limited to the example above in terms of high availability or performance gain. It is worth mentioning the NLBS (network load balanced service, was once known as WLBS), which basically scales (or clusters) IP services such as TCP, along with UDP and GRE across 1 or more servers and runs on all servers in parallel. The NLBS is provided with Windows 2k AS and Data Centre Server and is also part of the Microsoft Application Centre product. Essentially, NLBS assigns a single IP address amongst the servers in your server farm, the set-up of which is done via the network properties screen of each server partaking in the farm and selecting the network load balancing properties. This does require a good understanding of IP networking and I highly recommend getting your network administrator involved early in the planning stage. It is very important to understand that routing requests based on SSL session IDs or cookies, arent possible. This takes NLBS out of the running completely if your Web site uses sessions (35). Even so, the DBA needs to consider the client affinity settings for preserving client state and test accordingly. The affinity settings are: 1. none many requests from a client can go to any node in the NLB cluster 2. single - many requests from a client return back to the same node 3. class c multiple requests from the same TCPIP class-c address range will use the same node in the NLB cluster. The incoming requests are balanced in terms of the percentage of requests they receive and of course overall server availability.

183

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

If you were wondering, NLBS is the clustering technology is used in web-farms.

NLBS

NLBS installed on each node in the cluster and clients connect via a single, static IP and servers communicate via a private network (intra-cluster comms), isalive check approx every 1sec.

In this scenario, (see CLB later), we may be running our business layer (typically COM+ components) on each individual web-server in the NLB server cluster, communicating directly to the database cluster. After talking to a few DBAs regarding the use of NLBS at the database server, they have, apparently, used these scenario(s) below to achieve fail over:
Connections from Web Server Farm

100% load Database Server 1 Database Server 2

MyDB

Transactional Replication or Log Shipping

MyDB

Network Load Balancing Service (NLBS)

The concept is: a) NLBS all TCP/IP database connections to database server one b) We transactionally replicate or log ship to database server two c) The two servers are not running in a cluster, which is implicit from b) d) On failure of server one, database server two is prepared for cutover of the NLBS. e) Database server two may be used for reporting and other functions as need be whilst server one is taking the load.

184

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

I am not convinced of this option and it will require a fair amount of thought before running with the architecture.

CLB (Component Clustering)

Taking the clustering model another step further, we have COM+ clustering, or component load balancing. This feature is only available with Microsoft Application Centre, which is licensed separately per node (you may be surprised at the cost of your cluster implementation at the end of the day!). The CLB service is installed on all servers in the existing web server NLB cluster, we also form a new application server (business layer) cluster managed by the same CLB. The web-server CLB maintains a COM+ routing list per node in the NLB cluster; the list contains all nodes in the application server cluster. The routing list servers are polled every 800ms (approx), logs the response time of each server, ranks them and CLB distributes the COM instantiations in a round robin fashion amongst the nodes. We proxy (DCOM) the COM+ components to the web servers which have the IP of the applications server cluster. The CLB of course manages node failure within the application server.

Web Server Farm (NLBS)

COM+ Proxy

COM+ Proxy

COM+ Proxy

COM+ Proxy

Routing Cluster (CLB)

Application Centre Cluster (CLB)

COM+ COM+ COM+ COM+

Some keynotes regarding this architecture: This model can be taken another step further with a separate COM+ routing cluster, this is not discussed and is not recommend for web-based applications. Microsoft has some excellent documentation related to NLB and CLB clusters, which can be found in the references.

Microsoft Clustering - The Big Picture

185

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

We have concentrated primarily on SQL Server, but to bring clustering in context with a complete highly available and scalable solution we have also discussed (albeit briefly) CLBS and NLBS. The following diagrams show a complete solution utilising Microsoft technology that the SQL Server DBA should be familiar with.

Web Server Farm (NLBS)

COM+ Proxy

COM+ Proxy

COM+ Proxy

COM+ Proxy

Routing Cluster (CLBS)

Application Centre Cluster (CLBS)

COM+ COM+ COM+ COM+

Database Server (MSCS)

SAN or SCSI/Fibre connected Disk Array

186

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Federated Databases
I personally do not regard database federation as a form of high availability, but it is pushed by Microsoft as a key technology to enhance performance. The architecture for federation can vary significantly between installations. An example is shown below.
Client

Web Server 1 IIS

Web Server 2 IIS Load Balancing Service (NLBS)

COM+

COM+

Database Server 1

Database Server 2

MyDB

MyDB

DPV A-K Data Linked Server

DPV L-Z Data Distributed partitioned view(s).

Database Servers with their Member Databases forming the federation can be clustered to increase high availability. Database instance names must be carefully considered.

Table data is horizontally partitioned amongst the member databases which reside on two or more physical database servers. Linked servers (database links) are created between each database server instance and distributed partitioned views (DPV) are created on each member database, for example:

187

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- Database Server 1 (dbserver2 is the name of the linked server) CREATE VIEW dbo.salesdata AS SELECT * FROM mydb.dbo.customersales UNION ALL SELECT * FROM dbserver2.mydb.dbo.customersales -- Database Server 2 (dbserver1 is the name of the linked server) CREATE VIEW dbo.salesdata AS SELECT * FROM mydb.dbo.customersales UNION ALL SELECT * FROM dbserver1.mydb.dbo.customersales With such simple views, they are naturally updateable and support all forms of DML. There are ranges of conditions imposed on the view and the underlying table that are described in the books online within SQL Server. At the table level with each member database we apply check constraint conditions to ensure the partitioning occurs correctly in terms of data integrity. The key to the entire architecture is the business object layer, namely the COM+ routines. The business layer must be federated aware. This can be done in a variety of ways: a) Stored Procedures if the COM+ routines utilise stored procedures for all DML, the procedures can encapsulate the necessary logic to distribute data over member databases. b) COM+ routing call a custom built routing component that has the necessary smarts to communicate to the appropriate member database. The trick here is the connection string used by the COM+ routines. It can be very difficult to move an existing application to a federated model but it is not impossible. If the application has been architected well (yes that would be another ebook), and data access is typically via stored procedures, then utilising distributed partitioned views within the procedures should be completely seamless to the application.

188

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Log Shipping
Conceptually, log shipping in SQL Server and Oracle are the same. The process in SQL Server works at a database level, not at a global instance level as you will see later. The aim here is the create a warm standby server in which one or more databases are permanently in recovery mode. The source database ships transaction log backup files to the destination server and we restore (with no recovery option) the logs sequentially (in order). If there is a failure on the source server we attempt to recover the last of the transaction logs, ship it to the destination server and complete the database recovery. Once done, we send client connects to the new server and they continue to work with little or no data loss.
Windows Domain

Server A (primary)
DatabaseA Client Connections Backup Transaction Log to network share

Server B (standby)
DatabaseA

Restore Log with norecovery on destination server

NOTE - we have not diagrammed the full database backups that must also be log-shipped to start the process. The DBA can use a custom written script or the SQL Server Wizard to do the shipping. The shipping in most cases will be to a server on the same Windows 2k network domain, but there is no reason why VPN tunnels over large distances or other remote scenarios are not utilised.

189

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The supplied wizard provides all necessary steps to setup log shipping for selected databases. Before doing so remember the following: document the process before running the wizard how will the client application components detect the failover? Databases must be in full or bulk-logged recovery mode ensure instance names are the same on source and destination servers pre-determine the log backup schedule for the source database pre-setup the transaction log backup directory via a network UNC path (referred to by both source and destination DTS jobs) create DTS packages on both servers and use the transfer logins job to facilitate the transferring of login information between shipped servers. Must run the wizard via a login that has sysadmin system privileges Once the log shipping DTS (maintenance plan) has been created, go to the Management Folder in EM and select properties of the log-shipping job to alter the process in place. See reference (56) for a documented How-To for log shipping in a SQL Server 2k environment using the Wizard.

190

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Manual Log Shipping - Basic Example

Here we will discuss a working example that was coded using two stored procedures, a linked server, two DTS packages and (optionally) pre-defined backup devices. This example revolves around a single server with two named instances: Primary Database Instance - SECA\MY2NDINSTANCE Standby Database Instance - SECA\MY3RDINSTANCE i. Setup disk locations from primary to destination server and test with SQL Server service account.

Backups on primary server are dumped to \primary and are then shipped via a simple xcopy to the \standby directory from which they are restored.

ii. Pre-determine what account will be used for doing the backups, establishing the linked server and restoring backups on the primary and standby databases. I recommend that you create a new account with sysadmin rights on both servers to facilitate this. iii. Setup Linked Server on primary server to destination server

We will link from the primary server over to the standby server via the account specified in step 2. Data source is the server/instance name we are connecting too.

191

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In this case we are using the SA account which is not best practice, but will suffice for this example.

Apart from the SA account mapping above, no other mapping will be valid.

Ensure remote procedure call options are set to facilitate the calling of t-sql stored procedures on the standby server from the primary server.
* If you are using EM, and have registered the server under an account no mapped in the linked server, then the linked server will no work for you. Check your EM registration before attempting to view tables/view under the linked server within EM.

iv.

Setup database backup devices on primary server This is completely optional and really depends on the backup model you are using. SQL Server allows you to pre-create backup devices, once created, we can use the logical name for the backup device (which maps to a physical file) rather than the using the physical path and filename for the backup. This makes scripting much nicer and easier to read/change. Here we create two devices, one for full backups and the other for logs. See management folder in EM and the backup item to create these.

v.

Check primary server database recovery model. Select properties of the database via EM. Ensure the model is in line with the backups you will be log-shipping.

192

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

vi.

Write two stored procedures that will reside in the master database on the standby server for recovery of the FULL and LOG backups. The standby restore option is what tells SQL Server that the database is in warm standby mode.
CREATE PROCEDURE dbo.StandbyServer_restore_full_database_backup AS SET NOCOUNT ON RESTORE DATABASE logshiptest FROM DISK = 'e:\backups\standby\logship_primaryserver_full_backup.BAK' WITH RESTRICTED_USER, -- leave db in DBO only use REPLACE, -- ensure overwite of existing STANDBY = 'e:\backups\standby\undo_logshiptest.ldf', -- holds uncommitted trans MOVE 'logshiptest_data' TO 'e:\standbydb.mdf', MOVE 'logshiptest_log' TO 'e:\standbydb.ldf' GO

CREATE PROCEDURE dbo.StandbyServer_restore_log_database_backup AS SET NOCOUNT ON RESTORE LOG logshiptest FROM DISK = 'e:\backups\standby\logship_primaryserver_log_backup.BAK' WITH RESTRICTED_USER, STANDBY = 'e:\backups\standby\undo_logshiptest.ldf' -- holds uncommitted trans GO

vii.

Write two DTS packages on the primary server, one to do a full backup and the other a log backups. Package 1 Primary Server, Full Database Backup

BACKUP LOG logshiptest WITH TRUNCATE_ONLY WAITFOR DELAY '00:00:05' BACKUP DATABASE logshiptest TO logship_primaryserver_full_backup WITH INIT WAITFOR DELAY '00:00:05'

exec standbyserver.master.dbo.StandbyServer_restore_full_database_backup

193

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Package 2 Primary Server, Log Database Backup

BACKUP LOG logshiptest TO logship_primaryserver_log_backup WITH INIT, NO_TRUNCATE WAITFOR DELAY '00:00:05'

exec standbyserver.master.dbo.StandbyServer_restore_log_database_backup

viii. ix.
x. xi.

Test Full then Log DTS routines and debug as required Schedule DTS packages Monitor On failure of the primary, do the following
-- Login to primary server (depends on failure), and attempt to backup last database log file BACKUP LOG logshiptest TO logship_primaryserver_log_backup WITH INIT, NO_TRUNCATE -- Login into standby server restore database logshiptest with recovery Deleting database file 'e:\backups\standby\undo_logshiptest.ldf'. RESTORE DATABASE successfully processed 0 pages in 4.498 seconds (0.000 MB/sec). -- Ensure client connections are connection to the now live standby server.

194

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Some thoughts about this setup: a) The full and log backups are being appended to the same file, consider writing better backup routine on the primary server than produces separate files for each backup with a date/time stamp. Do this is a T-SQL stored procedure on the primary database and consider replacing the DTS copy command with a call to xp_cmdshell within the stored procedure. b) If using a), parameterise the two recovery procedures on the standby server to accept the file path/filename of the file to be recovered. The routine in a) will have all this information and can pass it to the standby server without any problems. c) Consider email on failure DTS tasks. d) Consider how you will remove backups N days old? e) Will the standby ever become the primary and effectively swap serve roles?

Online Object Redefinition

In Oracle 9i, we see the release of the oracle redefinition PL/SQL package (DBMS_REDEFINITITION). The process basically allows the DBA to stage schema alterations, for the DBMS to track the changes and to cut them over to the new database structure with a simple command and very little downtime to the end-user (if any). This is a very powerful option that is covered in good detail in the article Don't Shut Down That Database! Use Oracle 9i Online Object Redefinition Instead (34). There is no equivalent option in SQL Server.

195

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

Globalisation Services (NLS)

ith the advent of Oracle 9i, the NLS (national language support) services has been extended significantly and has been rebadged as Oracle Globalisation Support. Previous NLS implementations could only support one national language and character set per database, the architecture can now support many locales (languages, character sets, sort orders etc) loaded at runtime. This effectively means that single database instance implemented correctly can support error messages, sort orders, date/time, monetary, numeric and calendar conventions of any native language and locale, creating a truly global database. This chapter provides a quick overview of Globalisation Support with Oracle and expands this to encompass the equivalent SQL Server options. NOTE Oracle refers to locale as the national language and the region in which the language is spoken (11). This consists of language (national language of the user) and territory (region in which the language is spoken). We do not cover timezone specifics in Oracle.

196

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Oracle Architecture
A simplified overview of the Oracle 9i globalisation support architecture is shown below:
Loaded locale-specific data (eg. French, German etc)

Create database statement we define.. character set used to store data (OS dependent default value or defined), national character set (to manage nchar, nclob, nvarchar2 etc)

Multilingual Oracle Database (Unicode all or select columns)* NLS Are NLS Runtime Library (NLSRTL)

Locale specific server behaviour via DB init parameters (NLS_)

Client environment variables (NLS_) Alter session parameters..

ORA_NLS33 env variable, lxlboot.nlb server boot file and custom edited *.nlb and *.nlt language definition files. This allows the DBA to control language, territory, character set (how characters are mapping to their binary values) and linguistic sort options (collations) available to the oracle database on start-up (boot) of the instance.

SQL statement parameters.. Locale Builder (GUI) Locale specific controlled behaviour at the client as shown above. Client applications can utilise APIs to manipulate NLS parameters.

* Unicode is required for true globalisation and rendering of a content to many languages rather than a select few.

The NLS_LANG parameter allows the DBA and client to specify: a) Language for messages, sort order, day and month names b) Territory default date, monetary and numeric formats c) Character set the languages character set to be used (eg. US7ASCII) The DBA can check NLS parameters via the following dictionary objects: NLS_SESSION_PARAMETER NLS_INSTANCE_PARAMETER NLS_DATABASE_PARAMETER V$NLS_VALID_VALUES SYS.PROPS$ In order for an Oracle database to be global (multi-lingual applications rendering content in the countries native language), the database is created to encode stored data as Unicode (unique character for any character regardless of platform/OS/language etc). The Oracle database is encoded with a Unicode v3.1 UTF-8 compatible character set such as on its creation:

197

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

UTF8 (Unicode v3.0) ALU16UTF16, AL32UTF8 (Unicode v3.1) UTFE (EBCDIC formats)

If the DBA does not want to encode the entire database with the Unicode character set (for whatever reason), the table column data type NCHAR (or NVARCHAR2) can be used to store exclusively Unicode characters. As shown in the architecture diagram, the client and server can alter NLS_ parameters accordingly to utilise different cultural settings shared over a single database and rendered appropriately. NOTE NCHAR is stored as UTF-16. To manage sort (collation) options, Oracle caters for: Binary sort sequence based on numeric value of characters defined by the encoding scheme. Linguistic each character has a numeric value representing the characters position in proper linguistic order, the kinds of include: monolingual (eg. generic_m, thai_m, japanese_m) multilingual (eg. Arabic, latin, ASCII7) Multi-lingual indexes can be created to speed linguistic sort performance.

SQL Server Architecture

The SQL Server architecture, in some respects, encapsulates many of the features of Oracle but with significantly less control over the internals of collation(s) and associated parameter settings at the client and database level. This is the model SQL Server has opted for in many areas within the DBMS and is evident here with SQL Server 2000s version of Globalisation Support. NOTE - There are significant differences between earlier versions of SQL Server Collation and Sort Order. This chapter will not cover them but the DBA should be fully aware that it might not be a straightforward task moving to SS2k. First of all, SQL Server uses the term Collation rather than the Globalisation Support or National Language Support (NLS). Even so, the related concepts are similar in meaning to that of Oracles in most respects as all are based on ISO standards. Collation is the rules governing the proper use of characters for a language or character set. This includes the following three properties Sort Order for Unicode Data Types Sort Order for Non-Unicode Data Types Code Page used to store Non-Unicode Data o Uni-code data has a fixed storage structure for string data o Code page is sort order independent

198

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The sort order under SS2k is identical for both uni-code and non-unicode string data, the DBA can not specify different sort orders (unlike previous SQL Server versions) for such string data. The sort order defines rules for the interpretation, comparison and presentation of string data, this includes case, accent, kana sensitivity or simple binary order. NOTE Unicode can be encoded as UCS-2, UTF-8 or UTF-16 (and others). UCS-2 is used by NT v4.0 , SQL Server v7 and 2000, UTF-16 is used by Windows 2k. SQL Server 2k uses the Unicode 2.0 standard. The collation (language) of SQL Server exists at a numbers of levels: a) On installation of the SQL Server Instance (named or default) a. This sets the collation of the master, msdb, model and tempdb system databases b) On creation of a User (non-system) Database a. If not specified, uses the collation of the model system database. Use the alter database command to alter the tempdb collation for subsequent user database creations if need be. c) At a column level within a table object a. For string data types columns only (not using Unicode data types of course) d) Within expressions Altering the collation of a database does not automatically rebuild existing string data to the new collation (and possibly code-page), therefore, the DBA must BCP (or use DTS for import/export) out the data and reimport to take up the new collation (remember - the DBA may need to alter the collation for all table string columns individually). Before we drill too deeply into SQL Server Collation, let us diagram the architecture. The architecture is closely associated with the underlying operating system (known as Windows Locale) as shown below:

199

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Server / PC Windows Operating System Windows Locale (regional settings)

SQL Server language and its code-page associated with an equivalent Windows Collation (locale)

SQL Server Database Instance

(Collation and Sort Order used by all system DBs)

master

model

msdb

tempdb

TEMPDB collation used for temp tables/sorts etc unless specified by programmers. MODEL DB collation used if not specified for user databases on creation.

User DB (collation) TABLE String-Column (collation)

Data access method defines the rules for text manipulation. SQLOLEDB for example data is converted to Unicode using the collation of the actual data.

SS2k Server Network Utility (listener)

Defines the interpretation of character data, keyboard locale, date/time, numeric, currency etc..

Client

Internationally aware app

WINDOWS LOCALE

Some of the many key issues related to this diagram include: Collation & sort order on installation of the instance affects all system databases. The model database collation is used for all users databases that dont explicitly use a collation on its creation, this can be altered if need be. Temporary objects created in TEMPDB will use this system databases collation. The database is re-created on start-up from the model database, therefore, to alter simply change the collation on the model database and re-start the SQL Server instance. The DBA cannot alter sort order after the installation of the SQL Server instance. SQL Server can support only code pages that are supported by the underlying operating system. Multiple collations can use the same code page for non-Unicode data

200

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Code page is a character encoding system used when storing string data (char, varchar, text) for different alphabets (non-unicode), it defines the bit-pattern for each character. Client uses the code pages associated with the OS locale to interpret the character bit stream from the database. An international aware application may include further resource files to provide a truly international application without recoding. International system should utilise Unicode data to ensure all computers perform the correct translation when interfacing to the OS code page.

A SQL Server collation is associated with one and only one code-page (which defines string datas physical storage), for each code page, one or more Windows Locates (identified by the LCID or locale ID) are associated with it, and therefore, supports the code page translation. For example: SS2k Collation Latin1_General SS2k Code Page 1252 Windows Locale Afrikaans Basque Dutch(Standard) Italian <..etc..>
Windows Collation (locale)

SQL Server Collation

Code Page (non-unicode) or Raw storage (Unicode)

IMPORTANT - SQL Server can support only code pages that are supported by the underlying operating system. SQL Server will give an error if there are compatibility issues. The Collation Name is specified via a unique string, this is representative (to some degree) to language and the sort-order and sometimes the code page for the language that forms the colation, for example: CP[code-page] BIN CI CS AI AS WS KA Code Page Binary Case Insensitive Case Sensitive Accent Insensitive Accent Sensitive Width Sensitive (blank for insensitive, Unicode, full&half) Kana Sensitive (blank for insensitive)

Just to be totally confusing, dont rely on these coding to accurately reflect the actual collation. To get a list of valid collations: SELECT * FROM ::fn_helpcollations()

201

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

NB - fn_helpcollations() is a function in the master database. The :: must be used for the function to be included in the FROM clause of a SQL statement. Example: SQL_Scandinavian_CP850_CS_AS Finnish-Swedish, case-sensitive, accent-sensitive, kanatype-insensitive, width-insensitive for Unicode Data, SQL Server Sort Order 59 on Code Page 850 for non-Unicode Data When we say SQL Server Collation, we mean all collations valid within SQL Server. Note that SQL Server does present us with two key language designators, they being SQL and Windows, where the SQL designators are provided for backward compatibility to previous SQL Server versions. Some example SQL-specific code pages (also known as sort order) are: SQL_Latin1_GeneralCP850 SQL_Ukrainian_SP1251 SQL_Polish_CP1250 All unicode strings utilise the SQL-92 standard N'' literal prefix, for example: DECLARE @mystring NVARCHAR(50) set @mystring = N'mystring' All system table string data uses unicode data types.

Collation Installation (Server Level)

In many cases the DBA will utilise the default collation during the installation of the instance. In my particular case its Latin1_General_CI_AS. Always use the windows collations listed, rather than the backward compatible _SQL collations. The installation screen is simple and requires not further explanation. To retrieve the server (instance) collation: select convert(char, SERVERPROPERTY('collation')) The instance collation will affect the language for all system databases, including the model database. As new user databases are created they will, by default, get the model databases collation. To alter the instance collation you need to use rebuildm.exe on the installation CD. This is not a simplistic task, read the books online carefully and review the chapter on backup and recovery.

202

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Collation Database
Specifying the collation when creating a new database in EM is very simply and will not be covered. To specify the collation at a SQL statement level:
CREATE DATABASE [mydb] ON PRIMARY (NAME = N'mydb_Data', FILENAME = N'C:\mydb\data\mydb_Data.MDF' , SIZE = 1, FILEGROWTH = 10%) LOG ON (NAME = N'mydb_Log', FILENAME = N'C:\mydb\log\bb_Log.LDF' , SIZE = 1, FILEGROWTH = 10%) COLLATE Albanian_BIN

The default database collation is sourced from the model database. All table columns (string data only of course) will take on by default the databases collation setting. You can alter the database collation via: ALTER DATABASE mydb COLLATE Latin1_General_CI_AS This will not magically alter and port all subsequent column collation data to the new database value. Do not rely on selecting properties of the database via EM and get the databases collation. Typically, the collation is shown under the maintenance section of the property dialog. In the example below it is blank where we have used veitnamise over the Windows default for our installation, that being latin general (latin1_general_ci_as).

The routine: select DATABASEPROPERTYEX( 'mydb , 'Collation' )

-- returns NULL

This may be a bug under XP? Use the following command instead: exec sp_helpdb Some important notes: a) all users must be disconnected before attempting the alteration
Server: Msg 5072, Level 16, State 1, Line 1 ALTER DATABASE failed. The default collation of database 'pubs' cannot be set to Vietnamese_CI_AS_KS. b)

no schema bound object can be dependent on the instance collation

203

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

materialised views check constraints computed columns table level functions

-- Materisised view, created with the option WITH SCHEMABINDING Server: Msg 5075, Level 16, State 1, Line 1 The object 'myview' is dependent on database collation. -- Check constraint Server: Msg 5075, Level 16, State 1, Line 1 The object 'CK_emp_id' is dependent on database collation.

Collation Table Column

Column collation can be specified for all character data types. In most cases, consider using Enterprise Manager when editing or creating table to see how the collation is specified and the SQL syntax used. When scripting tables from Enterprise manager the default collation (which comes from the databases collation setting) is also specified. The collation value for the column within EM is shown as a property of the column. The following dialog is shown when altering the column. Remember that the databases collation is used by default.

The databases collation is used by default and affects the dialog value shown. Windows Collation and SQL Server designators are the options for the DBA.

The DBA can alter the collation via SQL as the examples show below:
CREATE TABLE dbo.mytable ( mycol char(10) COLLATE Vietnamese_CI_AS_KS NULL

204

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

) ON [PRIMARY] GO ALTER TABLE mytable ALTER COLUMN mycol char(10) COLLATE Latin1_General_CI_AS GO

Multi-lingual columns are not supported. The database collation is used if the column collation is not specified.

Collation Expressions and Variables

Variables, stored procedure parameters and string literals can also utilise the COLLATE option as shown in the examples below.
DECLARE @mystring varchar(100) COLLATE SQL_Latin1_General_Cp1_CI_AS SELECT 'hello world ' COLLATE Vietnamese_CI_AS_KS CREATE PROCEDURE MyProc @myvar varchar(10) COLLATE SQL_Latin1_General_Cp1_CS_AS AS <..etc..> select * from authors order by au_lname COLLATE Czech_CI_AI select * from authors where au_lname = au_fname COLLATE Czech_CI_AI

205

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Internationally Aware Applications

This is a DBA book, not an application development one, so we will not spend too long discussing the numerous issues to do with building international applications. Some of the many items to be considered include: a) Language a. Printing documentation b. Grammar c. String data in UI (menus, dialogs, other GUI controls) b) Locale a. b. c. d. Specifics Date/time format Numeric Currency Sort order / string comparison

The application architecture needs to divide UI (user interface) away from the application components (code block or business logic) in order to produce a localised product that is internationally aware.
International Enabled Application User Interface Component (UI) Application Component (Code Block) Localised Product Installation

The resource file (*.res, *.rc or *.resx) is the key for the UI, a separate resource file exists for all supported languages and assists in localising the installation for the client in terms on string data interpretation, bitmaps, icons and its link to the clients OS locale. Any further discussion on this topic is out of scope , suffice to say that initial design is everything and some first hand experience is required on the team to ensure a successful design an subsequent implementation.

206

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Chapter

OLAP
rom SQL Server v7, Microsoft bundled OLAP Services with its DBMS and was their first (and reasonably good) attempt at a move into the OLAP market space. In SQL Server 2000 the product has been renamed to Analysis Services and provides a range of enhancements that no-doubt pleased the OLAP gurus amongst us. In this chapter we will cover some of the basics to get the DBA up to speed with its administration. I say basics, as I am far from being an expert in this field, which I truly believe is a distinct skill in its own right, therefore, I do very little comparison to Oracle and only skim the surface of the Microsoft products capabilities. Do not let this put you off though as we cover a range of great tips and tricks.

Terminology
It is assumed that the reader has some previous Oracle experience with OLAP. To be honest, I have had little involvement with OLAP technologies under Oracle so will rely on you to draw the necessary comparisons between the two companies implementations.
Terminology OLAP Summary Online Analytical Processing Microsoft have separated its OLAP technology away from the DBMS and wrappered it into what they call Analysis Services (or OLAP Services in v7 of the DBMS). The product includes Microsoft utilises a star schema for their cubes. A cube in such a schema is based on a single fact table (or view) and 1 or more reference data (dimension) tables connected to it. Is a view or 1 or more cubes. Basically we can build a variety of physical cubes, and from there, create virtual cubes were we their fact table and associated shared dimensions can work together for form a single virtual cube. To the end-user there is no difference to standard cubes. Organised hierarchy of categories or levels that links to data within the fact table of the cube. The dimension hierarchy (i.e. tree) is divided into levels and each data element of the dimension is called a member. Analysis services support regular, unbalanced and ragged dimension hierarchies. Multiple dimension expression. Calculated field within a cube that is not physically part (calculated at runtime) of the underlying tables/views feeding the cube. They are written via MDX statements.

Star Schema Virtual Cube

Dimension

MDX Calculated member

207

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Cell level security

A cell is the smallest element of a cube, typically being the unique element in which all dimensions (where applicable) intersect.

Administration
Installation
The installation options are very basic, we are simply asked: a) what options we would like to install a. note that Analysis Manager requires decision support objects (dso) and client components. b) where our OLAP data directory (cubes and their meta-data bases) are to be stored (single directory location). and installation continues. On completion, analysis services are installed with the SQL Server group as shown below. The MDX (multi-dimensional expression) example VB application is a good place to trial MDX queries used in OLAP, which you will soon find out are very different from standard SQL. Always reboot after the installation, you may get some strange errors and its not worth the trouble dealing with them.

The directory structure is as follows, the OLAP repository is, by default, a series of Microsoft Access databases the core files being msmdrep.mdb and msmdqlog.mdb. The DBA should ensure that these files and associated cube directories are regularly backed up. A share is automatically created called MsOLAPRepository$ and is explained in more detail later. Core repository database files are stored here. In order to administer the OLAP repository and its cubes, you must be part of the olap administrators NT group that is installed with Analysis Services. Of course, if the server is not

208

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

part of your typical login domain then you need to create a trust and ask the server administrators to add your login to this group. Once this is done you can connect and administer the cubes from your PC. NOTE - Analysis Services does not have the same concept of multiple named instances, therefore, multiple analysis servers running on the same box. Within a single analysis server (single service) we run one or more OLAP databases. The following Access databases are used to manage analysis services:
Database Name msmdrep.mdb msmdqlog.mdb Purpose Analysis Services repository to store meta data of the servers objects. Can be migrated to SQL Server. Analysis Services query log. Also used during cube processing. Cannot be migrated to SQL Server. The documentation states that this log is only used when querying cubes, even so, I have received errors when processing cubes are not having read/write access to this database when managing the OLAP server remotely.

NOTE - Reinstalling analysis services will not destroy or reconfigure your repository although you must re-process all cubes in the repository. You may loose your sample OLAP database (foodmart) on re-installation. To allow Analysis Services to use over 3Gb (assuming the typical /3GB and other settings have been set in the boot.ini), edit the HighMemoryLimit registry string under KEY Local Machine\Software\Microsoft\Olap Server\Current Version and specify its decimal value accordingly. Once done, from within the Analysis Services GUI, select properties for the server and you can alter the minimum allocated memory settings.

Look through the properties of the server carefully. From here, you can administer the OLAP logs, active directory registration, cube processing configuration and other general properties. WARNING Interestingly enough, I completely removed Analysis Services, rebooted and re-installed. Before the removal the repository was migrated to one of my SQL Server instances. On reinstallation the repository was retained and naturally continued re-using the SQL Server instance. This may be a problem for you. Always re-check the registry after an uninstall and clear as required. The root is \Hkey_local_machine\ software\Microsoft\olap server.

209

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Administering from your client PC

After you have installed analysis service on the server, the DBA will typically install the developer edition on their desktop and administer the cubes remotely rather that at the server. If you are not too particular, use all the default installation settings and apply the appropriate service packs to bring the client up with the server you will remotely administer. Once done, we need to make some server changes to allow the client to connect and process cubes and dimensions. The first problem you may be presented with is this:

The first key change is adding your windows networking domain login name to the olap administrator domain group on the server in which we installed analysis services. The network administrator may need to create a 1-way read-only trust to your server if its not a member server of the domain you are logging into and/or is running in a different windows domain.

Edit the group, select members tab and add your domain login name to this NT group. The Administrator is naturally part of this group. In the example below we have added two domain users from a different domain (via trust) and the SQLServer user in which SQL Server runs.

Once done, we need to repeat a similar operation with the MsOLAPRepository$ share that is automatically created on the bin directory holding the Analysis Services binaries. This is very important as you will get a variety of errors when attempting to edit and process cube data structures. By default though, the OLAP Administrators group has permissions to the share so changes directly to the share are rare. NOTE Clients using MDX via OLE-DB (includes Excel Pivot control, Office 2k web components, ADOMD, etc) do not require access to the MSOLAPRepository$ share.

210

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The registry entries for OLAP include the remote access properties to manage your cubes, such as:

There is apparently a bug (Microsoft support doc# Q293782) regarding the administration of the cubes. Even with access to the OLAP Administrators group and the share, you may still have problems and will require Administrator group access. I have not come across this particular problem. Remember that cross domain access will require a trust between the domains.

Migrating the repository

If you are like me, the thought of using Microsoft Access databases for anything business critical is like skydiving without a parachute. The migration process is very basic, simply right click the registered analysis server you wish to migrate, select migrate repository and following the prompts. It assumes that you already have a SQL Server database created. When asked, select the appropriate database authentication scheme (if installing under the NT account SQL Server is running in then uses this.

211

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In the GUI dialog shown, it tends to highly recommend a straight migration to a SQL Server database rather than utilising SQL Servers meta-data services. The two types of migration are essentially the same in terms of database objects that are created, they being the tables server and olapobjects. The meta-data repository install option defaults to the MSDB database, which of course you can change. Personally, I am quite impressed with the SQL Server meta-data repository as it adds another reporting element to the repository. The model utilises the XML nature of the olapobjects table. Within EM we see something like the following (right click virtually anywhere in the tree and select browse options to get a custom view of the many properties):

212

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

If, after migrating you have connection problems to the database and have altered SQL Server login privileges accordingly, then go to the registry (regedit.exe) to fix the connectivity properties:

If, after you migrate the repository you can no longer administer the cubes from your PC, even though you are part of the OLAP administrators NT group; then create a SQL Server login for each user (or create a NT group at map the SQL Server login to it instead; whatever is easier) using Windows Authentication, and allocate the database privilege db_owner to the user in the OLAPRepository database you created. I believe db_writer and db_reader are also suitable if you are not keen on allocating db_owner. NOTE Once you migrate away from the Access databases, there is no going back unless you re-install.

Storage Models and Connectivity

From a DBA perspective, the actual involvement will vary from simple administrative tasks in a production environment (i.e. set-up, backup/recovery, processing cubes and dimensions and their associated schedules etc) to full blown development via the Analysis Services Manager and possible Visual Basic or C++ utilising DSO and ADOMD. We will provide a simple walkthrough in this section and no more.
OLAP Databases

You can only have one installed Analysis Server installed at any one time. Running inside of this we have zero or more OLAP databases. When created, the databases are logical entities stored within the OLAP repository (i.e. meta-data framework). For each database we define one or more data sources (virtually any OLE-DB compatible source or ODBC link) in which we source data from for cubes and/or dimensions.

213

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Cubes and Cube Storage Models

We have three different cube storage models in SQL Server: a) MOLAP data and aggregations stored in multi-dimensional structures on disk (data directory) b) HOLAP leaves data at the data source and stores aggregations in multi-dimensional structures on disk c) ROLAP (dimension ROLAP stored only for Enterprise Ed) - leaves data at the data source and also stores aggregations within the source database in which the fact table resides.
OLAP Connectivity

Connectivity to analysis services varies, but is typically via is OLAP data provider or In order for a client to task to analysis services, the MSSQLServerOLAPService must be running on the server, this serves as the listener for the OLAP server. Connectivity is via the MSOLAPXX.dll OLEDB provider from the client, as a connection string, this corresponds to the OLEDB provider Provider=msolap;. NOTE Using the OLAP provider installed with SQL Server v7 OLAP services will not work against Analysis Services cubes. This is especially evident when using Microsoft Excel and its pivot control (Office 2000 web component), where virtual cubes (discussed later) are not shown. Search for msolap.dll to get versioning information. Be aware that in Excel, to connection to an Analysis Servcies cube (SQL Server 2000 OLAP or OLAP v2.0), you need to use:

Security

We will not be going into any depth with programming OLAP, but even so, its worth spending a little time exploring cell level security.

214

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

As with most things Microsoft, the fundamental security framework revolves around the Windows Domain, its users and their NTFS and Domain (group-policy) privileges to determine what the network user can and can not do. This flows into OLAP which only supports Windows Authentication (can be very unfortunate for some installations). For the well-established Windows Network, this allows seamless single-sign on to not only your databases (depending on the application of course) but also to your OLAP cubes. As a basic element of security, access to cubes and virtual cubes is via roles within OLAP. The role can have any number of NT users or NT groups associated with it. On top of this, when allocated to a cube, we can control a variety of security properties to further restrict the groups users. We manage cubes roles via the Cube Role Manager dialog that pops up when selecting role properties for the cube. Note that you will get a different set of options if you select roles at the cube level vs the entire analysis services database level. IMPORTANT - remember that analysis services is a huge jump from OLAP Services released with SQL Server v7, and as such, many of the options simply dont exist or are only accessible via ADO-MD API calls using VB or C++. Security at the role level is very straight forward, but their comes a time where people will be asking you to secure your cube at the cell level. A classic example that we will use is the organisational chart, where we want staff members to only view their portion of data related to a specific division/branch/section combination that the user selects from the cubes organisationalchart dimension.
Example Cell Level Security

The user can select a specific Division, Branch or Section to get their current financial position in terms of budget and expenditure. It is critical to the business that people can only view data based on their position within the organisation, therefore, the CEO can see all divisions, the General Managers can see all data related to their specific Division, Directors are restricted at the Branch level and Managers at the section level.

In our SQL Server data warehouse, we have linked over to the HR system and extracted the current organisation chart to populate the org-chart dimension for the cube, and have also extracted out information related to all staff members and their position within the chart. These

215

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

have been mapped (magically) to their Windows NT logins for single sign-on to the OLAP cubes. So, with all that done, we know exactly what data User A can view in the cube and what they cannot. Within the analysis services, we select the cube of concern and edit their roles we have allocated to the cube. The roles are actually programmically created (shown later), and have a poor naming convention, but you get the idea:

In this case, role EISHROrg_012 has been created with one member. This has allowed to user to access the entire cube and all its data. Clicking on the lock, we can drill into cell security to further restrict the users access to the cubes underlying data.

Click on cell to edit and manage cell security, or move one level up and restrict users to specific dimensions.

Read Restriction Cell Security String (in MDX):

IIF ((

ancestor([HR Complete Org Chart].CurrentMember, [Org Chart Division Name]).Name = "DivisionA") AND ( ancestor([HR Complete Org Chart].CurrentMember, [Org Chart Branch Name]).Name = "BranchB" ), 1, 0)
Read Contingent: Unrestricted

216

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Read/Write: Unrestricted

where [HR Complete Org Chart] is the dimension name, and [Org Chart Division Name] and [Org Chart Branch Name] two of the three levels within the dimension that the user can filter on to view cube data. In this case, we are saying that if the user wants to read the data in BranchB which is part of DivisionA, then they must be a member of this role, otherwise a value of 0 (false) is returned and access is denied (no data shown to user). IMPORTANT Before testing the security for yourself, remove your NT login from the olap administrators NT group, if you dont the cell security will seem as though its not working for you when it actually is. In this particular example, we have generated the roles and applied the security strings to the cubes via a chunk of VB code. The class has been implemented as a COM (dll) and it is called via a scheduled DTS. The next section looks at the DSO routines to do this.

ADO MD & DSO

To programmatically manipulate and query OLAP cube data (apart from using the Excel Pivot Control and other 3rd party tools) the programmer can utilise ADO-MD (Active Data Objects Multidimensional). It essentially uses an underlying OLE-DB provider that is MDP (multi-dimension provider) compatible for OLAP data access and manipulation. To programmatically manipulate Analysis Services, the programmer needs to utilise the DSO (decision support library) of COM objects. With these and ADO-MD, you can basically re-create the Analysis Service Manager GUI if you like, just as using SQL-DMO you can rebuild Enterprise Manager for SQL Server. From the example for cell-security, we utilise DSO code (the DSO objects are installed on installation of Analysis Services) within a small VB COM to remove existing roles, then re-create them from information stored in our data warehouse. The code below as been simplified to show the essential DSO commands used: Dim Dim Dim Dim Dim Dim Dim sNewRole As String dsoServer As DSO.Server dsoDatabase As DSO.MDStore dsoCube As DSO.MDStore dsoRole As Role dsoCubeRole As Role varOLAPDatabase

varServer = "151.122.12.1" varOLAPDatabase = "MyOLAPDB

217

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

'Set up the connection to the OLAP server (not the DB) Set dsoServer = New DSO.Server dsoServer.Connect (varServer)
'Does the OLAP DB Exist on the server?

If dsoServer.MDStores.Find(varOLAPDatabase) Then Set dsoDatabase = dsoServer.MDStores(varOLAPDatabase) 'create the OLAP role, remove if it exists sRole = "EISOrg_001 If Not dsoDatabase.Roles.Find(sRole) Then dsoDatabase.Roles.AddNew (sRole) Else dsoDatabase.Roles.Remove (sRole) dsoDatabase.Roles.AddNew (sRole) End If 'add the user to the role strUser = "MYDOMAIN\myuser" dsoRole.UsersList = strUser dsoRole.Update ' NT security only
'Connect to the OLAP DB

'add cell security string to this role Set dsoCubeRole = dsoCube.Roles(sRole) dsoCubeRole.SetPermissions "CellRead", _ "IIF ((ancestor([HR Complete Org Chart].CurrentMember, [Org Chart Division Name]).Name = "DivisionA") AND (ancestor([HR Complete Org Chart].CurrentMember, [Org Chart Branch Name]).Name = " & Chr$(34) & " BranchB" & Chr$(34) & "), 1, 0)" dsoCube.Update End If dsoServer.CloseServer A thorough overview of the ADO-MD API can be found on MSDN. http://msdn.microsoft.com/library/default.asp?url=/library/enus/ado270/htm/ammscadoapireference.asp Programming Analysis Services can also be found on MSDN. http://msdn.microsoft.com/library/en-us/olapdmpr/prabout_84a4.asp

218

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Backup
To backup Analysis Services, the DBA must: a) Backup the Registry (\Microsoft\OLAP Server) b) Backup the repository data files, even if you migrate the repository to SQL Server you should backup the bin directory to ensure maximum recoverability. This includes the msmdrep.mdb database (unless you have migrated the repository to SQL Server). c) Backup the OLAP data-files If you select a ROLAP storage model for your OLAP cubes, this can complicate your backup as the aggregations will be stored in the data-source in which your cube is utilising to source its fact data. This is not a major concern as the cubes storage structure can be rebuilt, but this may be problematic with very large cubes. Within Analysis Service manager you can export your cube database, this is the primary method for backup that is probably the most reliable. This will export the aggregations, security privileges, not the actual processed cubes with their data. On restoring the OLAP database you will need to do a complete re-process of the OLAP database (repository). Look at the command line executable msmdarch.exe to archive a specific database into a single .cab file. The DBA should extend this backup to include the items discussed above.

219

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

10
Chapter

Scripts & Scheduled Jobs

nder UNIX, the Oracle DBA writes numerous scripts for managing database backups, instance re-starts, statistic collections, space management, the list goes on. These are Typically scheduled tasks via cron (AT in Windows) and are written in a variety of languages, including Unix scripting (whatever shell the DBA prefers), C, Perl etc. In the SQL Server environment, the DBA has a range of methods to use that typically benefit the DBA with a good knowledge of VB-Script and VB programming, these methods include: a) Creating and scheduling DTS packages (run via SQL*Agent) that call T-SQL stored procedures, VB-Script and any other COM or defined task they like to call. b) Custom written COM, COM+ or standard executables routines that utilise SQL-DMO, called via a scheduled VB-Script file scheduled via at from the command line or within a DTS. c) Simple VB-Script (.vbs files) scheduled via at from the command line. d) Consider Perl and other scripting languages, so long as they communicate via OLE-DB, ADO, ODBC etc. e) For those ill die without my grep Unix phreaks, seriously consider downloading a unixtoolkit for DOS (NT), there are some excellent ports and offer all the functionality required to re-produce some great scripts under Windows. Lets face it, you cant beat Unix for scripting. As I am not a VB programmer by any means, I tend to use DTS packages calling T-SQL stored procedures that I store in the master database. I then schedule them and SQL*Agent does the rest. To enhance them, I may opt for some simple vb-script or callouts to existing DLL classes. This is very simple to implement and with SQL*Agent it allows me to utilise its inbuilt notification service on failure and to historically track its progress. Some find that T-SQL procedures lack the features they require for more complex tasks. In this case, I would recommend writing COM classes that utilise the SQL-DMO API calls to interact with SQL Server. A range of examples are shown throughout this chapter. NOTE Consider 3rd party scheduling software if you are not utilising SQL*Agent and associated jobs, as the OS schedulers smallest granularity for running jobs is daily.

220

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

T-SQL Routines
With the power of the sysadmin server privilege (system or sys account in Oracle), the DBA can code some powerful T-SQL routines to assist with ongoing administrative tasks. Some of the key commands of interest: a) master..xp_cmdshell (to call out to an OS command) b) master..sp_OA* procedures - call external COM and COM+ components c) using temporary tables (table datatype and the create table #mytab statements) d) using dynamic SQL (sp_executesql and exec()) e) using SQL*Agent and Jobs

XP_CMDSHELL Example
Here are a couple of simple examples of using xp_cmdshell to perform file OS file manipulation and get a directory listing into a database table:
-- Move Unzipped files set @v_commandline = 'move /y ' + @v_ftpdest + '*.unl ' + @v_processeddest exec @result = master..xp_cmdshell @v_commandline -- Get File list from directory for further processing (like unzipping them) set @v_commandline = 'dir /B ' + @v_processeddest + '*.unl' insert into infile_list exec @result = master..xp_cmdshell @v_commandline

DTS
To call your DTS package you can: a) schedule the package to run at some predefined date/time b) review the command sp_start_job to force the running of a scheduled DTS package c) review the executable dtsrun.exe d) see the execute package task within the DTS package designer for cross calling of between separate DTS packages (is version dependent on each save of the package, so be warned). Remember that scheduled jobs are managed by SQL*Agent which utilities the MSDB database as its repository. There can be issues with a packaged jobs not running when scheduled. Some of the areas to try: a) the user in which the SQL*Agent service is running has insufficient database access

221

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

b) calling xp_cmdshell? And the user is not part of the sysadmin role? If so, if you fail to get up the proxy account the step will fail. This may have flow on effects resulting in this error. c) Try forcing the first schedule to run, and see if SQL Server will correctly set the data/time for the next scheduled run d) Check the error log (obvious enough).

ISQL / OSQL
The DBA can utilise the command line equivalents to query analyser (SQL*Plus GUI) to run scripts and pipe the results to a standard ASCII file. An example is shown below:
cd g:\scripts osql -S SECA\MYINSTANCE osql -S SECA\MYINSTANCE osql -S SECA\MYINSTANCE osql -S SECA\MYINSTANCE -Usa -Usa -Usa -Usa -Pking -Pking -Pking -Pking /Q /Q /Q /Q "EXIT(dbcc "EXIT(dbcc "EXIT(dbcc "EXIT(dbcc checkalloc('mydb1'))" -og:\scripts\mydb1_dbcc.txt checkalloc('mydb2'))" -og:\scripts\mydb2_dbcc.txt checkcatalog('master'))" -og:\scripts\master_catalog.txt checkcatalog('msdb'))" -og:\scripts\msdb_catalog.txt

Here we run a variety of DBCC commands and pipe the results to associated text files. Utilising the sa account is not recommended in such a way and where possible, always opt for Windows authentication for sysadmin access. From here, we can do a variety of things, such as using Perl to extract and massage data from the file and produce appropriate reports, do a call to some visual basic script or call an emailer to send the results directly to the DBA. In the example below, we are utilising a command line emailer called smtpsend.exe to email the file direct to the DBA:
cd g:\scripts smtpsend.exe -fMyDBServer -tckempster@amcon.com.au -h163.232.x.xxx -s"PROD DB LOGSPACE STATS" ag:\scripts\dblogspace.txt

VB Script
With a VBS file (*.vbs) or VB Script within a DTS, the DBA can utilise many of the classic VB commands, the key one being the file system objects library for OS file manipulation. I will leave it up to the internet to give you a large range of examples, even so, here are some handy commands to start you going. NOTE Variable declarations in VBScript are not explicitly typed (i.e. integer, float etc).
This is a comment, below is a CONSTANT Const SUCCESS = 0 declare variables Dim WshShell Dim FSO Dim oMsg Dim lSMTPServer

222

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Strings require double quotes lSMTPServer = "163.231.4.128" Example of using the date function logfile = "myfile_" & Day(Date()) & "Jan" & Year(Date()) & ".txt" Example of a case statement and message box Select case Month(Date()) Case 1 : msgbox Month 1 Case 12 : msgbox Month 1 End select Calling a OS command, in this case, running a SQL (script) file retcode = WshShell.Run("isql -S myinstance -d mydb -U userA -P password -i myscript.sql", WSNORMAL, true) Moving a file Set FSO = CreateObject("Scripting.FileSystemObject") Call FSO.MoveFile("myfile.txt", processed_file) Log a message in the NT event log WshShell.LogEvent SUCCESS, "Example Complete."

The .vbs file can be called directly via AT. If you have issues with running it then try using the wscript or script commands, for example: call %SystemRoot%\system32\cscript.exe //Nologo //E:VBScript "%SamplesDirectory%\myfile.vbs" "MYExample"

VB / COM Example
The example below is not really related to a specific SQL Server problem, but gives you an idea of the possibilities. We have written a small DLL that copies a series of files from one destination to another with a date-stamp, and is called via a small VBS that we schedule via AT. The key item here is that the experienced VB or C++ developer in the Microsoft realm can basically do anything they like in terms of scripted programming. Most will argue that this is going a little to far but offers a lot of functionality. Personally, the DBA will find that they can resolve most problems elegantly with T-SQL routines / DTS package scheduled via SQL*Agent.
-- INI File [database_backup] database_location="C:\database\MyDB.mdb" destination_folder="C:\database\backup\" -- VBS File Source Dim oDBBackup As dblib.DatabaseBackup Dim lngReturn As Long set oDBBackup = CreateObject("dblib.DatabaseBackup") lngReturn = oDBBackup.BackupDatabase()

223

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

set oDBBackup = Nothing -- COM+ Source File VERSION 1.0 CLASS BEGIN MultiUse = -1 'True Persistable = 0 'NotPersistable DataBindingBehavior = 0 'vbNone DataSourceBehavior = 0 'vbNone MTSTransactionMode = 2 'RequiresTransaction END Attribute Attribute Attribute Attribute Attribute VB_Name = "DatabaseBackup" VB_GlobalNameSpace = False VB_Creatable = True VB_PredeclaredId = False VB_Exposed = True

Option Explicit Implements COMSVCSLib.ObjectControl Public Function BackupDatabase() As Long Dim Dim Dim Dim Dim lngError As Long oFSO As Scripting.FileSystemObject oGen As dbutility.General -- another COM+ package of general utility functions strDBFile As String strDestination As String

On Error GoTo ERR_HANDLER Set oGen = CreateObject("dbutility.General") 'Extract information from INI file. strDBFile = oGen.GetINIFileSetting("database_backup", "database_location") strDestination = oGen.GetINIFileSetting("database_backup", "destination_folder") 'Backup the database. Set oFSO = CreateObject("Scripting.FileSystemObject") If Not oFSO.FolderExists(strDestination) Then Call oFSO.CreateFolder(strDestination) oFSO.CopyFile strDBFile, strDestination & Format(Now, "ddmmmyyyyhhmmss") & ".mdb", False FUNCTION_RETURN: 'Vote on transaction. If lngError = 0 Then If Not GetObjectContext Is Nothing Then Call GetObjectContext.SetComplete Else If Not GetObjectContext Is Nothing Then Call GetObjectContext.SetAbort End If Set oGen = Nothing Set oFSO = Nothing BackupDatabase = lngError Exit Function ERR_HANDLER:

224

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

lngError = Err.Number 'Call off to a custom emailer or log to NT event log or text file -- Call SendEmail("Database backup failed.", Err.Description) GoTo FUNCTION_RETURN End Function '******************************************************************** ' COM PLUS METHODS '******************************************************************** Private Function ObjectControl_CanBePooled() As Boolean ObjectControl_CanBePooled = False End Function Private Sub ObjectControl_Activate() ' End Sub Private Sub ObjectControl_Deactivate() ' End Sub

225

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

11
Chapter

Managing Databases

hroughout this chapter we discuss a wide variety of topics related to everyday database management. Where appropriate we will draw comparisons with Oracle but will not discuss in depth the differences, preferring to focus on practical SQL Server examples.

Database Checking Routines

The DBA has a variety of options in Oracle, including: a) DBMS_REPAIR b) DBVERIFY c) DB_BLOCK_CHECKING d) DB_BLOCK_CHECKSUM e) RMAN f) ANALYSE TABLE VALIDATE STRUCTURE CASCADE In SQL Server the DBA utilises one or more of the database consistency checking (DBCC) routines, some of the commonly used ones are: a) DBCC CHECKALLOC -- check disk space integrity for a database b) DBCC CHECKCATALOG -- check consistency of system tables in a database c) DBCC CHECKCONSTRAINTS -- check constraint integrity d) DBCC CHECKTABLE -- check table page integrity e) DBCC CHECKDB check allocation and structural integrity f) DBCC DBREINDEX -- rebuild table indexes g) DBCC INDEXDEFRAG -- defragment clustered and heaps indexes for a table h) DBCC OPENTRAN -- oldest open transaction dump i) DBCC SHOWCONTIG -- get fragmentation information of a table j) DBCC PINTABLE -- mark table as pinned in buffer cache k) DBCC UPDATEUSAGE update storage statistics Other DBCC commands have been discussed throughout this e-book. See books online for good coverage of these commands.

226

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Log Files (UDUMP BDUMP) ,

The Oracle alert log is found at the backupground_dump_dest parameter destination. In SQL Server this is called the error log and its destination is defined by the e startup parameter. This is also stored within the registry during instance start-up:

From within EM, we can right click the instance for global properties and view the start-up parameters rather then searching the registry:

The management folder in EM allows the DBA to view the contents of the log files. I say files because SQL Server will cycle through six different log files (default setting), the cycling of the files will occur on each re-start of the SQL Server instance, or via exec sp_cycle_errorlog. The DBA can control the number of cycled logs via a registry change, this change can be done within Query analyser if you like or via EM by right click for properties in the SQL Server Logs item under the Management folder. Exec xp_instance_regwrite N'HKEY_LOCAL_MACHINE', N'SOFTWARE\Microsoft\MSSQLServer\MSSQLServer', N'NumErrorlogs', REG_DWORD, 6 The DBA can view error logs within query analyser using: exec sp_readerrorlog [number of log file, values 0 to 6] The valid list can be retrieved via:

227

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

exec sp_enumerrorlogs
Archive# 0 1 2 3 4 5 6 Date 06/15/2002 06/14/2002 06/03/2002 06/03/2002 05/26/2002 05/12/2002 05/11/2002 23:59 21:25 21:36 10:29 14:25 16:27 15:54 Log File Size (Byte) 3646 21214 3063 49852 31441 5414 2600

There is no equivalent destination for user traces as defined by the user_dump_dest parameter. The error log for SQL*Agent is best managed via EM. With SQL*Agent shutdown, select its properties and you can control a range of log settings:

Location & filename of the error log. Include full error trace with log entries Use non-unicode file format

228

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Linked Servers (Database Links)

The equivalent to the database link in Oracle is the Linked Server. The GUI interface for it is located under the security folder within Enterprise Manager, which like all GUIs executes a variety of commands to create the database link. The linked server supports any valid OLE-DB provider; this includes a variety of bundled providers such as Oracle and OLE-DB and standard ODBC drivers (therefore, any source supporting ODBC connectivity drivers). An example of Oracle vs. SQL Server db link creation: CREATE DATABASE LINK mylink CONNECT TO myuserid IDENTIFIED BY mypassword USING 'sqlnetentry' -- entry points to instance ZEN exec sp_addlinkedserver N'MYORACLEDB' , @srvproduct = N'' , @provider = N'mylink' , @datasrc = N'ZEN' , @provstr = N'ZEN' NOTE Dont assume that creating the link without error means the link actually works. The SQL Server link includes a host of additional properties the DBA can manipulate. One of the key benefits is the mapping of user accounts over the database link. In Oracle, the database link is user centric, where we identified the CONNECT TO myuserid IDENTIFIED BY mypassword properties in which the connection is established. In SQL Server, we can map one or more accounts over the linked server; this allows the DBA to specify the local database user accounts that can utilise the linked server and define how this user account is mapped to the remote system over the link. An example is shown below later. SQL Server supports the three part convention for referencing database objects in SQL, namely [database].[owner].[objectname]. A linked server query requires four parts to be listed and we cant create synonyms for them either (although we can create local views that refer to the linked objects). So the convention now changes to [linkedserver].[remotedatabase].[owner].[objectname]. Oracle DBAs will hate this. Even worse, it can restrict you in SQL queries and other DML operations, for example: select mydb.dbo.mytable.col1 from mydb.dbo.mytable select mylinkedserver.mydb.dbo.mytable.col1 from mylinkedserver.mydb.dbo.mytable -- this is OK -- this is NOT

Server: Msg 117, Level 15, State 2, Line 2 The number name 'mylinkedserver.mydb.dbo.mytable' contains more than the maximum number of prefixes. The maximum is 3.

229

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

This presents some issues with inserts/delete/updates or complex recursive queries over the same tables and column names where aliasing tables is not possible. NOTE If creating a linked server to another SQL Server instance and you are still having connectivity problems, try: using the servers IP rather than the hostname, check remote server listener is configured, try using the source service client network utility to configure an alias to the remote server, ping or trace route to the server, verify providers. The registry entry HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC provides the hook to the associated Oracle driver when linking to an Oracle DBMS:

To get a list of linked servers, use: exec sp_linkedservers From there, you can get a list of databases over the linked server by using:
exec sp_catalogs @server_name = 'mylinkedserver'

You can go another step further and get a list of tables over the linked server via:
exec sp_tables_ex @table_server = 'mylinkedserver'

to get a list of privileges:

exec sp_table_privileges_ex @table_server = 'mylinkedserver'

To improve linked server performance, consider the following:

exec sp_serveroption 'MYSERVER\MYINSTANCE', 'collation compatible', 'TRUE' exec sp_serveroption 'MYSERVER\MYINSTANCE', 'lazy schema validation', 'TRUE'

Read the BOL for further information, and like all things, test for your installation:
exec sp_serveroption 'SECA\MY2NDINSTANCE', 'lazy schema validation', 'true' The 'lazy schema validation' option is ignored in this edition of SQL Server.

230

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Example - Databases in the same Instance

You do not need to create a linked server to establish connectivity between databases in the same SQL Server instance. If the login has database access to the other database (i.e. is a user in the source and destination databases within the instance) with the necessary object privileges, then we use the three part object lookup syntax to access the other database. For example: select * from myotherdb.dbo.mytable or select * from myotherdb..mytable

-- current user or dbo assumed

Example - Connecting to an SQL Server Instance

Connecting to another SQL Server instance is relatively straight forward (and you would hope so too!). The only trick here are those instances whose listener ports are very different from the defaults, therefore, we may need to create an alias via the Client Network Configuration utilities at the source server and set the appropriate settings in order to complete the connection to the remote server.

SQL Server provider

EIS-NT is the hostname, or we could have used the IP address.

This is the database name, we do NOT include the [ ] as we would normally do for database names with spaces.

231

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Map our local users logins to remote ones.

Security contextual information can be passed between servers. This is especially handy integrated security logins. Take care with these options; you can open up the linked server to all logins if you are not careful.

IMPORTANT - In the above example we map the SA account, dont assume that just because we listed the database in the linked servers creation that its only database we can access. Since SA on the remote server as sysadmin access and access to all other databases, we can utilise the linked server to access any remote database.

Example - Connecting to an Oracle Instance

IMPORTANT There may be issues with manipulating BLOB objects from Oracle with the Provider. Query the Microsoft support website for the latest information when resolving these issues. In this example we have a SQL Server 2k SP2 database and a Oracle 8.1.5 database installed on the same server. We are creating a database link via the standard OLE-DB Oracle provider from the SQL instance to the Oracle database.

232

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL*Net Connection string as entered in the tnsnames.ora file ZEN = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = seca) (PORT = 1521)) ) (CONNECT_DATA = (SERVICE_NAME = ZEN) ) )

-- return list of valid OLE-DB providers exec sp_enum_oledb_providers -- create the linked server exec sp_addlinkedserver N'MYORACLEDB' , @srvproduct = N'' , @provider = N'MSDAORA' , @datasrc = N'ZEN' , @provstr = N'ZEN' -- set linked server options as need be exec sp_serveroption N'MYORACLEDB', N'connect timeout', 0 The provider has specific properties are set in the registry: \SOFTWARE\Microsoft\MSSQLServer\Providers\MSDAORA In order to establish the connection between the two servers, we need to enter the login mappings via the security tab:

233

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

If the sa login is used to connect to SQL Server, this will map to the scott/tiger user in the Oracle instance and inherit this users security privileges if we query over the linked server.

For any other SQL Server user not listed above, we can define other connectivity properties. This can be treated as a catch all for all other connections.

NOTE You cannot profile (trace SQL via profiler) the addition of logins (linked servers or local users) to the instance. -- shows the linked server login server mappings we have created exec sp_helplinkedsrvlogin N'MYORACLEDB' -- attempt to get a list of tables on remote server via the linked server exec master..sp_tables_ex N'MYORACLEDB' , null, null, null, '''SYSTEM TABLE''' The following options should be set for your SQL Server session to ensure SQL-92 compatibility: SET SET SET SET SET SET SET SET ANSI_NULLS ON CURSOR_CLOSE_ON_COMMIT ON ANSI_NULL_DFLT_ON ON IMPLICIT_TRANSACTIONS ON ANSI_PADDING ON QUOTED_IDENTIFIER ON ANSI_WARNINGS ON XACT_ABORT ON -- auto rollback current trans on runtime failure

The SQL Server books online covers these options very well.

234

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

OPENQUERY, OPENROWSET, OPENDATASOURCE

Use OPENROWSET for once off, ad-hoc connections via OLE-DB to a remove server. In all cases, it is very similar to that of a permanent linked server connection. I have seen a good 2 to 3 fold increase in speed via OPENROWSET vs Linked Server lookup, but it does have a range of issues, one being security. An example is shown below from one linked instance to another on the same server:
SELECT aa.* FROM OPENROWSET('MSDASQL','DRIVER={SQL Server}; SERVER=SECA\MY3RDINSTANCE; UID=sa;PWD=king', select * from pubs.dbo.authors) AS aa ORDER BY a.au_lname, a.au_fname

We can specify a query, table name or view name within the OPENROWSET command. On the source and destination instance we see the following:
-- Source instance Execution Tree Sort(ORDER BY:([a].[au_lname] ASC, [a].[au_fname] ASC)) |--Remote Query(SOURCE:(<UNKNOWN>), QUERY:(SELECT a."au_id" Col1013,a."au_lname" Col1014,a."au_fname" Col1015,a."phone" Col1016,a."address" Col1017,a."city" Col1018,a."state" Col1019,a."zip" Col1020,a."contract" Col1021 FROM "pubs"."dbo"."authors" a)) -- Remote instance Execution Tree Clustered Index Scan(OBJECT:([pubs].[dbo].[authors].[UPKCL_auidind] AS [a])) declare @P1 int set @P1=2 exec sp_prepexec @P1 output, NULL, N'SELECT a."au_id" Col1013,a."au_lname" Col1014,a."au_fname" Col1015,a."phone" Col1016,a."address" Col1017,a."city" Col1018,a."state" Col1019,a."zip" Col1020,a."contract" Col1021 FROM "pubs"."dbo"."authors" a' select @P1

When compared to a linked server connection, we see the following, it is essentionally the same with additional schema locking on the remote server.
-- Source instance Execution Tree Remote Query(SOURCE:(MyLinkedServer), QUERY:(SELECT a."au_id" Col1002,a."au_lname" Col1003,a."au_fname" Col1004,a."phone" Col1005,a."address" Col1006,a."city" Col1007,a."state" Col1008,a."zip" Col1009,a."contract" Col1010 FROM "pubs"."dbo"."authors" a)) -- Remote instance RPC:Starting declare @P1 int set @P1=NULL declare @P2 bigint set @P2=NULL exec sp_getschemalock @P1 output, @P2 output, N'"pubs"."dbo"."authors"' select @P1, @P2 Microsoft SQL Server sa 2002-06-30 20:29:09.640 declare @P1 int set @P1=1 declare @P2 bigint

3736

51

235

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

set @P2=8670987119726518 exec sp_getschemalock @P1 output, @P2 output, N'"pubs"."dbo"."authors"' select @P1, @P2 declare @P1 int set @P1=-1 exec sp_prepexec @P1 output, NULL, N'SELECT a."au_id" Col1002,a."au_lname" Col1003,a."au_fname" Col1004,a."phone" Col1005,a."address" Col1006,a."city" Col1007,a."state" Col1008,a."zip" Col1009,a."contract" Col1010 FROM "pubs"."dbo"."authors" a' select @P1 exec sp_unprepare 3 exec sp_releaseschemalock 1

We also have OPENQUERY. This takes a pre-created linked server and a SQL string (if more than 1 query it used in the query string, only the first is actually returned). An example is shown below:
SELECT AA.* FROM OPENQUERY(LinkServer, 'SELECT * FROM pubs.dbo.authors') as AA

The first thing you will notice on tracing this command, is that no schema locking is done on the remote server which doesnt occur using a traditional 4-part query via the linked server. Finally, we have OPENDATASOURCE. Do not use this command, but rather create a proper linked server or utilise OPENROWSET. This command will execute a range of remote calls on the remote server that will affect overall performance. As an example:
SELECT * FROM OPENDATASOURCE( 'SQLOLEDB', 'Data Source=SECA\MY3RDINSTANCE;User ID=sa;Password=king').Pubs.dbo.authors

Will execute a range of calls such as:

exec exec exec exec exec [Pubs]..sp_tables_info_rowset N'authors', N'dbo', NULL [Pubs]..sp_columns_rowset N'authors', N'dbo', NULL [Pubs]..sp_indexes_rowset N'authors', NULL, N'dbo' [Pubs]..sp_check_constbytable_rowset N'authors', N'dbo', NULL, NULL sp_getschemalock @P1 output, @P2 output, N'"Pubs"."dbo"."authors"'

<etc>

With this simple query, total execution time doubled verses linked servers and open row source alternatives. In virtually all cases, use linked servers.

Remote Servers
The remote server option is a form of Linked Server but is SQL Server to SQL Server only and you can only execute stored procedures over the link. In most cases, utilise linked servers rather than remote servers.

236

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DTS Packages
Data transformation service (DTS) is a fantastic feature of SQL Server. The books online gives a overall description, that being an organized collection of connections, DTS tasks, DTS transformations, and workflow constraints assembled either with a DTS tool or programmatically and saved to Microsoft SQL Server, SQL Server 2000 Meta Data Services, a structured storage file, or a Microsoft Visual Basic file. We will drill into them throughout this section. The product is composed of: a) MSDB system database tables and views b) GUI management and editing utility that is embedded in Enterprise Manager c) Command line utilities to run saved packages (dtsrun.exe or the GUI version dtsrunui.exe) d) DTS Object Model (API) e) DTS Information Model (to store meta-data information about the running of a package and its transforms) f) DTS import/export wizard At the lowest level, a single DTS package is a range of record entries in the MSDB database table sysdtspackages, the schema is shown below:

sysdtspackages (the actual package code)

sysdtspackagelog

sysdtssteplog

sysdtscategories (DB storage type)

sysdtstasklog

NOTE The above tables are not applicable to meta-data services packages. They are stored in the MSDB database within the table prefixed with RTbl. The key tool for managing (i.e. creating, deleting, updating) DTS packages is Enterprise Manager. The GUI is not hard to understand and simply requires you to experiment with its variety of supplied active-x controls that, via their interactive dialogs, utilise SQL-DMO and the DTS object model to graphically present zero or more steps of programming logic. This may include any variety of tasks (packagedata column in sysdtspackages), such as loading data from a spreadsheet or another database, or processing a OLAP cube, calling an OS command, calling a VB COM we previously created or running some VB-Script, the possibilities are endless.

237

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

We will discuss meta-data services later in this section, suffice to say that a majority of packages will be local packages and if you need to generate meta-data about the running of a package, it must be stored as a meta-data services package (it is not an actual service in NT speak). Right click local packages and select the new package option. The designer is shown and exposes many of the underlying DTS object model calls to the end-user through the supplied connection and task active-x objects supplied, coupled with package properties etc. The DBA should simply experiment with the GUI via right-clicking properties, trying the supplied active-x controls. An example package is shown below.

Workflow (waiting, success, failure or completion), can be controlled via DTS API calls if need be to alter the execution flow dynamically and code execution loops for example. Right click in white pane for a variety of options, such as global variables, logging, concurrency control etc. This is a transform(ation). Defining how data from the source is placed into the destination, each field can be transformed as it comes in, logging properties are error counts set and much more. Standard OLE-DB connections, including Microsoft office products, ODBC to Oracle, Dbase5 etc. A transform can-not be created unless its to a connection. A variety of tasks, one being an active-x (vb or j-script) task, command line call-out to an OS command, OLAP cube refresh, FTP (cant FTP from the server, only too it unfortunately), SQL Mail mailer and a variety of others.

IMPORTANT Check the properties of all objects very carefully, especially the transformation task as it includes items such as automatic connection closing after data transformation. It is not until you save a package with actual package content will the packagedata column in sysdtspackages will be populated. Each save will create a new version of the package in the same sysdtspackages table, see the version id column (which is a unique-identity data type). You are presented with the dialog on save:

238

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL Server (local package) Meta Data Services (data warehouse and linage tracking repository)Generate VB source of the package (very handy for learning the DTS object model) or proprietary DTS structured file format. Meta data can only be collected for meta data packages.

To get the packages version (and go back versions!), right click the package and select version. This will run the following using the unique-identity column value within sysdtspackages:
exec msdb..sp_enum_dtspackages "%","%",NULL, 0x04,"AAB0B84F-7F4F-4432-BFDD-44FD8754801E"

NOTE To transfer a package to another server, consider save-as and entering the remote host-name and security information (if you have trouble connecting, use the client networking utility and create an alias then try again). Another option is saving the package as a structured file, to restore, right click properties of the Data Transformation Services folder in Enterprise Manager and select Open Package. Another unsupported method is simply bulkcopying the contents of the sysdts tables over to the new server, good luck with this one. I rare use the meta data services option so I wouldnt even consider direct system table transfers at that level. As the data model describes, there is a variety of log information associated with each package. This is very handy for drilling into runtime errors at each step within the DTS. Occasionally I have experienced flow control problems where steps run when they are not supposed to, the logs are handy when identifying such problems (re-create the workflow and/or transformation as need be). The package logging will not be activated until you enable it within the package designer, right clicking on the blank pane and selecting properties:

239

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

You can send log information to another server/instance, be warned that it can be very difficult to work out (at the remote server) where the log data is coming from. Name your packages carefully to assist with identification.

Click properties of the package to view the runtime log error information. If logging to a remote server or instance, you will NOT see the package listed, select properties as per normal and use the drop down list to pick the package name.

Some of the key stored procedure for DTS manipulation are, including viewing the log are:
exec exec exec exec exec exec sp_MShelpcolumns N'dbo.sysdtspackages', NULL msdb..sp_get_dtspackage msdb..sp_add_dtspackage msdb..sp_drop_dtspackage msdb..sp_enum_dtspackages msdb..sp_enum_dtspackagelog

To lock down DTS, the DBA can revoke execute access to these to prevent DTS creation and viewing. The security paradigm for DTS packages is crazy and Microsoft really need to this one up. For example, if User1 created a DTS package called user1 package, then any other user can run and schedule the package. The user can assign a password to the package that will prevent other users from running, scheduling and editing the package. Note that SQL Sever is not smart enough to associate this with your login and/or security group and it will also ask the creator of the package for the password.

240

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The save-as dialog allows the user to enter the appropriate password information. User password allows scheduling and execution, owner password allows editing.

If you want, say, user1 and user2 to edit the same package, then SQL Server takes a dive for the worst. Unfortunately, no matter what you do in terms on the passwords above or playing with windows or SQL Server authentication properties on the DTS package save, user2 will get the error:

On saving the package, SQL Server does a lookup using SUSER_SID (owner_sid column in sysdtspackages) and SUSER_SNAME (owner column in sysdtspackages), returning the security ID for the DB login name, no matter your authentication (trusted or not) to the instance via EM, this ID is entered in the owner column in msdb..sysdtspackages and is used as a lookup for users attempting to save the package. If the name is not the same then saving will fail.
User1 SUSER_SID: 0x680298C78C5ABC47B0216F035B3ED9CC User2 SUSER_SID: 0xF1C954D9C9524C41A9ED3EA6E4EA82F4

Under EM, the owner of the package is represented by the users windows login DOMAIN/username. The DBA can turn-off the check to allow many users to work on the same package by uncommenting the following code from a system stored proc:
Source : msdb..sp_add_dtspackage IF (@owner_sid <> SUSER_SID() AND (ISNULL(IS_SRVROLEMEMBER(N'sysadmin'), 0) <> 1)) BEGIN RAISERROR (14586, -1, -1, @name) RETURN(1) -- Failure END

Of course, this is somewhat risky and Microsoft Support will not help you out with further MSDB issues. To switch security, try to run the following with sysadmin privileges:

241

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- user1 get SID and Owner name select SUSER_SNAME() -- Owner Name select SUSER_SID() -- SID -- user2 get SID and Owner name select SUSER_SNAME() -- Owner Name select SUSER_SID() -- SID -- In this case, make user2 the new owner of the package(s) update msdb..sysdtspackages set owner = user2, owner_sid = 0x680298C78C5ABC47B0216F035B3ED9CC -- user2 where owner_sid = 0xF1C954D9C9524C41A9ED3EA6E4EA82F4 -- user1

There is an undocumented procedure that can change the owner as well, the syntax is:
sp_reassign_dtspackageowner [@name =] 'name', [@id =] 'id', [@newloginname =] 'newloginname'

See www.sqldts.com for more information specific to DTS. NOTE If you restore a MSDB database from another server, be warned that the jobs will need to be re-created and that each DTS package entry in msdb..sysjobs may also present errors related to MSX (remote server management). Try altering the originating_server column in sysjobs to get around the error. A scheduled package will run under the security context defined by the users system privileges. If the user does not have sysadmin fixed server role access, the package will execute via the sqlagentcmdexec proxy account (as per calls to xp_cmdshell). The proxy setup can be managed via EM by selecting properties of SQL Agent and going to the Job System tab. To kill off a running package, the user must have the process administrator fixed server role or sysadmin access.

242

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Diagrams (Data Model Diagrams)

All user databases include the ability to create diagrams, which are stored in the system table dtproperties. The diagram editor in Enterprise Manager allows the DBA to create physical data models, maintain tables, indexes and constraints (but not security privileges). From there, SQL Server can generate SQL scripts based on your changes to be saved and run later or to be applied immediately. A screen shot is shown below:

System table holds diagrams

Manage tables, indexes, add annotations, different views etc. Some classic Microsoft Access GUI feel in places.

NOTE Before printing diagrams, double check the printer and page size matchs what you set within the diagrammer, it has this ability to revert back to A4 when you were actually after a completely different setting. I found this tool to be an excellent feature, and although it is far from being a full featured modelling tool, it can generate scripts that are reliable time after time. This is particularly helpful when managing large data models and saves time when creating database alteration scripts. It should be noted that the scripts, are not optimal in terms of locking/blocking and the statement being used; the experienced DBA may be shocked what they see. This can have adverse effects for heavily used 24x7 databases.

243

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Transferring Diagrams
a) Identify the diagrams to be transferred select id, objectid, property, value from sourcedb.dbo.dtproperties order by 2 desc,1 Each diagram consists of 7 records, the DtgSchemaNAME property has the name of the diagram, use this in conjunction with the objectid column to locate the diagram. We need to transfer all 7 rows. b) Check the destination dtproperties table before transfer The objectid column of the source table is the most critical. We need to maintain its value to successfully complete the transfer. From the query above, check that the destination dtproperties does not already have the objectid. If it does, get the maximum id from the destination table and use it within your select statement. c) Transfer the diagram (from sourcedb to the destdb) The key here is the objectid column, it must be unique, the id column is an identity and does not need to be specified. If it is not, then editing one diagram will alter the other that shared the same objectid and dropping one will drop the other ! you dont want this sort of trouble. We wont risk this and will specify objectid value 2 as we are 100% sure its unique within the dtproperties table in the destination database.
insert into destdb.dbo.dtproperties (objectid, property, value, uvalue, lvalue, version) select 2 /* we know the object id 2 is unique in the destination table */, property, value, uvalue, lvalue, version from sourcedb.dbo.dtproperties where objectid = 12 -- this is the diagram we want to transfer d)

Same Name diagram? Then rename it with a simple update statement before or after the insertion is made.

244

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Row Chaining & Migration

You cannot have row chaining in SQL Server, if a row cannot fit in a free page allocated to the object, then another is allocated. This also means that, unless binary text blobs are used, a maximum row size is 8037 bytes. Row Migration is known as Page Splitting, and will primarily occur when an existing row is updated but can not fit in its current page.

Tablespace (File-group) Management

As the Oracle DBA can move tablespaces, so the SQL Server DBA can move and manage filegroups. To get filegroup information for your database, run the command: exec sp_helpfile This will return all necessary information to manage your filegroup and its associated datafiles. The examples below we will use the following database structure (as reported by sp_helpfile):

To alter a filegroups status: alter database fgexample modify filegroup fg1 readwrite alter database fgexample modify filegroup fg1 read-only select FILEGROUPPROPERTY ('fg1', 'IsReadOnly')
returns 1 or 0

You cannot place the PRIMARY filegroup into read-only (and other) mode; the entire database must be set if the primary filegroup is to change as its initial data file houses the sys files for the database. Also remember that the transaction log files have no concept of file groups. NOTE If you attempt to create a new database via EM and have query analyser open with the Model database set in context, you will get an unable to obtain exclusive lock error. To move a filegroups database files:

245

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

You need to use the sp_attach and sp_deattach commands to keep the entire instance up. Of course, the database will be down during this period and will invalidate logins to this database for the cutover period. To create a table or indexes in a filegroup: create table ourtest (col1 int) on fg1 To view the tables in a filegroup:
select o.name as TableName, s.groupname as FileGroup, u.name as Owner from sysobjects o, sysindexes i, sysfilegroups s, sysusers u where o.id = object_id(N'dbo.ourtest') and o.type in ('S','U') and o.id = i.id and i.indid in (0,1) and o.uid = u.uid and i.groupid = s.groupid -- and s.groupname = 'yourfilterhere' order by 1 desc

To shift a table or index between filegroups: Tables must be re-created, use EM to alter the table and see the generated code. For indexes, use the DROP_EXISTING clause, for example:
CREATE INDEX [myindex] ON [dbo].[ourtest] ([col1]) WITH DROP_EXISTING ON [FG2] -- the new filegroup

To drop a filegroup:
alter database fgexample remove filegroup fg1 Server: Msg 5042, Level 16, State 7, Line 1 The filegroup 'fg1' cannot be removed because it is not empty.

246

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Pinning Tables
As with Oracle, we can pin tables into the buffer cache and reduce the possibility of pages being aged out and adding to the IO round trips to the database files. An example is shown below:
USE mydb DECLARE @dbid INTEGER DECLARE @obid INTEGER SET @dbid = DB_ID('mydb') SET @obid = OBJECT_ID('dbo.mytable') DBCC PINTABLE (@dbid,@obid) or EXEC sp_tableoption 'mydb', 'mytable', 1

Check if table is pinned:

-- returns 1 (yes) or 0 (no) select OBJECTPROPERTY ( object_id(mydb),'TableIsPinned')

Take care when moving databases between servers where tables have been pinned. I have come across a variety of examples where, on servers with less RAM, the pinning can result in buffer cache errors and subsequent cache trashing (resulting in a reboot!).

Triggers
Some general information on triggers in SQL Server: a) SQL Server supports the following trigger types: a. INSTEAD OF [INSERT, DELETE, UPDATE], only one per action and as the name states, replaces the normal processing, can support text,ntext, image column data. b. AFTER [INSERT, DELETE, UPDATE], can have multiple per action, can specify order of action, trigger code cant refer to text,ntext,image column data. b) Set trigger firing order via sp_settriggerorder c) Run sp_helptrigger to view trigger information for a table d) You can enable and disable triggers, either named or ALL for a table, for example: alter table dbo.mytest disable trigger all e) An error in the trigger will rollback the entire transaction. Transactions in triggers are actually nested transactions to its parent. Cursors are also closed and deallocated in

247

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

context to the parent transaction. Consider the SAVE TRANSACTION (marked transactions) to do a partial rollback. f) Mutated triggers are supported where the trigger fires DML against its own table

Constraints
In SQL Server constraints are implemented for: a) primary keys b) foreign keys c) column check constraints d) default column values e) implement UNIQUE constraints We can alter constraints via: ALTER TABLE dbo.mytest [CHECK, NOCHECK] CONSTRAINT DF_mytest_col1 To check data integrity in term of foreign key constraint consistency, consider the command: DBCC CHECKCONSTRAINTS NOTE Always name your constraints. As with Oracle and its system naming convention, SQL Servers equivalent is no better and can result in major problems when attempting to run so called common scripts over many servers.

Long Running Transactions

Use the DBCC OPENTRAN command to view the longest open transaction. This command requires sysadmin access and the db_owner database role.
DBCC OPENTRAN (northwind) Transaction information for database 'Northwind'. Oldest active transaction: SPID (server process ID) : 53 UID (user ID) : 1 Name : user_transaction LSN : (27:380:2) Start time : Jul 24 2002 10:28:32:867AM

248

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

From here, use sp_who2, sp_lock or trace the session to determine the ongoing activity for the session. Consider the dbcc inputbuffer command for a dump of the SPID.

SQL Timeouts
A client timeout may be the result of: a) Dead-locking and/or blocking problems a. Isolation level settings from client connection b. Use of begin/end transaction in code (locks held for long periods of time) c. Unexpected lock escalation b) Incorrect SQL statement (missing join condition) c) Poor or inappropriate use of hints d) General connection properties a. Again, isolation level b. Lock timeout value (also see COM+ timeout setting) c. Close cursor on commit e) Using QUERY GOVENER within your application to restrict resource usage of a SQL statement(s). f) ASP session timeout as defined in IIS g) Database query timeout value a. @@LOCK_TIMEOUT set b. query wait (instance level) expiring due to held resources

Rename Database
Use the command master..sp_renamedb. Be aware that logins whose default database was set may also change, this can be a problem for the DBA when renaming to only replace with another version of the same database for testing purposes (or other reason). Check logins and verify that you do not need to alter their default database property. After renaming, it is a good idea to set the DBO only property if it is not to be used by end-users, this can assist with debugging login issues.

249

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

User Management
Login vs User
In SQL Server 2000, a single login maps to a single user in one or more databases within the instance. The login defines a basic set of properties for the connection to the instance, the EM screens encapsulate these options as shown in the screen shot below:

The core items of interest are: a) allocation of instance level system privileges (server roles) b) define the default database instance and language c) determine the method of authentication (mixed or integrated) a. authenticating domain must be available (via trust or the server is a member of the domain) d) define database access Defining database access will create a user entry in the sysusers table for the database. The syslogins table in the master database is loosely coupled to the databases sysusers table via the SID column. See orphaned logins next for more information. All security for the user should be granted by database roles to simplify overall privilege administration.

250

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Orphaned Logins
At times, the DBA will restore a database from one instance to another, in doing so, even though the login exists for the instance, the SID (varbinary security ID) for the login is different to that in the other instance. This effectively orphans the database user from the database login due to this relationship between master..syslogins and mydb..sysusers.

master database syslogins

user database sysusers

SELECT SUSER_SID('user1') 0x65B613CE2A01B04FB5E2C5310427D5D5 -- SID of user1 login, instance A 0x680298C78C5ABC47B0216F035B3ED9CC -- SID of user1 login, instance B

In most cases, simply running the command below will fix the relationship and allow the login to access the user database (must run against every database in which to login is valid against).
exec sp_change_users_login <see books online>

This will only work for SQL logins and not fully integrated logins (which is a down right pain). Write your own script (or search the web) to resolve this problem. If you are still getting errors, consider removing the user from the sysusers table in the restored database, and re-add the user to the The DBA can validate NT login accounts via the command: EXEC sp_validatelogins NOTE Do not use ALIASES; this allowed the DBA to map a single login to many database user. Microsoft released a great support document related to the scripting of logins, including the original password. The script is not complete in terms of all possible options, but is very handy: Example: (http://support.microsoft.com/default.aspx?scid=kb;[LN];Q246133)
/* sp_help_revlogin script ** Generated Nov 17 2002 12:14PM on SECA\MY2NDINSTANCE */ DECLARE @pwd sysname -- Login: BUILTIN\Administrators EXEC master..sp_grantlogin 'BUILTIN\Administrators' -- Login: user1

251

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SET @pwd = CONVERT (varbinary(256), 0x0100420A7B5781CB9B7808100781ECAC953CB1F115839B9248C3D489AC69FA8D5C4BE3B11B1ED1A30154D955B8 DB) EXEC master..sp_addlogin 'user1', @pwd, @sid = 0x680298C78C5ABC47B0216F035B3ED9CC, @encryptopt = 'skip_encryption' -- Login: user2 SET @pwd = CONVERT (varbinary(256), 0x01006411A2058599E4BE5A57528F64B63A2D50991BC14CC59DB0D429A9E9A24CA5606353B317F4D4CA10D19A2E 82) EXEC master..sp_addlogin 'user2', @pwd, @sid = 0xF1C954D9C9524C41A9ED3EA6E4EA82F4, @encryptopt = 'skip_encryption'

Change DB Owner
You can change the database owner with the command:
USE MyDatabase EXEC sp_changedbowner 'MyUser' GO

This requires the sysadmin instance privilege, and the user will logically replace the dbo user. You cannot run this command against the system databases. In 99% of cases, this command should not be used.

User Quotas & Account Expiry

SQL Server has no concept of user quotas or resource allocation restrictions as in Oracle. The best bet is to restrict the growth of the entire database. There is also no concept of account expiry, forcing password changes, account locking etc. The DBA should consider QUERY GOVENER, read up the books online for a good overview. This can only be set via the client on connection to the instance (we have no on-login triggers). The DBA can deny and grant login via: exec sp_grantlogin exec sp_denylogin exec sp_revokelogin Unfortunately this only works for Windows NT logins and/or associated groups:
EXEC sp_denylogin 'user2' Server: Msg 15407, Level 11, State 1, Procedure sp_revokelogin, Line 31 'user2' is not a valid Windows NT name. Give the complete name: <domain\username>.

OPS$ Logins (OS Authentication)

The equivalent to the OPS$ login for SQL Server is a fully integrated windows login.

252

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Change Object Owner

Unless you are the dbo (have db_owner or db_ddladmin privileges), you cannot create objects as another user, for example, if I am logged in as user1 cant issue the statement create table user2.mytable (col1 int). As its best practice to have all objects owned by DBO, the DBA may from time to time change the objects owner, we do this via the command: exec sp_changeobjectowner user1.mytable, dbo After running the command, refresh enterprise manager and check the object and its privileges. If its a stored procedure, make sure that the actual create statement is not referring to the old user. This is a classic problem that will ruin you plans for faultless scripting of the object.

Check object ownership

To check if a user owns objects, run the command:
-- Get user ID DECLARE @userid smallint SET @userid = (SELECT uid FROM sysusers WHERE name = 'myuser') -- Check ownership exec dbo.sp_MScheck_uid_owns_anything @userid

User Lock Timeouts

As there are no on login triggers and very few login properties that can be automatically set for a new connection, the DBA has to rely on the developer to do the work. One of these properties is LOCK_TIMEOUT. The value is in milliseconds and zero represents no wait time for locks. A timeout will cancel the SQL statement being executed and must be trapped by the developer:
SET LOCK_TIMEOUT 5000 SELECT @@LOCK_TIMEOUT -- 5 second timeout -- retrieve current value

I recommend that you do not use this setting as standard practice; it does not take into considerating a heavily worked instance or once off locking issues that can result in a ripple effect of application errors if the property was set. If you are in a multi-tier environment (i.e. asp, com+), in which their own timeouts can be set, then consider making the DBMS connection timeout the lowest to capture the DBMS error rather than at a higher level.

Transfer Logins between Servers

To transfer logins between servers and retain the logins password (SQL Logins), consider utilising the DTS task to transfer logins between servers, or the following SQL statements: select 'sp_addlogin @loginame = ' + name + ', @passwd = "' + password + '", @encryptopt = skip_encryption, @deflanguage = "' + language + '"' + char(13) + 'go' from syslogins where name in ('user1', 'user2')

253

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The key to this script is the skip encryption option. Note that we still need to: a) setup the login database user relationship (sp_adduser)

b) assign database user privileges

Change a Users Password

Use the command sp_password for SQL logins only (not applicable for NT logins of course), for example: exec sp_password NULL, newpassword, mylogin

Rename a Login
You cannot rename logins in SQL Server. Actually, you can, but this means changing some entries in the sys tables so good luck with that one. The process involves: a) altering server configuration to allow adhoc changes to system tables b) update user login name in master..sysxlogins c) update databases in which the user exists for dbo.sysusers

Killing Sessions
Within Oracle the DBA can kill user sessions via looking up v$session for the particular user or other status, and issuing the command (just examples of syntax): alter system kill session <sid>, <serial#> immediate; alter system disconnect session <sid>, <serial#> POST_TRANSACTION; At the OS level for rouge Oracle processes, the Oracle DBA on Unix can issue the kill command or on NT can use orakill.exe. Within SQL Server, each connections is allocated a spid (system server process identifier and worker thread). To identify them we can execute the system stored procedures: exec sp_who or exec sp_who2 The DBA can also use the current activity option under the Management Group folder and select Process Info.

254

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

NOTE - Be warned, I have experienced major performance problems when forcing the refresh of current activity via Enterprise Manager to a point where the CPUs hit a solid 50% for 5 minutes before returning back to me. This is not funny when running against a hard working production system. Once the SPID has been identified, we use the KILL command to remove the session. For example: select @@spid exec sp_who2 Kill 55 Kill 55 with statusonly ----get my sessions SPID, in this case its 51 from output, determine SPID to be killed issue kill request to DBMS get status of the request

SPID 55: transaction rollback in progress. Estimated rollback completion: 100%. Estimated time remaining: 0 seconds.

The DBA should reissue sp_who2 to monitor the SPID after the kill to ensure success. Also consider looking at and joining over the tables: sysprocesses syslock syslockinfo You cannot kill your own processes. Be very careful you do not kill system processes. The processadmin fixed system role will allow a user to kill SQL Server sessions. NOTE - Killing SPIDs running extended stored procedures or did a call-out to a user created DLL may take some time to kill and, in some cases, seem to have stopped but remain as a running process. There is no POST_TRANSACTION option as in Oracles alter session statement. Basically SQL Server will do an IMMEDIATE (disconnect and recover session without waiting for ongoing transactions) disconnect. To get further information about a session, consider the command: DBCC PSS (suid, spid, print-option) For example: --Trace flag 3604 must be on DBCC TRACEON (3604) --Show all SPIDs DBCC PSS DBCC TRACEOFF (3604) GO This is an undocumented command. For a good coverage consider see reference (46).

255

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Orphaned Sessions
An orphaned session has a SPID of 2, this may be caused by a variety of things (though rarely) and is typically linked to the distributed transaction coordinator (DTC). A DTC related problem will show up in the SQL Server log files with an error such as SQL Server detected a DTC in-doubt transaction for UOW <value>. If the transaction issue can not be resolved, then a kill statement can be issued over the UOW code. For example: Kill 'FD499C76-F345-11DA-CD7E-DD8F16748CE' The table syslockinfo has the UOW column. NOTE Use Component Services when appropriate to drill into COM+ classes and their instantiations to locate UOW to assist with killing the correct SPID at the database.

TEMPDB
The tempdb is utilised for a variety of reasons, such as: a) using create table #mytable ( clause to create temporary tables a. consider the table datatype over # temp tables. b) by the optimiser to complete a statement where necessary, typically shown as worktables c) many of the DBCC commands utilise temporary tables d) large sort and group by operations The DBA can manage all tempdb functions through EM, even so, here are some important commands to remember
Task Check DB Size Shrink DB Move files Command exec sp_spaceused DBCC shrinkdatabase(N'tempdb')
alter database tempdb modify file (name=tempdev,filename = 'c:\dbdata\tempdb\tempdb.mdf') go alter database tempdb modify file (name=templog, filename=' c:\dblog\tempdb\templog.ldf') go exec sp_tempdbspace (undocumented command) Using the alter command with the SIZE option Consider querying sysobjects in the tempdb database to monitor the creation of # and ## temporary table objects. Also condsider the SQLServer:Databases performance monitor counter and also profiler (filtering out the database by ID).

Space Usage Size of Usage of

You should keep the standard database options with tempdb. You may want to pre-empt the increase of the database size and alter growth rates as you feel appropriate. As a very general

256

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

indicator, my production DB with 800 online users, 4Gb data with lots of OLTP activity and standard DBCC functions at night, sees tempdb growing to a maximum of 800Mb. It is not recommended that you run DBCC commands (like CHECKDB) against tempdb. There is a good chance that errors will be returned depending on what is happening at the time. Remember that tempdb is rebuilt on instance restart, if you do suspect errors, it may be reported on instance startup or further investigation of the underlying file system. The Backup and Recovery section covers the movement of the tempdb datafiles.

Instance UPTIME
There is not specific function to get the uptime of your instance. Consider the query as the next best thing:
select datediff(day, login_time, getdate()) as DaysUptime from master..sysprocesses where spid = 1

Or check your SQL Server Logs.

Re-creating & Auto-Starting Services

To view the configuration of services running (and therefore assist with re-creating them), run regedit.exe and goto /HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services. There will be one entry per service, drilling through to our SQL Server instance service we see:

Consider exporting this portion of the registry after installation. To manually restart the SQL Server NT services, use the NET STOP/START command, for example:
NET STOP SQLSERVERAGENT NET STOP MSSQLSERVER NET START MSSQLSERVER NET START SQLSERVERAGENT

257

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Put this in a batch file and schedule via NTs AT command.

Raw Partitions
I hate raw partitions, they are difficult to use and are restrictive in terms of the underlying NT commands (copy, move, delete etc are not possible) and backups you can perform. I also question the real performance benefits you are getting and have yet to read any white paper that justifies the associated management issues. I am not covering raw partitions and ask you to spend more time with application performance instead. NOTE Raw partitions are unavoidable in Oracle RAC.

SQL*Mail
The SQL*Mail service utilises a MAPI compliant client to send and receive email. The client is typically Microsoft Outlook (not Outlook Express), and configuring Mail (profile) in control panel to setup either an SMTP mail account or hook into Exchange.
MAPI Interface SQL Server 2k (xp_sendmail, sp_processmail, xp_deletemail, xp_findnextmsg etc..)

I have had varied success with SQL*Mail, and at the end of the day, I opt not to use it, but have written a small COM component (DLL) and a stored procedure to call a SMTP send email method. There are numerous articles on the internet and at Microsoft for configuring sql*mail and not using it at all. Have a good read before progressing any further. NOTE This may sound like a contradiction, but I have read a variety of forum posts from DBAs regarding the real need for a MAPI client. To get the low down from Microsoft check http://support.microsoft.com/default.aspx?scid=kb;en-us;Q306962

Auto-growth and Space Management Tracking

Automatic data file growth
Automatic data and log file growth are default options via Enterprise Manager when creating a database. The default growth is by 10% with an unrestricted maximum size.

Space Tracking
Space management tracking is simple enough. The core commands for tracking database space usage are:

258

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- return log utilisation for all instance database dbcc sqlperf(logspace) with no_infomsgs -- get basic summary of database space usage exec sp_spaceused To encapsulate the DBCC command in a table for subsequent querying and historical tracking, utilise the T-SQL command below:
create table #logsize (db_name varchar(30) not null primary key clustered, logsize float, logused float, status integer) insert #logsize ( db_name, logsize, logused, status ) exec ('dbcc sqlperf(logspace) with no_infomsgs') select * from #logsize

At an object level for each database, lookup a variety of sys tables and apply the following logic:
Reserved space (sysindexes) = sum(reserved) where indid in (0, 1, 255) Data space used (sysindexes) = sum(dpages) where indid < 2 + sum(used) where indid = 255 (text) Index space used (sysindexes) = sum(used) where indid in (0, 1, 255) - data Unused (sysindexes) = sum(reserved) - sum(used) where indid in (0, 1, 255)

To get broad file growth data, do not forget master..sysfiles and DBCC SHOWFILESTATS.

Statistics and Histograms

When we select the cost based analyser in Oracle (which should be your only option), the DBA has core responsibility of collecting statistics to ensure optimal and appropriate execution plan generation. In SQL Server, cost based analysis is the only option for the optimiser and there is no concept of rule-based optimisation. Unlike Oracle, statistics collection is fully automated by default at the database level. This is configured via the GUI or command line as shown below:
exec sp_dboption mydb, auto create statistics, on exec sp_dboption mydb, auto update statistics, on

or via Enterprise Manager by selecting properties whilst selecting the database. A statistics collection is synonymous with histograms in Oracle. This is different to physical indexes which naturally have statistical properties (updated on the creation of the index and automatically thereafter based on DB settings). Indexes and statistics collections are critical in enabling the query optimiser to best determine the execution plan.

259

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

At the database level, the auto create statistics option is an interesting one. This tells SQL Server to create a statistics collection (stored in the databases sysindexes table) for SQL predicates not already indexed. For example:
CREATE TABLE [bb] ( [col1] [int] IDENTITY (1, 1) NOT NULL , [col2] [int] NULL , [col3] [int] NULL , CONSTRAINT [PK_bb] PRIMARY KEY CLUSTERED ([col1]) ON [PRIMARY] ) ON [PRIMARY] GO The following statistics collection was created in sysindex on execution of the query: (name field in sysindexes) _WA_Sys_col2_29572725

select * from bb where col2 = 2 select * from bb where col2 = 2 and col3 = 2

_WA_Sys_col3_29572725

Note that running exec sp_statistics bb will return a list of all indexes and statistics for our table in the example. I personally find this procedure useless for statistic collections as you may find with its output. Use exec sp_helpstats 'bb' to get a listing of statistics for the table, pass this into dbcc show_statistics ('bb', '_WA_Sys_col2_29572725') for example to get a further analysis of the collection. Note that the collection can be replaced with the index name of the table if need be as we will see later. NOTE Statistics cannot be created for text, ntext and image column datatyes. Both indexes and collections include histogram (distribution) statistical information in the statblob column within the sysindexes column. Throughout in this chapter we will cover both the autocreation and autoupdate statistics options, along with analysing the statistical data collected.

Building and Maintaining Histograms

Within Oracle, the DBA can create histograms via the command:
EXECUTE DBMS_STATS.GATHER_TABLE_STATS ('myuser','mytable', METHOD_OPT => 'FOR COLUMNS SIZE 10 mycol');

Funny enough, Oracle state that histograms must be carefully chosen as they can impact performance, and should, generally, only be used on columns with highly skewed data so as the cost-based optimiser can make well formed decisions over uniformity of the data. In SQL Server this thinking is somewhat different and statistics are used widely throughout the database instances with little impact. These statistics (especially if auto-stats option is on) can occur over uniformly distributed data for example and in most cases, seem to help the perhaps less complex optimiser compared in SQL Server.

260

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In Oracle, the histogram min/max values and distribution data is stored in buckets that the DBA specifies. In SQL Server the histogram and associated density group is stored in the statblob column that is an image blob datatype.
CREATE STATISTICS usrstat_mytable_col1 ON mytable (col1) WITH FULLSCAN

The DBA can utilise the WITH NORECOMPUTE option to prevent SQL Server from subsequent updates to the statistics. This is not recommended. NOTE A statistics collection is regarded as an index against the table although its not a physically one, therefore, query sysindexes where locating statistic collections. As the Oracle DBA will quickly realise, the SQL Server histogram is very simplistic when compared to Oracle.

Import and Export Statistics

There is no ability to export or import statistics as found within Oracle. The DBA may find an unsupported method, but there are no utilities, extended stored procedures or otherwise to cater for this. Moving or copying a database will of course bring with it all statistics.

Re-collecting Statistics & Monitoring Auto-Statistics Collection

Updating Statistics

The DBA can manually update statistics via:

UPDATE STATISTICS mytable (col1) WITH SAMPLE 70 PERCENT UPDATE STATISTICS mytable (col1) WITH RESAMPLE UPDATE STATISTICS mytable (col1) WITH FULLSCAN use mydb exec master..sp_updatestats -- update statistics for all objects in mydb

Such statistical information can be recollected for the entire table or index, the DBA can also selectively re-sample a specific index or collection within the object. The DBA can also create a Database Maintenance Plan to schedule the update statistics of statistics. Most DBAs will opt for this as its simple and works very well. Those requiring finer control will use a custom t-sql.
Monitoring Statistics

The most effective way to monitor the automatic updating of statistics is via Profiler. Select the following (not definitive by any means but a good start):
Events Objects, Auto-Stats Data Columns Event Class, Duration, Database ID, ObjectID, StartTime, EndTime, TextData

An example screen shot is shown below. The update runs asynchronously to the users execution of

261

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

the query. After some testing I still find it difficult to predict the actual logic being used, as a large row count change doesnt necessarily cause a stats update but different queries against the table with different predicates does. Keep an eye on regularity, associated locking (if any) and its impact on CPU time. Here is an example output, use OBJECT_NAME to get the actual table name:

Controlling Stats Collection at the Object Level

The DBA can turn off statistics collection at the object level if need be. There may be a variety of reasons why this is done, but it would be a rare occasion. On creation of an index the DBA can use the option below:
CREATE UNIQUE CLUSTERED INDEX [PK_mytab] ON [dbo].[mytab] ([col1], [col2]) WITH FILLFACTOR = 90, STATISTICS_NORECOMPUTE ON [PRIMARY]

CREATE STATISTICS mytab_stats ON mytab (col3) WITH FULLSCAN, NORECOMPUTE

This option is available within Enterprise Manager when editing your indexes and statistic collections. Be careful, as any manual or automatic statistic collection will skip these objects, which can be a problem when debugging performance issue or strange execution plans.

Analysing Indexes & Histograms (collections)

The DBA can analyse the distribution statistics for indexes or via:
DBCC SHOW_STATISTICS (mytable, mytableindex) -- sysadmin, db_owner or db_ddladmin fixed role users only

The command is very helpful when drilling down through and index to determine its selectivity and subsequently its usefulness. Classic examples are comparing existing non-clustered (heap) and clustered indexes to determine wether another index is better suited to range scans and therefore clustering. The results are divided into 4 sections,we discuss via an example: DBCC SHOW_STATISTICS (address, ix_address_locality) The core purpose of the command is to display information about index density. Providing the DBA with an insight into the key distribution within the index, and therefore its selectivity.

262

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Updated length Rows Rows Sampled Steps Density Average key -------------------- -------------------- -------------------- ------ ------------------------ -------------May 12 2002 4:17PM 117798 117798 200 2.3193457E-4 12.565909

All density -----------------------1.0090817E-3 8.489108E-6

Average Length -----------------------8.5659094 12.565909

Columns --------------------------------------------addr_locality addr_locality, addr_id

Use the formula all-density * rows = key selectivity to calculate wether the key combination is highly selective. In the case above, we have a composite index over addr_locality and addr_id, we are shown the two sets of figures for us to calculate the density for each combination of index lookup. In this case the values are: addr_locality = 118 rows (very poor) addr_locality, addr_id = 9 rows (ok) of course, perfect selectivity is a value of 1. This is still no substitute for a thorough understanding of your applications demographics and associated indexing requirements, but is still a very good guide. The other columns are self-explanatory. The next part of the statistics shows the sampled histogram, each representing a step. The columns of interest are: a) range high key (upper value of histogram step) b) eq rows (number of rows in sample with this value) c) avg range rows (avg duplicate values within the step range)

RANGE_HI_KEY ----------------------AJANA ALBANY ALEXANDER HEIGHTS APPLECROSS ARMADALE ATTADALE AUSTRALIND BALCATTA BALGA BALLAJURA

RANGE_ROWS -----------------------0.0 0.0 10.0 148.0 219.0 127.0 179.0 99.0 123.0 51.0

EQ_ROWS -----------------------6.0 1574.0 150.0 541.0 853.0 235.0 218.0 1364.0 199.0 457.0

DISTINCT_RANGE_ROWS -------------------0 0 1 8 3 5 2 6 1 3

AVG_RANGE_ROWS -------------0.0 0.0 10.0 18.5 73.0 25.4 89.5 14.142858 123.0 17.0

I use a tool call Diagnostics Manager from NetIQ, a great management tool for drilling into such statistics, here is a screen shot (versions will undoubtly differ):

263

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Please read the chapter on performance tuning and indexing issues to better understand and leaver off the statistics presented above.

Synonyms
There are no synonyms in SQL Server, the best you can do is (perhaps) use views. In all cases the owner of all user database objects should be dbo, then use roles to manage security.

Shrinking Database Files

The shrinking of database files is relatively common for SQL Server DBAs. The DBA should keep auto-growth parameters on for their database files but think carefully about the growth parameters to reduce unnecessary IO and huge jumps in un-used space. Anyhow, back to shrinking; the first option that should be turned off is the auto shrink database option (via EM, properties for the database):
ALTER DATABASE pubs SET AUTO_SHRINK OFF

So long as the DBA understands the application(s) running against the database, its growth spurt periods and that a database running with a full recovery model requires log backups to occur (or your transaction log file will keep growing), then a nightly, weekly or monthly database shrink is more than enough.

264

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Examples: -- 2 is the % max free space after shrink -- truncate free space only DBCC shrinkdatabase(N'pubs', 2, TRUNCATEONLY ) -- will move pages to beginning before shrink -- same as compress and truncate DBCC shrinkdatabase(N'pubs') -- shrink primary data file in current db (pubs) to 2Mb DBCC SHRINKFILE (N'pubs', 2) -- truncate free space only (no page movement) DBCC SHRINKFILE (N'pubs',TRUNCATEONLY)

WARNING If you shrink specific files, DO NOT press the OK button to close the shrink database dialog, if you do, it will run a shrink database for the entire database. Under the FILES option of the shrink dialog, the empty filegroup option also exists. This option only works for a single filegroup that has multiple data files. As you know, if you create an object on a specific filegroup, SQL Server will distribute the objects storage over the groups database files. The EMPTYFILE option will move (migrate) all data from the specified file to the other file sharing the same filegroup.
-- DB_DATA2 is the logical file name for my DB_DATA filegroup, all object data is move from this -- database file to DB_DATA1, therefore allowing me to drop DB_DATA2 DBCC SHRINKFILE (N'DB_DATA2',EMPTYFILE)

To track shrinking, consider the DBCC EXTENTINFO (mydb) command as discussed in the Microsoft Support document Q324432. This support document provides a great script for viewing object space allocation and fragmentation. I have adapted the end-query a little to suit my needs:
-- To hold DBCC command data for further analysis create table extentinfo ( [file_id] smallint, page_id int, pg_alloc int, ext_size tinyint, obj_id int, index_id tinyint, pfs_bytes varbinary(10) ) -- Stored proc to run DBCC create procedure import_extentinfo as dbcc extentinfo(pubs)

265

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

go -- Populate the table insert extentinfo exec import_extentinfo go -- Show possible fragmentation and reduction possibilities (run shrink later and compare the results) select FILE_NAME([file_id]) as DBFile, OBJECT_NAME(obj_id) as ObjectName, case index_id when 1 then 'ClusteredIndex' when 255 then 'Text/Image' else 'NonClustered' end as IndexType, ext_size as ExtentSize, 'actual extent count'=count(*), 'actual page count'=sum(pg_alloc), 'possible extent count'=ceiling(sum(pg_alloc)*1.0/ext_size), 'possible extents / actual extents' = (ceiling(sum(pg_alloc)*1.00/ext_size)*100.00) / count(*) from extentinfo where ext_size != 1 group by [file_id],obj_id, index_id, ext_size

266

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL DMO
SQL Distributed Management Objects (SQL-DMO) is a COM layer whose relationship to the SQL Server engine is such that it is an in-process OLE Server (in other words OLE automation compatible COM objects). As such, it exposes many of the core methods for SQL Server object and replication manipulation, both local and remotely. NOTE SQL DMO is fundamental to Enterprise Managers functionality. The DLL is called sqldmo.dll. Select properties of the DLL via the OS to get its version information. The object model is very thorough, and will cover close to all of your requirements for database manipulation. A simple search of www.google.com will provide you with more than enough examples to get started, especially for those experienced with VB. The DBA and/or developer can use SQL-DMO in a variety of ways as shown below. An excellent resource can be found at:
http://www.sqlservercentral.com/columnists/sjones/sqlserverdmoresources.asp

VB Example
Dim oServer As SQLDMO.SQLServer ' mixed mode login example.. Set oServer = New SQLDMO.SQLServer oServer.Connect vServer, "sa", "theSApassword" ' trusted connection Set oServer = New SQLDMO.SQLServer With oServer .LoginSecure = True .Connect End With

VB Script Example (.vbs)

Dim oSQLServer Set oSQLServer = CreateObject(SQLDMO.SQLServer) oSQLServer.LoginSecure = True oSQLServer.Connect set oBackup = CreateObject("SQLDMO.Backup") oBackup.Database = northwind oBackup.Files = c:\nothwind.bak oBackup.Action = 0 oBackup.SQLBackup oSQLServer Set oBackup = Nothing oSQLServer.Disconnect Set oSQLServer = Nothing

267

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

12
Chapter

SQL & T-SQL

hroughout this chapter we will discuss transact SQL (equivalent to PL/SQL) and common SQL commands with examples. I will not give too many direct comparisons to Oracle but will attempt to focus on providing you with good overall coverage to speed your introduction to SQL Server.

PL/SQL (T-SQL)
The core programming language comparable to PL/SQL is T-SQL (Transact-SQL). This will change in future releases of SQL Server when the .Net framework becomes embedded in the DBMS. The affect of this is that any .net compatible language can use used instead of T-SQL. The following sections summarise some of the core elements of T-SQL with comparisons with Oracle as required. [ ] = Optional code segment {} = One or more mandatory options

Comments and Statement End

Both Oracle and SQL Server use the same strings for comments.
-- this is a single line quote /* this is a multi-line quote */

You cannot nest multi-line quotes, attempting to do so will result in error. The single line quote can appear anywhere on the line, although once started all values after it until a newline will be regarded as comment text. T-SQL statements do not require the ; or any other character to denote the end of the statement. If used at the end of a DML statement the command will not error.

268

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Understanding GO
The GO statement defines the end of a batch of T-SQL, although itself is not a T-SQL statement. Using profiler (discussed later), we can see the affect that the GO statement has on batch submission of SQL and T-SQL statements to the relational engine:
use pubs declare @aa integer set @aa = 1 print 'this is a test' print @aa GO

use pubs GO declare @aa integer set @aa = 1 GO print 'this is a test' print @aa GO

Of course, each of the statements between the GO are compiled as a single execution plan.

Take care, variables are not persistent between batches, the above actually gives us an error for the below code, as @@aa is not declared within the batch. print 'this is a test' print @aa Be careful using the GO statement in stored T-SQL code, the GO statement will define the end of the code block so if you went a placed GO statements all through your stored procedure for example and saved it successfully to the database, you may find that the rest of the code is missing after the first GO statement. The entire stored procedure / UDF is a single block of code and you cannot change this via GOs.

269

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Block Structure
The Oracle block structure for functions, procedures, and anonymous blocks consists of the format: ORACLE [declare -- variable declarations] begin -- body code and other statements [exception -- handler code] end; SQL Server [begin] -- declaration and body code can be -- anywhere after the declaration -- of the procedure/function/trigger -- there is no enforced syntax [end] [GO]

It should be noted that although optional in this particular case, the BEGIN and END statement is used to define a logical block. The begin/end combinations are mandatory for multi-line control statements (2 or more lines, i.e. IF, loops) as we will see later. The begin cant be used without the end statement. The BEGIN and END statements do not affect a variables scope.

Declarations / Local Variables / Constants

The T-SQL variable or parameter must include the prefixed @ symbol to be valid. All declaration require the DECLARE statement. myvariable NUMBER; DECLARE @myvariable INT

You can declare multiple variables of the one DECLARE statement, for example: DECLARE @myvariable INT, @mysecondvar CHAR(1) You cannot set default (initial) values for the local variables:
DECLARE @error_count INTEGER = 0 Server: Msg 139, Level 15, State 1, Procedure myproc, Line 3 Cannot assign a default value to a local variable.

You cannot use all variable types for local variables:

Server: Msg 2739, Level 16, State 1, Procedure myproc, Line 3 The text, ntext, and image data types are invalid for local variables.

SQL Server T-SQL has no concept of constants, they are only string literals or other fixed values, like 10, Hello etc in code that is it. Take care with declarations. If you do not initialise the variable the default value is NULL. This will create adverse problems, for example:

270

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

DECLARE @error_count INTEGER print @error_count + 1 -- will return NULL set @error_count = 0 -- initialise first (basic good practice) print @error_count + 1 -- will return 1 The BEGIN and END statements do not affect a variables scope. Therefore, once a variable is declared ALL code below it (within the same batch of course, see the section on GO) can refer to it. Nested calls to other stored procedures or UDFs cannot use the variables declared.

Assigning values to variables & Create Table as

Oracle myvariable := 10; SELECT MAX(id) INTO @myvariable FROM mytable; SQL Server set @myvariable = 10 set @myvariable = (SELECT MAX(id) FROM mytable) select @myvariable = 10 SQL Server does support the INTO clause, but this is the same as the CREATE TABLE AS <select-statement> clause in Oracle. The syntax is also a little back to front, for example: -- create a new SELECT * INTO -- as above but SELECT * INTO This is a) b) c) d) e) f) table mynewtable based on myexistingtable with all data mynewtale FROM myexistingtable no data copied mynewtale FROM myexistingtable WHERE 0 = 1

a bulk/insert command. The new table will not inherit: primary or foreign keys triggers column formula and default properties indexes file-group, the table will be placed into the PRIMARY filegroup of the database role level permissions and directly allocated user permissions

It will inherit: a) identity property b) table structure

271

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Exception Handling (error handling)

There is no exception handling in T-SQL, therefore it can be close to impossible to re-direct runtime errors or other exceptions to custom written code blocks, or cascade up an error through nested calls. The best we have is shown via examples below. In many cases we will use the optional BEGIN/END statement. NOTE Remember, stored procedures can utilise the RETURN statement to stop code execution at the point of the RETURN, it is not restricted to user-defined functions.
-- Example 1 <any DML statement here> IF @@ERROR <> 0 BEGIN PRINT An error occurred with the previous statement END -- Example 2 DECLARE @error_count INTEGER <any DML statement here> set @error_count = @error_count + @@ERROR <any DML statement here> set @error_count = @error_count + @@ERROR IF @error_count <> 0 BEGIN PRINT An error occurred with the a range of statements END -- Example 3

BEGIN RAISERROR (20067, 16, -1, Critical Error) RETURN (1) END
-- Example 4 IF @@ERROR <> 0

IF @@ERROR <> 0

BEGIN GOTO CriticalError END <other code> RETURN 0 -- skip over labels below CriticalError: -- place labels at the end of your stored procs for ease of maintainability PRINT Critical error message Rollback Transaction -- optional of course, implementation specific RETURN 1

272

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- Example 5 declare @RowCount int, @ErrValue int <DML statement> select @RowCount = @@ROWCOUNT, @ErrValue = @@ERROR IF @ErrValue <> 0 GOTO CriticalError RETURN 0 CriticalError: BEGIN IF @@trancount <> 0 ROLLBACK TRAN raiserror @ErrValue set nocount off RETURN @ErrValue

END

NOTE Apparently the next version of SQL Server will bring with it exception handling for TSQL, similar to that used in VB. If you have issues with checking @@ERROR directly in an IF statement, then immediately after the command you are verifying, assign @@ERROR to a variable and use this for checking. Be warned that not checking @@ERROR immediately and executing other statements may affect the value returned from @@ERROR.

RAISEERROR and Error Messages

It is worth explaining raiserror and error messages in more detail. Within the master database we have the table sysmessages, which includes over 3782 predefined error and warning messages of varying severity. select * from master..sysmessages By passing in @@ERROR, we can translate and appropriate error code into its equivalent severity and description. The severity levels are:
Severity Level 0 to 19 RAISEERROR 0 to 18 can be used by any user. 19 to 25 require the NO LOG. sysadmin users only. Summary 1 to 9 = warning and process errors 10 = Status 11 to 16 = warning errors 17 = out of resource error 18 = non-fatal internal error 19 = internal sql setting exceeded Fatal system error.

20 to 25

We can maintain our own user-defined messages via the following stored procedures in the master database:

273

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

sp_addmessage sp_dropmessage sp_altermessage Example:

exec master..sp_addmessage 50006, 16, 'My Error parameter 1:%d, parameter 2: %s.' RAISERROR (50006, 16, 1, 100, 'MyTest') Server: Msg 50006, Level 16, State 1, Line 1 My Error parameter 1:100, parameter 2: MyTest.

NOTE - Be warned though in stored procedures, if you were planning on using raiserror to stop processing immediately you may be surprised, a classic example are foreign key errors on an insert or update and the stored procedure continues to run. Always force the exit on error via a RETURN statement and rollback otherwise the non-fatal error will be shown and the code execution continues. We can also log userdefined error messages to the SQL Server log and Windows NT event log via the extended stored procedure xp_logevent.

Non-SQL Output
Calling DBCC commands and other system stored procedures (xp_ or sp_) , typically requires the encapsulation of the result set into a temporary or static table for further manipulation. This practice may also required when calling the stored procedure and returning the results to a recordset from a VB or other application.
DECLARE @ExecStr varchar(50), @Qry nvarchar(255) -- create the tempory table (stored in tempdb), also consider the TABLE datatype CREATE TABLE #inputbuffer ( EventType nvarchar(30), Parameters int, EventInfo nvarchar(255) ) SET @ExecStr = 'DBCC INPUTBUFFER(' + STR(@@SPID) + ')' INSERT INTO #inputbuffer EXEC (@ExecStr) SELECT EventInfo FROM #inputbuffer

Take care with the EXEC[UTE] statement, which may result in the error:
Server: Msg 197, Level 15, State 1, Procedure mytest, Line 16 EXECUTE cannot be used as a source when inserting into a table variable.

This is a real pain for the new data type of TABLE.

274

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Cursors
As in Oracle, the cursor is pretty well standard for the modern DBMS. In terms of T-SQL, the cursor type and associated command set has a large number of options that is better covered in the books online. A very basic example is show below, you will probably use this example in a majority of cases for simple record processing:
Use PUBS GO DECLARE @empid varchar(20), @fname varchar(50) DECLARE mycursor CURSOR FOR SELECT emp_id, fname FROM employee OPEN mycursor FETCH NEXT FROM mycursor INTO @empid, @fname WHILE @@FETCH_STATUS = 0 BEGIN PRINT @empid + ' : ' + @fname FETCH NEXT FROM mycursor INTO @empid, @fname END CLOSE mycursor DEALLOCATE mycursor -- note the missing @ operator!

You will see this per FETCH NEXT statement:

If we altered data after the OPEN statement, the record will be reflected in the cursor. The cursor in this particular case does not pre-fetch the data (as the plan shows above). Read carefully in the books online about other options such as INSENSITIVE, STATIC and KEYSET that may change this, for example, an INSENSITIVE cursor sees a single operation generated for the whole statement rather than one for each fetch.

275

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In the CURSOR example above, we see the following locks:

Those cursor options that utilise the TEMPDB are not recommended (see STATIC cursors next though for an example where its required). On top of this, dont use options where you update through the cursor, use a standard SQL update/delete command. A simple test with the above statement and the SCROLL option for example:
DECLARE mycursor SCROLL CURSOR FOR SELECT emp_id, fname FROM employee

Build the cursor temporary table

Scan internal table and return row set.

This sees the appropriate locks taken out on the new tempdb object and an increase in turn-around time of about 30%, not great on the already slow standard (dynamic) cursor. In the discussions above, I have made a point of highlighting locking. One of the big issues with SQL Server performance is locking and blocking. Be very careful with cursors, especially options such as SCROLL_LOCKS. Unintended locking and the flow on effects in terms of overall performance and time spent tracing the problem is something you would rather avoid than deal with later. So long as you close and deallocate, there is no reason why you cant alter the cursors SQL statement, open and continue with your new set of work. Failure to deallocate will result in the error:
Server: Msg 16915, Level 16, State 1, Line 4 A cursor with the name 'mycursor' already exists.

276

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Be aware that many SET operations (such as setting dateformat) will result in error if a cursor has been declared before it:
Server: Msg 16958, Level 16, State 1, Line 9 Could not complete cursor operation because the set options have changed since the cursor was declared. Server: Msg 16917, Level 16, State 2, Line 11 Cursor is not open.

NOTE Always opt for a bunch of DML statements if you can to do a bulk of the work. Adversely, be careful trying out to completely code out cursors all together, depending on the code, you may find a WHILE loop with multiple DML statements even slower ! test carefully. In terms of cursor functions, the equivalents between Oracle and SQL Server are:
Oracle SQL Server

%FOUND %NOTFOUND %ROWCOUNT %ISOPEN STATIC Cursors

@@FETCH_STATUS @@FETCH_STATUS @@CURSOR_ROWS @@CURSOR_ROWS - value of zero

The DBA should spend some time reading up on the types of cursor. By default, all cursors are dynamic, this means that each FETCH will retrieve the value from the database as is, not from when the cursor was first opened. A classic example of where this can be a problem is when your cursor SQL statement is using a not exists or not in clause. If on a fetch of one record, you alter the data in the table that effectively cancels out the not exists/not in for subsequent records, then unless you used s STATIC cursor, the next fetch may return no records. If you are experience this sort of problem, consider STATIC cursors that pre-fetch the data into a temporary table.

UTL_FILE (File IO)

In SQL Servers native T-SQL language, there is no native stored procedure or extended stored procedures for file input/output. This will of course change with the release of .Net SQL Server (possibly called Yukon). Alternatively, we can: a) write an extended stored procedure in C++ or VB b) use the sp_OA* stored procedures in the master database to call out to a COM c) write a DTS and create an active-x step that utilises vb-script to call FSO (file-system objects) d) call out to the OS via xp_cmdshell a. possibly using bcp.exe to dump data from table or view we have created e) utilised a linked server (read only) this is far from complete but gives you some ideas.

277

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The linked server e) above is an interesting one. In the SQL Server Magazine, June 2002, Brian Moran showed that utilising sp_addlinkedserver, you could subsequently query an external file (this is also in the BOL). Here is an example:
exec sp_addlinkedserver textfile, 'Jet 4.0', 'Microsoft.Jet.OLEDB.4.0', 'c:\', null, 'Text' select count(*) from textfile...[cktest#txt] select * from textfile...[cktest#txt] exec sp_dropserver txtsrv -- get line count, file is c:\cktest.txt -- get the data (first line assumes to be the column header in -- query analyser

Procedures/Functions/Packages
Lets be honest here, T-SQL is the poor mans PL/SQL and probably always will be. This of course will all change when SQL Server Yukon is released and the power of VB, C++ and over 20+ other languages that are embedded in the .Net framework, enter the SQL Server DBMS. Anyhow, back to the topic, all we have in terms of code blocks in SQL Server are: User Defined Functions (UDF) Stored Procedures System stored procedures Extended stored procedures (wrappers to operating system DLLs in master DB) User defined stored procedures There are no T-SQL packages or concepts of private and public routines (control security via roles), so like I said, its the poor mans PL/SQL.
User Defined Functions (UDF)

As an Oracle DBA you will probably be surprised that the UDF is a new feature to SQL Server, I would have thought this was a fundamental but apparently not. Anyhow, the concept and code is very easy and there is no rocket science to it.
Basic Syntax CREATE FUNCTION [owner.]function-name ([zero or more parameters]) RETURNS [data-type] AS [BEGIN] [optional declarations] [{other code}] [END] GO Example -- in this example, the begin and end must be removed -- the columns returned must be named RETURN ([value or a select statement])

278

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

CREATE FUNCTION mytest() RETURNS table AS RETURN (select 'hello' as col1) GO

Some examples when using the functions:

print myfunction(1) select myfunction(col1) as newcol1, col2, col3 from mytable create table mytable (col1 int, col2 as (myfunction())) -- col2s data type is that of the return value for the function.

There are a variety of issues to be aware of with functions: a) cannot be indexed b) must be deterministic c) concerned with speed? I am, test this very carefully as even a stub call results in some interesting speed issues. d) cannot call extended stored procedures that return data directly from the function (must encapsulate the returned data into a temporary table first) e) has many restrictions, such as not being able to call dynamic SQL inside it, or creating temporary tables, eg:
Server: Msg 2772, Level 16, State 1, Procedure mytest, Line 5 Cannot access temporary tables from within a function.

User defined Stored Procedures

The stored procedure is very common amongst SQL Server developers, and true for Oracle database applications as well (but typically encapsulated in packages). The stored procedure is the key for enhanced system performance and maintenance of DML that would otherwise be in the COM+ tier (business layer). Like UDFs, the BOL provides thorough coverage of the syntax of the procedure. There are few restrictions in the stored procedure unlike the UDF, and are relatively simple to implement and call via VB. The basic syntax is:
create procedure [owner.]procedure-name [parameter-name datatype[,]] as set nocount on -- stops INPROC returns, see performance tuning chapter [all declarations here] go {procedure code}

By default all parameters are IN only (read-only), unless the OUTPUT clause is used, for example:

279

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

create procedure dbo.checkuser @username varchar(30), @isvalid INT OUTPUT as <etc> go

Like variables, all parameters must include the prefixed @ symbol. To specify a default value for parameters, use the syntax:
create procedure dbo.checkuser @username varchar(30), @isvalid INT = 0 OUTPUT as <etc> go

Like the UDF, the stored procedure can return values, but only integers via the statement RETURN [value]. The DBA can encrypt procedures (and functions) via the WITH ENCRYPTION option. This uses an internal encryption routine and is further covered in the security chapter. This makes the syscomments text column unreadable. Be warned that encrypted routines require script change management as they can not be scripted from EM. Finally, the last option worth mentioning is WITH RECOMPILE. This causes the recompilation of the execution plan every time it is called. This is not normal practise and should be carefully considered before using the option. To trace the recompilation, check for the event SP:CacheMiss, this is the only indication followed by the SP:Starting (being the actual routine being called). These events are found under the stored procedure group of events. An example trace:

I have had mixed success with the SP:Recompile event and I wouldnt recommend it for tracing recompiles.
Versioning Stored Procedures

It is possible to version procedures via:

create procedure mytest;1 as {code} go

To call, you must include ;1, therefore, its not really. You cannot overload stored procedures or functions, which is another pain to deal with.
Security & Ownership Issues

The DBA can grant create procedure or function access at the database level for which the user is a member of; but cannot deny/grant the alter statement. By default, unless the user is part of db_owner or ddl_admin database fixed roles, the user can not create or alter objects owned by different users (such as DBO).

280

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

It is very important that ALL objects in a database are owned by the dbo user before moving into production. During development, the DBA should let the developers (using defined logins), create procedures, functions and views freely. The DBA can then alter the owner via sp_changeobjectowner to dbo and allocate privileges via database roles, moving the routines through to test and then into production. When creating procedures and functions, avoid explicitly prefixing the owner, eg: create procedure myuser.mystoredproc <etc> Doing so can be a nightmare for ongoing administration, where the textual definition of the function/procedure is different to that defined in separate columns within sysobjects. The flow on effect is that EM shows a procedure as owned by dbo for example, but when editing the procedure, the create procedure statement has a different (original) user! not good when you rely on object scripting to move between servers as the script will have the wrong user. While all this is fine, there can be recompile and compile lock issues not prefixing calls to routines with the owners name. I am not convinced this is a problem in terms of performance. We cover these and other issues in the chapter on security.

Calling External Applications (command shell)

In the master database we have the extended stored procedure xp_cmdshell. This allows us to call out to any operating system command (executable, batch file etc). As such, this can be a major security risk that needs to be carefully controlled by the DBA. Assuming that userA in their database userdb, wanted to use xp_cmdshell to compress some files in some predefined directory. The programmer, of course, has a variety of methods but in this particular case must use a T-SQL stored procedure call. To give the user access to xp_cmdshell we: a) Run Enterprise Manager and connect to the instance b) Goto the user folder for the master database and chose the add user option a. Make userA a user of the master database b. Grant execute permission to xp_cmdshell for the user i. Optionally use a role rather than granting the privilege direct The user can then run:
exec master..xp_cmdshell c:\scripts\gzip.exe *.dat

By default, sysadmin users have xp_cmdshell execute access, and auto-proxies to the user running the instance. For everyone else, you may need to set the "sqlagent" proxy via:
-- what the proxy is set at currently EXEC master.dbo.xp_sqlagent_proxy_account N'GET' -- setup the proxy EXEC master.dbo.xp_sqlagent_proxy_account N'SET', N'MYNTYDOMAIN', -- windows domain of NT user N'SQLServer', -- agent NT username

281

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

N'password'

-- agent NT password

Remember, this is completely different from the service account in which SQL*Agent is running, selecting its properties via Enterprise Manager does not reflect the change above. Only setting this if you are presented with a SQL error related to the SQL*Agent proxy account ("error at line 140 in xp_shell.cpp, no proxy"). A way to deal with security is to encapsulate the calls to xp_cmdshell in another stored procedure that users are granted access to via the same paradigm mentioned above. Within this we can audit user activity, encapsulate further validation to prevent certain actions from occurring etc. In this way we only grant access to the wrappered stored procedure and not xp_cmdshell direct.

exec master..RunOsCommand gzip.exe c:\*.dat

Create procedure RunOsCommand @cmdstring nvarchar(255) as -- <do some auditing here> exec master..xp_cmdshell @cmdstring -- <return error messages etc> GO

Granted execute privileges to custom wrapped procedure.

xp_cmdshell call

sysadmin system privilege or if non-sysadmin, will run via the proxy account setup in SQL*Agent configuration.

DBMS_SQL
Please read the section on Dynamic SQL.

DBMS_RANDOM
In Oracle, the PL/SQL developer can use the DBMS_RAMDON package to generate random numbers. This is installed via: utlraw.sql prvtrawb.plb dbmsoctk.sql prvtoctk.plb dbmsrand.sql or review $ORACLE_HOME/rdbms/admin/catoctk.sql. To generate random numbers in SQL Server we use the pseudo-random number generator function RAND with a optional single parameter being the seed. select rand()

282

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

As described in the SQL Server documentation, if you decide to specify a seed, then consider utilising getdate() to assist with randomising the seed. A fixed value will return the same random value every time (of course).

DBMS_OUTPUT
The DBMS_OUTPUT package allows the DBA to write out messages to the console (i.e. SQL*Plus etc) whilst our stored procedure or other PL/SQL code executes. This is typically used when debugging large routines, tracking progress or even for character based reporting. Out of all the procedure calls you can make with the DBMS_OUTPUT package, the only equivalent in SQL Server is the print command. Which is not necessarily a bad thing. Example:
CREATE PROCEDURE exampleofprint AS print '===== This is an example ====' GO
print 'The time is ' + cast(getdate() as varchar)

Each single print command has a maximum of 8000 characters. Also note that print does not work in triggers.

DBMS_MAIL, UTL_SMTP
Within emails a) b) Oracle we can utilise the DBMS_MAIL package, or the UTL_SMTP package for sending via a PL/SQL stored procedure. In SQL Server we can: use SQL*Mail via a MAPI compliant emailer write our own DLL and call out to it via the sp_OA procedures in the master database

Search contents page for SQL Mail.

DBMS_AQ
No packaged routines exist in SQL Server for queuing. Search on the web for MSMQ for more information. The DBA can easily call its API if need be or talk to an MSMQ within the DTS prepackaged active-x controls for MSMQ.

PSP, PL/SQL Web Toolkit

There is no equivalent to the PSP and/or the PL/SQL web toolkit. The best we have is SQL Server returning XML directly from a SQL statement (FOR XML clause). The MS development tools have this area very well covered.

DBMS_METADATA
I rarely see DBAs utilising this package in Oracle as most use their own custom scripts (many sourced from web-sites) and typically provide all they require. Even so, this package provides high

283

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

level meta-data about the underlying database schemas rather than utilising the sys objects. In SQL Server, we have a similar built-in) meta-data views called the information schema. Listed below are a variety of information schema views available. The list is far from comprehensive but does provide a general guide.
Information Schema information_schema.tables information_schema.views information_schema.columns Column Summary Catalog (database), Owner, Object name, Object type (includes views) Catalog (database), Owner, Object name, View Source (if not encrypted), Check Option?, Is Updatable? Catalog (database), Owner, Table name, Col Name, Col Position, Col Default, Is Nullable, Data Type, Char Min Len, Char Octal Len, Numeric Prec <etc> Database Name, Schema Name, Schema Owner, Def Char Set Catalog, Def Char Set Schema, Def Char Set Name Catalog (database), Owner, Constraint Name, Unique Constraint Database, Unique Constraint Owner, Unique Constraint Name, Update Rule, Delete Rule

information_schema.schema information_schema.referential_constraints

There are no special security restrictions for the views and they are accessible through the public role. Be aware though that the schema views will only return data in which the user has access too. This is determined within the views themselves via the permissions() function. To view the code of any of the views:
use master exec sp_helptext 'information_schema.Referential_Constraints'

For further examples see reference (44) for great coverage.

DBMS_JOB
The equivalent to DBMS_JOB for the scheduling of T-SQL stored procedures, is to use the following MSDB stored procedures: a) sp_add_job -- add defined job to be run by SQL Agent b) sp_add_jobschedule -- creates a schedule for a job c) sp_add_jobserver -- sets target server to run the job d) sp_add_jobstep -- add step to the job (some work to be performed) e) sp_update_job -- change job properties f) sp_update_jobstep -- change job step properties g) sp_update_schedule -- alter execution schedule for job h) sp_delete_job -- remove job i) sp_help_job -- get metadata about job j) sp_help_jobhistory -- get execution history k) sp_help_jobschedule -- get metadata about job schedule l) sp_help_jobserver -- get metadata about job server m) sp_help_jobstep -- get metadata about the job step n) sp_add_category (optional) -- categorises the job o) sp_update_category -- alter categorisation

284

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

These of course map to a range of MSDB system tables, the data model of which is shown below:

sysjobservers

(+multi-server jobs)

sysjobs

sysjobsteps

sysservers

(linked servers)

sysjobschedules

sysjobhistory

An example of checking the existence, then scheduling a job is shown below. Remember that the SQL*Agent service is responsible for the running on the job.
DECLARE @JobID BINARY(16) DECLARE @ReturnCode INTEGER -- get the ID of our custom job SELECT @JobID = (select job_id FROM msdb.dbo.sysjobs WHERE (name = N'my custom job')) IF @JobID IS NOT NULL BEGIN EXECUTE msdb.dbo.sp_delete_job @job_name = N'my custom job' set @JobID = null END BEGIN -- add the job EXECUTE @ReturnCode = msdb.dbo.sp_add_job @job_id = @JobID OUTPUT , @job_name = N'my custom job', @owner_login_name = N'sa', @description = N'our custom job example', @enabled = 1, @notify_level_eventlog = 2, -- log NT event on failure @delete_level= 0 -- never delete, 2 means delete on failure, 1 is delete on success IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO JobError -- job step #1 EXECUTE @ReturnCode = msdb.dbo.sp_add_jobstep @job_id = @JobID, @step_id = 1, -- first step, dont use zero @step_name = N'step 1', @command = N'exec OurStoredProc_sp', @database_name = N'pubs', @database_user_name = N'', -- db user account to use when running step @subsystem = N'TSQL', -- default @cmdexec_success_code = 0, -- if OurStoredProc_sp returns 0 then OK @retry_attempts = 1, -- 1 retry @retry_interval = 1, -- wait 1 minute

285

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

@on_success_step_id = 0, @on_success_action = 1, @on_fail_step_id = 0, @on_fail_action = 2

-----

default is zero quit with success, 3 is goto next step default is zero quit with failure

IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO JobError EXECUTE @ReturnCode = msdb.dbo.sp_update_job @job_id = @JobID, @start_step_id = 1 -- ensure step 1 is what we start off with IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO JobError -- schedule the job EXECUTE @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id = @JobID, @name = N'our example job', @enabled = 1, -- enable it @freq_type = 4, -- daily @active_start_date = 20020713, -- 13 July 2002 start date @active_start_time = 210000, -- 9pm @freq_interval = 4, -- daily @freq_subday_type = 1 -- run at set time IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO JobError -- Add the server to run job against EXECUTE @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @JobID, @server_name = N'(local)' IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO JobError

END

RETURN JobError: PRINT Error with our job example

286

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Here is a screen shot from Enterprise Manager regarding the scheduled job.

DBMS_LOB
There are no built-in routines similar to DBMS_LOB. The SQL Server binary objects have a variety of restrictions in terms of SQL (cannot check equality in a SQL statement without using LIKE, T-SQL does not support TEXT datatypes etc). See the help of your associated language about reading/writing to SQL Server for more information about dealing with LOBs.

BFILE

There is no equivalent option to the BFILE type in SQL Server.

Tables
The syntax of Oracle and SQL Server for table creation and alteration is significantly different, primarily due to the large number of additional options included for space and concurrently management in Oracle. The syntax itself for SQL Server is simplistic and requires no explanation that cannot be covered in the books online. The only syntax option worth mentioning is the FILEGROUP clause:
CREATE TABLE mytable (col1 integer) ON MYDB_DATA

-- where MYDB_DATA is the filegroup name for the database

287

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The only issue here is when the table has a clustered index, where will the table reside? your index or data filegroup? When creating constraints (primary keys, foreign keys, defaults, unique, check), the DBA should explicitly name them. The system generated name can create issues with scripting and make meta data close to meaningless. Finally, when querying sysobjects for user tables, consider the SQL statement:
select o.name as "Table",i.rows as "#rows" from sysobjects o, sysindexes i where i.id = o.id and indid in(0,1) and o.type = 'U' and o.name <> 'dtproperties' order by o.name

User Defined (Inline) Functions

The user defined function is similar to that of Oracle in the respect that they are what they say they are. Even so, in SQL Server we support: a) Scalar functions return a scalar data type (singular), such as INT, FLOAT etc. b) in-line table functions returns type TABLE, used when the function contains a single SELECT statement for example, eg. RETURN(<select-statement>) c) multi-statement table functions as above but can contain a variety of other statements are we explicitly define the structure of the returning table. The function returning a table can be utilised in standard SQL statements such as:
select A.* from dbo.myfunction() as A -- can also be used in WHERE, HAVING, GROUP BY clauses

The DBA should be aware of the following UDF restrictions: restricted use of many non-deterministic functions (i.e. cannot use - getdate(), RAND etc). cannot use temporary tables, can declare table datatype though (cant index table types although you can declare a primary key that will create an index anyhow, table types are also not considered part of a transaction). cannot be used in a FOR XML clause does not support OUTPUT parameters if calling an extended stored procedure, it must not return a result set to the client that cannot be captured by the trigger code.

288

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The UDF will support recursion.

Indexes
The performance and tuning chapter covers indexes. In broad terms, the indexing schemes in SQL Server are very simple with few equivalents to Oracle:
Oracle Function Based Index Bitmap Index Reverse Key Index Partition Indexing Clustered Index Non-clustered index R-Tree indexing Compress option for indexes ONLINE and OFFLINE indexes VALIDATE STRUCTURE option Index organised table SQL Server Equivalent Indexed Views None None No concept of partitions None Heap (non-clustered index) None (no equivalent or Oracle Spatial in SQL Server) None None DBCC commands Clustered Index

Sequences
In SQL Server, the sequence is called an identity. There are no similarities what so ever between sequences in Oracle and identities as we will explore throughout this section. In Oracle, the sequence is a separate data structure in which the developer or DBA can select unique values from within PL/SQL code or DML statement. When given the privilege, many database users can concurrently select unique values from the sequence and as such, can service more than one database object. The Oracle sequence is much more feature rich as shown below:
Oracle parameter Increment by Start with Maxvalue Nomaxvalue Minvalue Nominvalue Cycle Nocycle Cache Nocache Order Noorder SQL Server equivalent Increment Seed <based on datatype maximum> N/A <based on datatype maximum> N/A N/A N/A N/A N/A N/A N/A

The first key difference is that a sequence in SQL Server is not its own database object as in Oracle. The identity is an extended property of a table column, and only one column can enable it. The valid data types for the column to include the identity property include tinyint, smallint, int,and bigint, therefore defining the maximum and minimum values for the column. Here is an example:

289

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

CREATE TABLE dbo.MyTable ( col1 int NOT NULL IDENTITY (1, 1) PRIMARY KEY, col2 char(10) NULL ) ON [PRIMARY] GO

NOTE The column using the identity property does not need to be a primary key but must be non-nullable. The DBMS will automatically populate the columns value, the programmer does not code a trigger or use the .NEXTVAL or .CURRVAL clauses as in Oracle to populate the tables column. When inserting data into the table, we simply do not specify the column in which the identity property has been applied and the DBMS will do the rest, ensuring a unique value is inserted. We can retrieve the identity value inserted using a couple of methods: SELECT @@IDENTITY SELECT SCOPE_IDENTITY() The difference is very important as shown in the example below:
CREATE TABLE [mytab] ( [col1] [int] IDENTITY (1, 1) NOT NULL primary key, [col2] [char] (10) NULL ) ON [PRIMARY] GO CREATE TABLE [mytabaudit] ( [auditid] [int] IDENTITY (100, 1) NOT NULL primary key, [col1] [int] NOT NULL , [col2] [char] (10) NULL ) ON [PRIMARY] GO CREATE TRIGGER [auditinsert] ON [dbo].[mytab] FOR INSERT AS insert into dbo.mytabaudit (col1, col2) select col1, col2 from inserted GO -- at this point, the trigger will audit insertions into mytab into the mytabaudit table insert into mytab (col2) values ('A') select @@IDENTITY select SCOPE_IDENTITY() value is 100 value is 1

As you can see, scope is vitally important to ensure consistency and data accuracy.

290

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The DBA can reseed, check the current value or reset the identity column value as need be via the command: dbcc checkident (<table-name> [, NORESEED | RESEED [, <newvalue>]]) eg: DBCC CHECKIDENT ('mytable, RESEED, 1) NOTE DO NOT assume that re-seeding will force SQL Server to magically fill in skipped sequences. If for example you had a table whose primary key is an identity, and you removed record 4 and then decided to re-seed back to 3, the new insert will do as you wanted to insert the new record with identity value 4. Unfortunately for you the next insert will fail with a primary key error as 5 is already taken. This command requires sysadmin or db_owner or db_ddladmin privileges. To view the current seed value for a table, use: SELECT IDENT_SEED(mytab) or use EM. To override SQL Server and force your own value into the column with the identity property set, you need to use the SET IDENTITY command before and after the insertion. For example:
SET IDENTITY_INSERT mytab ON insert into mytab (col1, col2) values (122, 'Hello') SET IDENTITY_INSERT mytab OFF

You MUST specify the column list for the insertion to work. SQL Server will continue on from the value you inserted, in this case 123. This option can only be set for one table at a time for the current session at any one time. Finally, there is a function called IDENTITY that can be utilised in SQL statements to generate 1 or more seeded sequence values from an initial seed and increment. Its very restrictive though as it can only be used when creating new tables via a SQL statement (create table as in Oracle). Here is an example: select col2, identity(smallint, 1, 2) as newcol into newtab from mytab If you try and use the SQL statement only without INTO, you will get:
Server: Msg 177, Level 15, State 1, Line 2 The IDENTITY function can only be used when the SELECT statement has an INTO clause.

To find all tables utilising the identity property, try this SQL statement:
SELECT u.[name] AS Owner,

291

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

FROM

o.[name] AS [Table], c.[name] AS [Column]

syscolumns c INNER JOIN sysobjects o ON o.Id = c.Id INNER JOIN sysusers u ON o.uid = u.uid WHERE c.status & 128 = 128 -- apply bitmap mask for identity property

What about UNIQUE IDENTIFIER?

Another option over IDENTITY is to use the unique identifier data type, which is actually stored as a 16-byte binary value. The mechanics of the data type and its associated functions are very different from identities and for 95% of cases, you will have no need for them. Another name for unique identifiers is GUID (global unique identifier) that is guaranteed to be unique no matter the server and/or location/time. The value is a large HEX string represented by the format xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, in which only no arithmetic functions are allowed against it. To allocate a value to a GUID variable, you need to use the NEWID() function as shown in the examples below:
DECLARE @myvar UNIQUEIDENTIFIER SET @myvar = NEWID() SELECT @myvar -- of course, different every time we run it 304DFB66-55D9-4BF3-A0A7-F2EDC7A04550 CREATE TABLE mytable ( mypkeycol UNIQUEIDENTIFIER DEFAULT NEWID() PRIMARY KEY mycol2 VARCHAR(20) )

Note one of the key differences is that we have no magic function to select the allocated NEWID value as we can with identities using @@IDENTITY.

NOTE - Classic uses of GUID are with replication, uniquely identifying rows and to assist with conflict resolution.

292

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Built in Functions
Numeric Functions
Some of these are not strictly numerical functions but have been listed due to their context of usage.
SQL Server Function % (modules operator) @@ERROR @@SPID ABS ACOS ASIN ATAN ATN2 CEILING COL_LENGTH LEN DATALENGTH COS COT COUNT COUNT_BIG DEGREES EXP FLOOR ISNULL ISNUMERIC LOG LOG10 N/A N/A N/A N/A N/A N/A N/A N/A N/A PI POWER RADIANS RAND* ROUND SIGN SIN SQRT Oracle Equivalent MOD SQLCODE UID ABS ACOS ASIN ATAN ATAN2 CEIL VSIZE COS COSH COUNT N/A N/A EXP FLOOR NVL N/A LN LOG(10) DUMP GREATEST LEAST COSH TANH TRUNC SINH TANH BITAND N/A POWER N/A ROUND SIGN SIN SQRT

293

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQUARE sysmessages TAN TOP USER_NAME

N/A SQLERRM TAN N/A USER

* random numbers are discussed later in this chapter.

In Oracle we can use TO_CHAR (numeric to a varchar2) and TO_NUMBER (varchar2 to numeric) functions for translation and formatting of numerical data. The TO_CHAR function offers a wide variety of number format masks for the resultant string value, this includes such things as currency, suffix and prefix minis signs, commas, scientific notation and much more. We have no equivalent functions in T-SQL for SQL Server, and are typically resolved via the application utilising VB or C++. The DBA should consider CAST and CONVERT functions where applicable.

String Functions
Some of the core string functions:
SQL Server Function REPLACE LOWER, UPPER SUBSTRING STR CHARINDEX PATINDEX REVERSE REPLICATE LEN LEFT, RIGHT ASCII STUFF LTRIM, RTRIM SPACE (no direct equivalent) SOUNDEX QUOTENAME N/A DIFFERENCE Oracle Equivalent REPLACE LOWER,UPPER SUBSTR N/A INSTR N/A N/A N/A LENGTH N/A ASCII TRANSLATE LTRIM, RTRIM LPAD, RPAD SOUNDEX N/A INITCAP N/A

294

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Date Functions
The SQL Server DBMS includes two datatypes for date/time data storage: a) Datetime accuracy of one three hundreds of a second Jan 1 1753 to b) Smalldatetime accuracy to nearest minute Jan 1990 to June 6 2079 IMPORTANT SQL Server has no equivalent to the TIMESTAMP or TIMESTAMP WITH TIME ZONE datatypes, do not confuse the SQL Server timestamp datatype with these. The closest option you have is the datetime datatype. As with oracle, the time component is embedded with the datatype and there is no specific time datatype.
SQL Server Function @@datefirst convert cast Oracle Equivalent(s) N/A TO_CHAR ROUND TO_DATE Summary See set datefirst above, retrieves current set value. This is not strictly for dates only, but is one of the core functions for formatting and converting datetime values. Note the covert function for dates returns the formatted string value, style 103 is DD/MM/YYYY. select CONVERT(varchar, getdate(), 103) set dateformat dmy print cast(07/01/2001 as datetime) Add some specified internal to an existing date. Returns that date difference between two dates, we can choose wether the date part is year, month etc. select datediff(mm, getdate(), getdate() + 100) Returns the string representation of a date part. some cases there is no difference to datepart. select datename(month, getdate()) Returns a specified portion of a datetime value. print datepart(yyyy, getdate()) print datepart(ms, getdate()) DAY returns the integer value of the day for a specified date. Also see DATEPART. Returns current local datetime. Formats are: 2002-07-22 09:39:34.057 or Jul 22 2002 9:42AM Not affected by logins language setting. Returns current local datetime in terms of GMT timezone. Returns zero or one if value is a valid date. MONTH simply returns the integer value of the month for the specified date. Also see DATEPART. In Oracle this returns the date of the first weekday specified that is later than the date. In Oracle this returns the last day of the month. In

dateadd datediff

ADD_MONTHS MONTHS_BETWEEN

datename

TO_CHAR

datepart

TO_CHAR EXTRACT EXTRACT Sysdate

day getdate

getutcdate isdate month N/A N/A

NEW_TIME N/A EXTRACT NEXT_DAY LAST_DAY

295

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

N/A

NEW_TIME

set datefirst set dateformat

N/A NLS_DATE_FORMAT

See Reference (41) for an example user defined function. In Oracle, this returns the clients datetime value, not the value of the server (which sysdate does). There is not equivalent in SQL Server. This function can also convert from one timezone to another in Oracle. Sets the fist day of the week. Default is 1 (Monday). Alters the interpretation of a string literal date format. Is affected by the language setting of the login. -- lang is ENGLISH, which is mdy, need to set it -- to prevent out-of-range errors declare @aa datetime set dateformat dmy set @aa = '31/07/2002' Be careful with setting these constants, they will invalidate a cursor declaration if its run after the cursor declaration and are set globally for the session (connection). YEAR simply returns the integer value of the month for the specified date. Also see DATEPART.

Year

EXTRACT

Date formats used by most of the functions above include:

Date Part Year Month Day Hour Minute Second Millisecond Week Quarter Century Standard ISO Year First day of ISO Year Format String yyyy, yy mm, m dd, d Hh Mi ss, s ms wk, ww qq, q N/A N/A N/A

There is no TRUNC() function is SQL Server, therefore, date comparisons over equality may require the use of datepart or cast functions to trim the time component. NOTE SQL Server has no concept of JULIAN dates, i.e. the number of days since Jan 1, 4712 BC, for continuous dating from a common reference point. There is no format string (J) for julian date conversion. In Oracle we can utilise, amongst other things, the NEW_TIME function to assist it converting datetime values between timezones. The closest we have in SQL Server is the getutcdate function, that returns the current datetime in terms of GMT (greenwich mean time). The servers local-time and the zone setting of the operating system can affect the value returned. Therefore, the timezone of the server will influence the result set. Apart from this, there is not magical NEW_TIME function available for time-zone conversions.

296

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL Examples with Oracle Comparisons

It is important to note that Oracle supports the ANSI and SQL-92 for its join syntax, with a move to ANSI where possible. The following table provides some examples in SQL Server with notes on Oracle as need be. Syntax Type
CROSS JOIN (cartisan product) INNER JOIN (equi-join)

SQL Server Example

SELECT FROM dept, emp SELECT dept.* FROM dept INNER JOIN emp ON dept.dept_id = emp.dept_id same as SELECT dept.* FROM dept, emp WHERE dept.dept_id = emp.dept_id SELECT dept.* FROM dept LEFT OUTER JOIN emp ON dept.dept_id = emp.dept_id

Notes
Combines ALL rows from the left the right tables. The inner join is identical to the equi-join with standard where clause predicates.

LEFT OUTER

RIGHT OUTER

SELECT dept.* FROM dept RIGHT OUTER JOIN emp ON dept.dept_id = emp.dept_id

FULL START WITH CONNECT BY Collelated SubQueries

SELECT dept.* FROM dept FULL OUTER JOIN emp ON dept.dept_id = emp.dept_id The DBA will need to consider additional columns to support the hierarchy, such as the parent record, optional depth of the record in the hierarchy etc. SELECT dept.* FROM dept WHERE EXISTS ( SELECT X FROM emp WHERE dept.dept_id = emp.dept_id) As per Oracle, no restrictions or differences in use. -- Row locks hint -- For update in cursor DECLARE mycursor CURSOR FOR

All rows in left table (dept) even when keys dont match. In SQL-92, this would be represented via: dept.dept_id(+) = emp.dept_id in the where clause. All rows in right table (emp) even when keys dont match. In SQL-92, this would be represented via: dept.dept_id = emp.dept_id(+) in the where clause. This is a RIGHT AND LEFT OUTER join merged into one. SQL Server has no equivalent syntax to Oracles connect-by-prior for hierarchies. Have fun coding your own. As per Oracle, all standard correlations are support in the WHERE clause, the user needs to alias tables as need be to remove any ambiguity. The sub-query can be used in the SELECT, FROM and WHERE clauses and most other DML. These are standard operations and are identical in SQL Server. In Oracle, FOR UPDATE locks rows based on the join clauses used. In SQL Server we have little control over such locking due to SQL

EXISTS, NOT EXISTS, IN, NOT IN FOR UPDATE

297

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

select col1, col2 from mytable FOR UPDATE OF col1, col2 TABLE collections -- table data type DECLARE @myvar TABLE (col1 int, col2 int) Insert into @myvar (col1, col2) values (1,2) -- local temporary table create table #mylocaltab (col1 int, col2 int) -- global temporary table create table ##mylocaltab (col1 int, col2 int)

Servers lock escalation, even so, we can get those via SQL Hints. In SQL Servers cursors, In SQL Server we can create 3 different types of table. The first being the TABLE data type (in memory complex memory structure) , the second type is local temporary tables (and finally global temporary tables (session wide). Such table structure support constraints like physical tables but not foreign key constraints. The table data type is preferable as local and global tables use the tempdb database, therefore result in physical IO. The virtual table (also known as derived tables) must be aliased as shown in the example. Any valid SQL statement is allowed.

Virtual/Derived Tables

No FROM clause

SELECT deptname, totalemps FROM dept as DD, (SELECT deptid, count(empid) totalemps FROM deptemps GROUP BY deptid) as AA WHERE AA.deptid = DD.deptid --get my SPID number SELECT @@SPID -- get SS2k version SELECT @@VERSION No example required. No example required. As per Oracle in virtually all cases, it also supports dynamic ordering, ie: SET @myvar = 2 SELECT COL1, COL2, COL3 FROM MYTABLE ORDER BY CASE WHEN @myvar = 1 THEN MYTABLE.COL1 CASE WHEN @myvar = 2 THEN MYTABLE.COL2 ELSE MYTABLE.COL3 END To randomly order records, consider the following code: ORDER BY NEWID() No example required. select item, salesman, sum(total) from sales

DISTINCT GROUP BY ORDER BY

A valid SQL statement in Oracle requires, at a minimum, the SELECT .. FROM clauses. In SQL Server you can skip the FROM clause where you are selecting data not from a table or view. As per Oracle, no example required. As per Oracle, no example required. As per Oracle, no example required.

HAVING CUBE

As per Oracle, no example required. In SQL Server, the CUBE operator create a data set that is a result of

298

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

group by item, salesman with cube Item Sales Person Total ----------------- ----------------- ------15" Monitor Veronica 350.0 15" Monitor NULL 350.0 Arm Chair Carl 120.0 Arm Chair Chris 120.0 Arm Chair NULL 240.0 Desk Carl 450.0 Desk Veronica 450.0 Desk NULL 900.0 Foot Stool Carl 25.0 Foot Stool Chris 25.0 Foot Stool NULL 50.0 NULL NULL 1540.0 NULL Carl 595.0 NULL Chris 145.0 NULL Veronica 800.0 select item, salesman, sum(total) from sales group by item, salesman with rollup Item Sales Person Total ----------------- ----------------- ------15" Monitor Veronica 350.0 15" Monitor NULL 350.0 Arm Chair Carl 120.0 Arm Chair Chris 120.0 Arm Chair NULL 240.0 Desk Carl 450.0 Desk Veronica 450.0 Desk NULL 900.0 Foot Stool Carl 25.0 Foot Stool Chris 25.0 Foot Stool NULL 50.0 NULL NULL 1540.0 No example required. No example required. N/A N/A No example required. N/A

all possible values based on the WITH CUBE columns specified (call dimensions). In the example, we can see that the CUBE expression has pivoted over the sum(total) column and produced a result set that covers all valid combinations of item and salesmen where data exists. The GROUPING clause can use used to determine if a NULL will be returned so the user can alter this value and return something easier to understand.

ROLLUP

When compared to the query above and the with c8be option, the rollup clauses uses the same paradigm but rolls up the summarised totals.

UNION UNION ALL INTERSECT MINUS ASC, DESC AS OF

SAMPLE

N/A

As per Oracle. As per Oracle. There is no equivalent in SQL Server. There is no equivalent in SQL Server. Sorting conditions for order by clause are as per Oracle, no example required. Oracle The FROM <list> AS OF [SCN, TIMESTAMP] clause is used to query data as it existed back in time, this is done via Oracles automatic undo tablespace and its settings to control Oracle flash-back. There is no equivalent in SQL Server. No SQL Server equivalent. Oracle can return a sampled

299

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

INSERT DELETE UPDATE

No example required. No example required. update emp set salary = 200000 from dept where dept..empid = emp..empid TRUNCATE TABLE mytable

TRUNCATE

subset of data from a table. Does not support joins. No SQL Server equivalent. As per Oracle, no example required. As per Oracle, no example required. SQL Server adds a fantastic little feature that Oracle does not. This is shown in the example. As per Oracle though, the table being updated can not be aliased. In Oracle, the truncate command covers the specification of table or cluster (clustered table and its indexes), wether to preserve or purge materialised view data and wether the already allocated storage will also be dropped or reused. The table can be truncated if relationships exist, but you still need to truncate in order of the key values. In SQL Server the command set is simple truncate <table>, as the DBA has no control over storage, and the DBA cant truncate whilst foreign keys exists. SQL server is much more restrictive on its use, requiring table owner or db_owner or sysadmin or db_ddladmin privs. Will return the top n records or top n percent of records of 100. Use the TOP clause in views if you want to also use an ORDER BY. If Order by is used the entire result set is sorted and the top n records are then selected. The set ROWCOUNT N constant can also be used to simulate a TOP n clause but will remain in effect until set to zero or the connection is lost.

TOP and TOP PERCENT [with ties]

SELECT TOP 10 * FROM MYTABLE

COALESCE

SELECT COALESCE (col1, col2, col3) FROM MYTABLE If col1 was null, but col2 was not, then the value of col2 would be returned. This continues for all records in MYTABLE. All values must be of the same data type.

Returns the first non-null value from a list of values.

300

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Partitioned Tables
There is no concept in SQL Server of partitioning; there seems to be no plans from Microsoft to support this great Oracle option in the next release of SQL Server

Parallel Query
Unlike Oracle, the parallel options in SQL Server seems to be a half hearted effort for multiprocessor machines. There is only one hint available (DOP), and from a lot of work with parallel queries, I have been a little disappointed with performance overall. But dont let me discourage you, I am not backing up my statement so you will need to try it yourself. At an instance level, the DBA can turn on and off the CPUs utilised by SQL Server. This is easily managed via Enterprise Manager by right clicking properties at the instance level:

From here, the DBA can control basic instance parallelism options:

Degree of parallelism (DOP, plan generation) EXEC sp_configure max degree of parallelism', 1 RECONFIGURE WITH OVERRIDE Contol what CPUs can be used by SS2k threads EXEC sp_configure affinity mask', 0x00000003 RECONFIGURE WITH OVERRIDE

Threshold for Parallelism (seconds) EXEC sp_configure 'cost threshold for parallelism', 5

The cost estimate or threshold parameter is somewhat vague, its like the recovery interval, you dont really know what its doing under the covers. This value is in seconds and is used by the optimiser to predicate a plans execution time; if it is greater than 5 seconds for example, it will consider utilising intra-query parallelism.

301

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To control parallelism at a SQL level, use the MAXDOP (DOP = degree of parallelism) hint, for example:
select * from products option (maxdop 3)

Take care to monitor CPU utilisation along with logical and physical reads. You may find a significant increase in CPU utilisation and with large, frequent queries you may see a reduction in buffer cache hit ratios if the queries are large and adhoc. Utilise the trace flag 8687 to disable parallelism globally for the DBMS on instance startup. Also note that many internal DBCC commands will utilise parallel operations where possible; consider the trace flag 2508 and 2528 to turning it off selected DBCC calls.

Parallel IO
With the installation of SQL Server 2k SP1 and above, the parallel affinity option has been enhanced to include to specify what CPUs are dedicated to run SQL Server IO operations. Without the option, SQL Server disk IO will be scheduled to any of the eligible CPUs as specified by the affinity mask option. The parameter is specified on instance startup: sqlservr.exe I0x00FF0000 Remember that this option goes hand in hand with the affinity mask option to split away CPUs dedicated to disk IO verses other thread tasks for the rest of the instance.

SQL Hints
As a general rule, do not use HINTS unless there is a real need too. They are hard to administer and at times understand, and if not used correctly, can create adverse performance problems due to changes in the underlying schema or row counts. This rules tends to flow between both RDBMSs. In Oracle, the hint applies at the start of the select, insert, delete or update statement. Select /*+ hint [text] */ Select --+ hint [text] where [text] are one or more additional options for the actual hint. There can be one or more hints specified that can work over any views/tables used in the from clause of the statement. In all cases, the cost based optimiser will be used (except of course when the RULE hint is used).

302

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

In SQL Server the optimiser is COST BASED (there is no other option) and as one would also expect, the range and type of hints vary significantly between two DBMSs. The SQL Server hint categories are: a) index hints a. INDEX b) join hints a. LOOP b. HASH c. MERGE d. REMOTE c) locking & isolation level hints a. granularity (one per table) i. PAGLOCK ii. NOLOCK iii. ROWLOCK iv. TABLOCK v. TABLOCKX vi. LOCKX vii. UPDLOCKX viii. READPAST b. Isolation (can be mixed with selected granularity hints) i. HOLDLOCK ii. NOLOCK iii. READCOMMITTED iv. REPEATABLEREAD v. SERIALIZABLE d) query hints (affects all operators in the statement) a. HASH GROUP b. HASH ORDER c. ROBUST PLAN d. FORCE ORDER e. MERGE UNION f. HASH UNION g. CONCAT UNION e) table hints (one per table) a. FASTFIRSTROW b. + index hints c. + granularity hints f) view hints a. EXPAND VIEWS b. NOEXPAND c. Selected index hints also apply g) other hints a. FAST n-rows b. MAXDOP degree-of-parallelism c. KEEP PLAN d. KEEPFIXED PLAN The syntax for specifying hints:

303

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

-- TABLE HINT SELECT <columns> FROM mytable WITH (<hint>) [WHERE <predicate>] -- QUERY HINT SELECT <columns> FROM mytable [WHERE <predicate>] OPTION (<option hint>) -- JOIN HINT SELECT * FROM orders LEFT OUTER <hint> JOIN customers ON orders.customerid = customers.customerid From the syntax above, we see that the hints themselves are broken down into two styles: 1) WITH i. Can be specified for each table referred to in the DML, within the brackets of the WITH statement we can specify multiple hints. SQL Server will report syntax errors but will not report on un-used hints unless you carefully check the execution plan. 2) OPTION i. Can only be specified once for each whole SQL statement, meaning that the option applied to the entire statement and any sub-queries specified. If a UNION is used, then the last statement can only include the option. 3) Join Hint i. Will enforce the join strategy between two tables. The CROSS JOIN will require the hint to be wrappered in brackets. NOTE - Using hints for remote queries (queries over linked servers, otherwise known as database links) are not supported in SQL Server 2k, you will get the error: Server: Msg 7377, Level 16, State 1, Line 1 Cannot specify an index or locking hint for a remote data source. Comparing HINTS is very difficult; the list below is by far complete and will require a significant amount of testing when porting SQL statements from one DBMS to another. It is recommended that hints are stripped when porting SQL and evaluate the need for a SQL Server equivalent as performance or locking issues dictate. We will only list Oracle hints where I believe there is a vague resemblance to an equivalent SQL Server one.
Oracle FIRST_ROWS(n) CLUSTER HASH INDEX_JOIN HASH FULL SQL Server FASTFIRSTROW OPTION(FAST n) INDEX(0) if the cluster index exists for the table INDEX(0) if the cluster index exists for the table INDEX(n) or consider the join hints to provide some control over index usage Consider the HASH join hint None, you cannot force a full table scan unless of

304

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

QUERY TRANSFORMATION hints

ORDERED USE_NL USE_MERGE USE_HASH PARALLELISM hints CACHE NOCACHE Locking Hints

course you have no indexes or statistics are out of date and the optimiser chooses not to use them. No equivalents in SQL Server, perhaps one could argue that the query hints such as FORCE ORDER, ROBUST PLAN etc are similar, but there are vague resemblances at best. FORCE ORDER LOOP join hint MERGE join hint HASH join hint OPTION(MAXDOP n) at query level And instance level parameters No hint, the lazywriter will manage the procedure cache and age plans as required. See books online for aging execution plans. Consider the recompile option, apart from that there are no specific hints. There are no equivalent options in Oracle for the SQL Server locking hints listed previously.

NOTE - The NOLOCK, READUNCOMMITTED, and READPAST table hints are not allowed for tables that are targets of delete, insert, or update operations. Microsoft Support Doc Q235880 describes issues with NOLOCK and READ UNCOMMITTED hints and the generation of transient error messages to do with the processes reading data that is being moved or altered by another user (dirty read).
Example Hint
BEGIN TRANSACTION SELECT * FROM PRODUCTS WITH (HOLDLOCK, ROWLOCK) WHERE PRODUCTID = 1

COMMIT TRANSACTION

305

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Data Types Compared

See table below, to review data types in SQL Server also consider this query: select * from master..systypes
SQL Server Character types Char Varchar Text Ntext Nchar Oracle Equivalent Char Varchar2 CLOB or LONG Nchar Oracle Notes Max 2000 bytes/row Max 4000 bytes/row Up to 232 - 1 bytes, or 4 gigabytes. Up to 2Gb data for LONG. Fixed for every row in the table (with trailing blanks). Column size is the number of characters. (The number of bytes is 2 times this number for the AL16UTF16 encoding and 3 times this number for the UTF8 encoding.) The upper limit is 2000 bytes per row. Default is 1 character. Variable for each row. Column size is the number of characters. (The number of bytes may be up to 2 times this number for a the AL16UTF16 encoding and 3 times this number for the UTF8 encoding.) The upper limit is 4000 bytes per row. Default is 1 character. Oracle Notes Fixed-length date and time data, ranging from Jan. 1, 4712 B.C.E. to Dec. 31, 4712 C.E. SQL Server Notes Max 8000

Nvarchar

Nvarchar2

SQL Server Datetime types Datetime

Oracle Equivalent Date

SQL Server Notes Date and time data from January 1, 1753, through December 31, 9999, with an accuracy of three-hundredths of a second, or 3.33 milliseconds Date and time data from January 1, 1900, through June 6, 2079, with an accuracy of one minute.

Smalldatetime

Date

N/A on conversion the DBA will need to consider the loss of accuracy

306

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQL Server Numeric types Bit Tinyint Smallint Int Bigint

Oracle Equivalent NUMBER(1,0) NUMBER(p,s) NUMBER(p,s) NUMBER(p,s) NUMBER(p,s)

Oracle Notes

SQL Server Notes

Big Int - 8 bytes in size versus its 4 byte int. whole numbers between -9,223,372,036,854,775,807 and 9,223,372,036,854,775,807

Decimal Numeric Smallmoney Money Real Float SQL Server Binary Types Image Varbinary Binary

NUMBER(p,s) NUMBER(p,s) NUMBER(p,s) NUMBER(p,s) Float Float Oracle Equivalent BLOB Or LONG RAW? BLOB BLOB Or LONG RAW? Oracle Equivalent Timestamp or RAW Oracle Notes Up to 232 1 bytes, or 4 gigabytes. Up to 232 1 bytes, or 4 gigabytes. Up to 232 1 bytes, or 4 gigabytes. Variable-length binary data with a maximum length of 8,000 bytes. Variable-length binary data with a maximum length of 2up31 1 (2,147,483,647) bytes. SQL Server Notes SQL Server Notes

SQL Server Other Types Timestamp N/A

Oracle Notes

%TYPE

Not really a data type as such but allows the pl/sql programmer to declare variables that have the same type of a tables column.

No equivalent in T-SQL.

SQL_VARIANT Sysname TABLE N/A N/A

N/A TYPE is of TABLE %ROWTYPE TYPE <name> IS RECORD

No equivalent in T-SQL. No equivalent in T-SQL.

307

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

The text data type can be very restrictive in terms of SQL and T-SQL. You can not declare data types of type text (or other binary types), LEN and other functions will not work:
Server: Msg 8116, Level 16, State 2, Line 1 Argument data type ntext is invalid for argument 1 of len function.

To get around this error, use the command:

SELECT MAX (DATALENGTH (MyTextColumn)) FROM mytable

Moving data from a ntext to a varchar will give an error:

Server: Msg 260, Level 16, State 1, Line 2 Disallowed implicit conversion from data type ntext to data type varchar, table 'mydb.dbo.mytablet', column 'mycolumn'. Use the CONVERT function to run this query.

When used in predicates, you may also face this error:

Server: Msg 306, Level 16, State 1, Line 1 The text, ntext, and image data types cannot be compared or sorted, except when using IS NULL or LIKE operator.

All fun for the developer and DBA alike.

308

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Dynamic SQL
It is very rare that the modern DBMS does not support dynamic SQL statements in some form, within Oracle we have a choice of two methods:
DBMS_SQL package

mysqlstr varchar2(100); numrows NUMBER; cursorhandle NUMBER; mysqlstr := 'INSERT INTO mytable VALUES(:col1,:col2)'; cursorhandle := DBMS_SQL.OPEN_CURSOR; DBMS_SQL.PARSE(cursorhandle, mysqlstr, DBMS_SQL.NATIVE); DBMS_SQL.BIND_VARIABLE(cursorhandle,':col1',10); DBMS_SQL.BIND_VARIABLE(cursorhandle,':col2', 'This is an example'); numrows := dbms_sql.execute(cursorhandle); DBMS_SQL.CLOSE_CURSOR(cursorhandle);
Native Dynamic SQL (requires compatible parameter set to 8.1.0 or higher)

mysqlstr varchar2(100); mysqlstr := 'INSERT INTO mytable VALUES(:col1, :col2)'; EXECUTE IMMEDIATE mysqlstr USING 10, 'This is an example';

These statements support all forms of DDL, DML and anonymous PL/SQL blocks. In SQL Server we also have a choice of two methods:
EXECUTE

declare @mysqlstr nvarchar(100); set @mysqlstr = N'INSERT INTO mytable VALUES(10,' + '''This is an example''' + ')' exec(@mysqlstr)
SP_EXECUTESQL (recommended over EXECUTE)

declare @mysqlstr nvarchar(100); execute sp_executesql N'INSERT INTO mytable VALUES(@col1, @col2)', N'@col1 int', N'@col2 varchar(50)', @col1 = 10, @col2 = 'This is an example'

The SQL in both cases are not parsed until the command is executed. It is also important to remember that if the database context is altered via a dynamic SQL statement, then it will only persist whilst the dynamic routine is run, for example:

309

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

USE master EXEC ("USE nothwind") SELECT * FROM mytable

goto master database for subsequent commands goto northwind database this will run under master, not northwind

The sp_executesql system stored procedure has a number of benefits over EXECUTE: supports parameter substitution generates procedure cache plans that will more likely benefit from reuse due to its parameterisation (when part of a loop condition or the stored procedure is called often with different values, then sp_executesql with parameters will benefit tremendously). In terms of transaction context, the following example shows that both calls will retain their context in terms of the current open transaction. Note that a Unicode string is required for both dynamic procedures.
CREATE PROCEDURE DynamicSQL DECLARE @mysql nvarchar(100) BEGIN TRANSACTION insert into mytable values (1) set @mysql = N'insert into mytable values (2)' exec sp_executesql @mysql exec(@mysql) insert into mytable values (3) select * from mytable ROLLBACK TRANSACTION select * from mytable GO AS

We will try the two dynamic SQL routines. Values 1,2,2,3 inserted as part of the single transaction All values successfully rolled back (table is empty)

310

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

T-SQL (pl/sql) Overview

Throughout this book we have provided numerous examples of SQL Server functionality and discussion of the Oracle equivalents as need be. Comparing T-SQL and PL/SQL is summarised below; be aware that PL/SQL is 100% more comprehensive in terms of a language and a 1;1 comparison is simply not particle. Here are some key points to remember either way:
Oracle <variablename> <type> [DEFAULT <value>]; mystring VARCHAR2(100); Dynamic SQL Stored Procedure DBMS_SQL EXECUTE IMMEDIATE CREATE OR REPLACE PROCEDURE myprod [(<parameters>)] IS [<local variables>] BEGIN <body code>; [EXCEPTION WHEN <code>; THEN <code>; [WHEN OTHERS] END; SQL Server DECLARE @<variable> <type> [DEFAULT <value>] DECLARE @mystring VARCHAR(100) sp_execute_sql EXECUTE CREATE PROCEDURE [owner].myproc[;version] <parameters> [WITH <options>] [FOR REPLICATION] AS [<local variables>] <body code> GO ALTER PROCEDURE myproc[;version] <parameters> [WITH <options>] [FOR REPLICATION] AS [<local variables>] <body code> GO CREATE FUNCTION [owner].myfunc (<parameters>) RETURNS <datatype> AS [<local variables>] <body code> RETURN (<variable or statement>) GO SET <dbname>, eg: SET master N/A IF <expression> [BEGIN] <code> [END] [ELSE] -- No FOR loops in SQL Server, only the -- WHILE loop

Variable Declaration

User Defined Function

Set current database Precompiler Statement IF

N/A PRAGMA <instruction> IF <expression> THEN <code> [ELSIF] END IF; LOOP <code>

LOOP

311

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

END LOOP; WHILE <condition> LOOP <code> END LOOP; FOR <index> IN <statement> LOOP <code> END LOOP; GOTO <label>; <<mylabel>> See Oracle Documentation.

WHILE <condition> BEGIN <code> END

GOTO LABELS PACKAGE

-- get scoping rules for both languages -- before porting code. GOTO <label> mylabel: N/A

312

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

14
Chapter

Networking SQL Server

ne of the big topics discussed a lot within Oracle mailing lists and news groups is SQL*Net and the Listener to manage DBMS connectivity. Throughout this chapter we will cover the SQL Server equivalent and associated connectivity issues the DBA may face from time to time.

Listener and ODS

As in Oracle, SQL Server has a listener which is managed per instance of SQL Server (default or named). The database instance itself is an instantiation of the listener (i.e. starting the instance will start the listener for the instance), which is very different to Oracle. This dependency can mean that incorrect listener settings may prevent the instance from starting. An example of the log entry for a starting instance:
2002-11-29 2002-11-29 2002-11-29 2002-11-29 2002-11-29 16:36:32.27 16:36:32.37 16:36:32.87 16:36:32.91 16:36:32.91 server server server server server Using 'SSNETLIB.DLL' version '8.0.534'. SQL Server listening on TCP, Shared Memory. SQL Server is ready for client connections. SQL server listening on 163.232.13.22: 2433. SQL server listening on 127.0.0.1: 2433.

due to this, any changes in the server network utility require the instance to be restarted (not great for 24x7 environments). The instance listener is managed via the Server Network Utility found under the Microsoft SQL Server group, this is installed once on the server. The utility allows the DBA to specify what protocols are enabled for each instance and provides the ability to alter various protocol specific settings (such as ports, encryption etc):

313

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Manage listener properties at a instance level on the server Enabled protocols the instance will accept connections from List of valid protocols the instance can utilise Get properties for selected enable protocol Global options for all enabled protocols.

NOTE - If a protocol is not available, re-check your installation by re-running the setup. To view the underlying DLL for each protocol click on Network Libraries. The ss prefix on the dlls represents a server based network library; the ssnetlib.dll (super socket) will also manage SSL encryption where appropriate (covered later in the chapter). The configuration and setup is very simple and requires little discussion. See the chapter on security for information about protocol encryption. One point noting though about TCP/IP connections, is the default port number of 1433. On installation this is setup unless you entered a value explicitly, if another instance is using port 1433 it will auto select another port on the first connection to the instance through that protocol. The DBA should not use default port 1433 as it is a classic port for hackers. NOTE Take a careful look at the first few lines of your SQL Server instance log, you will see information regarding the listener and the protocols and ports supported.

314

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Multi-Protocol
The multi-protocol library is worth mentioning on its own. The library utilises the NT remote procedure call (RPC) in addition to its network library. This protocol supports the standard IPC calls supported by NT, namely tcp/ip (nacn_up_tcp), ipx/spx and named pipes (nacn_np):

The enable encryption option is discussed later

On instance startup, we will see something like this in the SQL Server log:

The net library will utilise a random port making hacking a little harder but going through a firewall can be problematic. NOTE Does not support named instances.

Registry Entries
To verify the last instance connection from the client, goto the register entry last connect for either multiprotocol connections (rpcnetlib) or super sockets network library:

To get a good summary of registry entries related to the network library API, check out: http://msdn.microsoft.com/library/default.asp?url=/library/enus/dbnetlib/htm/dbnetlib.asp

315

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Client Network Utility

A client does not require the client network utility to be configured in order to connect to a SQL Server instance. So long as the client has a SQL network library installed utilising a protocol in which the instance is listening on (i.e. TCP/IP); the connectivity can be established via a suitable provider (eg. OLEDB, ODBC driver, db-library clients). All of which is installed with MDAC. So, what is it used for then? The client utility is used when: a) underlying server does not have the appropriate network libraries installed to facilitate communication with the server (installing the client will resolve the issue) b) to run SQL Server client tools (such as Query Analyser, Profiler, EM) c) want finer control over dbnetlib.dll (which talks to the SQL Server equivalent ssnetlib.dll the server will return a packet via UDP port 1434 listing instances and their listener protocols for the clients dbnetlib.dll to decide what to do with the client request for connectivity). A classic example of utilising the client utility is to specify an alias to a SQL Server instance. If the server is hosting multiple instances (all of which will talk on different ports for there defined listener protocols), you may need to create an entry clearly specifying the protocol and its properties in order to successfully connect. If you have issues connecting from your SQL Server client tools, then double check your client utility alias and its protocol and port properties (or create one if you have not). Another example is ODBC and the SQL Server driver. Without the client utility and an alias, I have found that connecting to a named instance on a non-default port will not work.

316

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

ODS
The ODS (open data services) API manages connections and disconnections to SQL Server as defined by the protocols enabled via the server network library utility for the instance. The ODS talks using the TDS (tabular data stream) which is an application level protocol; the TDS is encapsulated in the protocol stack as defined by the net library listener.
Net-Lib Listener Net library Protocol(s) ODS API Client Application
Native client library used to talk with the listener

TDS (tabular data stream) packets encapsulated within the transmission protocol

SQL Server Instance

The ODS API can be used by C++ developers to build extended stored procedures (discussed later) that talk natively to SQL Server and the client via TDS. The books online cover the API reasonably well with code examples and the associated header and object libraries.

Tracing TCP/IP port issues

Use the portqry.exe command line utility to assist in tracing issues with TCP/IP ports. This tends to happen when other services are using default ports of SQL Server (namely 1433). The utility is available in the operating systems resource kit.

List of Running Instances

Use the command osql.exe L or isql.exe L the instances may not necessarily be running. There are some great SQL Server hacking tools to search the network for running instances, such as sqlping, sqlscanner, sqlcracker, sqloverflowdata, sqldosstorm, freely available on the internet.

SSL of SQL Server Connections

In terms of the SQL Server listener, the DBA can select any protocol and force encryption that utilizes the SQL Server 2000 super socket network library extensions (ssnetlib.dll). In order for this to work, a server certificate must be installed for the service account in which SQL Server runs. To manage server certificates:

317

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

a) b) c) d) e) f) g) h) i) j)

run mcc.exe Add/Remove snap-in Add the certificates snapin You will be prompted to manage certificates for My user account Service account Computer account Select the local computer when asked what server to manage If you selected service account, you will be provided with a list of services to choose from OK

If you have issues with installation and error related to opening a connection to the server due to SSL security issues, then try and remove (via export then delete) all computer account certificates and try again. Once installed, re-import the certificates and re-start the instance (see Q309398). The installation and manage of certificates can be subsequently managed through the snap-in screen:

NOTE The certificate must be registered with the full DNS name of your server. Attempting to force encryption without a certificate gives you an error like:
2002-11-29 16:26:23.65 server 2002-11-29 2002-11-29 2002-11-29 2002-11-29 16:26:23.70 16:26:23.70 16:26:23.70 16:26:23.70 server server server server Encryption requested but no valid certificate was found. SQL Server terminating. Error: 17826, Severity: 18, State: 1 Could not set up Net-Library 'SSNETLIB'.. Unable to load any netlibs. SQL Server could not spawn FRunCM thread.

and the instance fails to start. NOTE - Only 40bit and 128bit certificates are supported. This may vary though based on your SQL Server release and other government restrictions.

318

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To enable encryption for the instance, open the server network utility and check the force protocol encryption option. Be aware though, it affects all enabled protocols.

It is important to note that client encryption is enabled via the client network utility, creating an alias and specifying encryption via the checkbox provided. This will encrypt all incoming traffic from the specific client to the instance. When setup at the server though, all inbound connections and data transfers to the server will be encrypted. The only protocol that can be set up individually for encryption is multi-protocol by selecting the properties button for the enabled protocol:

The multi-protocol encryption does not use SSL but the Windows RPC API and will not work for named instances. In virtually all circumstances the DBA should use certificates.

SQLOLEDB
Virtually all connections to SQL Server are via SQLOLEDB, and its rare that other providers are used, especially in the world of large scale VB development. The connection sting is simple enough, but care must be taken based on the libraries being exposed via the listener (Server Network Utility); this is especially relevant when different protocols are being used or you are changing port numbers.

319

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Some of the key elements of the connection string are:

Provider=SQLOLEDB; Data Source=server-name or IP-address or server-name\instance-name; Network Library= see list below (optional) Initial Catalog=database name; User ID=username; Password=password;

This information is typically stored in an INI file or within the registry by the developers. An example reg file to add an entry into the registry is: (save with a .reg extension and simply run it, Windows will prompt you to continue with the addition of the registry item):
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\AppName\DBConnStrings] "MyApp"="Provider=SqlOleDb;Data Source=myserver;Initial Catalog=mydb;User ID=myuser;Password=myuserpassword"

The network library parameter is very important when you need to hook into a specific protocol enabled within the listener. If you do not specify it then you may be using a protocol you did not intend to, or have difficultly connecting (specially when Multiprotocol is the only one enabled on the listener). The position of the network library option is also important within the connection string, I have had a range of issues with this and its a down right pain to debug. Use the parameter immediately after the data course parameter is used. The values for the parameter are:
Win32 Network Library Named Pipes TCP/IP Library Name for Network Lib parameter (dll names) dbnmpntw dbmssocn Summary

Multiprotocol (RPC) IPX/SPX Apple Talk VINES

dbmsrpcn

dbmsspxn dbmsadsn dbmsvinn

TCP 139, UDP 137,138 ports. No encryption. 1433 is the default port. No encryption. (can be forced in SQL Server 2k) NT RPC calls only. Uses random TCP ports (default). Supports encryption. 33854 default port

A simple search on google.com for oledb connection strings or even better check out www.connectionstrings.com (I didnt believe it, but yes, one exists). To save you the trouble of searching as I know you will be asking the question, to specify a specific port use the parameter data source=xxx.xxx.xxx.xxx,2433.

Network Monitoring
Within SQL Server we can utilise the sp_monitor command to track packets received, sent and in-error between now and its last execution. The results also translate to the @@ equivalents as shown below:

320

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

select @@PACK_RECEIVED select @@PACK_SENT select @@PACKET_ERRORS The default TDS packetsize is 4096 bytes and may be limited/altered by the network library connection to the instance. The DBA can control this for the instance as a configurable parameter (sp_configure):

Other command line utilities, such as BCP, include the a switch to control packet size, this can be handy when pumping through large amounts of data to the instance. Take care with the parameter, and very large settings may degrade performance. In terms of performance and monitoring, the DBA needs to have a clear understanding of: a) network card configuration and its throughout b) throughput (bandwidth) of the LAN/WAN c) OS configuration properties for the card d) How a TDS packet is routed e) What other applications are sharing the bandwidth Careful consideration of this typically comes into play with replication and/or distributed sites. The DBA may also need to check on the duplexing of backup files between servers during the day and its associated impact on overall network performance. The key to monitoring is via performance monitor (perfmon.exe). watch for: Some key areas to

a) SQL Server Network reads/sec or writes/sec high values measure how network intensive the instance is. Match up values with user connections and network queue lengths; and check application components being executed at the time to try and drill through to the issues. b) Utilise the Network Interface performance object and its associated counters carefully. There are a variety of counters and all should be used with a thorough understanding on the issues outlined above. Consult with your network administrator on possible network settings (server and domain controller configuration) to assist with further tuning. They usually have a host of utilities to monitor traffic.

321

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

SQLDiag
Utilise SQLDiag.exe to retrieve information about your network libraries. An example dump may include:
Registry Information -------------------SOFTWARE\Microsoft\MSSQLServer\Client\ConnectTo: ----------------------------------------------DSQUERY: DBNETLIB 163.232.xx.xx: DBMSSOCN,163.232.xx.xx myserver: DBMSSOCN,163.232.xx.xx,1433 SOFTWARE\Microsoft\MSSQLServer\Client\DB-Lib: -------------------------------------------AutoAnsiToOem: ON UseIntlSettings: ON Version Information ------------------ntwdblib.dll: 8.00.194 ssmsad60.dll: N/A ssmsde60.dll: N/A ssmsrp60.dll: N/A ssmsso60.dll: N/A ssmssp60.dll: N/A ssmsvi60.dll: N/A ssnmpn60.dll: N/A dbmsadsn.dll: 8.00.194 dbmsdecn.dll: N/A dbmsrpcn.dll: 8.00.380 dbmssocn.dll: 7.00.819 dbmsspxn.dll: 7.00.819 dbmsvinn.dll: 8.00.194 dbnmpntw.dll: 8.00.194 sqlsrv32.dll: 3.80.0528.00 odbc32.dll: 03.52 odbc32 driver: 03.80.0528

MTS / Oracle Shared Server

There is no equivalent to MTS (Oracle Shared Server) in SQL Server.

322

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

15
Chapter

Security

T
a) b) c) d) e) f) g) a. b. c. d.

he topic of security can be as simple or as complex as you like, and with any DBMS, this tends to cover a whole range of issues. Throughout this chapter we will provide a general overview of the issues within the SQL Server environment and couple these with some best practice tips.

Secured Installation
How thorough the DBA is with installation is a tricky question. First of all, if you are new to SQL Server, then install your instances under a user that is part of the Administrator NT role. It is important to keep the installation user separate from other NT users as you can tighten security later down the track. For those still wanting more, then: Create a new NT user to run the service for the instance(s) Grant login as service rights Grant administrator group privileges to this user Create DB data and log file directories for database files to be created in Login as this users and install the SQL Server instance Once installed, shutdown the MSSQLServer service Login as Administrator of the server and alter the new users rights as follows Full control of SQL Server binaries (install directory) Full control of all directories to store database files (mdf,ndf,ldf) For the above directories, remove the "everyone" group privilege Full control of the registry keys:

HKEY_LOCAL_MACHINE\Software\Microsoft\MSSQLServer

323

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

HKEY_LOCAL_MACHINE \System\CurrentCOntrolset\Services\MSSQLServer or MSSQL$<INSTANCE> for named instances HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Perflib And associated service registery keys for SQLAgent$InstanceName, MSSearch, and MSDTC h) i) Revoke administrative rights to the SQL Server user account Login as the SQL Server user and attempt to re-start the database, debug as required via the Windows Event viewer and the SQL Server and SQL Agent logs

Consider using EFS (encrypted file system) for the directories used by the instance whilst logged in as the service user account. This NT feature will be discussed later. The DBA should be using an NTFS file system in all cases to utilise its fine grained file level security features. If the instance is partaking in replication, use a domain user rather than a local user account. Note that EFS will only work for NTFS file systems.

324

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Logins and Users

In Oracle, the DBA creates users and then allocates appropriate statement and system level privileges to control what they can run/execute or create/modify/store. In SQL Server, the equivalent is the login account. The DBA should be the only user ever creating/deleting/altering login accounts within the instance, no matter what the reason, be it development, test and especially production servers. As such, we should not have: logins with null or blank passwords the login and password matches 1:1 should include a mix of numeric and characters of a reasonable minimum length should consider windows authenticated user vs sql authenticated

This is discussed throughout the book, but here is a summary:

Fixed Server (instance) Roles.

SQL Server Instance

DatabaseA (loginA maps to userA, recorded in sysusers) DatabaseB (loginA maps to userA, recorded in sysusers) LoginA (master..syslogins table) DatabaseC (loginA maps to userA, recorded in sysusers)

Windows Login (via trusted domain) or SQL Server Login (mixed)

Fixed database roles + user defined roles.

Authentication Modes
The SQL Server DBA can make use of only two forms of end-user authentication to the instance, that being: Windows (integrated) security same as OPS$ logins for Oracle Mixed Mode security (username / password maintained by the instance) as per standard user accounts in Oracle

325

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Oracle adds another layer of functionality with Oracle Advanced Security; there is no equivalent option in SQL Server. The problem with mixed mode user authentication is that username/password combinations are passed free-text to SQLServer. This can be circumvented by enforcing SSL encryption which is discussed in the chapter on Networking. For Windows authentication, the user authentication is handled by the underlying NOS and as such, is very secure. The authentication mode used by SQL Server is defined at installation time but can be altered via a right click on properties within Enterprise Manager and selecting the security tab. Ideally, all user logins to the SQL Server user databases use Windows Authentication. This should be implemented as: Application database roles hold object privileges, database login allocated to these Logins in SQL Server associated with Local NT Groups on the database server Local groups created on the server Pointing to global groups on the domain controller With NT users added to these global groups on the domain controller Providing integrated logins to the DBMS

The use of NT Groups offer a lot of flexibility but care must be taken when providing access (ie. the name of them should be self documenting to prevent mistakes). Apart from security, this also allows you to provide other team members with the responsibility for managing security and even better, can utilise the NT password expiration and management routines (that are missing for mixed logins). What you use is really dependent on the application itself and how it is architected. Even more so, the underlying network is a key factor in determining what path to take. I have always opted for mixed mode authentication, and a small number of secured NT logins for the application (that COM+ components instantiate themselves under) with a couple of mixed mode authentication DBA logins for general administration.

BUILTIN/Administrator Login
When the instance is installed, the administrator group and its users will instantly have sysadmin access to your DBMS via the BUILTIN/Administrator SQL Server Login account. To alter this and we can: a) b) Remove the BUILTIN\Administrators login from within SQL Sever Under NT, create another group called SQL Server Administrators or something like that and place the SQL Server user that starts the SQL Server service in it. Grant access to any other server administrators access if they need sysadmin access. In SQL Server, re-create the NT login account linked to the new SQL Server Administrators NT group (dont re-create BUILTIN\Administrators).

c)

326

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

By default, the service will be started via the local system account and its SQL Server connection via windows authentication. As such, altering the BUILTIN/Administrators account and revoking sysadmin access will prevent the service from running jobs. Funny enough, you will see no errors in the SQL Server log or SQL Agent log in relation to this. Jobs will simply not start and their next run date will be unknown. In the BOL, it clearly highlights the fact that the SQL Agent server connection must have sysadmin access. If you alter the BUILTIN/Administrator login and revoke sysadmin, remember that this adversely affects the SQL Agent database connection that must be altered accordingly.

Using the SA login

Never give your SA account password to a developer or server administrator, full stop. At a bare minimum, you should always set a password for the SA account even when you select pure windows authentication. The SA account is the master account for the entire SQL Server instance, in SQL Server 2k it can not be removed, you can not revoke sysadmin access, and by default it has DBA account for all databases in the SQL Server instance. The DBA should: a) b) c) d) never use it scheduled or regularly executed non-scheduled DTS jobs alter the password regularly guard it religiously ensure backup and recovery documents clearly document its use, how to get the password and associated responsibilities of using the login

What makes the SA account powerful is simply the sysadmin (System Administrator) fixed server role. Never grant this privilege to any other user, there is simply no reason to use it other than for DBA administrative tasks. The DBA can not remove the SA account, just like the Oracle DBA cannot remove the SYS account. The SA account is inheriantly the dbo user.

Duplexing a Login to Multiple Users

In SQL Server it is possible to login with one name and be connected to a database with another (strange I know). This is called aliasing logins and should never be used. The routine used to do it is:

327

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

use mydb EXEC sp_addalias 'mylogin', 'myuser' Where mylogin is the login name I used, which maps to the user name myuser. This is a nightmare to administer and also for security reasons it should never be used. To locate these users, look at the command sp_helpuser and the UserNameAliasedTo column that is returned.

Finding NULL Passwords

In SQL Server the DBA cannot manage passwords too well; mainly because of the fact that Microsoft pushes windows logins (OS managed login) rather than SQL Server logins (database managed login). It is rare for the security conscious DBA that null or blank passwords exist, but to find them run this query: select name, password from master..syslogins where (password is null or password = ) and name is not null

DBO Access
I can think of no valid reason what so ever for DBO privilege access for any standard (ie. non sysadmin or backup admin) user in production. There should be no DDL changes, the auditing is all in place, and any calls to a predefined SQL Server extended stored procedure has been sorted out the appropriate access granted. With proper role management DBO is not an option for any application. The only problem here can be third party software that utilises SQL Server as its security or other repository in order to run. I have come across a few products that prompt to for the SA password, then end up creating a database with no other user but SA, only to spend hours reading documentation and testing to change it.

User Quota and Resource Restrictions

In Oracle we have a variety of commands to lock, unlock, expire passwords, set quota limits, control password reuse, grace times etc for resource and access management. We have nothing of the sort in SQL Server, apart from Windows Authenticated user that will of course utilise password management features of the operating system.

Password File
The Oracle password file will store passwords for users with administrative privileges. There is no equivalent option in SQL Server.

Development Server Privileges

328

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

To allow developer access to creating stored procedures as dbo and granting execute access to the appropriate roles the DBA has created and documented, consider giving your developers access to the database roles:
db_ddladmin db_securityadmin

These privileges should be revoked in the test and of course production server environments. Lookup grant in the books online for a excellent matrix of database and instance level fix role privileges.

Application Security Connection Strings & Authentication

The following suggestions are based on this server set-up:

The first aim is to hide and/or secure the connection string used at the Web-Server in order to connect to the database. This does not stop a hacker from installing their own COM/COM+ objects that talk to your existing methods and utilise the business logic exposed through them. But you do want to stop at all costs complete access to your user database through an insecure connection string placed in an INI file on the server. The second aim is to determine the access method to the database. We can either use mixed mode or windows authentication. It is important to understand that as remotely possible it may seem to the programmers, security is of utmost importance in production and a text file based INI file is not acceptable. Altering data over a period of time (or even once!) is the worst type of hacking as it can span many backups and it the implications can be extremely costly to the organisation. NOTE - The ideas below can be cumulative, ie. they do not necessary describe a complete solution, but a variety of ideas for securing general connectivity to the database.

329

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Authentication Example

Client to Web Server o Consider HTTPS where appropriate. I have worked with the Athelon SSL accelerator (hardware) with great reliability and performance. Such hardware can selectively encrypt/decrypt and supports hundreds of simultaneous SSL connections per second but is a very expensive option and is another point of failure. Remember that once on the Internet, network sniffers can make short work of passwords and sensitive business data (to and from your server). The authentication mode at the webserver and the associated web-pages and folders is the key to web-site security (authentication modes summarised below). In the example above, a common method of authentication is via basic or integrated authentication to the web-site and all associated pages. The user has an account in Active Directory to facilitate the login and may be internal and external users (typically grouped and segregated). Authentication to SQL Server may be via common userid/password combination (shared account) for all users or fully integrated.

Integrated, Digest and Basic Authentication o Under IIS, if all are selected then the highest to the lowest level of security will be used in order based on what the client initially accepts. An NTFS partition must be used for Digest and Integrated. Basic

330

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Standard HTTP over readable text stream (base 64 encoded string) to the server unless HTTPS is used or a lower level protocol encryption is employed. User is only prompted for login ID and password Used by many organisations to prevent from being promoted with domain (see integrated security). o Digest Sends a hashed password over the network rather than the encoded password in Basic. User/password prompted must be a valid NT user and has appropriate NTFS permissions on the object(s) it is trying to access. Works via a challenge/response scheme and checksum. o Integrated Uses underlying Windows cryptographic data exchange with IE. This enabled single-sign-on where appropriate if the user is already logged into the domain IIS is talking over. User prompted for login, password and domain. This can be confusing for end-users where we are authenticating via a Domain. I have yet to see a program that utilises integrated security and pre-enters the domain field in the login prompt. COM+ (from a security context - there are also numerous other performance and feature benefits to the programmer) o Roll-based security and authentication Can administratively construct authorisation policies for an application down to a method level Enables you to manipulate security information from all upstream callers in the execution chain of your component. o o o Client authentication Client impersonation and delegation Software restriction policies

Obtaining the DB Connection string and Connecting to the Database

Here the objective is to: a) Store connection strings securely at the webserver

331

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

b) c) d) e)

Encrypt communication between webserver and database server Ideally, encrypt data where appropriate between client and webserver (out of scope for this section) Use only Windows Authentication for SQL Server login, deny local server login for these accounts. Avoid using the global.asa

Presented below are a series of diagrams that provide a catalyst for ideas in achieving this security. The examples all assume a COM+ (Component Services) business layer as a default for Enterprise Computing. Substitute this for COM as required. I do not recommend straight DB connection from ASP (global.asa variables etc) to the database for any sort of internet based application. General notes on the examples below: 1. Consider application roles in SQL Server to set and configure security context. Remember though only 1 role at a time as security privileges are not additive. 2. Add registry entries and protect via regedit32.exe for database connection strings. The COM+ package will run as a specific local domain user and connects to the database via integrated security. 3. As per 1, but rather than using the registry with encrypt the INI file via Windows 2k encrypted file system (EFS). As the encryption is based on a specific user profile this can be a little more difficult to manage if the COM+ package runs as a different user. 4. Set-up a series of User DSNs that are only accessible by the Windows user that created them. These typically map to a 1:1 SQL Server login. 5. Rather than using the registry, use Active Directory (Win 2k) for other 3rd party LDAP directory to store connection strings. Access is restricted via Windows Authentication and is always read-only. 6. Consider a global security or management database. Such a database includes a range of information for connectivity, login/password (encrypted) combinations, security contexts and lookups to establish connections and use the application. 7. COM+ a. De-encrypt routines (ideally talking to a lower level crypto API) to deencrypt and pass connection strings and passwords to establish connectivity. b. COM+ supports shared memory areas that can use used to collectively manage connectivity settings and logins amongst packages and associated methods.

332

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

c. Supports integrated security at a package level to individual users and/or groups.

Views and Stored Procedures

Over the years I have talked to a variety of people about application design and how data models are simplified via views and stored procedures to access and manipulate data. Although fine, it can be a hard sell to programmers, analysts and project managers. Why? because many programmers want complete CRUD access to virtually all schema tables, they want to code all DB access via the application and believe views are only for reports and simplifying complex lookups. When under pressure, the time to sit back and think security and performance is a tough ask. This is very simplistic view of course as security at this level in the database is typically dictated by: a) Cost to implement in terms of time and effort

333

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

b) c) d) e) f)

Understanding of the requirement for such access (definition of the security requirements) Architecture of the application Need for simplicity (unnecessary complexity?) Skills to implement Ease of maintenance

I like to take the approach of not exposing more data than you have to in selected areas within the application. Exposure to CRUD (create,read,update,delete) level can be implemented by views, stored procedures, roles, table priviligies, instead-of triggers and other object level and column privileges. The use of views is an excellent method of implementing security and hiding schema relationships and relational database complexities. Each table has an equivalent simple or complex view; the views may include instead-of triggers to re-direct DML to other "hidden" tables for complex views, for auditing or data partitioning. If instead-of triggers to do appeal to you, another option for managing CUD (create/update/delete) access is to grant select only access to the views and implement all CUD via predefined stored procedures. Therefore, the database users view of the actual database structure may be quite different from that of the schema owner and how they manipulate it is restricted by the stored procedures (business rules).

Another approach is only using stored procedures for insert, delete, update and select access and access to the database schema is only via these stored procedures. This can have significant performance benefits also due to caching of procedure execution plans. On top of this, you can lever the benefits of OPEN XML. Getting back to security, you can effectively deny all access to tables, and implement all CRUD through stored procedures. The only problem with this is a T-SQL one, which is its poor exception handling (ie. none!). The use of views and stored procedures also assist in application maintenance (no embedded SQL statements in your COM+ and ASP code) and makes impact analysis of schema change and performance tuning much simpler.

334

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

NOTE - It is important to remember that you do not need to grant CRUD access to the tables and views and stored procedure is referring to, only to the procedure itself.

Virtual Private Databases

The Oracle virtual private database option is managed via the DBMS_RLS (row level security) package. This option allows fine grained access control (FGAC) to table and view data, making sure a user only sees data they are allowed to see, no matter how they access the data (SQL*Plus or a prewritten Application). There is no equivalent option in SQL Server.

Label Security
The Label Security option use the virtual private database feature to implement row level security. By creating policies, row data can be tagged (based on sensitivity) to assist in managing and controlling access to table data.

Locking Down Extended and System Procedures

Securing xp_cmdshell (OS Shell)
The extended stored procedure xp_cmdshell allows you to shell out and execute an valid operating system command. By default, all users allocated to the fixed system role sysadmin (ie. SA account) have execute access. This is a real tough command to administer. Why? every project I have worked on to date has some need for it. Because of the fact that SA is the only user with sysadmin access, rather than creating a special account or looking at other work-arounds, the SA account is used to execute the task. Classic examples are stored procedures wrapped up in DTS jobs or supposed global administrative functions. This is problematic because: a) b) c) now application databases are using the SA account and rely on it to run their jobs. altering the SA password has application impact xp_cmdshell will be executed under the security context in which the SQL Server service is running

Points a) and b) are obvious and the fix is a simple matter of explicitly granting execute access to xp_cmdshell via a role and allocating that role to a managed and secure database user whose actual login is rarely used (ie. DTS connectivity only to run the command shell tasks). Point c) is the very important. If the database user is a member of sysadmin then, more than likely, the user will have local administrative privileges to the server, as this is the user running the MS SQLServer and associated services. This is even more the reason why not to use the SA account. If the user is not a sysadmin but has been granted

335

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

execute access as described above, then the SQL Server Agent proxy service user will be used instead. The SQL Server Agent proxy account can be altered via xp_sqlagent_proxy_account (undocumented in BOL) which defines the account used to run the SQLServerAgent service. This may be your SQLServer NT user if you are not using the Administrator account (which is bad security practice), so you may decide to alter this to another user with restricted access rights for finer control to the operating system. In the end, xp_cmdshell should be carefully evaluated before using it. Ideally it should be totally disabled (revoke execute permission) for ultimate security. Look at using isql jobs scheduled via AT and look closer at the options available to you via DTS.

Securing xp_cmdshell (OS Shell)

The security check-list from www.sqlsecurity.com mentions a variety of other extended stored procedures to lock down where possible. In the end, I believe its better to disable them rather than simply removing them and loosing functionality in Enterprise Manager or other 3rd party products. With strict role management and user security they should not be accessible. The list is (not definitive by any means):
OLE Automation
Sp_OACreate Sp_OADestroy Sp_OAGetErrorInfo Sp_OAGetProperty Sp_OAMethod Sp_OASetProperty Sp_OAStop

Registry Access
Xp_regaddmultistring Xp_regdeletekey Xp_regdeletevalue Xp_regenumvalues Xp_regread Xp_regremovemultistring Xp_regwrite

Other routines
sp_sdidebug xp_availablemedia xp_cmdshell xp_deletemail xp_dirtree xp_dropwebtask xp_dsninfo xp_enumdsn xp_enumerrorlogs xp_enumgroups xp_enumqueuedtasks xp_eventlog xp_findnextmsg xp_fixeddrives xp_getfiledetails xp_getnetname xp_grantlogin xp_logevent xp_loginconfig xp_logininfo

336

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

xp_makewebtask xp_msver xp_perfend xp_perfmonitor xp_perfsample xp_perfstart xp_readerrorlog xp_readmail xp_revokelogin xp_runwebtask xp_schedulersignal xp_sendmail xp_servicecontrol xp_snmp_getstate xp_snmp_raisetrap xp_sprintf xp_sqlinventory xp_sqlregister xp_sqltrace xp_sscanf xp_startmail xp_stopmail xp_subdirs xp_unc_to_drive

Data Transformation Services

Disabling the following via a role or revoking PUBLIC execution to lock down data transformation tasks (all in the MSDB database): sp_add_dtspackage sp_enum_dtspackages sp_add_job sp_add_jobstep save dts into sysdtspackages open dts package adds new job executed by the SQLServerAgent service add new step for a job created above

It should be very rare that DTS creation and editing in production is required unless of course during emergency fixes. You may have routines that dynamically create jobs, add job steps (classic examples are backup routines) where the above may need to be rethought, but in a majority of cases DTS can be safely locked out. By default in SQL Server 2k the DTS job is encrypted but unless you specify a password they remain editable. It is important to remember that SQL Servers access model for DTS packages is, well, limited. Meaning that only members of sysadmin can edit and save any package they like, but all other users are restricted only to the packages they initially created themselves. This holds true even when you try and give users dbo access to the MSDB database in a vain attempt to share packages with other database users. To get around some of these issues read the chapter on Managing Databases.

System, Object Privileges and Roles

Within Oracle, we have System Privileges (80+) and Object Privileges, many of these privileges do not extend to the data dictionary objects, for example 'update any table' or 'select any table' privileges will not flow over to the data dictionary tables.
337

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Within the SQL Server architecture these privileges are wrapped up into server roles that exist at a variety of levels within the instance. These are shown below: a) b) c) d) System Privs - Instance Level - unknown as a "fixed server role" System Privs - User database level - unknown as a "fixed database role" Object Privs - permissions allocated to distinct object or to other roles Statement Privs - ability to perform a "statement", namely create table

etc.

The roles of course are similar to Oracle roles, where a fixed role allocation can provide to user with numerous bundled system and object privileges. Example fixed server roles (instance level) are: Example fixed server roles (database level) are: Example statement privileges are: BACKUP DATABASE BACKUP LOG CREATE DATABASE CREATE DEFAULT CREATE FUNCTION CREATE PROCEDURE CREATE RULE CREATE TABLE CREATE VIEW CREATE INDEX CREATE STATISTICS <..etc..> Example object level privileges are: GRANT ALL ON mytable TO myrole GRANT INSERT,DELETE,UPDATE,SELECT ON mytable TO myuser Typically, such statement privileges are naturally inherited from fixed server roles and other fixed database roles. For example, CREATE DATABASE is granted automatically with the sysadmin and dbcreator fixed server roles. On top of this, you may also get extended privilege (WITH ADMIN in Oracle) to grant this statement privilege to other users as the sysadmin and securityadmin fixed server roles can do. This can become somewhat complex and its well worth exploring in details before allocating fixed server and fixed database roles to any user. We will not compare every Oracle system privilege with SQL Servers statement and other privileges, but, we will compare SQL Server fixed server and database roles with Oracle.

338

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Fixed Server Roles (Instance)

The fixed server roles in SQL Server are basically made up of the SYSDBA and SYSOPER accounts and the grant DBA privilege in Oracle. The permissions are high level ones that are really aimed at the DBA or the System Administrator. In no circumstance should you allocate any other user to these accounts. I do not grant any user these privileges, think long and hard account your senario before giving an account access to these (typically the db_owner database privilege is the highest I would grant). The roles are:
Role Name Bulk Insert Administrators Database Creators Disk Administrators Process Administrators Security Administrators Server Administrators Setup Administrators System Administrators Used for? Similar to the import/export privilege in Oracle but via the bulk insert DML statement. Can create user databases. DBA should never allow this for backup and recovery purposes. Can manage any database file group data files May be allocated during development only to allow developers to kill off running DTS jobs. Manage user account, create databases, read error logs. Can alter instance settings, DBA only. Manage linked servers and startup permissions. DBA only (sysadmin role)

To get an better idea as to what is included in the privilege, use EM, Security logins select a user, properties, Server Roles tab, select a role and click properties, for example:

Fixed Server Roles (Database)

For each database the following fixed roles exist:

Role Name Db_owner (dbo) Oracle Equivalent Fixed Role CONNECT Used for? DBO owner account (DBA only)

339

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Db_datareader Db_datawriter Db_ddladmin

RESOURCE DBA N/A N/A CONNECT RESOURCE DBA SYSDBA SYSOPER EXP_FULL_DATABASE IMP_FULL_DATABASE CONNECT RESOURCE DBA CONNECT RESOURCE DBA N/A N/A

Read access to any table or view but not stored procs or UDFs. Write access to any table or view. Alter or create or drop any user definable database object. Can issue backup, checkpoint, DBCC commands. Deny, grant and revoke permissions. Can add or remove users. As per data reader by deny the privilege. As per data writer by deny the privilege.

Db_backupoperator

Db_securtityadmin

Db_accessadmin Db_denydatareader Db_denydatawriter

Oracle recommends you dont use the predefined roles, but build a custom set of privileges.
User Defined Roles

The name explains it all, they are the same in Oracle. They are simple enough to manage and require no explanation. The DBA cannot alter fixed database and instance roles.
Object Privileges

The management of object level privileges is Oracle and SQL Server are essentially the same. The DBA can grant the basic select, insert, update, delete, execute, and references (DRI) privileges to tables, views, stored procedures and user defined functions. At a lower level, the DBA can also grant virtical security at a column level via:
GRANT UPDATE ON [dbo].[mytable] ([mycolumn]) TO [myuser_role]

Removal of privilege is via the REVOKE command:

REVOKE REFERENCES ON [dbo].[efs] FROM [public] CASCADE

Use the DENY command if you want to retain the permission in the role and/or user but close off access temporarily. The CASCADE option will revole the privilege for all users part of the role, in this case the public role. The public role should be left alone where possible, meaning that the DBA should create their own version of the role and allocate privileges through that. The public role includes these standard permissions begin tran, print, rollback tran, set, commit tran, raiserror, save transaction.

340

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

No database user in SQL Server can create objects with being explicitly granted the permission to do so. The DBA should carefully consider the implications of granting the the db_ddladmin database role or explicity granting the create table or view statements. The objects by default will be owned by the user unless explicitly prefixed with dbo. This is unlike oracle where we would need to revoke access to say, create table etc after allocating the 'connect' role.

Auditing
Application Auditing
There are no SYS$AUD objects as in Oracle to assist the SQL Server DBA with auditing of user databases and its objects; this is a right pain as the classic fix is typically a range of triggers that log inserts, updates, deletes against the table(s). Here is an example of some simple auditing in action:
Source Table : dbo.MyTable (trigger on this table) Audit Table : dbo.MyTable_Audit Audit actions column on audit table: 1 = Insert, 2 = Update, 3 = Delete CREATE TRIGGER dbo. MyTable_Audit_Trigger ON dbo.MyTable FOR INSERT, UPDATE, DELETE AS DECLARE @insert_count int, @delete_count int, @current_time datetime SET SET SET SET NOCOUNT ON @current_time @insert_count @delete_count = GETDATE() = (SELECT COUNT(*) FROM INSERTED) = (SELECT COUNT(*) FROM DELETED)

/* A new record is inserted */ IF @insert_count > 0 AND @delete_count = 0 BEGIN INSERT INTO dbo.MyTable_Audit ( col1_code, col2_desc, update_count, last_updated_on, last_updated_by, Audit_Action, Modified_By, Modified_Time ) SELECT access_type_code, col2_desc, update_count, last_updated_on, last_updated_by, 1, last_updated_by, @current_time FROM INSERTED END /* A record is updated */ IF @insert_count > 0 AND @delete_count > 0 BEGIN INSERT INTO Dbo.MyTable_Audit

341

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

( col1_code, col2_desc, update_count, last_updated_on, last_updated_by, Audit_Action, Modified_By, Modified_Time

) SELECT

col1_code, col2_desc, update_count, last_updated_on, last_updated_by, 2, last_updated_by, @current_time FROM INSERTED

END /* A record is deleted */ IF @insert_count = 0 AND @delete_count > 0 BEGIN INSERT INTO dbo.MyTable_Audit ( col1_code, col2_desc, update_count, last_updated_on, last_updated_by, Audit_Action, Modified_By, Modified_Time ) SELECT col1_code, col2_desc, update_count, last_updated_on, last_updated_by, 3, last_updated_by, @current_time FROM DELETED END

If you are using identity columns on your trigger tables as well as the sourse table, then the application must use the @@SCOPE_IDENTITY() function to source the correct identity value from an insertion. The last thing you want to do is to capture the identity value from the trigger table.

342

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Expanding on the trigger and tables above, the scenario we are using is this:

USERA USERA (OS Authenticated via Basic Authentication on Web Site, 1 of 8000 users) Mycom.ini COM+ component (business layer) connects to DBMS via USERX in INI file Application logs in via single (or very few) actual DBMS accounts as defined by INI file. USERX select only access

mytable

mytable_audit

Trigger fires

This creates some issues the DBA must be aware of, namely: if triggers are off (disabled) whilst initially migrating data into the table, then you will not record the first insertion audit entry for the record. This may have flow on issues later down the track. The application layer must manage issues with concurrency in a stateless web environments. This is typically done by columns for each table such as update_count, last_updated_on, last_updated_by which can also be used for auditing. As in the example above, last_updated_by must be passed in from the application and not sourced from the DBMS. If you dont update the record before deleting it, then you will capture the wrong user who actually performed the deletion, as we are simply using the last_updated_by column to record the user who performed the operation. As mentioned in c), we can not use the USER_NAME() system function as all users stream through a single (common) login via the applications business layer.

The DBA should work closely with the developers and establish the auditing schemes early on in the project cycle, especially in terms of recording update by and updated on information. The DBA should ask: retention period of audit table, how will it be purged or moved? how the end users will interact (search) over the audit data and the associated indexing issues. How will the added indexing affect OLTP performance? will you create a separate file-group to store audit data tables?

343

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

how roles will be used to control CRUD access to the audit tables consider writing a generic audit display framework for your application rather than writing custom reports have you considered third party applications? remember the text, ntext and image restrictions within triggers and communicate these to the analysts/developers

Profiler (trace)
The DBA can also audit via profiler, letting it run for a period of time collecting a variety of events for the DBA to review (use sp_trace rather than the GUI, use the GUI to generate the script for you). Although fine, this method may not be practical and may drive down performance slightly. The trace utility will not trace sp_addlogin and associated routines dealing with passwords.

Instance level auditing

Apart from application based auditing and using profiler, the DBA can audit some basic DBMS events. Within EM, right click and select properties for the instance. Click on the security table and we get:

exec xp_instance_regwrite N'HKEY_LOCAL_MACHINE', N'SOFTWARE\Microsoft\MSSQLServer\M SSQLServer',N'AuditLevel', REG_DWORD,2

A restart may be required for the property to take affect due to the registry change. To read the current value, use the command:
exec xp_instance_regread N'HKEY_LOCAL_MACHINE', N'SOFTWARE\Microsoft\MSSQLServer\MSSQLServer', N'AuditLevel'

344

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

C2 Level Auditing
For detailed information on C2 level auditing in SQL Server see the books online. I would not recommend it unless you have explicit requirement to do so.

Encryption / Obfuscation Options

Networking Options
See the chapter on Networking for more information on protocol encryption. Basically, SQL Server supports 40 and 128bit SSL encryption of its listener protocols and of course standard NOS encrypted Windows authentication (kerberos), whereas Oracle supports a large range of other encryption methods through the Oracle Advanced Security option. Most will agree that SSL is the better way to go.

Encrypted File System

In the Windows 2000 server environment, the DBA can utilise the EFS (encrypted file system) option to secure database files and database backups files at the operating system file level. This option facilitates file encryption via the Windows Crypto Architecture feature and associated public key infrastructure features. Before using this option, the DBA should have their instance service(s) running under a custom NT user rather than the Administrator account, see Secured Installation previously discussed. Login as the service account and shutdown all instance services. Once done, goto the directory you want to EFS encrypt via Explorer, right client properties, advanced button and check the encrypt contents to secure data option:

IMPORTANT Backup all files before attempting encryption.

345

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

You may be prompted about encrypting the this folder only or this folder and all subfolders, and then asked about encrypting the parent folder (does all existing and any new files in the folder) or just this file, in most cases you will want the entire folder:

Only the user that encrypted the files can de-encrypt. To check on the status of encryption via the command line, utilise the cipher.exe command with no parameters. The U of course meaning un-encrypted. Take a close look at the command options via the /? Switch, namely the /E (encrypt) and the /D (deencrypt) switches. Note that windows binaries cannot be encrypted.

WITH ENCRYPTION option

The WITH ENCRYPTION option can be used with stored procedures and views to encrypt their contents. I wanted to believe that SQL Server 2k under Windows 2000 was using the Windows Crypto API and as such, was using a key complex enough to make hacking very difficult. Then I came across this article from www.SecuriTeam.com:

346

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

http://www.securiteam.com/windowsntfocus/6Q00H2A3FY.html

I have to say that security specialists and hackers are some of the most sarcastic people I know! but with good reason. The articles example, which I have tested under SQL Server 2k, shows you all the code of a stored procedure created with the WITH ENCRYPTION option. Here is an extract from this article regarding SQL Server 2k encryption:
Here is how stored procedure (and view and trigger) encryption works on SQL Server 2000: 1. Take the database's GUID (generated when the db is created), the object id (from sysobjects) and the colid (from syscomments) and concatenate them. 2. Hash the key using SHA. 3. Use the SHA hash as an RC4 key, generate a sequence of bytes equal in length to the stored procedure text. 4. XOR this stream of bytes against the stored procedure text.

The example given in the article works with any user that has priviligies to create and alter procedures and select access against syscomments. To help mitigate the problem: a) b) c) Deny SELECT access to PUBLIC against syscomments Deny create and drop procedure commands See previous notes on restricting DBO access and sysadmin access to your database.

With all this said, I believe it is still worth the effort to encrypt code. It is though, imperitive that your test t-sql code execution thoroughly to determine the CPU impact this may have on execution (though I would imagine it to be very small).

T-SQL
There are no native encryption or obfuscation options available to the T-SQL developer with SQL Server. In Oracle the developer may utilise the dbms_obfuscation_toolkit, but in SQL Server the developer must either code their own, or utilise another language and its API calls (OS or language dependent). Some DBAs actually regard database encryption as bad practice. I tend to disagree and can think of a variety of reasons where, especially one way encryption, would be required. The trick here is private key management and exposure of the de-encryption functions/routines. This requires very careful planning and realisation of the security risks and possible litigation if data is cracked. Developers in .Net will find the new system calls very helpful.

347

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

INDEX
A ado-md 217 alert log 227 Archived redo log 7 authentication 325 B Backup history of 95 no more space 96 backup set 78 BFILE 287 Block size 25 boot.ini 11 Buffer cache 12 BUILTIN/Administrator 326 BUILTIN/Administrators 53 bulk insert 66 C cell security 215 CHECKALLOC 104 CHECKDB 104 Checkpoint 18 Ckpt 20 Client Network Utility 316 collation database 203 Collation Column 204 COM+ 169 COM+ isolation level 171 COM+ proxy 172 command shell 281 component load balancing 185 Connection 17 Control file 4 Corrupt indexes 104 Counters performance monitor 142 D Data dictionary 5 E Editions 37 Encrypted file system 44 encryption multi-protocol 319 error log 227 Explain plan 137 Extended stored procedures 18 Extents 26 F Federated databases 187 File IO statistics 160 Fillfactor 26 for xml 283 Fragmentation 30 FREELISTS 26 full backup 80 Functions user defined 278 Data transformation service 237 Data types in SQL Server 306 Database connection 17 Database links 229 DATABASEPROPERTYEX 85 Datafiles 5 Date functions 295 DBCC 65 DBCONTROL 85 DBMS_JOB 284 DBMS_LOB 287 DBMS_MAIL 283 DBO privilege 328 dbrepair 104 DBVERIFY 65 DbwN 20 Deadlocking 126 developer security rights 329 diagrammer scripting 90 differential backup 81 dso 217 Dynamic SQL 309

348

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

G Global allocation map page 27 GUI tools 58 H High Water Mark 33 Hints Comparison 304 HINTS 302 I IAM 28 identity 289 Index tuning wizard 134 index types compared 289 Indexes composite 152 fragmentation 156 function based 152 rebuilding 158 skewing 155 types of 150 information schema 283 installation security of 323 Installation 37 Instance 1 Named 4 J JServer 66 K Kill user sessions 254 Killing users 92 L Label Security 335 Latch statistics 140 Lgwr 20 License Manager 40 Licensing 38 Linked Server 229 Listener 68, 313 log miner 69 log shipping 189 login vs user 325

M maintenance plan backups 75 MDAC 2 media set 78 metadata 283 Moving master database 55 MSDB and MODEL databases 55 TEMPDB database 57 MTS 322 Multi-protocol 315 N network load balancing 183 Network monitoring 320 Northwind re-install 105 Numerical functions 293 O Object change owner 253 check ownership 253 ODS 317 olap administer 210 backup 219 installation 208 repository migration 211 security 215 storage models 213 Open Data Services 317 OPENQUERY 236 OPENROWSET 235 OPS$ login 252 Optimal Flexible Architecture 48 orphaned session 256 P Package 278 pad_index 26 Page free space 27 Page splitting 140 Page types 25 Parallel affinity option 301 IO 302 query 301 password file 328 passwords finding null 328

349

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

Performance monitor 142 Pmon 20 Procedure cache 15 Profiler example events 131 saving scripts 133 Q Query Analyser 68 R RAID 46 raw partitions 258 Read-ahead 13 recovery interval 72 recovery model 73 Redo log buffer 14 Relational engine 35 REPAIR_REBUILD 104 Resource File 206 Restore emergancy mode 102 file added 101 file group 101 filelistonly 96 full backup 103 loading mode 100 logical names 96 master database 97 MSDB and MODEL databases 99 point in time 104 rebuildm.exe 106 suspect database 99 tempdb 105 with file move 100 RMAN 75 rollback 110 Rollback segments 7 S SA account 327 scripting 220 Scripting 89 security auditing 341 c2 auditing 345 data transformation services 337 encrypted file system 345 object permissions 340 tracing 344 via views and stored procedures 333 WITH ENCRYPTION 346

xp_cmdshell 335 Segments 35 sequence 289 SERVERPROPERTY 40 Service account 43 Service accounts 53 services re-creating 257 set nocount 172 Setup 42 Shared global allocation map page 27 SHOWCONTIG 30 Shutdown instance 84 sp_spaceused 28 Space Management 26 SQL Loader 66 SQL Trace 67 SQLDiag.exe 322 SQLOLEDB 319 STATSPACK 69 Stored procedure 279 stored procedures 172 String functions 294 Striping 49 Suspect Database 99 SVRMGRL 65 SYS login equivalent 327 SYS$AUD 341 T table 287 Tablespace 22 tablespace backup 83 Tablespaces 6 temporary 7 Task manager 169 TDS 321 tempdb 256 Temporary tablespace 23 timezones 296 Tracing 130 black box 139 system functions 134 trace flags 147 transaction log backup 82 TRUNC() 296 truncate transaction log 83 U UMS 11 unique identifier 292 update statistics 261 uptime 257

350

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

User database ownership 252 orphaned 251 resource quota 252 user accounts quotes, resource restrictions 328 user defined function 288

V Versions 37 Virtual address space 11 virtual private database 335 W Wait statistics 160 Worker threads 21

351

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

References
Please take the time to refer to these references from DBA experts around the world.
(1) What is the Maximum page size in SQL Server 2000?, Steve Jones, 7/2/2002, http://www.sqlservercentral.com/columnists/sjones/pagesize.asp (2) SQL Server 7: Some Useful Undocumented DBCC Commands, Alexander Chigrik, 22/10/2001,http://www.sqlservercentral.com/columnists/achigrik/sqlserver7someusef ulundocumenteddbcccommands.asp (3) Oracle Concepts: The Instance and the Database, Joe Lax, http://www.devx.com/dbzone/articles/laxseries/lax082001/lax082001.asp (4) INF: Job to Monitor SQL Server 2000 Performance and Activity, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q283696 (5) http://support.microsoft.com/default.aspx?scid=kb;EN-US;q272428 (6) Memory Management, Can a readme file contain extra memory, Kalen Delaney, March 2001, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=16522 (7) Technology Comparison Discussion, Microsoft, http://www.microsoft.com/sql/evaluation/compare/discussion.asp (8) How to Migrate Unix based Oracle Applications to Windows and SQL Server, Microsoft, http://www.microsoft.com/windows2000/migrate/unix/unixoracle.asp (9) MS SQL Server 2000 Books Online (10) Oracle9i Database : The Power of Globalisation Technology, An Oracle Whitepaper [February] [2002] (11) Best Practices for Globalisation using the Oracle 9i Internet Application Server, Simon Wong, July 12 2001, v1.0 (12) Oracle 8i/9i and Microsoft SQL Server 2000 A comparative Study from a Developers and DBA Perspective, Sriram Kumar, April 2002. (13) SQL Server: Enterprise vs. Standard, Rahul Sharma, 19/4/2002, http://www.sqlservercentral.com/columnists/rsharma/enterprisevsstandardedition.asp #_ftnref3 (14) http://msdb.microsoft.com/library/en-us/vbcon98/html/ (15) Data Concurrency and Consistency, Oracle Documentation, Oracle 9i database concepts, http://otn.oracle.com/docs/products/oracle9i/doc_library/release2/server.920/a96524 /c21cnsis.htm#2841 (16) PRB: How to recover SQL Server After a Tempdb Database is Marked Suspect, Microsoft Support, http://support.microsoft.com/search/preview.aspx?scid=kb;enus;Q288809 (17) Transferring a SQL Server Diagram, Brian Knight, 24/5/2002, http://www.sqlservercentral.com/columnists/bknight/transferdiagram.asp (18) Managing Rollback/Undo Segments in AUM (Automatic Undo Management), Note:135090.1 (19) Creating, Optimizing, and Understanding Rollback Segments, Note:62005.1 (20) Inside SQL Sever: Parse, Compile, and Optimize, Introduction to the query processor, Kalan Delaney, December 1999, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=7446 (21) Oracle Bulletin Checkpoint Tuning and Error Handling (22) Query Recompilation in SQL Server 2000, Thomas Davidson, Microsoft Corporation, May 2002 [http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnsql2k/html/sql_QueryRecompilation.asp] (23) www.sql-server-performance.com (24) www.sqlteam.com (25) Optimizing SQL Server Performance by using File and Filegroups, Alexander Chigrick, 27/9/2001, http://www.sqlservercentral.com/columnists/achigrik/fileoptimtips.asp

352

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

(26) Improving text and image column performance, Brian Knight, 1/4/2002, http://www.sqlservercentral.com/columnists/bknight/textinrow.asp (27) Microsoft SQL Server 7.0 Performance Tuning Guide, Henry Lau, Microsoft Corporation, October 1998, http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnsql7/html/msdn_sql7perftune.asp (28) Microsoft SQL Server 7.0, Storage Engine Capacity Planning Tips, Microsoft Coorporation, March 1999, http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnsql7/html/storageeng.asp (29) What rules of thumb you live by when programming?, SQL Server Central forum posting(s),http://www.sqlservercentral.com/forum/topic.asp?TOPIC_ID=2772&FORU M_ID=61&CAT_ID=5&Topic_Title=What+rules+of+thumb+you+live+by+when+progr amming%3F&Forum_Title=Anything+that+is+NOT+about+SQL%21 (30) Inside Microsoft SQL Server 7.0, Book Sale Site, http://mspress.microsoft.com/prod/books/2394.htm (31) INF: How to Use KEEPFIXED PLAN to Disable Stored Procedure Recompilations, Microsoft Support, http://support.microsoft.com/search/preview.aspx?scid=kb;enus;Q276220 (32) Minimizing the Number of Log Backups, Andy Warren, 11/5/2001, http://www.sqlservercentral.com/columnists/awarren/logbackup.asp (33) Logins, Users, and Roles Getting Started, Andy Warren, 20/11/2001, http://www.sqlservercentral.com/columnists/awarren/loginsusersandrolesgettingstart ed.asp (34) Dont shutdown that database! Use Oracle 9i Online Object Redefinition, Chris Lawson, Copyright 2002 Database Specialists, http://www.dbspecialists.com/presentations/online_redef.html (35) Network Load Balancing Service, Tony Norhrup, http://www.mcpmag.com/reviews/Services/article.asp?EditorialsID=1 (36) http://www.microsoft.com/ntserver/downloads/bin/nts/clustering_apps.exe (37) INF: SQL Virtual Server Client Connections Must be Controlled by Clients, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;EN-US;q273673 (38) http://www.microsoft.com/israel/events/presentations/teched2002/inf305.ppt (39) New SQL Server 2000 Data Types, Brian Knight, 14/5/2001, http://www.sqlservercentral.com/columnists/bknight/sql2knewdatatypes.asp (40) INF: How to enable Analysis Services to use 3 Gb of RAM, Microsoft Support, http://support.microsoft.com/search/preview.aspx?scid=kb;en-us;Q295443 (41) Intro to User defined functions, Garth Wells, 8/1/2001, http://www.sqlteam.com/item.asp?ItemID=1955 (42) Information Schema Views, Brian Kelley, 9/1/2002, http://www.sqlservercentral.com/columnists/bkelley/informationschemaviews.asp (43) Keep SQL Server Up and Running Availability enhancements in SQL Server 2000 can keep your business going 24x7, Kalen Delaney, Dec 2000, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=15742 (44) DBCC SHOWCONTIG, SQLMag.com Forum Post, http://www.sqlmag.com/Forums/messageview.cfm?catid=3&threadid=6878 (45) Understanding MS SQL Servers SHOWPLAN Output, Microsoft, http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/sql /maintain/featusability/showplan.asp (46) http://webtool.rte.microsoft.com/tutor/perf_NT4.htm (47) Transact-SQL Language Reference Guide, Randy Dyess, http://www.transactsql.com/Manuscripts/tslrg.html (48) Microsoft T-SQL Performance Tuning, Part 1: Analysing and Optimizing T-SQL Query Performance on Microsoft SQL Server using SET and DBCC, Kevin Kline, 2002, Quest Software (49) Understanding Execution Plans Part 1, Robert Marda, 26/7/2002, http://www.sqlservercentral.com/columnists/rmarda/understandingexecutionplans.as p

353

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

(50) INF: Moving SQL Server Databases to a New Location with Detach/Attach http://support.microsoft.com/default.aspx?scid=kb;en-us;Q224071 (51) Microsoft SQL Server Administration FAQ, http://vyaskn.tripod.com/administration_faq.htm (52) MS SQL Server Performance Tuning and Optimisation for Developers, Part 3, Configuring SQL Server for Performance, Adam Shapiro, 1997, http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/sq l/maintain/optimize/dat412ef.asp (53) Chapter 9 Server Architecture, Microsoft, http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/sq l/proddocs/intro/part3/75515c09.asp (54) Oracle 9i Application Server: Availability, Scalability, and Manageability of J2EE and Web Cache Clusters, An Oracle Whitepaper May 2002. (55) Clustering Web Accelerators, Presented at IEEE WECWIS, San Diego, June 28th 2002, James Feenan, Patrick Fry, Ming Lei. (56) How to Setup Log Shipping Microsoft SQL Server 2000, Purna Gathani. (57) Like without model, Jasper Smith, 28/7/2002, http://www.sqlteam.com/item.asp?ItemID=10060 (58) The FROM clause, Bill Graziano, 25/6/2002, http://www.sqlteam.com/item.asp?ItemID=9867 (59) INF: Effect of Database File Additions or Deletions on Database Recovery, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q286280 (60) Quest Knowledge Expert for Oracle Administration v6.1.1 (61) Developing Databases with Microsoft SQL Server 7.0 Transaction Processing (62) How to Diagnose and Fix Wait Locks, Cathan Kirkwood, 21/11/2002. (63) Stored Procedures and Caching, Brian Kelly, 28/1/2002 (64) Managing Extended Properties, Bill Vaughan, July 2001, SQLMag.com (65) Optimizing Stored Procedures to avoid recompiles, http://www.transactsql.com/Articles/OSPR.html, Randy Dyess 2002 (66) Version Control for Stored Procedures, Andy Warren, 10/5/2002 (67) Understanding parallel queries in SQL Server v7.0, Scott Mauvais, May 2000, http://www.mauvais.com/PubsCached/ZD-Parallel.htm (68) Documented and Undocumented Trace Flags for SQL Server 2000 and 7.0, Randy Dyess, 2002, http://www.sqlservercentral.com/columnists/RDyess/traceflags_printversion.asp (69) Indexed views in SQL Server 2000, Doug Carpenter, 18/10/2002 http://www.sqlteam.com/item.asp?ItemID=1015 (70) INF: How to Automate an Archive or Backup of OLAP Database, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;en-us;294287 (71) HOW TO: Setup and Troubleshoot a Linked Server to Oracle in SQL Server, Microsoft Support, http://support.microsoft.com/search/preview.aspx?scid=kb;en-us;Q280106 (72) INFO: Limitations of Microsoft Oracle ODBC Driver for OLEDB Provider, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;en-us;Q244661 (73) Transact-SQL.com, http://www.transactsql.com/sprocs/sp_tempdbspace.html (74) Index maintenance, Baya Pavliashvili, 06 Aug 2002,http://searchdatabase.techtarget.com/tip/0,289483,sid13_gci841586,00.html (75) Using the SQL Server Transaction Log to Improve Database Availability and Performance, Lev Vaitzblit, http://www.dbta.com/transactionlog/index.html (76) How to restart SQL Server service at regular intervals?, http://vyaskn.tripod.com/restart_sql_server_service.htm (77) How to troubleshoot orphan users in SQL Server databases?, http://vyaskn.tripod.com/troubleshooting_orphan_users.htm (78) FIX: AUTO-SHRINK Option or DBCC SHRINKDATABASE May Slow Down SQL Server 2000 Performance, http://support.microsoft.com/default.aspx?scid=kb;en-us;296386 (79) Introducing Indexed Views SQL Server 2000 unviels another path to peak performance, May 2000, Kalen Delaney (80) What's in an ADO Connection String?, John Peterson, http://www.asp101.com/articles/john/connstring/default.asp

354

S Q L

S E R V E R

2 0 0 0

F O R

T H E

O R A C L E

D B A

HOWTO: Set the SQL Server Network Library in an ADO Connection String, Microsoft Support, http://support.microsoft.com/default.aspx?scid=kb;en-us;238949 (82) Securing your SQL Server, Brian Knight, 14/5/2001, http://www.sqlservercentral.com/columnists/bknight/securitypitfalls.asp (83) SQL Server Security FAQ, http://www.sqlsecurity.com/faq.asp (84) Administering User and Security, Kevin Cox and William Jones, July 1997, http://www.windowsitlibrary.com/Content/77/10/toc.html (85) Are SQL Server userids and passwords passed in clear on the network?, Neil Pile, 5/2/2000, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=14357 (86) http://www.microsoft.com/sql/techinfo/administration/2000/security.asp (87) Permissions in SQL Server 7.0, Michael Deignan, June 1999, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=5392 (88) Multiple Vulnerabilities in Microsoft SQL Server 2000 and 7.0, Ken Pfeil, 2/1/2002, http://www.ntsecurity.net/Articles/Index.cfm?ArticleID=23639 (89) SQL Server Security Checklist, SQLServerSecurity.com, http://www.sqlsecurity.com/DesktopDefault.aspx?tabindex=3&tabid=4 (90) Microsoft SQL Server 2000 Security, Microsoft, http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/sql /deploy/confeat/c05ppcsq.asp (91) SA to be denied access, SQL Server Central Forum, http://www.sqlservercentral.com/forum/topic.asp?TOPIC_ID=1978&FORUM_ID=5&CA T_ID=1&Topic_Title=SA+to+be+denied+Access&Forum_Title=Administration (92) Advanced Security Concepts, Nelson Howell and Ben Forta, http://local.15seconds.com/issue/971130.htm (93) http://www.microsoft.com/ntserver/techresources/webserv/iissecure.asp (94) K-026: Microsoft SQL Server Admin Login Encryption Vulnerability, March 22 2000, http://ciac.llnl.gov/ciac/bulletins/k-026.shtml (95) Microsoft SQL Server Password Recovery, Password Crackers Inc, http://www.pwcrack.com/sql.shtml (96) How to I encrypt fields in SQL Server? Neil Pike, 5/2/2000, http://www.sqlmag.com/Articles/Index.cfm?ArticleID=14035 (97) The SQL Server 7 hackers toolkit, Tom Sager, Xephon, http://www.xephon.com/oldcode/Q016A01 (98) dSQLSRVD, SQL Server SysComments Decryptor, securiteam.com, http://www.securiteam.com/tools/6J00S003GU.html (99) Analysis of Microsoft SQL Server 2000 Stored Procedure Encryption, securiteam.com, http://www.securiteam.com/windowsntfocus/6Q00H2A3FY.html (100) Implementing row level security in SQL Server databases, Narayana Vyas Kondreddi, http://vyaskn.tripod.com/row_level_security_in_sql_server_databases.htm

(81)

355