Thanks to our Sponsors :

PASS SQL Saturday – Holland - 2014

 Understanding

Query Modes in Tabular Model

SQL Saturday Holland 2014
Julie Koesmarno
http://mssqlgirl.com | signal@mssqlgirl.com | @MsSQLGirl
Julie Koesmarno
10 years experience with SQL Server

Power BI & Tabular Model Enthusiast

Co-chapter Leader of

Volunteer at &

PASS Virtual Chapter Mentor

Microsoft Certified Trainer

Blogger – http://www.MsSQLGirl.com
Near Real-Time Strategy
Near Real Time Strategy
Session Takeaways
• Recognize the needs for near real time
• Show options available in Tabular Model
• Show differences in Direct Query and In Memory mode
Really Real Time?
The definition
Real-Time
— adj
denoting or relating to a data-processing system in which a
computer receives constantly changing data, such as
information relating to air-traffic control, travel booking
systems, etc, and processes it sufficiently rapidly to be able
to control the source of the data
Source: Collins Dictionary
Source: http://www.flickriver.com/photos/swissmiss/tags/switzerland/
real-time business requirements

a second?
a minute?
a jiffy?
an hour?
a day?
Why?
 what’s the urgency?
Source: http://health.howstuffworks.com/medicine/surgeries-procedures/about-weight-loss-surgery-ga.htm
what’s the urgency?
A Quick Recap
xVelocity, Tabular Model and SSDT
 xVelocity

xVelocity in-memory xVelocity Relational

analytics engine

Power Pivot Tabular ColumnStore Index

Vertipaq vs ColumnStore White Paper http://www.sqlbi.com/articles/vertipaq-vs-columnstore/

Tabular Development Ecosystem

Develop

Workspace Tabular

Deploy

Configure

Deployed Tabular
Query
SQL Server Data Tools (SSDT)
• Use at least the VS2012 version
• Can use VS2013 but be careful for early teething issues
Setting Up The Project
Clock-wise rule of thumb

1. Configure Model.bim properties

2. Configure Project Properties
3. Import from Data Source

(Don’t forget to add the Solution to Source Control)

Model.bim Properties
Build Action
• Compile
• None
Copy to Output Directory
Data Backup
• Do not backup to disk
• Backup to disk (creates ABF file)
DirectQuery Mode
• Off
• On
File Name
Model.bim Properties
Workspace Database*
• [Project Name]_[User]_[GUID]
Workspace Retention
• Unload From Memory
• Keep In Memory
• Delete From Workspace
Workspace Server
• Recommended local server
Deployment Options
Processing Option
• Default, Do Not Process, Full
Transactional Deployment:
• False, True
Query Mode
• In-Memory
• Direct Query
• Direct Query with In-Memory
• In-Memory with Direct Query
Deployment Server
Server (should be Dev / Test)
Database
Cube Name

Direct Query Impersonation

Settings
• Default
• Impersonate Current User
Data Source Types
Relational Databases

SQL Server, Azure SQL Database

Microsoft Access

Oracle, Teradata, Sybase, Informix, IBM DB2, ODBC / OLEDB

Multidimensional Sources

Analysis Services Multidimensional

Data Source Types
Data Feeds

Microsoft Reporting Services Report

Azure DataMarket Dataset

Atom Feeds from a public or corporate provider

Text Files

Microsoft Excel

Plain text files

Data Source Types
Data Feeds

Microsoft Reporting Services Report

Azure DataMarket Dataset

Atom Feeds from a public or corporate provider

Text Files

Microsoft Excel

Plain text files

Design Modes
In-Memory and DirectQuery
Tabular Solutions
Authoring
(SSDT) Deployment Option

Database
Properties

Connectivity
Tabular Solutions
Query Mode:
• DirectQuery
DirectQuery Mode:
• DirectQuery with In-Memory

=
• On
• In-Memory with DirectQuery
• Off
• In-Memory
Design Mode
• DirectQuery
• In-Memory Database
Properties

Connection String:
• DirectQueryMode=DirectQuery
• DirectQueryMode=In-Memory
 Tabular Solutions
SSDT Deployment /
Connection String
Type (Authoring) Database Properties
DirectQuery Mode
DirectQuery Mode Query Mode

In-Memory Off (default) In-Memory (default) In-Memory (default)

DirectQuery On DirectQuery DirectQuery (default)

DirectQuery with In-Memory DirectQuery (default for DQ/IM)

Hybrid On
In-Memory with DirectQuery In-Memory (default for IM/DQ)
Tabular Model Architecture
Third Party Application Excel Reporting Services Power View

Tabular Model

DirectQuery In-Memory

SQL Server Databases Non SQL Server Files OData Cloud Services
Databases
Tabular Model Architecture
(Roughly)

Third Party Application Excel Reporting Services Power View

Not quite right DirectQuery

Tabular Model

In-Memory !
SQL Server Databases Non SQL Server Files OData Cloud Services
Databases
Querying

DAX DAX / MDX

DirectQuery In-Memory

VertiPaq SE
SQL

Process
In-Memory

Query
Role
In-Memory
Row Level Security
VertiPaq SE

Process
Data Source Impersonation
In-Memory

Query

In-Memory xVelocity in-memory analytics engine

VertiPaq SE Full features of Tabular Model

Process
DirectQuery

Query Role
DirectQuery DirectQuery Impersonation
• Impersonate Current User
• Default (Data Source login)
SQL
Data Source Impersonation
• Windows User
• Service Account
DirectQuery
SQL Server engine
Query No Calculated Columns
DirectQuery Some DAX Functions are not supported in DirectQuery:
• No equivalent calculations in relational engine
SQL • No equivalent SQL expressions
• Performance of converted expressions
• E.g. TOTALYTD, SAMEPERIODLASTYEAR
More information on BOL: http://msdn.microsoft.com/en-us/library/hh213006.aspx
Hybrid Mode

DirectQuery design
Client tool flexibility of In-Memory
How?
Implementing it in In-Memory
In-Memory

Query
Mode Database Table Partition
Process Default × × ×
In-Memory
Process Full × × ×
Process Data × ×
VertiPaq SE
Process Clear × × ×
Process Defrag ×
Process
Process Add ×
Process Recalc ×
In-Memory
Multiple mutually exclusive partitions
Query Eliminate unnecessary processing and processor
load on the AS servers
In-Memory

VertiPaq SE Current
June 2013
May 2013 In-Memory
Process
Process April 2013 Partitions
…
How?
Implementing it in DirectQuery
DirectQuery
Real time support
Query
Always connects to database backend
DirectQuery

SQL
DirectQuery
One partition in Tabular Model
Query
• Can take advantage of Partitioning and
DirectQuery
ColumnStore Index technology on
SQL Server 2012
• Partitioning / ColumnStore management
SQL done at the source database level
How?
They Hybrid Option
Hybrid Mode
Hybrid Mode

Design Mode = DirectQuery

Connectivity : DirectQuery or In-Memory
Hybrid Mode
One partition for DirectQuery
Processing Option: Never process this query
Multiple Partitions for In-Memory (No Overlap)

Current
June 2013

DirectQuery May 2013 In-Memory

Partition April 2013 Partitions
…
Hybrid Mode
Only expose the “real time part” in the Direct Query partition
Excel can read the In-Memory Partitions
PowerView can read just Direct Query partition for real time

DirectQuery
Current
Partition
June 2013
May 2013 In-Memory
April 2013 Partitions
…
Wrap Up
Factors To Consider
data sources
complexity

scalability security
reporting tools
The Keys To Success

requirements
restrictions

purpose tools
creativity
53 |
Remember

Design time
Further Reading
Using Direct Query in Tabular BI Semantic Model - http://bit.ly/1usZnzZ
Cathy Dumas (Microsoft whitepaper)
Hardware Sizing a Tabular Solution - http://bit.ly/1uKXUl0
John Simon, Heidi Steen (Microsoft whitepaper)
Securing the Tabular BI Semantic Model - http://bit.ly/1v6DaGE
Cathy Dumas (Microsoft whitepaper)
Incremental Processing in Tabular Using Process Add - http://bit.ly/1wN0Lf0
Marco Russo
Hybrid Tabular Mode – in 2 parts - http://bit.ly/1ytnsJG
Julie Koesmarno
Julie Koesmarno
SQL Server / BI Consultant
Microsoft Certified Trainer

www.mssqlgirl.com
@msSQLgirl
signal@mssqlgirl.com
Thank you Holland!