QlikView Publisher

How to size your QlikView Publisher Server

Version
Date
Author

1.0
08 September 2014
Shane M Spencer

PUBLIC

Contents
Introduction ............................................................................................................................................ 3
QlikView Distribution Service (QDS) & QlikView Batch (QVB) ................................................................... 4
Concurrent / Simultaneous Reloads & The Reload Process ...................................................................... 4
Disk, RAM & CPU Requirements .............................................................................................................. 6
Sizing Your (Publisher) Server .................................................................................................................. 7
Memory Metrics...................................................................................................................................... 8
Network Metrics ................................................................................................................................... 10
Process Metrics ..................................................................................................................................... 11
Key QlikView Processes ......................................................................................................................... 12
Web Server Metrics ............................................................................................................................... 13
Other Metrics ........................................................................................................................................ 14
List of Perfmon Counters ....................................................................................................................... 15
Perfmon Data Collector Template.......................................................................................................... 17

PUBLIC - Page 2 of 7

Introduction
Correctly sizing the Publisher Server seems to be something that is overlooked, or at best based purely
on guess work. However, it is important to understand your requirements so that your server has
enough capacity (but not large amounts of excess) and to ascertain if you should have a dedicated
Publisher Server or whether it can reside on the same physical machine as QlikView Server.
The size of the QlikView Publisher will depend on several factors: the size of the data being loaded; how
the data is loaded / document design; the number of Documents being reloaded; the number of
concurrent tasks being run.

PUBLIC - Page 3 of 7

QlikView Distribution Service (QDS)

The Distribution Service is the component that is responsible for performing the preparation and
delivery of the QlikView files. A subcomponent of the QDS is the QlikView Batch (QVB) which is
responsible for the actual Reload.
In many ways these could be considered to be the very core of the QlikView Publisher.

QlikView Batch (QVB)

When a Task is run (i.e. a reload is performed via QMC) QlikView Distribution Service will spawn a QVB
processes that consumers RAM and CPU for the duration of the Task then it will disappear. For each
running Task there will be an individual QVB process.
This can be observed in Windows Task Manager of the Publisher Server. i.e.

PUBLIC - Page 4 of 7

Concurrent / Simultaneous Reloads

Although many Tasks can be set to run at the same time it is more likely that only a few tasks will
actually be consuming RAM and CPU the rest will be queuing for these to finish. The maximum number
of Concurrent Reloads / Simultaneous Distributions can be set in QMC
i.e. QlikView environment without Publisher licence.

i.e. QlikView environment with Publisher licence.

The Reload Process

In simple terms, when a task is started the QlikView Distribution Service will open the (empty) document
in Memory, load in the data (hence consuming RAM and CPU), and then write the reloaded Document
to Disk. It is important you have enough resources to cope with your reload concurrency or that you
throttle back your reload concurrency to avoid exceeding the capacity of these resources.

PUBLIC - Page 5 of 7

Disk Requirements
As a reloaded document is being written to disk it does not over-write the existing document until the
process is complete. That means there must be enough free capacity on the disk (where the document
is Distributed to) to temporarily write the document to; once the write is complete the space consumed
by the older version of the document will be released.
An incorrectly sized environment is likely to use the Paging File (Disk that is used as Virtual Memory) so
there should be enough Disk space to allow this to grow also.

RAM Requirements
Whilst a Task is being executed a corresponding QVB process will be consuming Memory. The amount it
will consume will depend not only on the amount of data being loaded but also the design of the
document (as temporary tables may be created).
For example we have observed the following:
A QVW of 6.25GB consumes about 25GB of RAM when loaded in to Memory of the QlikView Server. The
reload process (QVB) for this same document (temporarily) consumes about 47GB of RAM at its peak. A
pre-requisite task that consolidates about 2 months of data in to QVD(s) consumes well over 100GB (and
runs for over 5 hours) when we streamlined the task to run quicker we shaved 45 minutes off the batch
but peak Memory consumption was about10-20GB more.
If there is not enough available Memory (RAM) for the reload Task(s) then Windows Server will
automatically start using the Paging File (Disk that has been allocated as an extension to the servers
virtual Memory). This is extremely bad, especially for In Memory applications such as QlikView, as
reading from Disk is 1000x slower than Memory.

CPU Requirements
Whilst a Task is being executed a corresponding QVB process will be consuming CPU. Insufficient
resources will mean Tasks are queuing for Processor (CPU) time.

PUBLIC - Page 6 of 7

Sizing Your (Publisher) Server

It is relatively easy to calculate your Disk sizing requirements; however the CPU and RAM requirements
of a Publisher Server are far more difficult to calculate. Yet these are paramount if you are to
understand if you need a dedicated Publisher Server (separate from QlikView Server) and also so you
can ensure that the Server is sized correctly with appropriate maximum Reload Task concurrency, no
matter what type of deployment you go with.
Therefore to understand your requirements you must Performance Test each reload task (running each
task in isolation) whilst monitoring CPU, Disk and Memory metrics. Windows Perfmon can be used to
collect data about CPU, Memory and Disk Utilisation. See the following document.
http://community.qlik.com/docs/DOC-7050
Logical Disk space can easily be monitored by either the % Free Space or Free Megabytes counters. It
is in the final stages of the Task writes the Document to disk so it is at this point that close attention of
disk space is required. If the documents are written to a NAS share rather than a mounted disk then
Perfmon cannot monitor this.
Processor (CPU) Utilisation can be monitored at Server level using % Process Time (and also System
Processor Queue Length) and at a Process level using the % Process Time for QVB.
Memory (RAM) Utilisation can be monitored at Server level using % Committed Bytes in Use and/or
Available Megabytes and more accurately at Process level using Private Bytes for QVB (and QVS if the
Publisher is on the QlikView Server). Paging File % Usage should also be monitored to spot when/if
paging occurs.
To ensure you have enough Memory (RAM) you will need 5-10GB for OS and other processes. The rest
of the servers physical Memory can be used by QlikView Server (QVS) and/or QlikView Publisher (QVB).
If the Memory required to run your tasks (concurrently) and host your documents in Memory exceeds
the servers physical Memory (less 5-10GB) then you should consider some or all of the following
options:

reducing your max reload concurrency

having a separate QlikView Server(s) and Publisher Server
installing more memory / migrating to a bigger server

n.b. To estimate your base Memory footprint requirements (of QVS) excluding cached results, multiple
the size on disk of you individual documents by 4.25. A more detailed analysis will need to be done to
understand the amount of memory QVS requires to support user activity but thats another story

PUBLIC - Page 7 of 7