Professional Documents
Culture Documents
TenAsys Corporation
1400 NW Compton Drive, #301
Beaverton, OR 97006 USA
+1 503 748-4720
fax +1 503 748-4730
info@tenasys.com
www.tenasys.com
Copyright © 2006, TenAsys Corporation. TENASYS, INTIME, and IRMX are registered trademarks of TenAsys Corporation. 060803
Other trademarks and brand names are the property of their respective owners.
Virtualization Background
Virtualization of computer hardware has been used for many decades. The most widely
noted early examples are those implemented by IBM on their mainframe hardware giving
their customers a means to easily upgrade from old “iron” to new “iron.” In this case a
primary goal of virtualization was to allow legacy applications to run on newer machines,
alongside applications designed for the new OS and hardware. [1]
Most modern VMMs operate without the need for CPU instruction set emulation, since the
instruction set of the virtual machine is identical to, or is a subset of, the underlying
hardware. The guest OS and applications running inside each virtual machine are native to
the underlying processor instruction set. Also, instead of perfectly emulating a specific
hardware platform, the typical VMM presents a virtual computing system that can be
replicated multiple times on a single hardware platform and contains sufficient virtual I/O
to support common client and server applications.
The INtime RTOS is a 32-bit protected-mode operating system, as is Windows XP, the
desktop OS which “shares” a single hardware platform with INtime. Running two specific
protected-mode operating systems on a single platform is sometimes referred to as “static
virtualization.” [3] In this case Windows boots first and then INtime inserts itself between
the system hardware and Windows. Memory and I/O hardware designated for use by the
INtime RTOS are “hidden” from Windows and dedicated for use by RTOS processes. All
standard user interface I/O, such as the video, keyboard, and mouse, remain under the
ownership of Windows. Windows is unaware that the INtime RTOS and its processes exist,
even as it is relegated to the lowest-priority task in the INtime task list.
The static virtualization method used by INtime succeeds where the conventional approach
to virtualization used by traditional VMMs simply will not work; that is, for real-time
applications. Like a general-purpose OS, such as Windows or Linux, a conventional VMM
must be “fair” in its approach to scheduling CPU time for virtual machines and allocating
I/O resources among the virtual machines. The conventional VMM is targeted at solving
problems for a corporate IT network: maximizing the use of server resources and simplifying
the deployment and maintenance of client desktops. The “IT virtual machine” presented by
a conventional VMM is not capable of supporting the deterministic scheduling and
dedicated I/O required of real-time applications and their operating systems.
The I/O presented by a conventional IT virtual machine usually consists of a CPU, RAM,
disk, video, keyboard, mouse, and a network interface. A few other generic I/O devices may
be available to “install” inside the IT virtual machine, but due to the difficulty associated
with multiplexing a wide range of I/O between multiple virtual machines, most virtual
machine I/O is limited to these basic devices. Despite these limitations, this I/O
complement satisfies a large percentage of IT server and desktop applications, making this
form of virtualization popular with corporate IT groups as a means to quickly buildup and
teardown server applications and client desktops.
The downside to such VMM trickery is a decrease in performance and limited guest OS
compatibility. For example, binary files of a guest OS may be modified to trap supervisor-
level CPU instructions, requiring the VMM to emulate these instructions. Instruction
emulation slows down the execution speed of the guest OS, and the need to “fix up” binary
files limits your guest OS options to those that have been certified for use with the VMM.
Likewise, problems exist in the area of address translation and interrupt masking where,
again, the software solutions result in performance overhead that cannot be tolerated by
real-time applications.
Low interrupt latency, direct access to specialized I/O, and the assurance that a VMM won’t
“time slice away” the determinism and priority of real-time tasks are all key requirements of
a real-time virtual machine. The combination of multi-core CPUs and Intel VT are an ideal
platform on which to move beyond multi-OS real-time systems that utilize static
virtualization and build a real-time hypervisor based on dynamic virtualization.
A real-time hypervisor is a VMM that uses hardware virtualization technology to isolate and
simultaneously host general-purpose operating systems and real-time operating systems.
Unlike static virtualization (described above), the dynamic virtualization implemented by a
real-time hypervisor uses an “early start” technique, taking over complete control of the
hardware platform. This means that guest operating systems are allowed to “boot” only after
the real-time hypervisor has constructed a virtual machine for them.
There are two significant advantages to building a multi-OS real-time system by using
dynamic virtualization rather than static virtualization:
A wide range of operating systems, both general-purpose and real-time, can be
supported.
The boot sequence for each guest OS is under the control of the hypervisor.
This second advantage means it is possible to restart one guest OS while other guest
operating systems continues to run without interruption.
A key difference between the TenAsys real-time hypervisor and conventional VMM
solutions is how physical resources are allocated to each virtual machine. Resources, such
as CPU cycles, RAM, I/O, and interrupts, must be allocated by any VMM. In the simplest
case, a conventional VMM evenly multiplexes these resources between the virtual
machines. A conventional VMM may allow priorities to be assigned to specific virtual
machines, but these “tuning parameters” only slightly modify the bias of a system that, by
design, attempts to fairly distribute physical resources among all the virtual machines on a
given hardware platform.
When determinism and performance are more important than equal access, the processor
virtualization features can be used to isolate resources for use by a specific virtual machine
and its guest OS rather than to create virtual I/O for shared access between multiple virtual
machines.
Similar arguments can be made for access to RAM. In a conventional VMM some or all of
the memory in each virtual machine could be swapped to disk, in order to more efficiently
allocate limited physical RAM among multiple virtual machines. An RTOS, and all of its
real-time applications, must always be resident in physical RAM. The TenAsys real-time
hypervisor guarantees that each real-time virtual machine is locked into physical RAM, and
is never swapped to disk. This is the only way to insure that every real-time event is
serviced consistently, with deterministic timing.
Exclusivity of I/O doesn’t apply only to a real-time virtual machine. Graphics intensive
applications need access to real hardware for maximum performance. For example, a virtual
frame buffer may be too slow and inadequate in features for an application that renders 3D
moving images. In that case, the virtual machine containing the GPOS needs direct access to
the physical frame buffer and its control I/O.
Dedicating the physical video frame buffer to a single virtual machine guarantees optimum
video features and performance for the GPOS and applications running inside that virtual
machine, at the expense of leaving all other virtual machines with no output console. A
serial console or a virtual frame buffer can be presented to the remaining virtual machines,
depending on their video output needs, with the output of the virtual video devices
displayed in a window on the user interface of the virtual machine containing the dedicated
video buffer. This approach requires a video helper function residing on the virtual machine
that contains the physical frame buffer.
For those systems where virtual frame buffers have sufficient performance and features for
all virtual machines hosted by the hypervisor a simple keyboard switch (e.g., Ctrl-Alt-Fn)
can swap between virtual displays, showing the output of any one virtual buffer at a time on
Many systems exist today that depend on real-time code written many years ago. These
legacy real-time applications continue to be used because they work; they are proven, they
are reliable, and they may even be certified for an application (e.g., medical equipment,
defense, and aerospace). Unfortunately, these legacy applications may also be running on
obsolete hardware or be in need of an update, such as a graphical user interface or access to
an enterprise network. Rewriting a proven and/or certified real-time application is rarely
desirable or economical.
The TenAsys real-time hypervisor can directly host legacy real-time applications or host a
legacy RTOS and its application(s). With minimal modification to existing code, new
features can be added to the legacy application by simultaneously hosting a GPOS, thereby
enabling developers to maximize code preservation and maintain performance. Interprocess
communication, facilitated by the real-time hypervisor, is used to augment the legacy
application by adding new features via processes running in parallel virtual machines.
Running legacy RTOS code in a virtual real-time machine is also how legacy real-time code
can be migrated from obsolete hardware to modern embedded platforms. Because I/O can be
virtualized, it is possible to simulate old hardware devices, minimizing rewrite of proven
legacy code. For example, a VMEbus system could be converted to a less expensive single-
board computer system by intercepting I/O requests to VMEbus I/O and redirecting it to
equivalent on-board I/O devices.
Interrupt latency
The ability to monitor interrupts at all times, regardless of the state of their mask bits inside
a virtual machine, insures that real-time interrupt latencies are minimized and that virtual
machines cannot interfere with each other, even if they share hardware devices on a single
interrupt line.
The use of the TenAsys real-time hypervisor is not limited to just “dual-OS, single platform”
applications. Three or more virtual machines can be hosted by the real-time hypervisor. For
example, a single GPOS in one virtual machine and two real-time virtual machines, each
containing a dedicated RTOS.
Multi-Core CPUs
Multi-core processors are composed of multiple CPU cores in a single-socket package. Intel
Core microarchitecture multi-core processors access their external bus via a shared level-
two cache, called the Intel® Smart Cache. Multi-core technology runs at lower clock speeds
than single-core processors, reducing overall power consumption—an important
consideration for embedded applications. [5]
From a software application perspective, multi-core systems look just like a multi-socket,
symmetric multiprocessor platform (SMP); a server hardware platform that has been
available for many years. The key differences between multi-socket SMP platforms and
single-socket multi-core platforms are cost, size, power consumption, and availability to the
embedded market. Most general-purpose operating systems, such as Windows XP and
Linux, are designed to operate on and take advantage of SMP machines.
Real-time virtual machine solutions derive maximum benefit from multi-core processors.
For example, on a dual-core processor the TenAsys real-time hypervisor dedicates one CPU
core to the RTOS and the other to the GPOS; providing real-time applications with 100% of
the CPU instruction cycles of that core dedicated to the RTOS. The CPU cycles of the
remaining core are used exclusively by the GPOS virtual machine. Contention for key CPU
resources, such as pipelines and the FPU, are avoided, maximizing performance and
responsiveness of both operating systems. Coordination between the RTOS virtual machine
and the GPOS virtual machine is managed by the real-time hypervisor which uses built-in
interprocessor interrupt mechanisms, eliminating inter-OS context switch times.
Removing contention for resources in a multi-OS platform has a dramatic impact on real-
time performance metrics, such as interrupt latency. TenAsys has measured a 10 to 1
improvement for interrupt latency on dual-core platforms compared to equivalent clock
speed single-core platforms, latencies measured in the 10-30 microseconds range on single-
core systems are reduced by an order of magnitude to 1-3 microseconds on dual-core
systems. With such low latencies real-time application control loops can execute in the 50-
200 microsecond range with very high precision. Multi-core processors mean faster scan
times can be implemented and higher quality controllers can be deployed.
Industrial Control
Take, for example, a Windows-based industrial control application where the real-time tasks
run on a PLC engine that must operate continuously, and with very precise timing.
Windows is used as a man-machine interface to set important parameters, monitor system
status, provide disk logging, and act as an interface to the factory enterprise network.
Normally, two hardware platforms would be employed because the Windows OS can
guarantee neither continuous operation nor precision timing to satisfy the needs of the PLC
engine.
Combining the two functions on a dual-core hardware platform saves time, money, and
space and simplifies development and communication between the two parts of the system:
the Windows enterprise interface and the PLC control engine. The TenAsys real-time
hypervisor hosts each part in a separate virtual machine, where each virtual machine has a
dedicated CPU core to insure timeliness of response and ample performance.
Because Windows and the PLC engine reside in separate virtual machines, the PLC engine is
assured of its need for continuous operation, regardless of the state of Windows. This means
that Windows can be rebooted, for whatever reason, without impeding operation of the PLC
engine. By dedicating a CPU core to each virtual machine, the real-time integrity and
performance of the PLC engine is also maintained.
Legacy Real-time
Legacy real-time systems are a prime application for the TenAsys real-time hypervisor.
Proven real-time applications can be hosted in a guest real-time virtual machine with little
or no modification—enabling developers to maximize code preservation and maintain
performance while using the real-time hypervisor to add new features via a parallel general
purpose operating system. Helper functions can be added to the real-time virtual machine to
communicate with the GPOS virtual machine for coordinating between the existing legacy
code and new feature code. Shared memory or a virtual Ethernet channel can be used to
communicate between the two environments. If necessary, obsolete hardware can be
simulated because the VT-x technology allows the hypervisor to isolate I/O accesses.
DSP Replacement
Conclusion
The net gains from the application of TenAsys real-time virtual machine technology on Intel
multi-core processor platforms are elimination of redundant computer and communication
hardware, faster communication and coordination between RTOS and GPOS subsystems,
improved reliability and robustness, re-use of proven legacy applications, and simplified
development and debugging. The ability to reboot one OS while the other runs without
interruption promotes significant cost savings by condensing systems comprised of separate
GPOS, DSP, and real-time hardware subsystems onto a single, multi-core, hardware
platform.
Multi-core processors easily support multiple operating systems and high-performance, low-
latency, real-time applications by dedicating a CPU core to the RTOS. The CPU instruction
cycles of the RTOS core are available 100% of the time for its real-time applications.
Contention for key resources, such as CPU cycles, pipelines, and the FPU, are avoided. Intel
Virtualization Technology is used to remove I/O contentions by isolating and dedicating
those devices that belong to the RTOS for exclusive use by real-time applications. I/O that
can be shared, like keyboard, mouse, and enterprise network interfaces, are presented as
virtual devices for use by all virtual machines.