MSc Dissertation

OpenRouter
An OpenCOM-based Component Framework for Structuring Software Routers

Author: Kevin Lee BSc (Hons) Supervisor: Dr Geoff Coulson Date: 13th Sept, 2002

I certify that the material contained in this dissertation is my own work and does not contain significant portions of unreferenced or unacknowledged material. I also warrant that the above statement applies to the implementation of the project and all associated documentation. In the case of electronically submitted work, I also consent to this work being stored electronically and copied for assessment purposes, including the department's use of p lagiarism detection systems in order to check the integrity of assessed work.

Date: 13/9/2002

Signed: Kevin Lee ________________

Abstract This project is concerned with evaluating the viability of using component frameworks and reflective techniques to build software routers. This dissertation reviews related research, outlines a generic framework for designing software routers in general then focuses on IP routing. The project involves the design and implementation of a system which uses the low level packet filtering facilities of the Windows NT (5) Networking Stack together with a Component Framework to structure a number of components into a software router. The report evaluates the developed system to see if it achieves the ideal of allowing a high performance modular router to be reconfigured at run-time. This report shows that the system has increased functionality, ease of programmability and modularity over existing monolithic software routers and it performs on par with existing systems.

Acknowledgements I would like to take this opportunity to thank a number of people who helped during the duration of the dissertation and in the preparation of this report. I would like to thank Geoff Coulson for the guidance given in supervising me for the duration of the project. I would also like to thank Geoff and Gordon Blair for there excellent module during the MSc Course, which led me into this field. I would like to thank Stefan Schmid and Tim Chart for supplying the Lara++ code and allo wing me to use it and Paul Grace for helping when I got stuck with OpenCOM. I would finally like to thank Danny Hughes, Phil Greenwood and Matthew Pierce for there general comments during the dissertation and during the final stages of the write-up.

Contents

Table of Figures.......................................................................................................................................................7 1 INTRODUCTION...............................................................................................................................................8 2 BACKGROUND..................................................................................................................................................9 2.1 Routing............................................................................................................................................................9 2.1.1 What is Routing?..................................................................................................................................9 2.1.2 Router Hardware ................................................................................................................................11 2.1.3 Router Programming/Configuration ...............................................................................................12 2.2 Component Based Programming............................................................................................................ 13 2.3 Component Frameworks........................................................................................................................... 13 2.4 OpenCOM ................................................................................................................................................... 14 2.5 P1520........................................................................................................................................................... 16 3 REQUIREMENTS ........................................................................................................................................... 17 4 THE NETKIT PROJECT.............................................................................................................................. 18 5 RELATED RESEARCH................................................................................................................................ 20 5.1 NodeOS........................................................................................................................................................ 20 5.2 Lara++........................................................................................................................................................ 21 5.2.1 Overview 21 5.2.2 Evaluation22 5.3 Router Plug-ins.......................................................................................................................................... 22 5.3.1 Overview 22 5.3.2 Evaluation23 5.4 Click ............................................................................................................................................................. 24 5.4.1 Overview 24 5.4.2 Evaluation25 5.5 NetBind........................................................................................................................................................ 25 5.5.1 Overview 25 5.5.2 Evaluation26 5.5 Vera .............................................................................................................................................................. 26 5.5.1 Overview 26 5.5.2 Evaluation27 5.6 Bowman....................................................................................................................................................... 27 5.6.1 Overview 27 5.6.2 Evaluation28 6 DESIGN .............................................................................................................................................................. 29 6.1 Component Framework Design............................................................................................................... 29 6.2 Case Study: IPv4 Component Framework ............................................................................................ 33 6.3 State Management..................................................................................................................................... 36 6.4 Configuration and Deployment............................................................................................................... 37 6.5 Enabling Dynamic Reconfiguration ....................................................................................................... 38

7 IMPLEMENTATION ..................................................................................................................................... 39 7.1 Overview...................................................................................................................................................... 39 7.2 System structure......................................................................................................................................... 40 7.3 Packet filter................................................................................................................................................. 44 7.4 Buffer mechanism ...................................................................................................................................... 45 7.5 Threading.................................................................................................................................................... 46 7.6 Remote management ................................................................................................................................. 46 7.7 Project Structure and Build process....................................................................................................... 47 7.8 System Operation....................................................................................................................................... 48 7.8.1 Configuring the system.....................................................................................................................48 7.8.2 Dynamic reconfiguration ..................................................................................................................49 8 EVALUATION ................................................................................................................................................. 50 8.1 Performance Evaluation........................................................................................................................... 50 8.1.1 Lara++ packet filter overhead..........................................................................................................50 8.1.2 OpenRouter Performance .................................................................................................................52 8.1.3 OpenRouter Extensibility .................................................................................................................53 8.2 Qualitative Evaluation.............................................................................................................................. 55 8.3 Requirements Evaluation.......................................................................................................................... 57 9 CONCLUSIONS ............................................................................................................................................... 60 9.1 Project Review............................................................................................................................................ 60 9.2 Retrospect.................................................................................................................................................... 61 9.3 Future work................................................................................................................................................. 61 9.4 Personal Conclusions................................................................................................................................ 62 REFERENCES ..................................................................................................................................................... 63 GLOSSARY .......................................................................................................................................................... 65

Working Documents Available at URL: WWW.KEVIN-LEE.CO.UK Including the current version of all software developed and used and electronic copies of this report

Table of Figures Figure 2.1 an IP Version 4 packet..................................................................................9 Figure 2.2 a router connecting a LAN to the Internet ..................................................10 Figure 2.3 a router connecting two sections of the Internet.........................................10 Figure 2.4 a router within the Internet .........................................................................10 Figure 2.5 an OpenCOM component ...........................................................................14 Figure 2.6 P1520 Architecture .....................................................................................16 Figure 4.1 NETKIT reference architecture programmable networking ......................18 Figure 5.2 Lara++ Architecture ...................................................................................21 Figure 5.3 Extended Integrated Services Router .........................................................23 Figure 5.4 Router Plug- ins System Architecture and Control Communication ..........23 Figure 5.4 a click element ............................................................................................24 Figure 5.5 an example click configuration...................................................................25 Figure 6.1 Component Framework Design..................................................................30 Figure 6.2 Component Template .................................................................................31 Figure 6.3 Inheritance hierarchy of router designs ......................................................32 Figure 6.4 IPv4 Component Framework Design .........................................................33 Figure 6.5 Example configuration interface ................................................................34 Figure 6.6 Push interface .............................................................................................35 Figure 6.7 Push Parameter Struct.................................................................................35 Figure 6.7 a system configuration template .................................................................37 Figure 7.1 OpenRouter System Overview ...................................................................40 Figure 7.2 Implementation of IPv4 Component Framework .......................................41 Figure 7.3 OpenRouter Data Flow...............................................................................42 Figure 7.4 Push function..............................................................................................43 Figure 7.5 Implemented System Interface ...................................................................44 Figure 7.6 Structure of Implemented System ..............................................................47 Figure 8.1 Packet Filter Overhead System Architecture Test......................................51 Figure 8.2 Lara packet filter overhead tests. ................................................................51 Figure 8.3 OpenRouter performance test. ....................................................................52 Figure 8.4 the Null Push Method Implementation.......................................................53 Figure 8.5 Extensibility test results..............................................................................54 Figure 8.6 Ethernet Switch configuration....................................................................55

1 INTRODUCTION Recent years have seen a massive growth in the Internet and the number of nodes with access to it. These nodes largely communicate with each other using the Internet Protocol (IP) which gives each node a unique IP address. These nodes are grouped together into smaller subnets, which are linked together with routers which allow communication between these subnets. With more and more nodes on the Internet, there is a need for more and more routers to route traffic on the Internet. And with the advent of Multimedia and distributed computing, routers are being asked to do more and more complex tasks. These include QoS technologies like intserv, diffserv and RSVP, in-band media filters that can alter multimedia in the network, and other mechanisms like firewalls, security mechanisms and even application-level mechanisms for tasks like multicast. It is essential therefore that a router be as efficient, as flexible and as configurable as possible. There are a number of different types of architectures for routers, and as such a number of different standards for programming the software on them have emerged, both proprietary and forum based. Unfortunately this has lead to a certain level of interoperability problems with actually programming the devices, with some devices utilising closed proprietary APIs and some using internationally standardised APIs. This is not such a problem for simple routers which just blindly route packets between subnets but when a certain level of intelligence is needed for deploying more complex infrastructures like resource reservation schemes it may be difficult to implement and deploy on a number of different devices. A common standard across types of devices is therefore needed. Another factor in the design of IP routers is that as the Internet expands and its uses increase there will be a need to change a routers characteristics after its deployment, as taking a router down will inevitably be an expensive operation. This is because routers are often central points in any network and taking it down involves rerouting large amounts of traffic, or even worse dropping traffic. If the router was a edge router of a commercial network a significant amount of business could be lost. This project aims to address these issues by developing a component-based modular software router which is dynamically reconfigurable after deployment. It consists of an evaluation of a component framework built using Lancaster Universities OpenCOM component model which have reflective capabilities in order to enable run-time reconfiguration. Section 2 presents a background of the technologies used in this project and the field in general. Section 3 outlines the requirements of the project. Section 2 outlines and discusses the NETKIT project which this project is an investigative precursor for. Section 5 then discuses a number of papers and systems related to this project. Sections 6 and 7 then detail the design and implementation of the system. Section 8 discusses the testing and evaluation of the system and its presented ideas and theories. And finally section 9 presents the conclusions of the report as a whole.

2 BACKGROUND This section will try to give the reader a better understanding of the field and the technologies involved. This project involves routing so the first section will give the user a concise but as detailed explanation of routing as possible. It will present the different techniques for routing and how a router is designed and configured and also detail some of the current advances in routing. Section 2.2 and 2.3 detail the techniques of component-based programming and component frameworks respectively and how they are used. Section 2.4 describes in detail the Component model used in this project OpenCOM. Section 2.5 gives an overview of the IEEE P1520 architectural framework for designing programmable network devices. 2.1 Routing 2.1.1 What is Routing? A routers primary task is simple: to route packets from one network to another network based on a set of rules which it is assigned. However this task gets a lot more complicated when the router is asked to route Internet traffic because of the massive number of networks on the Internet. It is also more complicated because of the increasing speed at which these networks are working at. And more recently routers are being asked to do an increasingly large number of additional tasks including resource reservation and multimedia transcoding. As stated, a routers task is to route packets from one network to another, however if the focus is IP routing then it is clear that a router does more than simply route packets from one network to another network, it must actually calculate the fastest route to a packets destination based on the information it has. An IP packet as seen in diagram 2.1 contains a number of fields to aid routing, including a source address and a destination address, as well as other fields to aid Quality of Service (TOS field) and error correction (checksum).

Figure 2.1 an IP Version 4 packet.

A router performs the action of routing by utilising a routing table, which specifies where IP packets should be sent based on by looking at the destination address in the IP packet header. A routing table will indicate which network adapter would be used to forward the packet and hence which network the packet should be sent to. IP routers can also be more specific when routing traffic, for example it could route traffic from one source through a faster connection than traffic from another source because of a service- level-agreement. There are different types of routers ranging from a router which attaches a single network to the Internet (figure 2.2), a router which connects two sections of the Internet (figure 2.3) and a router which routes traffic in the core of the Internet (figure 2.3). Thus different requirements are placed on a router depending on its location on the global network.

Figure 2.2 a router connecting a LAN to the Internet

Figure 2.3 a router connecting two sections of the Internet

Figure 2.4 a router within the Internet

10

2.1.2 Router Hardware So, a router is a physical device that joins networks together and routes packet between these networks. This job can be done by several of different types of computer. The most common type of routing equipment for entirely hardware based routers specifically programmed to do a single task at deployment time. For example a router may be programmed to forward traffic from network A onto the Internet based upon a dynamically generated routing table created by a routing algorithm. This type of router is extremely efficient, reliable and proven technology usually based on propriety hardware and is such can be very expensive. Another common type of router is that based upon existing personal computing technology, primarily IBM-compatible Intel-based PCs. These can be put together for practically nothing, just inexpensive PC components with good network adapters. This together with a public licence operating system like NetBSD [NetBSD, 02] can produce an extremely efficient router for a minimal outlay. This is becoming more popular as more complex network configurations are emerging and as the pace of technologies emerging and the power of the software available increasing. This type of router is extremely flexible as the routing configuration is entirely in soft ware, there are also large amounts of open source software available to aid the system configurator. Because neither the hardware based router or the PC based router are configurable enough in the case of the hardware based router or powerful enough as in the case of the PC-based router. An emerging category of routers is that of network processors, like the Intel IXA1200 (Internet Exchange Architecture) network processor architecture [Intel, 02]. These network-processor based routers are multi-processor, multi- network interface and are extremely powerful, often capable of processing multiple streams of packets from different networks at line rates even for gigabit networks. They are much more focused on trying to deliver raw performance whilst having a large increase in functionality over existing router, including Quality of Service (QoS) functionality and advanced management interfaces. They are unfortunately very expensive (start from around 20,000 (2002)) and are still largely experimentally as they have only been available for a few years. They are also difficult to program, which often has to occur using assembler, which is timeconsuming although this produces efficient code and there is still little software available in the public sector. The design of software for network processor-based systems this has been the focus of a large amount of research recently, for example [Spalink, 01] and [Campbell, 01]. This dissertation will mainly focus on PC-based routers in regards to the fact that the implementation is based upon a Microsoft Windows NT (5.1) packet filter and the system is build using the Microsoft WIN32 compiler. However this report will also attempt to show that the design presented here will also apply to network processor architectures like that of the Intel IXA1200.

11

2.1.3 Router Programming/Configuration The amount of programming and configuration required by a router depends on what type of router it is (refer to previous section) and what is the task it is going to be doing. A hardware-based router as described in section 2.1.2 typically has a low level operating system residing in an EPROM or other flashable memory device in the router. This normally runs a either a web server, a network management software host like SNMP or a device discovery protocol like Universal Plug and Play (UPNP). In this way the system configurator will connect the device to network and power and use a standard PC to configure the router to the specific requirements using a scripting language. Suc h scripting languages are developed by the hardware vendor and are usually highly specific to the device in question. Unfortunately these languages are largely closed source, which is not available to most researches in the field; hence a division has appeared between the academic community and the commercial router vendors. The current favoured method of configuring software routers based on PC technology as mentioned in the previous section is by utilising a UNIX operating system and a kernel routing driver which is configured using a user-space utility. There are many operating systems capable of performing this type of function, including Solaris, NetBSD, FreeBSD and Linux. This type of function is primarily performed by a UNIX-based Operating System because of the ability to control resources better than other operating systems (i.e. Microsoft Windows NT) and because of the maturity and experience with those systems, as well as there increased stability. The final category of routers is network processor based routers. These are highly experimental and as such not much software is available. If a system deployer wishes to configure a network processor based router he usually has to either purchase specific software from the hardware vendor or employ a team of programmers to produce a customised piece of software. There has been some success in programming this type of router by academic researchers in the field including click [Karlin 01] and NetBind [Campbell, 01], as discussed in section 5.

12

2.2 Component Based Programming Component Based programming is a type of programming whereby implemented systems are built up of a number of discrete modules called components. These individual components each perform a discrete task, or each provide a specific function that the system as a whole needs. A component is a small part of a larger system performing a unified task. The use of components in the programming of a system has a number of advantages over other methods of programming. These include the ability to upgrade components, interchange different versions or replace components with new implementations. Examples of types of components include graphical interface components like menus and buttons, modular operating system components like memory managers or device drivers or networking software like the one described in the dissertation. 2.3 Component Frameworks A Component Framework is a collection of rules and interfaces (contracts) that govern the interaction of a set of components plugged into them [Szyperski, 98]. A specific Component Framework is targeted at a specific domain and embodies rules and interfaces that make sense for that domain. For example a component framework for a router would embody rules and interfaces for routing, buffering and processing a specific type of packet, i.e. IP packets. Component Frameworks are life support environments for plug- in components and help in configuring and dynamically reconfiguring areas of the system. They define the software architecture, a set of components and their interaction mechanisms. A dynamical reconfigurable component framework allows the reconfiguration of its set of components at runtime without having to shut down the system and risking the integrity of the network as a whole. Component Frameworks are used as a tool to structure software in a wide range of fields. They can be used to develop Application software including web browsers and Integrated Development Environments (IDE) like those built using the XPCOM Cross Platform Component Object Model being developed by Mozilla, [Mozilla, 0]. The software on communications software can benefit from being structured in a modular way. Programmable network devices can also be structured by using component frameworks, and the software router presented in this dissertation also uses a component framework to structure its components and support the dynamic reconfiguration.

13

2.4 OpenCOM There are a number of component technologies which have been developed in the last few years (refer to section 2.2), including Suns JavaBeans [Sun, 02], Microsofts COM [Microsoft, 02] and Mozillas XPCOM [Mozilla, 0]. These technologies are all capable of encapsulating the mechanisms and functionality of discrete routing components, and indeed all have been used in such systems. However, when designing a dynamically reconfigurable router there are a number of additional considerations which must be taken into account. The most important being identified by [Kon, 00], that without this type of explicit dependencies it is not always possible to determine the implications of removing or replacing a component. One such component technology which supports the idea of explicit dependencies has been developed at Lancaster Universities Computing Department, called OpenCOM. OpenCOM is a lightweight and efficient in-process component model, built atop a subset of Microsofts COM [Coulson, 00]. It also implicitly includes a mechanism for making the dependencies explicit between components. This mechanism is illustrated in figure 2.5 which shows an OpenCOM component (taken from [Coulson, 00]).

Figure 2.5 an OpenCOM component

The mechanism for explicit dependencies involves the use of receptacles. A receptacle specifies a interface which the component in question requires from another component before it can start. For example, if an OpenCOM component exposed receptacles of type A and B it would require components with interfaces of type A and B. This enables reconfiguration as when replacing a component or group of components the dependencies are explicitly encoded in the component so the component(s) will not be left without a requirement. OpenCOM also introduces basic mechanism- level functionality for reconfiguration, including mutual exclusion locks to serialise modifications of inter-component connections [Coulson, 00].

14

COM is a good basis for a lightweight component model because it is standardised, well understood and widely used [Coulson, 00]. It is language independent so it can be developed for a wide range of systems and it is also much more efficient than other component models like JavaBeans or the CORBA [OMG, 02] Component Model (CCM). OpenCOM ignores the higher- level features of COM such as distribution, persistence, security and transactions [Coulson, 00], and utilities a number of more low- level features. OpenCOM only utilises COMs core features because the component models design is intended to aid the implementation of higher level features in the middleware environment constructed from components. OpenCOM uses COMs binary- level interoperability standard which utilises the vtable data structure allowing function calls and object access. It also uses Microsofts Interface Definition Language (IDL) allowing skeleton and stub creation, COMs globally unique discovery technique (known as GUIDs) and COMs IUnknown interface for interface discovery and reference counting. It also adds support for Pre- and Post- method call interception which enables the injection of monitoring code [Coulson, 00], and adds support for other reflective techniques. The OpenCOM runtime manages the inter-component connections and the OpenCOM system itself. If a component is replaced it will insure that all the depend encies (receptacles) have been satisfied before starting it. OpenCOM is still a highly experimental component model, it has however been used to build the OpenORB v2 which is a CORBA compatible component-based middleware platform. OpenORB is a general purpose middleware platform based on a set of flexible and configurable component frameworks (CF). It basically adds the notion of component frameworks to support the nesting of components onto the functionality provided by OpenCOM. The designers note that as OpenORB is built on OpenCOM and that receptacle and interceptor based invocations should not overly affect the performance of the system then you would expect that OpenORB would perform on a par with a equally coded system without OpenCOM [Coulson, 00]. Tests comparing OpenORB using raw RPC invocations show that it does performs on a par with Orbacus which is well known as one of the fastest CORBA compliant ORBS. And it was only slightly beaten by GOPI which is an experimental ORB build using C in UNIX and is what OpenORB was based upon, except GOPI is less flexible. This therefore suggests that OpenCOM performs relatively will when pared with Component Frameworks for networking tasks, this makes a good case for using OpenCOM in other similar fields to middleware, including in this reports case, software routers. OpenCOM has been used successfully for this project for the reasons specified and it will be the core technology for the newly commissioned NETKIT project [Coulson, 02] as described in section 4.

15

2.5 P1520 In the design of router software architectures like the one presented in this dissertation it is wise to loosely base it upon an industry standard in order for it to become acceptable in the field and for it to be as compatible as possible. One such standard currently being developed is that of the IEEE P1520, which is being back by a number of important router hardware and software manufacturers including Bell Atlantic, Hitachi, Intel, NEC and Nortel Networks. This section tries to give a quick overview of the draft standard which will then be discussed in later sections. The IEEE P520 [P1520, 98] is a set of draft standards which describe an architecture by which all programmable routers and switches should be designed. One such draft, P1520/TS/IP-003 specifies a framework for designing IP based routers. The architecture is based around a four layered approach, as illustrated in figure 2.6 taken from [P1520, 98].

Figure 2.6 P1520 Architecture

The lowest layer in the architecture represents the underlying facilities of a programmable router, including network interface cards/memory/cache and processor as well as any low- level machine software. The data flow occurs at this level as indicated in the diagram. The Control and Manageme nt (CCM) interface allows higher level layers to access the low level facilities in the hardware layer. It also provides an abstraction over the specific architecture of the hardware device in question. The layer above the hardware layer contains a software representation of routing resources including any abstract buffer mechanisms, abstract functions and interface representations. It exposes the L-interface which is effectively an abstract router resource interface upon which any router can be programmed without the need to be aware of the specific hardware details. The layer which utilizes the Lower interface and which exposes the upper interface contains the majority of router functionality, all the programmability for in-band and out-of-band packet processing occurs there. The upper interface exposes the functionality of the router implementation and allows higher level management to occur in the top layer of the framework.

16

3 REQUIREMENTS In the proposal and during the initial stages of development of this project a number of requirements have emerged as being important. There are two main aims to the project; these are the first two requirements. 1. To design and implement a component framework to use as the basis in the construction of a software router. 2. To investigate the viability of dynamically reconfiguring routers at run-time with the use of component frameworks. There are also a number of secondary requirements for the system, fulfilling these requirements would produce a system with added value. 3. To enable the deployer to deploy the system as efficiently as possible with the minimal amount of training possible and with a minimal amount of time. 4. The implemented system based on the designed component framework must have comparable performance to existing monolithic and active routers. 5. The Component Framework must be platform independent. 6. To create a design that is simple and efficient. 7. To create a generic component framework design that can be applied to a number of different network protocols. 8. To try and limit the number of copies that the system makes. 9. The system must deal with some aspects of integrity management 10. The System must add enough additional useful functionality for it to be useful 11. The System must be compatible with P1520

17

4 THE NETKIT PROJECT

This dissertation is in association with a larger project investigating programmable networking software design; this project, known as NETKIT has been commissioned to look at a number of issues of designing programmable networking software utilising a component-based approach. The NETKIT project acknowledges that while there has been significant work in the field of active networking and in the design of component-based programmable networking software, there remains a need for generic toolkit support to facilitate programmable networking [Coulson, 02]. It argues that a generic toolkit is needed because of the number of emerging requirements on programmable networking software, such as manageability, configuration flexib ility and the ability of reconfiguration. As well as the need to support an ever growing number of services like Intserv, Diffserv, other QoS technologies, firewalls, in-band filters and virtual networks. The NETKIT manifesto [Coulson, 02] identifies four specific stratum of networking software, illustrated figure 4.1 taken from [Coulson, 02].

4: coordination 3: application 2: in-band functions 1: hardware

Figure 4.1 NETKIT reference architecture programmable networking

The four stratums in figure 4.1 describe the four distinct layers of networking programming which the project is trying to encompass in a single component-based toolkit. This dissertation focuses primarily on stratum two and some aspects of stratum 3, and attempts to apply some of the technologies developed during the OpenORB project detailed in [Coulson, 00] and [Coulson, 01]. OpenORB is a component-based middleware platform residing in stratum three and four of figure 4.1, it is built using Lancaster Universities Computing Departments OpenCOM reflective component model (discussed in section 2.4), which this dissertation will also be utilising.

18

The NETKIT project will focus on a number of specific issues. Support for run-time reconfiguration, in a running system without overly disrupting that system. This will mean developing new strategies for this, as the field is so weak at run-time reconfiguration. The development Additional component frameworks to supplement those of OpenORB and this dissertations OpenRouter, in order to evaluation the NETKIT strategy. Ensuring a system which is language-, platform-, and paradigmindependent [Coulson, 02].

This dissertation is meant as an investigative precursor to the NETKIT project, and focuses on a small part of tha t project, and evaluates whether component-based programming and reconfiguration techniques can be used to structure efficient programmable networking software.

19

5 RELATED RESEARCH After an extensive period of research a number of papers where identified as being important in terms of the field and in terms of the project itself. There are a number of different fields and aspects of those fields which are involved in this project; as such the papers presented here are from a number of different fields. 5.1 NodeOS NodeOS is draft standard specified by the DARPA Active Networking (AN) working group which defines an architectural model for active networks. In the words of the working group it lays out the fundamental tenets and objectives, defines the major components and interfaces which make up an active node [NodeOS, 98]. An active node in the context of NodeOS is simply a network host/router or other active network enabled device on the active network. The model described in the draft standard tries to set out a general framework for designing active network software. It states that each active node runs an operating system and one or more Execution Environments [NodeOS, 98]. An Execution Environment is a packet processing virtual machine that will process packets based upon the task it is configured to do. Each Execution Environment acts completely independent of In the draft there are a number of levels of architectures defined. The first is that a node provides a base functionality [NodeOS, 98], whose resources and configuration is managed by the node architecture. The second is a unique principle in modern networks, a network architecture deals with global matters like addressing and end-to-end management of resources [NodeOS, 98]. Within the NodeOS framework a specific protocol is defined called the Active Network Encapsulation Protocol (ANEP) which defines a packet header to allow special packets that are destined to be executed on an active node to be recognised by the network as a whole and be routed to the correct place. The Node Operating System is a layer between the Execution Environments and the underlying physical resources [NodeOS, 98]. Thus it acts similarly to other Hardware Access Layers like the Windows 2000 HAL. It also has the ability to support multiple Execution Environments rather than just a single one, adding flexibility to the framework as a whole. The standard is simple and efficient and also tries to give examples of possible uses of this technology. This standard is now used in a variety of research systems most prominently in the Bowman system which defines an operating system for a active network node, reviewed in section 5.6.

20

5.2 Lara++ Lara++ is an active networking system being developed at Lancaster University which uses a component based active router architecture to enables extensible network programming based on a progressive service composition framework. 5.2.1 Overview The Lara++ system focuses on the fact that the killer application of active networking will be the provision of services which are not known to either the hardware manufacturer or the vendor at the time of deployment [Schmid, 00]. And it identifies that this requires a higher degree of flexibility than other active router architectures. The Lara++ system utilises a component based active router architecture which enables extensible network programming based on a progressive service composition framework. The Lara++ system aims to be flexible by trading off performance, believing that the type of router this system is targeted at is one located at the edge of the network and thus requires a high degree of flexibility rather than raw performance (within limits). It is also designed to allow multiple services to co-exist within the framework whilst trying to maintain a high degree of flexibility. The system itself provides a flexible and dynamically programmable platform for the development of active applications and network services based on the concept of service composition [Schmid, 00]. It is constructed by layering active networking functionality on top of a existing router OS, which because the Lara++ architecture is general could be any suitable OS (they have currently implemented it on windows NT 5.0 and Linux). The architecture (illustrated in figure 5.2 taken from [Schmid, 00]) consists of; the NodeOS which provides low- level service routines and system policies, Execution Environments which form the management unit for resource access and security policies, Processing Environments which provide a safe and efficient execution environment for groups of active components and finally Active Components which comprise the actual active programs . The NodeOS provides access to low- level system service routines and resources. It forms an independent but tightly integrated layer on top of the router OS. The Core components of the NodeOS are the packet Interceptor and injector (used in this project) which interface with the network stack. The packet classifier which determines the type of computation required. There is also a unique memory mapping mechanism designed to avoid expensive copy operations.
Figure 5.2 Lara++ Architecture

21

Lara++ components can be either active which perform active computation or passive which provide merely static functionality that can be used by the active components. They are implemented using Microsofts COM component model so they are well understood and relatively simple to design, and utilise the well-known IUnknown interface to access component facilities. 5.2.2 Evaluation Lara components are different than what this dissertation calls a component although they share a similar structure, both being based on Microsoft COM. In this dissertation a component is a discrete part of the system, making up the overall system. A Lara component is a block of active code which can be run on a execution environment in the Lara++ system, a OpenRouter component The tests that have been carried out show that the use of PEs introduces a sizable processing overhead, but that the system has the benefits of the safe execution of potentially malicious active code. A number of optimizations have increased the efficiency of the system, including a user- level scheduler to reduce the performance hit associated with active scheduling. In general, the Lara++ system shows that with the NodeOS principles mentioned in section 5.1 it is possible to product a efficient and secure active router execution n environment based on Microsofts COM technology

5.3 Router Plug-ins Router plug- ins is a high performance, modular, extended integrated services router software architecture for the NetBSD operating system kernel; it was developed at Eth Zurich in Switzerland and at the Applied Research Laboratory in Washington University, 5.3.1 Overview Router plug- ins focuses on the increasingly important task of dynamically upgrading router software in an incremental fashion. The authors describe it as a high performance, modular, extended integrated services router software architecture in the NetBSD operating system kernel [Descasper, 98]. The architecture allows code modules called plug- ins to be dynamically added and configured at run-time. One of the novel features of the design is the ability to bind different plug- ins to individual flows. The system is based upon an Extended Integrated Services Router (EISR) in figure 5.3 taken from [Descasper, 98], which features additional components to a conventional monolithic Best-Effort architecture. Including a packet 22

scheduler, a packet classifier, security mechanisms, and QOS based routing.

Figure 5.3 Extended Integrated Services Router

The goals of the Router Plug- ins framework are the implementation of algorithms into modules called plug- ins, extensibility; so that new plug- ins can be dynamically loaded at run-time, flexibility; in that instances of plug- ins can be bound to specific flows, and importantly the system should provide for a very efficient data path, with no data copying, context switching or interrupt processing. Figure 5.4, taken from [Descasper, 98], shows the system architecture and the control communication between different components. Here the IPv4/IPv6 consists of an implementation of IPv4/IPv6 which contains the few components required for packet processing which do not come in the form of dynamic modules. The plug- ins are visible at the bottom-right and include, in this case types for dealing with IP options, packet scheduling, packet classification and security.
Figure 5.4 Router Plug-ins System Architecture and Control Communication

The plug- in control unit (PCU) manages and is responsible for the plug- ins. The Association Identification Unit (AIU) implements a packet classifier and glues the flows and plug- in instances together. The plug- in manager is a user space utility used to configure the system. The primary goal of the architecture as the authors put it was to build a modular and extensible networking subsystem that supported the concept of flows [Descasper, 98]. The authors justify this because the deployment of multimedia data sources and applications will produce longer lived packet streams with more packets per session than in common in todays environment. Also in general integrated services router architectures should support the notion of flows and build upon it. 5.3.2 Evaluation The authors performance tests show that the system has an overhead above an unmodified NetBSD kernel of 8% and a slight decrease in packet throughput, with the advantage of a very modular design. These tests however show that on Integrated Services platforms, a very flexible and modular architecture can be introduced with almost no additional processing costs.

23

5.4 Click Click is a software architecture designed to enable the building of flexible and configurable routers. Click was developed at the Laboratory or Computer Science at MIT. 5.4.1 Overview Click is a software architecture designed to enable the building of flexible and configurable routers [Karlin 01]. A click enabled router consists of a number of packet processing modules called elements. These individual elements implement simple router functions like packet classification, queuing, scheduling or network device access. For example, one such elements job would be to decrement an IP packets time-to- live field.
Figure 5.4 a click element

An element has a number of important properties. For example the element in figure 5.4 taken from [Karlin 01], has an element class which specifies the code that should be extracted when a packet is processed. It also has a number of ports; an output port on one element goes to a input put on another element, and each type of port can also have different semantics. A configuration string contains optional arguments that can be passed to the element at router initialisation time. Each element also has a number of method interfaces which allows elements to communicate at run-time. Click configurations can take a number of different forms as the y are modular and easy to extend. Different configurations considered include an Ethernet switch and an IP router, each with its own characteristics, like QoS or ECN. A Click router configuration is a directed graph with elements at the vertices and with the packets themselves flowing along the edges of the graph. For example figure 5.5 (taken from [Karlin 01]) shows a click configuration for a simple Ethernet switch, the Ethernet devices are at the top and bottom, there are queues for the outgoing packet to the devices, the modules in the middle perform simple operations on the packets.

24

Figure 5.5 an example click configuration

Click itself runs as a kernel thread inside a Linux 2.2 kernel. The kernel thread runs the router driver itself whic h then loops over the task queue to run each task. However as there is generally only a single thread running each router there is little support or possibility of support for QoS in the Click system. The Click framework includes several features that make individual elements more powerful and complex configurations easier to write. Pull connections allow the modelling of packet flow which is driven by transmitting hardware devices. Flowbased router context allows an element to locate other interesting elements. 5.4.2 Evaluation A standards compliant IP router built using click has sixteen eleme nts on its forwarding path, and using conventional PC hardware, this click IP router is capable of achieving a forwarding rate of 333,000 64-byte packets per second, indicating that such a structured architecture can achieve good performance. Although more support for QoS would be desirable and would lead to a more interesting and useful system.

5.5 NetBind NetBind is a high performance, flexible and scalable binding tool for dynamically constructing data paths in network processor-based routers, developed at the COMET group in Columbia University. 5.5.1 Overview The architects of NetBind describe it as a high performance, flexible and scalable binding tool for dynamically constructing data paths in network processor-based routers [Campbell, 01]. The system attempts to balance the flexibility of network programmability against the need to process and forward packets at line speeds. Data paths constructed using NetBind seamlessly share the resources of the same network processor. The system focuses on the fact that the higher line rate supported by a network processor-based router means that typically there is a smaller set of instructions that can be executed in the critical data path. A way to extend the functionality in the system is to utilise dynamic bindings, but this technology is not suitable for network processors because of the overhead this generates. The paper proposes one possible solution to this problem is to optimize the code produced by a binding tool once the data path composition has taken place. NetBind can create packet-processing pipelines through the dynamic binding of small pieces of machine language code. A binder actually modifies the machine language code of executable components at run-time. This results in components that can be seamlessly merged to create single pieces of code. 25

This process of dynamically generating machine code reduces the number of binding operations required for constructing data paths that are normally required for dynamic binding. In fact for fast path data composition the number of binding operations is so reduced that it is comparable to packet forwarding times.

5.5.2 Evaluation The current NetBind implementation focuses on the Intel IXP1200 network processor discussed in background section 2.1.2. It consists of a set of libraries that can interface with the IXA1200 micro engines, modify the IXA1200 instructions, create processing pipelines and can perform higher level operations. The NetBind system addresss the gap between using synthesised kernel code for constructing modular paths, and general-purpose processor architectures. The systems binding technique is optimized for network processor-based architectures and minimises the binding overhead in the critical path and thus allowing network processors to forward packets at line rates. 5.5 Vera VERA is a system developed at Princeton University which tries to create a virtual router architecture for Standard PCs and advanced network processors alike. 5.5.1 Overview VERA is an Extensible Router Architecture which attempts to address two issues, that of the increasing number of services a router must support and the increasing diversit y of router hardware which is available, by developing a system which is hardware independent and allows extensible services. The architecture aims to be extensible by allowing new functionality to be easily added to the router [Karlin 01]. It also aims to be a fully compliant router by supporting RFC1812 [Baker, 95] which defines a set of criteria for IP version 4 routers to support. As with other router architectures, VERA also aspires to be highly efficient on the particular implementation hardware. The VERA system itself consists of a router abstraction, a hardware abstraction, and a distributed router operating system [Karlin 01]. The router abstraction provides an interface which the actual routing functions can be performed. The hardware abstraction provides just enough hardware details for the router functions to be performed without making the router architecture hardware dependent. The distributed router operating system sits between the hardware abstraction and the router abstraction providing a mapping between the two and also managing communication between the abstractions.

26

Currently VERA has been implemented on a single processor architecture which employs a number of Network Interface Cards (NICs), and has been used to implement a number of different types of routers.

5.5.2 Evaluation VERA has shown impressive performance results on a simple single processor machine but has yet to be proven on a network processor-based system which it has partly been designed to run on. The Distributed Operating System contains a number of interesting features. The network processor based Systems organisation is based around a processor hierarchy, where different levels of packet processing time is executed on different levels of the processor hierarchy. The initial classification occurs on the processor controlling the network interface, if the processor requires further or more complex processing then a higher level processor will be used. The master processor controls the system and deals with any resource reservation or processing overflow. This type of architecture at the core of the operating system should make the task of deploying the system on different architectures. The Thread assignment and scheduling takes place on a per flow basis called a path by the authors and initialised by the createPath function. This clearly supplies core support for Quality of Service based threading mechanisms. Another interesting feature is the use of a packet header to describe the details of each packet and to reduce the number of expensive copy operations (this dissertation uses a similar idea, see section 6.3). So to conclude, VERA looks to be a much more practically based and promising router architecture than other routing architectures, including Click (section 5.4) and Router Plug- ins (section 5.3).

5.6 Bowman

5.6.1 Overview Bowman [Merugu, 00], utilises the System V UNIX operating System and layers active networking functionality on top in order the produce a extensible platform for active networking. Bowman is a software platform based on the NodeOS (section 5.1) designed to allow the implementation of execution environments within active nodes. The Bowman Node OS with the addition of a number of extensions to the host operating systems functionality acts like a hardware abstraction layer (HAL) over which platform independent Execution Environments can run. The Node OS, the additional functionality modules, and the Execution Environments can all be loaded dynamically at run-time.

27

An execution environment provides some form of end-to-end service which can be configured programmability. They can be either monolithic or multi-a- flow based; monolithic meaning that an individual Execution Environment has a single process per flow, whereas the multi-a-flow approach uses a single control process per flow which can spawn multiple packet flows. The Execution Environments actually contain control-code and sets of rules that implement the user-network interface supported by the Execution Environment [Merugu, 00]. Bowman utilise channels as communication end-points that support sending and receiving packets via an extensible set of protocols [Merugu, 00]. By using an extensible set of protocols accessible using channels, bowman enables the ability to perform a number of processes on packets including forward error correction and compression. Bowman also provides a number of state storage mechanism which expose a interface which enables the dynamic creation of registries which provide bowman flows with the ability to share data with other flows. 5.6.2 Evaluation Bowman has been specifically implemented for the CANEs Execution Environment although it is the architects believe that it provides a general platform over which other Execution Environments ma y also be implemented, however this has yet to be fully proved. Bowman successfully manages to, by the use f a dynamic HAL- like layer (the NodeOS) provide varying degrees of abstraction over hardware to the Execution Environments. For example the authors note that providing active applications with fine-grained control over scheduling requires a greater degree of access to output and processor decisions than is traditionally provided in a general-purpose operating system [Merugu, 00]. Tests show that bowman performs well under that of other systems. For example with a packet size of 400 bytes, Solaris has a throughput of 60Mbs whereas bowman achieves just 25mbs. However when the packet size is increased to 1000-1400 bytes, Bowman performs on a par with Solaris, which seems to indicate that the differences in performance is due to the decision mechanism in the respective systems rather than the ability of the systems to handle raw packet throughput. As with any system like Bowman, which is experimental and primarily focused on flexibility and configurability, Bowman cannot match mature routing products which are not as flexible. However Bowman achieves a good balance between these issues with a well performing system.

28

6 DESIGN As discussed in section 2.1, a routers primary task is to route packets from one network to another network based on a set of rules which it has. The challenge laid out in the requirements of this dissertation is to design a component-based dynamically reconfigurable IP router in which to evaluate a number of techniques. The first step (6.1) in the design of this system is the defining of a component framework as discussed previously in section 4.3. An example (6.2) of an IPv4 router will then be used to detail a specific component framework. Issues of efficient state management will then be highlighted together with how this design solves them. Then system configuration and deployment will be discussed before highlighting some issues regarding the mechanisms for dynamic reconfiguration. 6.1 Component Framework Design The designed component framework contains a number of discrete components, and defines the rules for how these components are built, how they interact with each other and with the framework and also how any processing occurs. A component framework contains a number of components, and there is a number of different ways which these components can be connected together to form a system. [Johnson, 02] uses a technique to structure OpenCOM-based middleware components; this technique which is called a pipeline consists of a linear structure with additional pipelines branching out from the main pipeline. Components in the data path can decide the direction which a data packet may take. In contrast [Schmid, 02] uses a much more dynamic mechanism to structure component connections for a active component space. Each component can be connected to any other components and loops can even be introduced depending on the mechanisms of individual components. These highly dynamic structures are fine for systems which are not heavily performance critical, but for software routers which are highly performance critical there is to much overhead associated with this type of structure. So in designing this component framework the first consideration was to make it as simple as possible without sacrificing functionality. The component framework design (figure 6.1) consists of a chain of components, linked together in a linear fashion. This is the simplest design possible other than a monolithic design, and should present a minimal performance hit over a monolithic router. This linear design may seem restrictive but it is also possible using OpenCOMs multi-pointer receptacles to create multiple paths similar to [Schmid, 02], this however should be discouraged because of the overhead associated with the decision mechanisms needed and additional inter-component connections. This linear component structure within the component framework is the in-band packet processor; every packet will be processed by each component in the chain. Any out-of-band configuration, like resource reservation or QoS setup would occur by a user level application accessing the component framework system interface which would then intern configure the system by using the configuration interfaces of the individual components.

29

Figure 6.1 Component Framework Design.

The component framework consists of a packet filter and a number of components. A packet filter is effectively the entry point to the system; it is linked to a external packet interceptor mechanism specific to the operating system and the networking stack being used. This is the single component which has to be tailored to a specific platform; this place helps to maintain platform independence. A system developed using this component framework template should theoretically be able to run on any platform, as apart from the packet filter, the components depend on networking standards rather than platform facilities. So to deploy an existing system on another platform, all that would be necessary would be to replace the single packet filter component. Each component (figure 6.2) declares a number of interfaces and receptacles (defined in section 2.4). The interfaces consist of a number of COM and OpenCOM interfaces to enable OpenCOM functionality, a unique configuration interface which exposes a components specific functionality to the component framework and external entities. There is also a single push interface on each component which corresponds to a receptacle on each component. The decision to use push was a early decision in the design process, which follows on from the decision to use a linear structure of components. The packet filter component will receive a packet from the network (or more likely from the operating system) and then push a processing request (push function call) onto the next component in the chain. This is a simple and elegant approach to packet processing. Experimentation was also given to the possible inclusion of a pull interface in the framework, but in the implementation process (detailed in section 8), but I found that pull was unnecessary as I could not find a legitimate reason where it wo uld be more efficient to use than Push.

30

In this way each component appears to be identical to the component framework apart from the configuration interface.

Figure 6.2 Component Template

The design presented so far has been a template specification for any type of router using a component framework, a packet filter and a number of components linked together in a linear structure. In order to design a component framework for a specific type of router (i.e. an IPv4 one) the framework illustrated in figures 6.1 and 6.2 must be extended to support the specific type. If we call the illustrations in figures 6.1 and 6.2 the root template for a generic router specification, we can then derive other router specifications from that template. Thus the root template for any type of software router based on this dissertations design is the same. More specifically, if the focus is on a specific router type, i.e. an IP one, then the root template can be used as a basis for another specification, hence the IP router inherits general functionality from the template router. In tern a more specific router would inherit specific functionality from the IP router, like and IPv4 one or an IPv6-IPv4 tunnelling router. Thus a tree of router templates is built up in a structure as in figure 6.3 below. So an IPv4 router will share the functionality defined for a general IP router and a root router. This means in practice that the root router has a set of rules which define the set of components and the behaviour of those components, a general IP router has this functionality and also extra rules, components and behaviours specific to IP routing.

31

Figure 6.3 Inheritance hierarchy of router designs

The implementation discussed in section 7 is IPv4 based so I will next present the component framework design for IPv4.

32

6.2 Case Study: IPv4 Component Framework The component framework design in figure 6.4 is for a generic IPv4 router and is based upon the design specified in the previous section for any generic router. It additionally specifies the exact interfaces a component must implement to allow the IPv4 component framework to utilise it. It also specifies the minimum set of components that must exist within an implemented framework to comply with the specification.

Figure 6.4 IPv4 Component Framework Design

The components in figure 6.4 are these minimum set of components, in that without these minimal components the framework would be meaningless. They include components that would be present in any IP router. The packet filter has access to the network subsystem of the operating system or network card (NIC) directly and filters IP version 4 packet by inspecting the version field in the IP header (if a IP header is present) of a packet. It will build the packet structure (introduced below), and pass a pointer to it onto the next packet in the chain. Note that each component does not know where it is getting a packet from and where it is going to send it next, a component simply performs a specific action. The buffer component, as the title suggests buffers (if necessary) specific packet processing requests for individual packets, this is to support the notion of queues to allow Quality of Service (QoS) to be catered for. [Spalink, 01] notes that the queueing of packets is the primary complexity in the router, and the routers performance is greatly influenced by the queuing discipline strategies. Thus in any system developed using this component framework, a large percentage of the development time should be spent developing an efficient as possible buffer component; this largely depends on if the router will be expected to be performing any QoS actions.

33

The remainder of the components perform a number of relatively simple and discrete tasks. The CRC check component performs a check on the integrity of the packet by utilising the IP header 16-bit checksum, the component will then decide whether to allow the packet to continue (if its integrity is okay), or to inform the other components to ignore it by setting the structure flag drop to 1 (figure 6.7). The TTL (Time to live) component, performs actions on the TTL field of the IP packet, ignoring the packet similar to the CRC check if the TTL field is set to one, it will also decrement the field by one. The Pre-scheduler and Post-scheduler perform the scheduling functions of the system which although needed for general operation are primarily used to allow Quality of Service. The Routing component is the final component in the chain of required components; it is the only component that requires any state. It is at the end of the chain, as it is the second most complex component after the buffer component, which may not be needed on all occasions, it will not do any computation it the drop flag is set. There may also be any number other of components in the framework; the best place for these would be after the buffering and integrity/TTL checks and before the scheduling and routing, as indicated by the dotted lines on figure 6.4. Example additional components are firewalls, multimedia transcoding components, and encryption/decryption/security components. Each of the components in the IPv4 Component framework is an OpenCOM compatible component, which means that it must support a number of OpenCOM interfaces (section 2.4), the COM IUnknown interface discovery mechanism and the COM binary standard. These facilities are needed in order to be able to utilise OpenCOMs reconfiguration abilities. Each component must also expose a customized configuration interface which will allow the component framework and any external programs to access specific component functionality, or any state held within the component (discussed in section 6.3). Figure 6.5 shows an example configuration interface for the Routing component visible in figure 6.4.
interface IIPv4RoutingTable_Config: IUnknown { //add a new route. By configuration program. HRESULT addRoute([in] byte routeInterface[4], [in] byte requestedlocation[4]); //remove a route. By configuration program. HRESULT delRoute([in] byte routeInterface[4], [in] byte requestedlocation[4]); //get a route. HRESULT getRoute([in] byte requestedEndpoint[4], [out] byte *routeInterface); //get human readable version of the routing table. HRESULT getRoutingTable([out] unsigned char *routingTable); } Figure 6.5 Example configuration interface

34

As well as exposing a custom interface, a component must also implement the push interface and receptacle, which allows the component to be added to the component framework. The push interface aka IPv4Push (figure 6.6) contains a single method, illustrated below. interface OCMR_IIPv4Push : IUnknown { //push a packet to the implemented component HRESULT push([in] OCMR_packet *inPacket); };
Figure 6.6 Push interface

This method has a single parameter (for simplicity) which is a pointer to a structure containing a number of data types and pointers, illustrated in figure 6.7.

Struct OCMR_packet { int sizeOfEthernetHeader; int bufLen; int sendPacket; //interface to send packet on. TIPv4Address routingInterface[4]; PIPv4Header pIPv4Header; PTcpHeader pTcpHeader; unsigned char *buffer; int schedparam; int drop; }
Figure 6.7 Push Parameter Struct

Passing a just pointer means that the only overhead in using a component based approach for this is the time it takes for a COM method to be called and the time it takes for a pointer to be passed on (discussed more in implementation section 8.1.2). The use of a structure to hold a number of data types which have been pre-defined has one draw-back, the fact that it limits the systems future compatibility and feature addition-ability by constraining the variables at deploy-time. This problem may be solved by using a number of generic abstract data types which be used to store different types of data. But using this type of meta-data would also mean that all the components in the system would have to be informed of this change, which would require a certain amount of state information, which is undesirable (section 6.3).

35

6.3 State Management This design tries to follow the guiding principle of P1520 (discussed in section 4.5) to dissociate the maintenance of state information from the algorithms that manipulate state in a network [P1520, 98]. In the case of this system, this means that the component framework should not store any state information within the components themselves. In general it is a bad i ea for a router to maintain to much state information as this d would require a number of read/write operations to memory which could hinder the performance of the router and hence slow its packet processing abilities which is the point of the router. A software router only needs a certain amount of state information to process packets. The first and most obvious is the routing table, there is unfortunately no way around having this data. Route information however does not change very frequently, because of the use of wildcards, and subnets, e.g. forward all packets from subnet X to subnet Y if Destination Z is the Destination address, so the routing table could simply be stored in an area of memory for read access. The second piece of state information needed by a software router is its configuration, i.e. it needs to know to state of all the connections in the system, which bits are connected to which. The component framework represents the state of the system in a meta- model which allows components to be connected, disconnected, started, stopped and replaced. The actual implementation of the meta- model is left up to the specific implementation designer, in order to try and not constrain the developer from optimizing the implementation. This system gets around the need for an excessive amount of state information by passing around any state information about a packet with the packet itself. So if a component was dynamically replaced it would not be necessary to extract any state information and pass it on to the replacement component, to continue processing the packet. As mentioned before a component is invoked by a previous component, and a pointer to the structure is passed on, this also contains a pointer to a byte array containing the packet itself. This technique is similar to that employed by [Karlin 01], which uses an internal routing header to describe the packet currently being processed and to reduce the amount of data copied from place to place. The difference is that this system uses a component based a functional approach whereas the one employed by VERA uses a path mechanism so an actual header is necessary rather than just a data set. As mentioned earlier, this reliance on a single struct of data types to hold means that there is a requirement to know the needed data types at implementation time, or at least at deployment time.

36

6.4 Configuration and Deployment Up to now the design has focused on the internal structure of the component framework, but for any system based on it to be viable as a software router, it must be possible for it to be configured externally by a configuration program. This is achieved by wrapping the component framework up into a single virtual component and giving it an interface which can be accessed by external programs. This can be either locally using a configuration program or on another computer on the network using a number of different possible distributed techniques (as discussed in implementation section 7.5), this also makes dynamic reconfiguration possible. An example template of a system configuration interface can be seen in figure 6.7. It consists of a number of functions which allow the initialisation, configuration and reconfiguration of an OpenRouter Based system. Interface SystemCF { initOpenCOMRuntime(); loadComponentsConfig(FILE Config file); connectComponents(); startComponents(); startSystem(); startProcessing(); stopProcessing(); exitSystem(); replacecomponent(int componentID, int componentReplace ID); getSystemConfig(*config systemconfig); }
Figure 6.7 a system configuration template

So, by writing a implementation of the interface in figure 6.7, and exposing it using a distributed technology like CORBA [OMG, 02] or DCOM [Microsoft, 02] the system could be managed from practically any location (security considerations allowing of course). This also increases the platform independence of the system, as once deployed on a particular architecture the system could be configured remotely from a number of platforms using the distributed nature of the technology. This also opens up the possibility of the mass management of networks, for instance a new Quality of Service technology could be deployed simultaneous ly over a network consisting of hundreds of routers in a matter of minutes without major disruption.

37

6.5 Enabling Dynamic Reconfiguration In the design of a dynamically reconfigurable router a number of issues are important in order to insure that the router maintains a stable configuration and thus does not fail in its operation. The task is made easier with the decision to use OpenCOM as the underlying component model because of its mechanisms for explicit dependencies (see section 2.4) which result in more stable reconfiguration operations. It is therefore left to any system implementer to ensure that the dependencies It is possible for flexibility reasons to allow components to have inter-relationships via functionality exposed by the configuration interface illustrated in figure 6.5. For instance a buffer component could hold state information regarding the size of the packet buffers currently being used and exposes this information by a function in its configuration interface. Another component, like a scheduler might access this method to discover the size of the buffers and then use this information to tune the performance of the system in general. It is important that if these inter-component non-receptacle-based relationships occur, which is inevitable as not all functionality can be encapsulated in the Push struct parameter (figure 6.7) that the component programmer, and all subsequent programmers be aware of this and document it fully. This is because, if for instance in the example in the previous paragraph the buffer component was replaced by a new buffer component without the buffer size state information then although the OpenCOM runtime or the component framework wont pick the problem up because it is not related to the r ceptacle mechanism the system will still fail when the e schedule component tries to perform the function. In this case hopefully the system wouldnt fail completely, as COMs vtable structure and QueryInterface mechanisms would return an exception which would be caught. This obviously could lead to some components having a legacy which rendered them vital to a system and completely irreplaceable or upgradeable. So implementing routers based on this component framework must be highly organised, and try to avoid any non-receptacle-based inter-component dependencies.

38

7 IMPLEMENTATION The implementation detailed here occurred over a six month period from March to September 2002 with most of it being done during June and August. The attempt was made to as far as possible to develop a system capable of demonstrating the ideas detailed in the design (section 6), satisfy the stated requirements (section 3) and perform as well as possible given the restricted time- frame. 7.1 Overview The implemented system was developed based on the component framework illustrated in figure 6.4 and using the component structure illustrated in figure 6.2. Because of the time constraints of a MSc project, a fully fledged RFC1812 [Baker, 95] compliant router was not feasible, however a large amount of evaluation-able implementation was achieved, including a working MAC address bridge necessary as a low level feature in any router, and a fully working component framework structure with benchmark-able implementations. ---change to show off more and a number of components where implemented. The system was implemented under Microsoft Windows XP build 2600, primarily because of the use of the Lara packet interceptor (section 7.3) and the Microsoft 32-bit C/C++ Optimizing Compiler Version 12.00.8804 was used together with the Microsoft Interface Definition Language compiler version 5.01.0164 becaus e of the use of COM and COMs IDL language in OpenCOM. The system was programmed using C/C++ and compiled and built using a custom MakeFile (discussed in section 7.6) and the Microsoft Compiler. The development machine was a standard IBM-compatible PC, using an X86 processor in an AMDVIA system. The network adapters used where either Intel or Realtek chipset based and where connected to a 100 Mbs network The next section details the implemented system structure, and then a number of important issues are discussed. The system structure and build process is then described. Some issues which appeared during the implementation are discussed before a number of details about the operation of the system itself are illustrated.

39

7.2 System structure Figure 7.1 shows the overall implemented system structure. It shows that the component framework is not the only factor in the implementation of a software router. The overall implementation utilises the facilities provided by the Windows NT Networking stack (refer to [Solomon, 00]) which provides the control functions necessary to access a network interface card (NIC) at a higher-level than the data- link layer. Packets enter the network stack at the bottom and progress through the layers and can be read by any driver placed within the stack which can then signal a userspace program or kernel driver that a packet it is interested in is present. Any packets to be sent can be injected into the network stack which will then send them on the correct network interface. The Lara miniport (discussed in detail in [Schmid, 00]) comprises a packet interceptor and a packet injector, the interceptor enables the OpenRouter system to access the low level functionality of the Windows NT networking stack. Using the interceptor means that the OpenRouter system can gain access to all the packets that are received on the network adapters on the router machine as soon as they enter the network stack. The Lara miniport also contains a packet injector which allows the OpenRouter framework to inject/insert packets into the networking stack which will then send them to a specific network interface and onto the network. The implications of using the Lara interceptor are discussed in section 7.3.

Figure 7.1 OpenRouter System Overview

40

The implemented component framework (figure 7.2) is almost identical to the designed component framework in figure 6.4. This obeys the instructions in section 6.5 that in the implementation of routers using the designed component framework, they must be highly organised in the design.

Figure 7.2 Implementation of IPv4 Component Framework

There are a few differences, the most notable being the lack of the schedulers. Schedulers can be complex things to implement and often involve mechanisms which must access resources and functions which can not be encapsulated within a single component. For example a scheduler might be a multi-queue diffserv scheduler which would therefore need access to the buffer mechanisms. I felt that this would lead to an over-complicated prototype system, and would lead to unnecessary implementation work and that the time was better spent looking at specific issues like those in the next few sections. Above each component in figure 7.2 is the corresponding component name used in the implemented system, the project structure of the system is discussed in greater detail in figure 7.8. As mentioned, the buffer component handles the buffering of the packet processing requests; it does this by buffering the struct passed by the Push function call made by the Core component. The CRC check component performs an integrity check on the packet. The TTL component deals with the time to live field of the IP packet header. The firewall component has the option to drop a packet in the event that its rules indicate it to. The routing table component can do what routing the system programmer/configurator c onfigures it to do. This includes MAC address routing and IPv4 Routing. During the operation of the system, there are a number of function calls, data copies and buffering which occur, these are illustrated in figure 7.3.

41

Figure 7.3 OpenRouter Data Flow

The diagram shows that the OCMR_IPv4Core component encapsulates the user-space functionality of the Lara Packet interceptor and injector which interfaces with the Lara kernel-space miniport. It receives a packet from the Lara miniport and then builds the packet structure before passing the packet reference onto the OCMR_IPv4Buffer component which can then buffer it depending on its buffer policies. In this implementation, it is the buffer component which has the control over the processing of the packets. Once the processing thread is started by the initialisation program or by a remote distributed client packet can be processed. If there are packets awaiting to be processed then the buffer component processing thread will perform a functio n call on the next component in the chain, by using the push receptacle/interface already set up when the system if configured. Each component in the chain then has the responsibility to call the next component as illustrated in figure 7.4 which shows an example of the push method implementation for each component.

//declare receptacle OCM_SINGLE_RECEPTACLE(OCMR_IIPv4Push) //push method STDMETHODIMP push(OCMR_packet *inPacket) { 1. look at struct 2. determine what processing to be done 3. perform some processing. 4. forward packet on m_pSR_OCMR_IIPv4Push->push(inPacket); //return when all subsequent components return return 0; }

Figure 7.4 Push function

As explained in section 6.4, for any router to be viable it must be capable of being externally configurable, this occurs in the implemented system by wrapping the component framework up into a component-like entity by using a single DCOM interface. The DCOM interface is in Figure 7.5, it illustrates a number of details when using COM and DCOM. The first is that all interfaces inherit the functionality of COMs IUnknown interface as indicated by the import statement on the first line. Each COM/DCOM object has a globally unique identifier or GUID, this is based on a number of unique features of the development machine including the MAC address of the network c ard. The interface contains a number of methods which allow the initialisation, configuration and reconfiguration of the system.

import "unknwn.idl"; [ object, uuid(D30D1FE8-1E06-4740-A249-3C61924F2D0E), dual ] interface ISystemInterfaceS : IDispatch { [id(0)] HRESULT initOpenCOM(); [id(1)] HRESULT loadComponents(); [id(2)] HRESULT connectComponents(); [id(3)] HRESULT startComponents(); [id(4)] HRESULT startSystem(); [id(5)] HRESULT startProcessing(); [id(6)] HRESULT stopProcessing(); [id(7)] HRESULT exitSystem(); [id(7)] HRESULT replacecomponent(int componentID, int componentReplace ID); }

Figure 7.5 Implemented System Interface

Now that an overview of the system structure has been presented, the next few sections will discuss a number of important features of the implementation. 7.3 Packet filter To allow the system to access packets from the network it is necessary for a specialised piece of software called a packet filter to be used. A packet filter simply allows a program to access packets that are received at the router machine. It is also necessary for that packet filter to filter out packets which are of no interest to the system and only forward packets that are wanted, for example the software router detailed here is only interested in IPv4 packets, so it is not interested in IPv6 packets. During the research period for this project a number of possible technologies where found to perform this function. These included the OpenBSD [OpenBSD, 02] packet filter which runs under the OpenBSD operating system and the Linux packet filter [Linux, 02] which runs under Linux. Both are open source and available under an open source licence. Unfortunately, because of the early restriction of using OpenCOM, which is based on COM which although is in design, platform independent it is practically only Windows NT (5) based, and OpenCOM itself has only been implemented on Windows NT (5). So a packet filter w required for as Windows NT, which unfortunately is Closed Source. In the end a compromise was reached, to use an already implemented packet filter which was part of another system.

44

This system which comprises the packet filter is Lara++ (detailed in section 5.2). The Lara interceptor has a number of advantages over the other packet filters available. It uses the Windows NDIS framework and is therefore inherently windows based which fits well with the fact that the component model for this system was going to be OpenCOM see section 2.4. The Lara packet interceptor/injector performs the packet filtering functions needed by the OpenRouter system. It can efficiently intercept packets; filter out relevant packets; in this case IPv4 ones and pass them onto the component framework for processing. The Lara packet injector also allows the system to send packets on different network interfaces, and thus perform the necessary functions of a router.

7.4 Buffer mechanism The packet buffer mechanism in the OpenRo uter system is implemented within the OCMR_IPv4Buffer component. It consists of a single FIFO queue of pointers to the struct (figure 6.7) which represents a packet, packets are added at the bottom and can be taken off the top. The OCMR_IPv4Core component receives the packet from the Lara++ miniport; it then builds the packet structure and passes this on to the buffer component (OCMR_IPv4Buffer), the buffer component then adds it to the bottom of the queue. The buffer component also has a thread which checks the size of the queue, and if there are any packets in the queue it will retrieve them and push the packet struct onto the next component in the chain. Special care had to be taken in programming the buffer component as the program is multi- threaded, as the queue could be read simultaneously if mutual exclusion locks are not present, this would inevitably lead to an invalid access exception. So, simple mutual exclusion locks where also placed on the FIFO queue, when a thread wishes to either add, read or remove items from the queue it must call gainaccess() and giveaccess() methods which block more than one thread from accessing the queue at the same time. The implementation of the buffer mechanism as a FIFO queue is not tied to the component f amework design. The buffer mechanism could be changed to a multir queue or even a priority queue mechanism. This could be achieved by simply changing the implementation within the buffer component.

45

7.5 Threading An important consideration when designing any large system especially when dealing with networking and/or performance is the threading policy which the system uses. Different router implement ations use different threading mechanisms, including thread per- flow, thread-per packet or single thread. This system uses a hybrid singlethread approach. The Lara packet interceptor utilises a single thread to receive packets from the network, the IPv4Core component uses a single thread to then receive these packets and build a descriptor struct for them. The Core component then passes the struct onto the first component in the chain, which in this case is the IPv4 Buffer component, which will add the struct to the FIFO queue; this thread can then process the next packet. The buffer component then has the final thread, a single thread for processing all packets. It tests for any packets (structs) present in the queue, if there are it will retrieve the point to the one on the top and forward this to the next component, which will process the packet and forward it on again, all under the same thread. When the final component has finished with it, it returns, and then so do all the components in turn. When the component next to the buffer component returns, the packet has ben fully processed by the system, the buffer component can delete any memory which the packet used and any reference to it. The buffer component can then continue by querying the queue and processing further packets. 7.6 Remote management As discussed previously, for any router to be viable it must allow itself to be configured and (in this case) dynamically reconfigured remotely. The implemented system has two methods of configuration, one using a Win32 executable on the local computer and one using a DCOM client and server. Both contain the same invocations to configure the system (see section 7.8) but do it locally or remotely depending on the strategy. The DCOM implementation for configuring the system utilises a local server and the DCOM registry and a remote client. The DCOM server contains the implementation which configures the system and exposes the system interface to the remote client by utilising the DCOM registry which is part of the Windows registry. The DCOM registry registers and holds details of the GUID for a particular DCOM server/service, so when a request is received from a client the DCOM registry looks up the unique GUID in the registry and starts up the DCOM server based on the registry. The DCOM service is then ready to receive requests or comnmands from the DCOM remote client. The remote client simply loads the client service and using a proxy containing local versions of the server interface contacts a remote DCOM registry and can then communicate with the server. There where a number of problems in allowing the system to be used by a DCOM service. The main one was that OpenCOM and the OpenRouter system utilise the OpenCOM run-time which runs a COM in-process server, whereas DCOM runs a outof-process server. This meant that the structure of the system was an out-of-process server which wrapped round a in-process server, this lead to some interoperability problems which where solved by changing some COM configuration variables. 46

7.7 Project Structure and Build process With a system which utilises a number of different technologies, especially experimental ones, it is important that they development is highly organised. Figure 7.6 shows the structure of the implemented system.

OCMR (OpenCOM Router or OpenRouter files and MakeFile in root) | | ---bin (binary files, where the executables and DLLs go when they are built) | | ---dcom (dcom project files) | | ---include (files to be included in project build, including definitions of structs etc) | | ---Lara++ (the Lara++ packet interceptor/injector and miniport driver) | | ---netperf (the net performance tools used in the system performance tests) | | ---OpenCOM (the OpenCOM runtime and libraries | ---config (OpenCOM configuration libraries) | ---interfaces (OpenCOM interfaces) | ---metaenc (OpenCOM meta encapsulation libraries, not used but still needed by OpenCOM) | ---metaenv (OpenCOM meta environment libraries, not used but still needed by OpenCOM) | ---misc (Miscellaneous OpenCOM interfaces/libraries) | ---OCM (The OpenCOM Runtime)
Figure 7.6 Structure of Implemented System

Each distinct technology is compartmentised into a folder and each has its own MakeFile. The OpenRouter system itself is in the root folder as indicated on the diagram. A master MakeFile in the root folder builds all the contributing libraries and OpenRouter.

47

The first step in building the system is to compile the interface definition language (IDL) interfaces which produce the OpenCOM stubs and other necessary files for compiling dynamic link libraries (DLL). The MakeFile can then be used to build all the different components which make up the system. Different components need to be linked with varying degrees of components. For example the OpenRouter enabled components have to be linked with the OpenCOM libraries and also link in some OpenRouter helper libraries. The packet filter component (OCMR_IPv4Core) must be linked with the OpenCOM and OpenRouter libraries and the Lara++ libraries. All the components in the system must them be registered with the Windows NT (5) registry; this is done us ing the regsvr32 tool which uses the information in the type library and the GUID (globally unique identifier) to register the location and name of the library. This will allow the OpenRouter system find the libraries it needs and to dynamically load them.

7.8 System Operation This section briefly describes how a system programmer would configure and deploy an OpenRouter-based component framework system and also how dynamic reconfiguration would take place practically. 7.8.1 Configuring the system In order to configure a system the programmer would at first need the set of components they wished to use and a packet filter for the specific platform that they choose to use. All the individual components must be able to do the specific task that they have been labelled as being able to do. Assuming that a version of the component framework had been implemented fo r the network protocol they required they would have to write a binding program or an implementation object that basically done the same thing but for a distributed system (described in section 7.6). This program created an initial system configuration, and can also accept incoming requests to perform actions on the system, including dynamic reconfigurations. The initial stage of the configuration program must do a number of tasks, these are: 1. Initialise COM. 2. Create an instance of the OpenCOM runtime. 3. Query the OpenCOM run-time for the control interface 4. Create an instance of OCMR_IPv4Core component (assuming IPv4) 5. Obtain the OCMR_IPv4Core interface 6. Create an instance of the OCMR_IPv4Buffer component 7. Obtain the OCMR_IPv4Buffer interface 8. Create instances of other components 9. Obtain interfaces of other components 10. Connect OCMR_IPv4Core to OCMR_IPv4Buffer 48

11. 12. 13. 14. 15. 16.

Connect OCMR_IPv4Buffer to next component Connect all other components together in linear chain Start the OCMR_IPv4Core component Start the OCMR_IPv4Buffer component Tell the OCMR_IPv4Core component to start receiving packets Tell the OCMR_IPv4Buffer to start

The system should then start processing packets as it is designed to do. What system actually does depends on what the implemented components are programmed to do.

7.8.2 Dynamic reconfiguration In order to complete a reconfiguration a number of steps must take place. If the system is assumed to be initialised then in order to perform a dynamic reconfiguration the following steps must be taken. 1. Stop the buffer component from processing packets. 2. Stop the component wishing to be removed. 3. Stop the components either side. 4. Disconnect the components either side (both its interface and its receptacle). 5. Insert and initialise the new component. 6. Connect the new component to the dangling receptacle and interface. 7. Start the components either side. 8. Start the new component. 9. Start the buffer component. 10 The system can continue. One interesting details about the way this is done, is that the system doesnt actually stop receiving packets from the Lara++ miniport and from the network. The system can actually continue to buffer the packets whilst the reconfiguration is taking place, therefore no disruption apart from a gap in transmission occurs. This of course changes if one of the core components like OCMR_IPv4Core or OCMR_IPv4Buffer are the ones being replaced.

49

8 EVALUATION A number of tests have been carried out in order to investigate the implemented system and to try and prove that it achieves the goals set out in the requirements. The first section (8.1) details a number of tests of some configurations of the implemented system. Section 8.2 then try to evaluate the system based on a number of system scenarios. Section 8.3 tries to evaluate the system based on the requirements stated in section 3. 8.1 Performance Evaluation All performance tests where carried out using a high-spec IBM-compatible personal computer connected to a 100Mbs duplex switched hub via category 5 network cable. The computer used to test the OpenRouter system is an AMDVIA based system, with a AMB Athlon XP 1800, 512MB 2100 DDR a Chaintech 7KJD mainboard with a AMB Northbridge and a VIA Southbridge. In the tests either a Intel pro/100+ card or a Realtek 8139C network card was used, although the effect was negligible. The system ran Windows XP build 2600, and was optimized when the tests where running, i.e. minimal software, no firewall etc. 8.1.1 Lara++ packet filter overhead The use of the Lara++ packet filter in this system does not give an ideal solution for a packet filter in this system. It was not designed as a performance- heavy packet filter but more as an experimental tool to execute active network code on, which is not performance critical. It uses the features of the Windows 5.1 Networking stack, so as such it is not specially optimized for packet filtering. It is also not been specifically designed or optimized for this specific system. However when tests where ran with the OpenRouter system which contained just the Lara++ packet filter and a number of OpenRouter components, the results where favourable. The System configuration simply intercepts the packet and copies it to a buffer. So it is basically testing the performance overhead in the system utilising the Lara packet filter and hence determine the maximum performance of the OpenRouter system whilst using the Lara Packet Filter. Figure 8.1 below shows the test architecture.

50

Figure 8.1 Packet Filter Overhead System Architecture Test

The Netperf Performance tool [Netperf, 02] was used to test the performance of the system. A server program was run on the OpenRouter enabled system, and the client was run on another PC linked together by a 100Mb network and 100Mb network interface cards (NIC). The client generates and sends UDP and TCP traffic for 10 seconds to the server which acknowledges receipt of the data by replying, the time it takes for the replies to reach the client is measured and this is used to calculate the bandwidth of the network. It can also be used to test the performance of any piece of software or hardware which is placed in the path of the performance tools messages, which is how it is being used in this case. The first test carried out with the Netperf tool was a control test which just tested the speed of the network, and operating system overhead and network stack overhead without the lara++ packet filter enabled. The results are in figure 8.2 below, all the throughput readings are in fact averages over 10 tests. Command issued on windows XP machine IP = 192.168.0.3 netperf H 192.168.0.2 meaning TCP STREAM TEST to 192.168.0.2
Recv Socket Size bytes Send Socket Size bytes 8192 Send Message Size bytes 8192 Elapsed Time secs 10.00

Throughput 10^6bits/sec 83.06

test 1 8192 (Control) test 2 8192 (Lara++) test 3 8192 (debug)

8192

8192

10.00

52.70

8192

8192

10.00

1.96

Figure 8.2 Lara packet filter overhead tests.

51

The control test (test 1) was taken place with receive and send buffer sizes of 8192 bytes and with messages of size 8192 bytes. Over a 10 second period the throughput achieved was 83.05 megabit per second, which is reasonably close to the bandwidth of the network. The second test was taken place using the configuration described above including the Lara++ packet filter and a minimal OpenRouter system. Over the series of 10 repeated tests the result was that a value of 52.70 megabit a second was achieved. This shows that although using the Lara packet filter had measured effect on the performance of the benchmarks it did not completely cripple it. It is possible therefore to conclude that the use of the Lara++ packet filter in the system as a whole proves to be a reasonable decision for a prototype system. Interestingly, when the above test was first ran, the results where very poor (test 3), this was discovered shortly after to be because of the fact that there was a number of comments printing of system status information, this shows the massive overhead associated with debugging information.

8.1.2 OpenRouter Performa nce This test details the results of testing the performance of the system using the Netperf bench marker (as discussed in the previous section) for testing the OpenRouter system. This test uses the same overall architecture (diagram 8.1) with the difference that instead of just testing the Lara++ packet filter overhead, the test will also include the OpenRouter component framework as well. To do this I use a public Boolean in the OCMR_IPv4Core library to only allow the Lara++ interceptor to release the current packet and receive a new one when the OpenRouter component framework has finished processing the packet. This is basically replicating the situation that would occur if the system was being used as an in-band IP router on a real network. The component framework was configured with the components as illustrated in figure 7.2. The results are illustrated in figure 8.3. Command issued on Windows XP machine IP = 192.168.0.3 netperf H 192.168.0.2 meaning TCP STREAM TEST to 192.168.0.2 Recv Socket Size bytes test 4 8192 (OpenRouter) Send Socket Size bytes 8192 Send Message Size bytes 8192 Elapsed Time secs 10.00

Throughput 10^6bits/sec 35.56

Figure 8.3 OpenRouter performance test.

52

The test produced an expected result, that without the use of kernel level buffering accessible by the OpenRouter system, that forcing of the system to act as a performance-centric in-band system means that the performance drops to a third of what can be provided by the network. This can be attributed to a number of factors, including the overhead associated with the Lara++ interceptor, the overhead in the Windows Networking stack, the time it takes to determine whether to process the component or not and as discussed in the next section the overhead associated in using OpenCOM enabled compone nts. 8.1.3 OpenRouter Extensibility When the system is configured and ready to process packets, the components within the component framework are all connected together (in a chain) using OpenCOMs explicit dependency mechanism together with the COM vtable structure. The test detailed here, tests the systems performance based on the same configuration as used in the previous test except with the addition of a number of N ull component to test the extensibility of the system. These Null components are added one at a time and a set of performance tests are performed, in this way the effect of adding additional components can be measured and quantified. Each of these Null components is basically an OpenRouter template component, in that it contains only a single line of implementation in the Push method implementation as indicated in figure 8.4 below, which shows the Push method. // Interface push implementation STDMETHODIMP OCMR_IPv4Null::push(OCMR_packet *inPacket) { //print debug info #ifdef DEBUG cout << " in OCMR_IPv4Null" << endl; #endif //do nothing. //forward packet on. m_pSR_OCMR_IIPv4Push->push(inPacket); return 0; }

Figure 8.4 the Null Push Method Implementation

The Null Push method implementation literally just forwards on the request for processing onto the next component in the chain, however there are still other mechanisms that are at work when a call is made, including vtable lookup, receptacle checking etc, refer to [Coulson, 01]. Figure 8.5 shows the results of the series of tests, again the performance figure is the average of number of tests.

53

Recv Socket Size bytes

Send Socket Size bytes

Send Message Size bytes

Elapsed Time secs

Throughput 10^6bits/sec

test 5 8192 8192 8192 10.00 31.48 (same as OpenRouter test, additionally done as extra control) test 6 8192 8192 8192 (additional 1 Null component) test 7 8192 8192 8192 (additional 2 Null component) test 8 8192 8192 8192 (additional 3 Null component) test 8 8192 8192 8192 (additional 4 Null component) test 10 8192 8192 8192 (additional 5 Null component) test 11 8192 8192 8192 (additional 10 Null component) test 12 8192 8192 8192 (additional 15 Null component) test 12 8192 8192 8192 (additional 20 Null component) 10.00 10.00 10.00 10.00 10.00 10.00 10.00 10.00 29.35 28.11 28.89 27.34 26.16 23.67 20.54 19.56

Figure 8.5 Extensibility test results

From the tests, it is possible to conclude that the system in its current state (experimental) is not extensible when five or more extra components are added to the minimal components specified for the specific framework (see figure 7.3). This doesnt really affect the validity of the system as the component framework will only ever be used with the minimal components necessary. A possible solution to this extensibility problem would be to run multiple micro-component frameworks on different processors/distributed systems with each performing a smaller task only consisting of 2-3 components. These component frameworks could be embedded in one large on to produce a seamless system that could run on a multi-processor platform, or even a network-processor based platform like the Intel IXA1200 [Intel, 02]. Another possible solution would be to optimize the invocations between OpenCOM components which cause the extensibility problem. This could be done by dynamically hard-coding at compile-time the function address usually discovered using the vtable structure. This has already been done by Microsoft for COM and is currently being looked at; refer to [ Sivaharan, 02]. With this optimization it is probable that the OpenRouter system may be extremely extensible. 54

8.2 Qualitative Evaluation

During the later stages of implementation and the evaluation of this project a number of system scenarios where implemented. Whilst these where mostly unable to be performance tested because of there incompleteness or because they where unoptimized they still contribute to any evaluation of the system. Some of these are presented here: Ethernet Switch The system was placed in the configuration illustrated in figure 8.6 below, which consisted of a single PC with the OpenRouter system with two network interface cards (NICs) each connection to a different network with a single PC on each of these networks. In this way the network interface on PCs 1 and 4 where completely isolated which allowed the routing properties of the implemented system to be tested.

Figure 8.6 Ethernet Switch configuration

A simple ping utility was used to create sample traffic for the system to be tested on. The Routing component of the OpenRouter enabled machine tested each packet for the MAC Source to be from interface 4 and for the Destination to be from interface 1, or the other way round. When a packet was discovered to meet these criteria, it was forwarded onto the other network. In this way the system configuration successfully routed traffic between computers on different networks based on the MAC address.

55

IP router The system was placed in the same configuration as illustrated in figure 8.6 above, except that the IP addresses where u rather than the MAC addresses to route sed traffic. The routing table component was configured to forward all traffic from machine 4 to machine 1 which had the correct Source and destination IP addresses. For example in one test performed machine 4 was configured as machine 3 as the gateway, a web browser on machine 4 was configured to use machine 1s IP address as a web proxy. A web proxy was installed on machine 1 and set to allow requests and perform them via its Ethernet (and dial- up) connection to the Internet. The OpenRouter system was ran on the PC with network interface cards 2 and 3. The web browser on PC 4 could successfully browse the web using the configured setup. This test is also illustrated in figure 8.6. Firewall The OpenRouter system can also be configured to act as a firewall. In this configuration the system does not actually do any routing, but uses its packet processing ability to decide whether a packet should be allowed to reach the higher level in the networking stack. If it is decided that the packet should not be allowed, the Lara SendDropPacket function can be used to drop the packet. This configuration would also work if the system had a NAT server or third party router present. That stem could perform the task specified and the OpenRouter system could filter out packets based on a set of rules, The OpenRouter system would certainly be fast enough for this. This is much more secure than some firewalls which actually allow other processes on the network stack to access the packets before it may drop them. And it would also be more configurable, with the introduction of new rules and types of rules as components without having to stop the system. Remote Configuration Using DCOM All the tests and configurations mentioned in this evaluation can all be performed using the DCOM interface rather than a local executable. Using DCOM to remotely configure the OpenRouter system simply adds an additional component to the system.

56

8.3 Requirements Evaluation This section attempts to evaluate the system based on the requirements detailed in section 4. The primary requirement of the system is to design a component framework to use as the basis in the construction of a software router; this is detailed in section 6. The design tries to be general enough to apply to a number of different types of routers whilst being specific enough to form a basis for the implementation which is detailed in section 7. The ability of the design to form the basis as a software router was proved in sections 8.1 and 8.2 which show some examples and scenarios of the system working. Further more, the implementation was almost fully true to the concept set down in the design, so the design looks to apply very well to IP routing. Whether the design applies to any other type of router i.e. ATM, is a question which cannot be answered without further investigation into the field. The secondary requirement is to investigate the viability of reconfiguring routers at run-time. This was more conclusively positive, it is very clear that using OpenCOMs facilities it is relatively straight forward to dynamically reconfigure components. The main influences in whether a dynamic configuration is viable are the policies used within the system for buffering, threading and the general level of coupling the system has. For instance excessive use of the configuration interface or of other methods apart from the receptacle based ones may render any dynamic reconfiguration dangerous. The 3rd requirement, to enable the deployer to deploy the system, as efficiently as possible has not really been addressed by this project; this would definitely have to be looked at in any future work, as deployment can be a big issue. The 4th requirement concerns the attempt to try and develop a well performing system. Although we cant compare this system with other implemented routers because a fully implemented router can take years to develop, it is possible by looking at the figures presented in 8.1 to predict that with further development and optimization the system would perform on a par with other systems. This system would also have the added advantage of being highly configurable and given it uses components, flexible. Requirement 5, states that the component framework should be platform independent. As discussed in section 6.1 the component framework is platform independent because it relies on network protocol standards rather than platform facilities. So to allow the component framework to execute on another platform, all that would be necessary would be to replace the packet filter and change some build options. The

57

actual component framework might even be able to run on some existing platforms, the only limiting factors being the implementation of a packet filter and the platform dependence of OpenCOM. As well as the IPv4 component framework being platform independent, as the component framework is simple and based on the hierarchal structure in figure 6.3 it should be relatively easy to apply to the general framework presented in design section 6.1 to any other network protocol. So although the developed system is based on IPv4 routing it is a reasonably trivial matter to convert it to IPv6. In fact a single line in the packet interceptor would convert the packet filter to filter IPv6 packets, if protocol = 6 would intercept IPv6 packets. In terms of code portability, the actual code was Written in c/c++ using only standardised functions. However OpenCOM completely relies on the Windows COM libraries, so the system is dependent on Windows until an implementation of OpenCOM for another Operating System is developed. There is the possibility of using OpenCOM under an emulator in a UNIX platform, like WINE [WINE, 02]; however this would almost certainly result in a performance overhead. A level of System independence was achieved by isolating system dependent mechanisms with a single component, (OCMR_IPv4Core), thus to implement the system on a different platform only this component need be replaced. Requirement 6 states that the design should be simple and efficient. A number of design decisions have made the design extremely simple and efficient, the most obvious being the use of a linear structure of components in the component framework. Also the use of a hierarchal inheritance model for describing the OpenRouter systems was a big step towards representing and categorising different configurations. Requirement 7 states that a number of generic components that work for a number of network protocols should be developed during the course of the project. The requirement became less relevant as the project developed as instead of creating full components that can work with multiple network protocols a hierarchal structure of component frameworks templates and component templates emerged. Thus if a new component is needed to be created, the programmer will either use the root template or a template for that specific ne twork protocol. The 8th requirement, to try and limit the number of copies that the system makes is highlighting the importance of limiting the number of copies when designing/implementing a performance centric system like the one discussed in this dissertation. The numbers of copies have been limited as far as possible, but limiting them further would require the reimplementation of the packet filter and buffer mechanism, which would be desirable in any further implementation anyway.

58

The 9th requirement highlights one of the important considerations when designing a dynamically reconfigurable system, that of trying to maintain its integrity for the lifetime of the system. This is important, the more changes that are made to the systems configuration, the more uncertain we can be about the stability and integrity of the system. The design of the component framework attempts to deal with this problem by trying to limit the communication between components to just the regulated push interface and receptacle combination. The 10th requirement states that the system should add enough useful functionality for the system to be useful. It is hard to verify if this requirement has been met due to the fact that the system is really trying to specify a component framework which allows developers to then implemented, there is no functionality as such in the design. The final requirement states that the system should be compatible with P1520. P1520 has a relatively vague layered model, as discussed in section 2.5. As far as can be analysed from the draft paper, the system developed here seems to map onto P1520 relatively straightforwardly.

59

9 CONCLUSIONS

9.1 Project Review This project was concerned with the design of a component framework for softwarebased IP routers. This document presented the design, implementation and evaluation of the system. The presented design was simple and generic and allowed the further design of an IPv4 router specific component framework. This IPv4 router design was then implemented and a number of specific system scenarios where developed. The findings from these scenarios together with qualitative and requirements evaluations formed the basis for a complete evaluation of the project. The evaluation tried to use a number of different techniques to evaluate the system based on performance, qualitative evaluation on system scenarios and also project goals which have been achieved. It showed that a component-based approach can be used to create a system that will perform well with the addition of being highly configurable and flexible. It also showed that with the use of the facilities provided by a component model like OpenCOM it is possible to achieve efficient dynamic reconfiguration. The main point that the evaluation was aiming at was that the project succeeded in producing a successful prototype implementation that has the potential to fulfil all the requirements and it has also highlighted a number ways in which the system can contribute to the field of component-based routers. Before the project started and during the proposal stage, it was assumed that the system was going to use an emulated packet filter as implementing a full blown real packet filter seemed a lot of work. So it is extremely good that the implemented system actually uses a real packet filter and deals with real packets. To actually get the system routing in any way was a resounding success, and especially successful that in one set of experiments the system was successfully routing based on MAC addresses and then based on IP addresses, which although hard coded was a great success. In terms of project credibility, the project maps onto p1520 quite well so in the longer term it should be possible to get it accepted by industry. Overall I think that the project can be said to have been a great success, it attempted to use existing and experimental technologies to create a new type of system and it succeeded in answering various questions and introducing new ideas to the field.

60

9.2 Retrospect If I had the chance of doing the project again, there would be a number of things I would like to do differently, although I would not change the project aims as a whole. In the design of this type of performance critical software it is imperative that much more emphasis should be placed on practical issues involving the network as well as part-theoretical work involving the component framework. I would therefore have spent more time investigating possible I would like to have spent more time investigating the properties of the component framework. There are a number of different ways in which the design of the component framework here could be extended to include other ideas. These include points like multiple component frameworks and ha ving a meta- model of rules to build the component framework. I would also have liked to spend a longer period of time researching the field, as I got a lot of ideas and influence from reviewing the papers I did look at. The papers included in this project where the most relevant ones, but there where a lot more which gave general guidance to the project not just specific technical guidance. 9.3 Future work This project was not only meant to stand on its own in terms of its findings, but also meant to aid in the various stages of the NETKIT project as detailed in section 4. As such, any such future work is not just theoretical but will also actually be investigated in the early stages of the NETKIT project itself. There are a number of things which in the later stages of this project have become clear that it would be beneficial to investigate them further. The program that initialises configures and reconfigures the system and the DCOM interfaces are basically ways in which the rules of the component f amework are r described. For example, both programs connect the components together in a specific order, which is hard coded into the program. This would be done much more sensibly and configurably by using a general program which loaded a set of rules from a file or database and based on the request of the system administrator built the system dynamically based on the rules. One of the major issues in this project and definitely something that should be looked at further is maintaining/improving the system/platform independence of the component framework and the OpenRouter system as a whole. This would probably be best done by defining a complete interface for the system. The developed system was actually tied to the Lara++ packet interceptor mechanism. A possible way forward may involve the development of a separate abstract layer which the OpenRouter system accesses the low level functionality of the system. Thus when porting the system to another platform the abstract layer would have to be written but the OpenRouter system could remain constant. This abstract layer is analogous to the software representation layer in the P1520 draft.

61

With the defining of a more distinct system framework interface, more interesting possibilities would arise. For instance, the designed and implemented system presented in this dissertation only utilises a single component framework, but if more than one component framework is used then the possibilities for configurability are huge. There could be multiple simultaneous component frameworks, each handling a different protocol, or there could be embedded component frameworks acting like individual components. With actual sections of the system being compartmentalised, they could be run as independent threads, even on different processors in a multiprocessor router like the Intel IXA1200 [Intel, 02], which could produce an extremely well performing system. Adding addition support to the component framework itself could make it more attractive as a software router. Additional support could include inherent security or distribution, although this may overcomplicate a system which is trying to be as simple and efficient as possible. One interesting possibility with this system, because of its flexible nature and because of the possibility of wrapping the system up in DCOM interfaces and controlling it remotely is that you could control more than one system at the same time. This could lead to some real breakthroughs in the way in which networks as a whole are monitored and configured. An administrator could be presented with a graphical representation of an entire network which consists of a number of OpenRouter enabled routers, he could then instruct a number to support a new QoS technology, and this could then be done instantly without the routers having to be taken down. The conclusion that should be made in respect to future work with this project is that there is a great breadth of possibilities and that the success of this project should be built upon.

9.4 Personal Conclusions During the development of this system I expanded my knowledge of the field and of computing in general. In the early stages of the project I learned a lot about how research works and how to read research papers efficiently. The design process was extremely enjoyable as it was the first real opportunity that I have had to design something based on cutting-edge research. During the implementation stages of the system my knowledge of C and C++ programming increased massively whilst I also learned component based technologies. I enjoyed greatly the system integration phase when I managed to get the Lara++ code integrated and working with the OpenRouter system. Overall this project has been extremely beneficial to me both in technical terms and software engineering terms.

62

REFERENCES [Baker, 95] Baker F., Requirements for IP Version 4 Routers: RFC 1812, Internet Request for Comments, June 1995. [Campbell, 01] Campbell A T., Chou S T., Kounavis M E., Vassilis D S., Vicente J., NetBind : A Binding Tool for Constructing Data Paths in Network Processor-Based Routers, COMET group, Columbia University and Intel Corporation [Coulson, 00] Coulson G., Blair G S., Clarke M., Parlavantzas N., An Efficient Component Model for the Construction of Adaptive Middleware., DMRG, Computing Department, Lancaster University and Dept of Computer Science, Tromso University, 2000. [Coulson, 01] Coulson G., Blair G S., Clarke M., Parlavantzas N., The Design of a Configurable and reconfigurable middleware platform, DMRG, Computing Department, Lancaster University and Dept of Computer Science, Tromso University, 2000. [Coulson, 02] Coulson, G., Blair, G. Hutchison, D., Ueyama, J., Irvin, Y., Lee, K., Surajabli, B., NETKIT: A Software Component-Based Approach to Programmable Networking, Lancaster University, 2002. [Descasper, 98] Descasper D., Ditta Z., Parular G., Plattner B., Router Plugins, A Software Architecture for Next Generation Routers, Computer Engineering and Networks Laboratory, ETH Zurich, Switzerland, and Applied Research Laboratory, Washington University, Proceedings of the ACM SIGCOMM, Sept 1998. [Intel, 02] Intel IXP1200, www.intel.com, Intel Corporation 2002. [Johnson, 02] Johnson L., Open middleware for the rapid development and experimentation of configurable multicast protocols, PhD thesis, Lancaster University Computing Department. [Karlin 01] Karlin S., Peterson L., VERA: An Extensible Router Architecture, Department of Computer Science, Princeton University, 2001. [Kohler, 99] Kohler E., Morris R., Chen B., Jannotti J., Kaashoek M.F., The Click Modular Router, Laboratory for Computer Science, MIT, Dec 1999. [Kon, 00] Kon F., Roman M., Liu P., Mao J., Yamane T., Magalhaes L C., Campbel R H., Monitoring, security and Dynamic Configuration with the dynamicTAO Reflective ORB, IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware 2000) [Linux, 02], Kernel Korner: Inside the Linux Packet Filter, http://www.linuxjournal.com/article.php?sid=4852, Gianluca Insolvibile, 2002 [Merugu, 00] Merugu S., Bhattacharjee S., Zegura E., Calve rt K., Bowman: A Node OS for Active Networks, College of Computing, Georgia Tech., Atlanta et al., 2000. 63

[Microsoft, 02] COM, Microsoft Corporation, http://www.microsoft.com/com/, 2002. [Mozilla, 01] Mozilla Organization, http://www.mozilla.org/projects/xpcom. XPCOM project, 2001,

[NetBSD, 02] The NetBSD Foundation, www.netbsd.org, 2002. [Netperf, 02] NetPerf, Network Performance tool, netperf.org, 2002. [NodeOS, 98] Architectural Framework for active Networks, DARPA AN working group Draft, 1998. [OMG, 02], CORBA, www.corba.org, the object management group, 2002. [OpenBSD, 02] The OpenBSD Packet Filter HOWTO, http://www.deadly.org/pfhowto/, Wouter Coene, 2002. [P1520, 98] Programming Interfaces for IP Routers and Switches, an Architectural Framework Document, IEEE pin Working group, Doc ref. P1520/TS/IP-003 [Schmid, 00] Schmid S., Finney J., Scott A.S, Sheperd W.D., Component-based Active Network Architecture, Lancaster University DMRG, 2000. [Schmid, 02] Schmid S., Chart T., Sifalakis M., Scott A C., Flexible, Dynamic and Scalable Service Composition for Active Routers, Distributed Multimedia Research Group, Lancaster University 2002. [Sivaharan, 02] Sivaharan, T., Optimizing OpenCOM local bindings, MSc Dissertation, Lancaster University Computing Department, 2002. [Solomon, 00] Solomon, D A., Russinovich M E., Inside Microsoft Windwos 2000 Third Edition, Microsoft Programming Series, Microsoft Press, 2000. [Spalink, 01] Spalink T., Karlin P., Peterson L., Gottlieb Y., Building a Robust Software-Based Router Using Network Processors, Department of Computer Science, Princeton University, 2001. [Sun, 02] Sun Microsystems, http://java.sun.com/products/javabeans/, 2002. [Szyperski, 98] Szyperski, C., Component Software: Beyond Object-Oriented programming, Addison-Wesley, 1998. [WINE, 02] http://www.winehq.com/, Wine Is Not an Emulator, 2002

64

GLOSSARY API Byte C C++ CF COM Component DLL ECN Flag IDL IP IXA Java OCMR Application Programming Interface 8 bits arranged in one word A powerful programming language Object Oriented C (with some differences) Component Framework Component Object Model A unit that forms a discrete entity, forms include a binary component like a Microsoft COM DLL. Dynamic Link Library Early collision notification A Boolean that indicates a true or false case Interface Definition Language Internet Protocol, version 4 or version 6 Intel Internet Exchange Architecture A byte code based, heavily typed programming language by SUN. OpenCOM Router, often prefixed on components and variables to highlight that the component is an OpenRouter component or is a OpenCOM Router component Open Component Object Model Open Object Request Broker Object Request Broker Personal Computer, assuming something around an 800Mhz+ 128MB RAM, network card A discrete entity that can be plugged into a system to give it new functionality Quality of Service

OpenCOM OpenORB ORB PC Plug- in QoS

65