Published: March 2014 Author: Indranil Dutta, Greg Stemp, James Birkinshaw
Abstract: This document describes the operational processes, tasks, and tools required for you to maintain a Lync Server 2010 environment. It explains how to manage Lync Server 2010 according to the Microsoft Operations Framework (MOF) model and it will help you design an efficient operational management environment, which includes implementing processes and procedures to maintain an efficient working environment.
This document is provided as-is. Information and views expressed in this document, including URL and other Internet website references, may change without notice. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. Copyright 2014 Microsoft Corporation. All rights reserved.
Table of Contents 1 Introduction and Overview ......................................................................................................... 1 2 Best Practices for Lync Server Environments ........................................................................... 1 2.1 Capacity and Availability Management ............................................................................... 2 2.1.1 Capacity Management .................................................................................................. 2 2.1.2 Availability Management ............................................................................................... 3 2.2 Change Management.......................................................................................................... 4 2.2.1 Managing the Timing of Changes ................................................................................. 6 2.3 Configuration Management ................................................................................................. 6 2.3.1 Implementing Configuration Management .................................................................... 7 2.3.2 Tools Used for Configuration Management .................................................................. 7 2.3.3 Relationship with Change Management ....................................................................... 7 2.4 System Administration ........................................................................................................ 8 2.4.1 System Troubleshooting ............................................................................................... 8 2.4.2 System Troubleshooting Process ................................................................................. 8 2.4.3 Issue Management Tools ............................................................................................. 9 2.4.4 Centralized vs. Decentralized Administration ............................................................. 10 2.5 Service Level Agreements ................................................................................................ 10 2.5.1 External Customers .................................................................................................... 10 2.5.2 Internal Customers ..................................................................................................... 11 2.5.3 Typical Criteria ............................................................................................................ 11 2.6 Documentation .................................................................................................................. 11 2.6.1 Document Management Systems .............................................................................. 11 2.6.2 Databases ................................................................................................................... 11 3 Using Administrative Tools ...................................................................................................... 12 3.1 Using Topology Builder ..................................................................................................... 12 3.2 Using Lync Server Control Panel ..................................................................................... 12 3.3 Lync Server 2010 Logging Tool ........................................................................................ 13 3.4 Lync Server Management Shell ........................................................................................ 14 3.5 Lync Server Best Practices Analyzer ................................................................................ 14 4 Operations Management ......................................................................................................... 15 4.1 Standard Procedures ........................................................................................................ 16 4.2 Emergency Procedures..................................................................................................... 16 4.3 Daily Tasks ........................................................................................................................ 16 4.3.1 Performing Physical Environmental Checks ............................................................... 17 4.3.2 Performing and Monitoring Backups .......................................................................... 18 4.3.3 Checking Disk Usage ................................................................................................. 19 4.3.4 Checking Event Logs .................................................................................................. 20 4.3.5 Monitoring Lync Server 2010 Performance ................................................................ 21 4.3.6 Monitoring Operating System ..................................................................................... 24 4.3.7 Central Management Store Replication Status .......................................................... 26 4.3.8 Monitoring Network Performance ............................................................................... 26 4.3.9 Scanning for Viruses and Checking Virus Definitions ................................................ 28 4.3.10 Viewing and Analyzing Monitoring Server Reports ................................................ 29 4.3.11 Validating Voice Number Normalization and Routing ............................................ 32 4.3.12 Validating Address Book Access ............................................................................ 33 4.3.13 Validating Address Book Web Query ..................................................................... 36
4.3.14 Viewing Status of Pools .......................................................................................... 38 4.3.15 Monitoring Back End Lync Server 2010 Storage Performance .............................. 38 4.3.16 Monitoring Group Chat ........................................................................................... 39 4.3.17 Validate Domain Name System (DNS) Settings for DNS Load Balancing ............. 40 4.3.18 Running Synthetic Transactions ............................................................................. 41 4.4 Weekly Tasks .................................................................................................................... 96 4.4.1 Archive Event Logs ..................................................................................................... 96 4.4.2 Create Reports ........................................................................................................... 96 4.4.3 Incident Reports .......................................................................................................... 96 4.4.4 Check IIS Logs and Performance ............................................................................... 97 4.4.5 Generate Database Reports ....................................................................................... 97 4.4.6 Check for Security and Lync Server Updates............................................................. 97 4.4.7 Run the Lync Server 2010 Best Practice Analyzer..................................................... 98 4.4.8 Review SLA Performance Figures ............................................................................. 98 4.4.9 Review System Center Operations Manager Management Pack and Quality of Experience Reports .............................................................................................................. 98 4.4.10 Generating and Viewing Database Reports for Enterprise Pools .......................... 98 4.4.11 Running Bandwidth Utilization Analyzer ................................................................. 99 4.5 Monthly Tasks ................................................................................................................. 100 4.5.1 Viewing Status of Global Settings for a Forest ......................................................... 100 4.5.2 Viewing Edge Server Settings .................................................................................. 115 4.5.3 Check Lync Server 2010 Server Certificates ............................................................ 117 4.5.4 Check Trunk Configuration against a Phone Number .............................................. 117 4.5.5 Check voice normalization rule ................................................................................. 117 4.5.6 Test Telephone Number against a Voice Policy ...................................................... 118 4.5.7 Test telephone Number Against a Voice Route ....................................................... 118 4.5.8 Test Voice Configuration .......................................................................................... 118 4.5.9 Test Voice Rules, Routes, and Policies .................................................................... 118 4.5.10 Security Checks .................................................................................................... 119 4.5.11 Capacity Planning ................................................................................................. 119 4.5.12 Disaster Recovery Test ........................................................................................ 119 4.6 As-Needed Tasks ............................................................................................................ 120 4.6.1 Backup (and Restore) Policies or Configuration Settings ........................................ 122 5 Operations Checklists ............................................................................................................ 122 5.1 Daily Operations Checklist .............................................................................................. 123 5.2 Weekly Operations Checklist .......................................................................................... 130 5.3 Monthly Operations Checklist ......................................................................................... 132 6 Monitoring Lync Server 2010 with System Center Operations Manager .............................. 135 6.1 Introduction to the Microsoft Lync Server Management Pack ........................................ 135 6.1.1 Getting the Latest Management Pack and Documentation ...................................... 135 7 Operational Dependencies .................................................................................................... 135 8 Troubleshooting ..................................................................................................................... 138 9 Key Health Indicators (KHIs) ................................................................................................. 138
1
1 Introduction and Overview This document describes the operational processes, tasks, and tools required for you to maintain a Microsoft Lync Server 2010 communications software environment. It explains how to manage Lync Server 2010 according to the Microsoft Operations Framework (MOF) model and it will help you design an efficient operational management environment, which includes implementing processes and procedures to maintain an efficient working environment. There is also a Lync Operations guide in the TechNet Library, but this guide differs in providing specific tasks at daily, weekly, and monthly intervals, and checklists to track ongoing operational issues. Both guides can and should be used in the ongoing operation of your Lync implementation, and where appropriate, this guide refers to the TechNet Library document. Understanding MOF MOF is a collection of best practices, principles, and models that provide organizations technical guidance about the management of IT assets, such as daily Lync Server 2010 operations. Following MOF guidelines can help you achieve mission-critical production system reliability, availability, supportability, and manageability for Microsoft products. For more information, see Microsoft Operations Framework 4.0. Learning about best practices for Lync Server 2010 We recommend that you implement practical and proven procedures to manage Lync Server 2010. Using tried, tested, and documented methods of managing operations may be more efficient than developing your own methods. Separating operations into daily, weekly, and monthly processes Document the required operational tasks that you will regularly perform. Documenting how you perform tasks helps to ensure that your information is preserved when there is a change in your operational environment such as when new technologies are deployed or staff changes occur. We recommend that operational tasks be separated into manageable workloads where tasks are performed on a daily, weekly, and monthly basis. Daily tasks would focus efforts on the functioning of a system, and monthly tasks would focus more on ensuring the long-term health of a system. This document is applicable to both environments deploying only instant messaging/presence (IM/P) components or IM/P with Enterprise Voice. When tasks or checklist items are specific to Enterprise Voice, this is mentioned and if your environment does not include Enterprise Voice the portion may be skipped. Deploying the tools required for operating Lync Server 2010 Many tools are available to help troubleshoot issues, automate tasks, and help monitor and maintain the Lync Server 2010 environment. Define a standard set of tools for your organization so the tasks performed by the operations team are performed accurately, efficiently, consistently, and in a controlled manner. You should also implement processes to track incidents and major configuration changes. 2 Best Practices for Lync Server Environments For the benefit of readers not already familiar with the basics of server management in general, we provide an overview of server management practices. Readers already familiar with server management may choose to skip this section. Best practices are recommendations that are based on the knowledge and experience that IT professionals have gained across many environments. They provide standard procedures for typical tasks that your Lync Server administrators must accomplish daily, and list the tools that they should use to manage a Lync Server environment. Typical tasks for Lync administrators include the following: Operations Management and Monitoring of a Lync Server 2010 Environment
2
Capacity and Availability Management Define how and what to measure to predict future capacity requirements and to report about the capacity, reliability, and availability of your systems. You must make sure that servers that are running Lync Server are sized to handle the load on the system, and that unplanned downtime is kept under the levels defined in the service level agreement (SLA). Additionally, you will have to upgrade hardware to continue to meet the defined requirements. Change Management and Configuration Management Control how changes are made to your IT systems. This should include testing, application feedback and contingency plans, documentation of all changes, and approval from management if issues occur. Keep a record of your software and hardware assets and their configurations. System Administration Outline standard methods for doing administrative tasks such as database administration and site administration. Security Administration Have a detailed policy and plan that protects data confidentiality, data integrity, and data availability of your IT infrastructure. This includes day-to-day activities and tasks that are related to maintaining and adjusting the IT security infrastructure. System Troubleshooting Outline methods for dealing with unexpected issues, including steps to prevent similar issues in the future. Service Level Agreements Maintain a set of goals for the performance of your IT systems and regularly measure performance against these goals. Documentation Document standard procedures, such as configuration information and lessons learned, and make them available to the staff members that need them. As changes to the configuration are made, update the documentation accordingly. 2.1 Capacity and Availability Management The purpose of capacity management and availability management is to measure and control system performance. We recommend that you implement capacity management and availability management procedures so that you can measure and control system performance. You need to know if the system is available and if it can handle the current and the projected demands by setting baselines and monitoring the system to look for trends. 2.1.1 Capacity Management Capacity management involves planning, sizing, and controlling service capacity to help ensure that the minimum performance levels specified in your SLA are exceeded. Good capacity management helps to ensure that you can provide IT services at a reasonable cost and still meet the levels of performance defined in your SLAs with the client. These criteria can include the following: System Response Time This is the measured time that the system takes to do typical actions. Examples include the time it takes for the audio/video server role to process audio/video traffic, the time that it takes for a client to create and join a conference, or the time taken for presence to be updated in all watcher clients. Storage Capacity This is the capacity of a storage system, whether it is a content database, a backup device, or a local drive. Examples include the maximum amount of storage space to be provided per site and the amount of time that backups should be stored before being overwritten. Adjusting capacity is frequently a case of ensuring that enough physical resources are available, such as disk space and network bandwidth. The following table lists typical resolutions for capacity-related issues. Operations Management and Monitoring of a Lync Server 2010 Environment
3
Typical resolutions for capacity-related issues Issue Possible resolution Remote users having poor audio/video performance Check to see if appropriate bandwidth is available on the WAN links and if QoS is enabled and properly configured. Check QoE data. Overall response of the Lync environment is slow. Run tests to check that the existing front-end servers are capable of dealing with the load. Introduce a new front- end server if required. Check SQL database response times and fix the causes for the delays (for example, improve disk I/O).
Troubleshooting in greater detail is covered in the Lync Server Networking Guide. Most of the Call Quality Methodology discussion in that document is transferable to Lync 2010. Capacity is affected by system configuration and depends on physical resources such as network bandwidth. For example, if a Lync environment is configured to perform a full backup nightly , care must be taken to help ensure that the impact on the interactive performance experienced by end users is minimized. Capacity management is the process of keeping the capacity of a system within acceptable levels and addresses the following issues: Reacting to changes in requirements Capacity requirements have to be adjusted to account for changes in the system or the organization. For example, if your environment decides to start using Enterprise Voice, the number and placement of Mediation Servers and public switched telephone network (PSTN) gateways will be crucial. Should you be doing Session Initiation Protocol (SIP) trunking or direct SIP, the overall design will substantially be modified to provide the best Enterprise Voice performance. Predicting future requirements Some capacity requirements change predictably over time. By tracking trends you can plan upgrades in advance. For example, available bandwidth between various Lync sites will need to be monitored to create a baseline. This baseline will allow you to predict when you will need to add additional bandwidth to these links as user count in these remote sites increases with time. 2.1.2 Availability Management Availability management is the process of ensuring that any IT service consistently and cost effectively delivers the level of consistent, reliable service that is required by the customer. Availability management is concerned with minimizing loss of service and with ensuring that appropriate action is taken if service is lost. In a Lync environment, you may be concerned about whether the Enterprise Voice service is available, whether users can join scheduled conferences, and so on. An SLA defines an acceptable frequency and length of outages and allows for certain periods when the system is unavailable for planned maintenance. If you have to provide reports to your management about the availability of systems, or if you have financial or other penalties associated with missing availability targets, you must record availability data. Even if you do not have such formal requirements, it is a good idea to at least know how frequently a Operations Management and Monitoring of a Lync Server 2010 Environment
4
system has failed in a certain time period; for example, system availability in the last 12 months and how long it took to recover from each failure. This information will help you measure and improve your teams effectiveness in responding to a system failure. It can also provide you with useful information if there is a dispute. Measures related to availability are as follows: Availability This is typically expressed as the time that a system or service is accessible compared to the time that it is down. It is typically expressed as a percentage. (You may see references to three nines or five nines. These refer to 99.9 percent or 99.999 percent availability.) Reliability This is a measure of the time between failures of a system and is sometimes expressed as mean (or average) time between failures (MTBF). Time to Repair This is the time taken to recover a service after a failure has occurred and is often expressed as mean (meaning average) time to repair (MTTR). Availability, reliability, and time to repair are related as follows: Availability = (MTBF MTTR) / MTBF For example, if a server fails twice over a six-month period and is unavailable for an average of 20 minutes, the MTBF is three months or 90 days and the MTTR is 20 minutes. Therefore, Availability = (90 days 20 minutes) / 90 days = 99.985 percent Availability management is the process of ensuring that availability is maximized and kept within the parameters defined in SLAs. Availability management includes the following processes: Monitoring Examining when and for how long services are unavailable. Reporting Availability figures should be regularly provided to management, users, and operations teams. These reports should highlight trends and identify areas that are doing well and areas that require attention. The report should summarize compliance with targets set in the SLAs. Improvement If availability does not meet targets that are defined in the SLAs or where the trend is toward reduced availability, the availability management process should plan remedial steps. This should include working with other responsible teams to highlight reasons for outages and to plan remedial actions to prevent a recurrence of the outages. Capacity and availability measurements are repetitive tasks that are ideally suited to automated tools and scripts such as Microsoft System Center Operations Manager, which is discussed later in this document. 2.2 Change Management Changes to your IT environment are inevitable. Changes include new technologies, systems, applications, hardware, tools, processes, and changes in roles and responsibilities. An effective change management system lets you introduce changes to your IT environment quickly and with minimal service disruption. A change management system brings together the teams involved in modifying a system. For example, deciding to take advantage of the Office Web Apps. This is an integrated Lync Service application that enables users to read and edit documents within a browser. The implementation of this service, after you have gone into production, requires the involvement of several teams: Test Team This team load-tests the Office Web Apps on a test server, in the process providing information about the expected usage patterns and expected performance of the production servers. Operations Management and Monitoring of a Lync Server 2010 Environment
5
Lync Administrators This team determines the deployment strategy and scripts the installation where possible. The team is responsible for ensuring that the change is deployed on the production environment, and it is responsible for administration afterwards. The team must understand the effect of the changes and incorporate them in procedures before the changes are put into production Network Team This team is responsible for changes to firewall rules that allow access from the Internet to the internal Lync pool servers. The team is also responsible in working with the Lync administrators for ensuring that the amount of available bandwidth can support the additional load. Security Team This team assesses security and minimizes risks. The security team must review known vulnerabilities and help to ensure that security risks are minimized. User Acceptance Team This team is composed of users who are willing to test the system and offer feedback for improvements. The change management process defines the responsibilities of each team and schedules the work to be performed, incorporating checks and tests where they are required. Change controls will vary depending on the complexity and expected effect of a change. They can vary from automatic approval of minor changes, to change review meetings, to full project-level reviews. To illustrate this better, the groups of changes are discussed in this section. Major Changes Major changes have a global effect on the system and may require input from various teams. An example of this is upgrading to Lync Server 2010. Major changes affect many different teams and perhaps different systems. The change management process will probably include one or more change review meetings to inform the teams that will be involved in the change or be affected by the change. Significant Changes Significant changes require significant resources to plan, build, and implement. Appropriate change controls should be introduced to help ensure that the effect of the change is understood, deployment procedures are tested, and the rollback and contingency plans are ready. An example of a significant change is deploying a new cumulative update. Minor Changes Minor changes do not significantly affect the IT environment, for example, modifying certain Lync policies via the Microsoft Lync Server 2010 Control Panel. Standard Changes Standard changes are performed regularly and are well understood and documented. The change management process should review all changes to the procedure, but it should not, for example, be involved in creating every content database. The following example of change management examines how different teams interact and the actions that are performed when a new service pack is deployed. These actions are organized and managed by the change management process. Raise a change request The security team has assessed the latest service pack and confirmed that it resolves a possible vulnerability in the production system. The team raises a change request to have the new cumulative update applied to all servers running Lync Server. Service pack release notes review The Lync administrator team reviews the service pack release notes to identify the effect on the system. A series of lab tests is done The Lync administrator team must perform test updates on a server in a non-production environment to decide whether the service pack can be applied successfully without affecting any of the installed applications and server systems. If there are third-party or internally created applications that interface with Lync Server in a production environment, these should be also tested. These tests can also be used to estimate the time required to perform the upgrades. Operations Management and Monitoring of a Lync Server 2010 Environment
6
Users are informed of the outage The Lync administrator team, communications team, or user help desk informs all affected users about the planned maintenance cycle and how long the service will be unavailable. A full backup of Lync is performed before the upgrade The Lync administrator team must make sure that there is a valid backup in place to be able to revert to the original system state if the service pack installation fails. We recommend that the backup be restored to a standby server to have this system readily available if there are issues. The cumulative update is deployed The Lync administrator team does the installation during the planned maintenance cycle. 2.2.1 Managing the Timing of Changes We recommend that you implement a procedure for scheduling changes to avoid disruptions in overlapping sections of your work. For example, two teams may both be planning a minor change to a system. One team may be applying a cumulative update on a pool while another team is migrating legacy users into that pool. Neither team is affected by the changes that the other team is planning, and each team may not necessarily know about changes that the other team is planning. If both changes occurred at the same time, there could be issues implementing the changes. Also, if there are issues after the changes have been applied, for example, if the user migration fails, it may be difficult to decide which change should be rolled back. There should be regular maintenance periods set up between IT and management to test the changes and accept them. 2.3 Configuration Management Configuration management is the process of recording and tracking hardware and software assets and system configuration information. It is generally used to track software licenses, maintain a standard hardware and software build for client computers and servers, and define naming standards for new computers. Configuration management generally covers the following categories: Hardware This category tracks the pieces of equipment that the IT organization owns, where equipment is located, and who uses equipment. This information enables an organization to plan and budget for upgrades, maintain standard hardware builds, report on the value of IT assets for accounting purposes, and help prevent theft. Software This category tracks software that is installed on each computer, the version numbers, and where the licenses are held. This information helps plan upgrades, ensure that software is licensed, and detect the existence of unauthorized (and unlicensed) software. Standard Builds This category tracks the current standard build for the client computers and servers and whether the client computers and servers meet this standard. The existence and enforcement of standard builds helps support staff because the staff is required to maintain only a limited number of versions of each piece of software. Cumulative Updates and Hotfixes This category tracks which service packs are tested and approved for use and which computers are up to date. This information is important to minimize the risk of computers being compromised and to detect users who have installed unapproved updates. System Configuration Information This category tracks the function of a system, the interaction between system elements, and the processes that depend on the system running smoothly. For example, third-party proxy server may be configured on a single server. The proxy servers dependence on this server should be understood and contingency plans may be required if there is a Operations Management and Monitoring of a Lync Server 2010 Environment
7
failure. If the proxy server can be configured to also communicate with another front-end server, dependencies and contingency plans will probably change. 2.3.1 Implementing Configuration Management After you determine the purpose of your configuration management exercise and decide what items need managing, you need to implement configuration management by collecting data and reporting data. The simplest approach for small organizations is to collect data manually (number and model of client computers, operating system, software installed) and save it in an Office Word or Office Excel document. For larger, more complex, and constantly changing systems, the discovery of assets and collection of detailed information must be automated. Decide what information is relevant to your organization and record it in a database. The configuration management database is a useful tool for support staff and management in the following areas: Security Audits The database enables you to identify servers running Lync Server and client computer systems that need to have hotfixes applied or that have missed the installation of a service pack or the latest antivirus updates. Software Installation Identifying client software versions and tracking them will aid administrators in planning version updates and new installs as well as assisting with licensing documentation and compliance. Configuration Information If you maintain an up-to-date list of all settings that have been modified from their default, then you will be able to troubleshoot issues quickly and more effectively. Planning Upgrades If a capacity review reveals that additional storage space is required on your Lync database servers, its important to know if each server has an internal RAID controller. If they do, then are they the same model? Do they have the same number of disks installed? The configuration management database will indicate the type of disk that can be installed, the number, and the upgrade path in each case. 2.3.2 Tools Used for Configuration Management There are many tools to discover, audit, and report assets. Some of these tools are discussed in this section. Automated Scripts You can write simple scripts to report items like the operating system, service pack level, and existence of software on a specific set of computers. You can write these scripts to an organizations exact requirements; however, the required number of scripts and their complexity can make scripts expensive to create and maintain. Automated Tools Depending on the size of your business and your organizational needs, you may want to consider using automated tools. Tools such as System Center Configuration Manager incorporate standard report templates (such as service pack level) and also enable you to create customized reports, for example, for a custom application. The System Center Operations Manager can also be used to report on hardware and software configurations. 2.3.3 Relationship with Change Management Configuration management is closely related to change management. Configuration management identifies the need for change and identifies and records that a change has occurred. For example, the Operations Management and Monitoring of a Lync Server 2010 Environment
8
configuration management database can be used to identify servers that require a hotfix. Change management then defines the process for applying the hotfix. Conversely, if a new cumulative update is rolled out, the change management process should supply this information to the configuration management system. The configuration management tools will probably need to be configured to identify the new software so that they can discover and track where and when the software is deployed. 2.4 System Administration System administration includes the day-to-day administrative tasks, both planned and on-demand, that are required to keep an IT system operating smoothly. Typically, system administration tasks are covered by written procedures. These procedures help to ensure that the same standard tools and methods are used by all support staff. In a Lync environment, typical system administration tasks include backing up and archiving pools, monitoring logs, creating and managing users, and updating antivirus software. 2.4.1 System Troubleshooting An organization must be prepared to deal with unexpected issues and should have a procedure to manage issues from the point at which they are reported until their resolution. Information about how support staff diagnosed an issue should be recorded and used in the future to avoid unnecessarily repeating completed work. 2.4.2 System Troubleshooting Process The following diagram shows the system troubleshooting process and the interactions with other operations roles. System Troubleshooting Flowchart Operations Management and Monitoring of a Lync Server 2010 Environment
9
Classify and Prioritize This task is typically performed by the service desk. For example, an issue may be grouped as a software issue or a hardware issue. The issue is then routed to the appropriate support team for investigation. The rules for determining the priority of an issue, together with the time to respond and time to resolve, are typically defined in the SLA. Investigate and Diagnose The appropriate support team diagnoses the issue and proposes changes to resolve the issue. If the solution is simple and does not require change control, the solution can be applied immediately. If the solution is not simple, a request for change should be raised and the proposed work should be managed by the change management process, frequently under a fast-track procedure. Any changes that are made should be recorded using the configuration management process. Close and Record After testing the resolution, the issue should be closed. If there are lessons to be learned from the issue, an entry should be created in the knowledge base. Review and Trend Analysis Periodic reviews of recent issues should be performed to identify issue trends. For example, if your users are experiencing frequent issues with slow logons to their Lync sites, network bandwidth issues may be the cause. Issue resolution times and the effect of any outages on system availability should be reviewed and compared with the SLA. The person who liaises with the customer on service issues, such as an account manager, should be informed of any significant issues. 2.4.3 Issue Management Tools Service desk tools enable staff to record, classify, and prioritize new issues. Tools will then provide the workflow processes to manage the issue service request through investigation and diagnosis, often by more than one support team. Tools, which will frequently provide reports about resolution times and historical trends, may also include a knowledge base database, which can be used to search through past issues. Operations Management and Monitoring of a Lync Server 2010 Environment
10
The Microsoft Knowledge Base is a useful record of support issues that have been encountered by Microsoft. For more information, see the Microsoft Support website (http://go.microsoft.com/fwlink/?linkid=14898). Third-party software typically requires customization to suit the organizations needs, such as the organization of teams, reporting requirements, and measures required by the SLA.
2.4.4 Centralized vs. Decentralized Administration Roles and responsibilities for performing system administration tasks depend on whether the organization follows a centralized model, a decentralized model, or a combination of both. Centralized Model In a centralized model, one or several administrative groups maintain complete control of the whole Lync Server environment. This administrative model is similar to a data center where all administration tasks are performed by a single information technology group. Roles and responsibilities within the team should be defined according to experience and expertise. Decentralized Model Decentralized organizations are located in several geographic locations and have servers running Lync Servers and teams of administrators in different locations. For example, there may be local administration staff and one or more servers running Lync Server 2010 for each country/region. Alternatively, there may be a pool of servers running Lync Server 2010 and an administrative team for North America and one for Europe. Sometimes, you may want administrators to be responsible only for their own geographical area and restrict permissions to administer resources in other areas. Lync Server also allows you to delegate specific administrative tasks to specific individuals or groups by using role-based access control (RBAC). RBAC allows administrators to delegate specific user rights and permissions to other administrators to perform a subset of the administrative tasks possible. With RBAC, the users ability to carry out specific administrative tasks depends on the RBAC roles that are assigned to the user. RBAC provides a list of cmdlets that the user is allowed to run based on the RBAC roles that the user is a member of. 2.5 Service Level Agreements The SLA is a document that defines the services that your customer expects from you. The complexity and content of this document depends largely on whether customers are internal (within your environment) or external. 2.5.1 External Customers If your customer is external, the SLA may be part of a legal contract with financial incentives and penalties for performance that falls inside or outside defined levels of service. Defining these levels of service should be part of the overall contract negotiation. As with all contracts, its important that both parties understand expectations. The SLA defines these expectations. The contents of the document should change infrequently and only because of negotiations with the customer. Operations Management and Monitoring of a Lync Server 2010 Environment
11
2.5.2 Internal Customers If your customer is internal, you may still want to define the services that are expected of operations teams and of IT systems. The SLA may be created by the operations staff and intended as a set of goals for the availability of IT services within your organization. Alternatively, performance levels may be set by management and used as benchmarks when assessing staff performance. 2.5.3 Typical Criteria SLAs include sections that define criteria of minimum levels of availability, support, and capacity. Availability Define the hours and the operating systems on which sites and other Lync services will be available. Any routine maintenance that affects service availability should be defined. Define external factors that affect service, for example, the loss of Internet connectivity. Support Define the hours when support for a system will be available. Specify methods for customers to contact support staff, how incidents are grouped, and target time to respond and to resolve the incident. Define frequency and content of feedback to the customer. Capacity Define the maximum allowed size of Lync sites and the steps to take if the limit is exceeded. Define the maximum allowed time to do standard tasks, such as the time to retrieve a document from a document library. Define the maximum number of users per Lync pool and agree to a process to increase capacity if more users are added. 2.6 Documentation The MOF model is composed of many service management functions. Documentation about how and when tasks are performed can be shared with members of the same team or with other teams. The method of storing and sharing documentation can vary according to the type of function. For example, the procedures for system administration may be stored as Word documents because they are likely to be printed and referenced frequently. Configuration management information may be automatically generated and stored in a database for easy searching and indexing. Some documentation may be sensitive and should be restricted. 2.6.1 Document Management Systems A documentation management system acts as a central repository for documents and helps to ensure that only the latest revision of a document is available. You can also consider archiving the older version of the document for reference purposes. Lync Server provides functionality suitable to this task. 2.6.2 Databases Several tools and management functions have been discussed that are suited to using databases. The configuration management process is likely to use automated processes that store large amounts of data that require indexing and searching. Support staff may search a database of past issues and resolutions when troubleshooting new issues. It is likely that there will be different databases being used for different purposes. Decide if these databases should be linked or consolidated. For example, if the service desk identifies several issues with a common theme (such as new software causing an issue with a particular network card), the support staff can query the configuration database to predict how many computers might be affected. Operations Management and Monitoring of a Lync Server 2010 Environment
12
3 Using Administrative Tools The Lync Server 2010 administrative tools consist of the following: Lync Server Deployment Wizard Use to deploy Lync Server 2010. Lync Server Topology Builder Use to add any components to your deployment. Lync Server Control Panel Use for ongoing management of your deployment. Lync Server Logging tool Use to troubleshoot problems in your deployment. Lync Server Management Shell Use to manage your deployment at the command line. Lync Server Best Practice Analyzer Lync Server 2010, Best Practices Analyzer is a diagnostic tool that gathers configuration information from Lync Server 2010 environments and determines whether the configuration is set according to Microsoft best practices. The administrative tools are installed by default on each server on which you installed Lync Server 2010. Additionally, you can install the administrative tools on other computers, such as administrative consoles, to facilitate planning and deployment for your organization. You also use most or all of these tools to manage your deployment. For details about installing Topology Builder and the other administrative tools, see Topology Builder Requirements for Installation, Publishing, and Administration in the Deployment documentation and Install Lync Server Administrative Tools in the Operations documentation. When you manage your deployment, you use the following two tools primarily: Topology Builder and Lync Server Control Panel. Note In Lync Server 2010, the Lync Server Management Shell is a new method of administration and management. Lync Server Management Shell is a powerful management interface, built on the Windows PowerShell command-line interface that includes a comprehensive set of cmdlets that are specific to Lync Server 2010. With Lync Server Management Shell, you gain a rich set of configuration and automation controls. Topology Builder and Lync Server Control Panel both implement subsets of these cmdlets to support management of Lync Server 2010. The Lync Server Management Shell includes cmdlets for all Lync Server 2010 administration tasks, and you can use the cmdlets individually to manage your deployment. For details, see Lync Server Management Shell in the Operations documentation. 3.1 Using Topology Builder If you need to make changes to your topology after your initial deployment (for example, to add a server to your topology), you must run Topology Builder to make the changes, and then publish the topology again prior to deploying the new component in your topology. Documentation on Topology Builder can be found in the deployment documentation at http://technet.microsoft.com/en-us/library/gg398788(v=ocs.14).aspx. 3.2 Using Lync Server Control Panel You can use Lync Server Control Panel to perform most of the administrative tasks required to manage and maintain Lync Server 2010. Use either of the following procedures to open Lync Server Control Panel so you can make changes to the configuration settings for your deployment. Note You can use a user account that is assigned to the CSAdministrator role to perform any task in Lync Server Control Panel. You can use other roles to log on to Lync Server Control Panel to administer specific functionality, dependent on the task you need to perform. For example, you can use Operations Management and Monitoring of a Lync Server 2010 Environment
13
CSArchivingAdministrator to administer Archiving in Lync Server Control Panel. For details about roles, see Role-Based Access Control in the Planning documentation. For information about the roles that can be used to perform a specific procedure, see the documentation for the procedure. To open Lync Server Control Panel using the administrative URL 1. From a user account that is assigned to the CsAdministrator role or other role that has appropriate user rights and permissions for the task to be performed, log on to any computer in your internal deployment. 2. Start Lync Server Control Panel: Open a browser window, and then enter the Admin URL provided by your organization. To open Lync Server Control Panel on a computer running Lync Server 2010 1. From a user account that is a member of the CsAdministrator role or other role that has appropriate user rights and permissions for the task to be performed, log on to a computer on which you have installed Lync Server 2010. 2. Start Lync Server Control Panel: Click Start, click All Programs, point to Administrative Tools, point to Microsoft Lync Server 2010, and then click Lync Server 2010 Control Panel. Further documentation about Lync Server Control Panel is available at http://go.microsoft.com/fwlink/?LinkId=392057&clcid=0x409. 3.3 Lync Server 2010 Logging Tool Microsoft Lync Server 2010 Logging Tool integrates a range of logging and tracing functionality. It facilitates troubleshooting by capturing logging and tracing information from the product while the product is running. Logging Tool is installed with the Lync Server administrative tools. You can use Logging Tool to run debug sessions on any Lync Server role. Lync Server 2010 Logging Tool generates log files on a per-server basis, so it must be actively running and tracing on each computer for which you want to generate a log. Note: To view and analyze log files on a computer other than the one on which the logs were captured, you can run the Lync Server 2010 Logging Tool on the other computer by installing the Lync Server administrative tools on that computer. OCSLogger.exe is the file that runs Logging Tool, which by default is installed to %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing. You can run OCSLogger.exe on computers running the Windows XP, Windows Vista, or Windows 7 operating systems as well as on Windows Server 2003, Windows Server 2008 SP1, and Windows Server 2008 R2. To run OCSLogger.exe on Windows Vista or Windows 7, you must run the application in the Run as Administrator mode. To Start Lync Server 2010 Logging Tool Do one of the following: Open a command prompt (Command Prompt, Windows PowerShell, or Lync Server Management Shell). Navigate to the folder where OCSLogger.exe is installed (by default, %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing), type OCSLogger.exe, and then press Enter. On the Start menu, open the Run dialog box. Type the full path and file name, by default %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing\OCSLogger.exe), and then click OK. On the Start menu, click inside the Search box. Type OCSLogger.exe, and then press Enter. Operations Management and Monitoring of a Lync Server 2010 Environment
14
For further details about the Logging Tool, see the Lync Server 2010 Logging Tool documentation in the TechNet Library at http://go.microsoft.com/fwlink/p/?LinkId=199265. 3.4 Lync Server Management Shell Lync Server 2010 introduces a large set of new and improved features compared to what was available in Microsoft Office Communications Server 2007 R2. One improvement is the way in which you manage your implementation. For example, theres a new user interface, called the Lync Server Control Panel, which represents a big shift from what most people are used to with the Microsoft Management Console (MMC). The other major improvement to manageability is the inclusion of Windows PowerShell. Windows PowerShell allows you to manage Microsoft applications from the command line. It includes a command-line environment, product-specific commands, and a full scripting language. Windows PowerShell was first introduced as a downloadable release for the Windows operating system late in 2006, and was incorporated as the command-line interface for manageability of Microsoft Exchange Server 2007. From that point it continued to grow, and it has been incorporated into most of the Microsoft Server products, the most recent of these being Lync Server 2010. Lync Server 2010 introduces close to 550 product-specific cmdlets that you can use to manage every aspect of your deployment. The following sections contain a list of cmdlets and their descriptions. This information is also available directly from the command line. Simply type the following at the Lync Server Management Shell command prompt: Get-Help <cmdlet name> -Full For example, to retrieve help from the command prompt on the New-CsVoicePolicy cmdlet, type the following: Get-Help New-CsVoicePolicy -Full Things to know about Windows PowerShell in Lync Server 2010: To run the Lync Server 2010 cmdlets, open the Lync Server Management Shell. Lync Server Management Shell is automatically installed on every Lync Server Enterprise Edition Front End Server or Lync Server Standard Edition. New and updated information, sample scripts, and help for getting started and learning more about Windows PowerShell and Lync Server cmdlets is available at the Lync Server 2010 Windows PowerShell Blog, http://go.microsoft.com/fwlink/?LinkId=203150. For detailed cmdlets documentation: Lync Server 2010 Cmdlets by Category at http://technet.microsoft.com/en- us/library/gg398306.aspx Lync Server 2010 Cmdlets Index at http://technet.microsoft.com/en-us/library/gg398867.aspx 3.5 Lync Server Best Practices Analyzer Lync Server 2010 Best Practices Analyzer is a required tool in the sections that follow. Make sure it is installed shortly after implementing Lync Server 2010. You can use the Best Practices Analyzer to identify and resolve issues with your Lync Server deployment. The Best Practices Analyzer gathers configuration information from Lync Server 2010 components. Operations Management and Monitoring of a Lync Server 2010 Environment
15
With the proper network access, the Best Practices Analyzer can examine servers running Active Directory Domain Services (AD DS), Exchange Server 2010 Unified Messaging (UM), and Lync Server. You can use the Best Practices Analyzer to do the following: Proactively perform checks, verifying that the configuration is set, according to recommended best practices. Automatically detect required updates to Lync Server 2010. Generate a list of issues, such as suboptimal configuration settings, unsupported options, missing updates, or practices that we do not recommend. Help you troubleshoot and fix specific issues. The Best Practices Analyzer provides the following features: Minimal installation prerequisites. Online documentation about reported issues, including troubleshooting tips. Configuration information that you can save for later review. State-of-the-art system analysis. Best Practices Analyzer uses a set of XML configuration files to determine the information to gather from your Lync Server environment. In addition to checking AD DS, it checks sources such as the Windows Server operating system registry and settings in Windows Management Instrumentation (WMI). Best Practices Analyzer compares the data it gathers with a set of predefined rules for the settings and configurations of Lync Server 2010 deployments. After comparing the collected data with the predefined rules, the tool reports issues. For every issue that it reports, Best Practices Analyzer provides information about what was found in the scanned Lync Server 2010 environment and the recommended configuration. Best Practices Analyzer also provides links to more detailed information about the specific issues. The Best Practices Analyzer gathers configuration information only from Lync Server 2010 components. You can use the previous version of the tool to scan Office Communications Server 2007 R2 and Office Communications Server 2007 environments. For details about downloading and installing the Lync Server 2010 version of the tool, see Lync Server 2010, Best Practices Analyzer at http://go.microsoft.com/fwlink/?LinkId=211230.
4 Operations Management Operations management involves administering an organizations infrastructure components, and includes the day-to-day administrative tasks, both planned and on-demand, that are required to keep an IT system operating smoothly. In a Lync Server 2010 environment, typical system administration tasks include backing up data, managing settings, monitoring system status and performance, and managing connectivity. Operations Management and Monitoring of a Lync Server 2010 Environment
16
4.1 Standard Procedures Several resources can help you define what standard procedures are required in the organization, and how to perform them. Because each organization is unique, you may have to further customize and adapt these resources to suit everyday requirements. Standard operational procedures change, and documentation occasionally needs to be revised. As changes are made, the change management process, as defined in the Service Management Functions of the Microsoft Operational Framework, should identify how each change is likely to affect how and when administrative tasks are performed. Use the change management function to update and control the procedural documentation. We recommend that operational tasks be separated into manageable workloads, where tasks are performed on a daily, weekly, monthly, and as-needed basis. Daily tasks would focus efforts on aspects that are critical to the functioning of a system and monthly tasks would focus more on ensuring the long- term health of a system. The tasks that must be performed can be separated into the following categories: Daily Tasks Weekly Tasks Monthly Tasks As-Needed Tasks When preparing documentation for operations management, use checklists to help ensure that the required tasks are performed at the appropriate time. For detailed information about preparing operations checklists, see the sample checklists located in Operations Checklists. Frequently, change management takes over where system administration finishes. If a task is covered by a standard procedure, it is part of the system administration function. If there is no standard procedure for a task, it should be handled by using the change management function. 4.2 Emergency Procedures Emergency procedures are on-demand procedures that fall outside the standard procedures described above and should be handled by the change management function (refer to the MOF for a description on the "change management" function). These procedures usually relate to: Implementing change as part of a troubleshooting effortto resolve an identified system issue. Resolve an identified security threatsuch as emergency security patches or software updates. Impact to the business should be evaluated to determine whether a change should be implemented in an ad hoc manner or channeled through the official change management procedures as defined by the Changing Quadrant guidance. We recommend implementing emergency change procedures. For example, document how to deal with these types of procedures and assist with reducing a state of confusion when an emergency change is presented. 4.3 Daily Tasks To help ensure the availability and reliability of the Lync Server 2010 deployment, you must actively monitor elements that are critical to the functioning of the system, which includes the physical platform, Operations Management and Monitoring of a Lync Server 2010 Environment
17
the operating system, and all important Lync Server 2010 services. Preventive maintenance and proactive monitoring will help you identify potential errors and issues that may negatively affect the Lync Server 2010 deployment. Monitoring the Lync Server 2010 deployment involves checking for issues with connections, services, server resources, and system resources. Windows Server operating systems 2008, 2003, and 2003 R2, together with Microsoft System Center Operations Manager 2007, and Lync Server 2010 provide you with many monitoring tools and services to help ensure that the Lync Server 2010 organization is running smoothly. When these technologies are implemented together, administrators will have the ability to receive alerts when or before issues occur. The key advantages to daily monitoring are: Meeting the performance and availability requirements of defined SLAs. Successfully completing specific administrative tasks, such as daily backup operations, and checking server health. Detecting and addressing issues, such as bottlenecks in the server performance, or need for additional resources before they affect productivity. Daily maintenance tasks assist the administrative team to define or establish a criteria or baseline for normal systems operations within the organization, and to detect any abnormal activity. It is important to implement these daily maintenance tasks so that the administrative team can capture and maintain data about the Lync Server 2010 infrastructure, such as usage levels, possible performance bottlenecks, and administrative changes. To help organize the performance of daily tasks, see Daily Operations Checklist. 4.3.1 Performing Physical Environmental Checks Before checking the performance, availability, and functionality of the Lync Server 2010 deployment, you should check the physical environment. For example, the server room temperature might need to be lowered, or a network cable might need to be replaced. For best results, perform the following physical environmental inspections: Physical security measures Physical security protection such as locks, doors, and restricted- access rooms must be secured. Check for any unauthorized and forced entries and signs of equipment damage. Temperature and humidity High temperature, poor air flow, and humidity can cause hardware components to overheat. Check temperature and humidity to help ensure that the environmental systems such as heating and air conditioning can maintain acceptable conditions and function within the hardware manufacturer's specifications. When new equipment has recently been installed, also check that air flow both to and from the servers is unimpeded and meets manufacturer spec. Devices and components The Lync Server 2010 organization relies on a functioning physical network and related hardware. Make sure that routers, switches, hubs, physical cables, and connectors are operational. The specifics on how to perform these checks will depend greatly on your installation site and the server hardware that was chosen. The first time you perform this check, refer to the hardware documentation and make a note of the desired parameters for future reference. Operations Management and Monitoring of a Lync Server 2010 Environment
18
Desired server space environment
Parameter Desired value or range Temperature Humidity Front of server faces Hot aisle / cold aisle Unimpeded exhaust clearance 4.3.2 Performing and Monitoring Backups Your business priorities should drive the specification of backup and restoration requirements for your organization. Performing backups of the servers and data is the first line of defense in planning for a disaster. Computers that run Lync Server 2010 services or server roles must have a copy of the current topology, current configuration settings, and current policies before they can function in their appointed role. Lync Server is responsible for ensuring that this information is passed along to each computer that needs it. The Export-CsConfiguration and Import-CsConfiguration cmdlets are used to back up and restore your Lync Server topology, configuration settings, and policies during a Central Management store upgrade. The Export-CsConfiguration cmdlets enable you to export data to a .ZIP file; you can then use the Import-CsConfiguration cmdlet to read that .ZIP file and restore the topology, configuration settings, and policies to the Central Management store. After that, the replication services of Lync Server will replicate the restored information to other computers running Lync Server services. The ability to export and import configuration data is also used when initially configuring computers located in your perimeter network (for example, Edge Servers). When configuring a computer in the perimeter network, you must first perform a manual replication using the CsConfiguration cmdlets: you will need to export the configuration data using Export-CsConfiguration and then copy the .ZIP file to the computer in the perimeter network. After that, you can use Import-CsConfiguration and the LocalStore parameter to import the data. You only need to do this once; after that, replication will take place automatically. Who can run this cmdlet: By default, members of the following groups are authorized to run the Export- CsConfiguration cmdlet locally: RTCUniversalServerAdmins. To return a list of all RBAC roles, this cmdlet has been assigned to (including any custom RBAC roles you have created yourself), run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Export-CsConfiguration"} All SQL 2008 Back End databases should be backed up as per SQL best practices using the DBExport command. Operations Management and Monitoring of a Lync Server 2010 Environment
19
Regular testing of the Disaster Recovery Plan for your Lync Server 2010 infrastructure should be performed in a lab environment that mimics the production environment as closely as practicable. Refer to the Monthly Tasks for details about Disaster Recovery Testing. Note that the backup frequency can be adjusted, based on your Restore Point and Recovery Point objectives. As a best practice, take regular, periodic snapshots throughout the day. Generally, you should perform full backups every 24 hours. Backup all SQL Back End databases using the dbexport command. 4.3.3 Checking Disk Usage Hard disks drives are a critical component of the Lync Server 2010 deployment. Without sufficient free disk volume, neither the operating system nor the Lync Server 2010 databases can function correctly. You must monitor the Lync Server 2010 back-end database statistics daily to help ensure that servers do not run out of disk space, and to prepare to add storage resources as required. Apart from checking space on disks hosting the operating system, program files, database, and transaction logs (Lync Server 2010 Back End), you should also monitor usage of the file system that includes disk space for file shares containing the following important data: Meeting content Meeting content metadata Meeting compliance logs Application data files (used internally by the application server component) Group Chat Server Web service and compliance folders (to store files uploaded to the Group Chat Web service) Group Chat compliance XML files (containing Group Chat compliance records) Update files (for Device Update Service) Address Book files Lync Server 2010 needs hard disk space to store its databases and transaction logs as well as files on file shares previously listed. You should monitor the disk space regularly to help ensure that the Lync Server 2010 deployment is not negatively affected because of insufficient storage resources. Compare and maintain statistical information about available disk space on each Lync Server 2010 volume and expected growth of the databases and transaction log files. This helps with capacity planning and adding storage when the storage resources are required. To accommodate troubleshooting and disaster recovery situations, we recommend that available free volume space be equal or greater than 110 percent of the size of database. You can check free disk space by using the following methods: 1. System Center Operations Manager System Center Operations Manager can be used to alert administrators when volume space is constrained. 2. Running a script Monitor disk space by running a script that sends you an alert message if the available hard disk space falls below 20 percent. You can find a sample script on Microsoft Script Center on TechNet , look at: Operations Management and Monitoring of a Lync Server 2010 Environment
20
http://gallery.technet.microsoft.com/scriptcenter/site/search?query=hard%20disk%20alert&f%5B0 %5D.Value=hard%20disk%20alert&f%5B0%5D.Type=SearchText&ac=5 3. Windows Explorer Use Windows Explorer to check for disk space on volumes that store Lync Server 2010 logs and databases. 4.3.4 Checking Event Logs You can use Windows Event Viewer to view event logs and obtain information about service failures, replication errors in the AD DS, and warnings about system resources such as virtual memory and disk space. Event Viewer is included with Microsoft Windows Server 2008 and 2012. In the Lync Server 2010 Logging Tool, when you end the debug session, click Analyze Log Files to view the log files by using the Snooper tool. Event Viewer maintains logs about application, security, and system events on the computer. Both Lync Server 2010 and Windows report warnings and error conditions to the event logs. Therefore, make sure that you review event logs daily. Use Event Viewer to: View and manage event logs. Obtain information about hardware, software, and system issues that must be resolved. Identify trends that require future action. A server that is running a Windows Server operating system records events in three types of logs: Application logs The Application log contains events logged by applications or programs. Developers determine which events to log. For example, a database program might record a file error in the Application log. Most Lync Server 2010 Server-related events appear in the Application log. Security logs The Security log records events such as valid and invalid logon attempts as well as events related to resource use such as creating, opening, or deleting files or other objects. For example, if logon auditing is enabled, attempts to log on to the system are recorded in the Security log. System logs The System log contains events logged by Windows system components. For example, the failure of a driver or other system component to load during startup is recorded in the System log. The event types logged by system components are predetermined by the server. Lync Server 2010 The logging tool records significant events related to authentication, connections, and user actions. After enabling diagnostic logging, you can view the log entries in Event Viewer. Note We do not recommend using the maximum logging settings unless you are instructed to do this by Microsoft Product Support Services. Maximum logging drains significant resources and can give many false positives, that is, errors that get logged only at maximum logging but are really expected and are not a cause for concern. We also recommend that you do not enable diagnostic logging permanently. Use it only when troubleshooting. Within each Event Viewer log, Lync Server 2010 Server records informational, warning, and error events. Monitor these logs closely to track the types of transactions being conducted on the Lync Server 2010 servers. You should periodically archive the logs or use automatic rollover to avoid running out of space. Because log files can occupy a finite amount of space, increase the log size (for example, to 50 MB) and set it to overwrite, so that the Lync Server 2010 Server can continue to write new events. Operations Management and Monitoring of a Lync Server 2010 Environment
21
You can also automate event log administration by using the following tools and technologies: System Center Operations Manager System Center Operations Manager 2007 monitors the health and use of Lync Server 2010 servers. Lync Server 2010 Management Pack for Operations Manager 2007 SP1 extends Operations Manager by providing specialized monitoring for servers that are running Lync Server 2010. The Lync Server 2010 Management Pack for Operations Manager monitors Standard and Enterprise Edition of Lync Server 2010. This release also incorporates the Quality of Experience (QoE) Management Pack which was previously a separate management pack. Monitored types are event log entries, performance counters as well as stateful monitoring of QoE. This version of the management pack only monitors Lync Server 2010, and cannot be used to monitor Office Communications Server 2007. The management pack provides the following features: Alerts indicating service impacting events Alerts indicating configuration, and other non-service impacting issues State monitoring of Lync Server services Product knowledge: Cause and resolution of identified issues For more information about Lync Server 2010 Management Pack, refer to Monitoring Lync Server 2010 with System Center Operations Manager. Event Comb The Event Comb tool gathers specific events from the event logs of several computers to one central location. It lets you report on only the event IDs or event sources it specifies. For more information about Event Comb, see the Account Lockout and Management Tools website. Event triggers o In Windows Server 2008 you have the ability to "Attach a Task to This Event" within the Windows Event Viewerwhere an administrator can either run a program, send an email message or display an on-screen message. For more information about this feature, see the Windows Server 2008 R2 topic Run a Task in Response to a Given Event. o You can also use command-line tools such as Eventtrigger.exe to create and query event logs and associate programs with particular logged events. By using Eventtriggers.exe, you can create event triggers that run programs when specific events occur. For more information about event triggers, see the Windows Server 2003 topic New command-line tools. 4.3.5 Monitoring Lync Server 2010 Performance Lync Server 2010 performance is affected by various factors such as user profiles, system architecture, software, hardware components, third-party integration points such as gateways and telephony equipment, network connectivity and performance, Windows Active Directory service configuration and performance as well as the Windows operating system functionality. At the core of a Lync Server 2010 deployments performance is the server software and hardware it is implemented on. As an example, it is essential that a front-end server has sufficient hardware resources to cope with the expected (short-term) user load. If a respective front-end server is required to provide services to 10 thousand users, then an adequately configured server is essential to meet the expected load requirements to ultimately help ensure the best possible end-user experience. Operations Management and Monitoring of a Lync Server 2010 Environment
22
Monitoring server performance is thus critical to gauge whether the implemented server infrastructure have adequate hardware resources for the day-to-day peak-load requirements. Monitoring server performance helps identify system bottlenecks allowing administrators to apply corrective action before the end-user experience is affected. The performance data should be used for long-term capacity planning. While detailed information on all performance objects and counters to be observed is linked to in Monitoring Lync Server 2010 with System Center Operations Manager 2007, a few performance counters you should follow can provide administrators a quick view of the system performance: To track overall system health of the front-end server, a good starting point is to check Processor\% Processor Time. The value should be below 80 percent at all times. To track the performance of the back end Microsoft SQL Server database software instance used by the Front End pool, monitor the following performance counters: LC:USrv 00 DBStore\Usrv 002 Queue Latency (msec) LC:USrv 00 DBStore\Usrv 0 04 Sproc Latency (msec) The healthy server at steady state should show <100 ms latency values. The throttling mechanism will kick when latency reaches 12 seconds, in which case the front-end server starts throttling requests to the back end, causing clients to start receiving a 503 Server too busy error. To track the processing time at the front-end server, monitor the following counter: LC:SIP - 07 - Load Management\SIP - 000 - Average Holding Time For Incoming Messages This is another throttling mechanism on the front-end servers, this time starting when processing time on the front end is high. If average processing time is more than six seconds, the server goes into throttling mode and only allows one outstanding transaction per client connection. To track memory issues at SQL Back End Server, monitor the following counter: SQL Server Buffer Manager\Page life expectancy A low value, below 3600 seconds (together with high latency writes/sec and checkpoint pages/sec) indicates memory pressure. For more detailed information about these and a few other counters that may be helpful in obtaining quick information about your front-end and back-end servers, see the Lync Server Team blog: http://communicationsserverteam.com/archive/2007/09/10/9.aspx. 4.3.5.1.1 Additional Counters to View There are a few key counters that are good indicators of overall health from the frontend server. This is by no means a comprehensive list and is not meant to identify root cause. These counters will let you perform a quick check on your server health. We recommend verifying these counters on each of the servers in the pool. It is important to understand what these counter values are when your server is healthy. A baseline is crucial to understanding what changed when the user experience is degraded. The front-end server can indicate issues that may be due to bottlenecks elsewhere in the system. This means it is the best place to start when looking at overall system health. Two additional counters to review first are: LC:USrv-00-DBStore\Usrv-002-Queue Latency (msec) LC:USrv-00-DBStore\Usrv-004-Sproc Latency (msec) Operations Management and Monitoring of a Lync Server 2010 Environment
23
The queue latency counter represents the time a request spent in the queue to the back end and the Sproc latency represents the time it took for the back end to process the request. If, for any reason, disk, memory, network, and processor on the back end are in trouble, the queue latency counter will be high. It can also be high if there is high network latency between the front end and the back end. The next question is: What is acceptable queue latency? At 12 seconds, the Front End Servers will start throttling requests to the Back End Servers. This means the servers will start returning Server too busy 503 errors to the clients. A healthy server should have less than 100 msec DBStore queue latencies at steady state, but during times where the server has just come online and users are all logging in at the same time, that counter can be quite high and you may even see it hit multiple seconds. You may have a load-balanced configuration, where you have a pool deployed with multiple front-end servers and a load balancer that is configured for "least number of connections." In this case, if one front- end server is restarted, then all users who attempt to reconnect will be pointed to the restarted server, since that server will have fewer connections compared to the other pool members. During this time, the respective front-end server may be overloaded while the other pool members are not. We recommend that you perform maintenance during off hours to mitigate the performance impact as users will not all be competing to connect to the server at the same time. If the above two performance counters are high, the most likely bottleneck is the SQL Back End Server. The next components to confirm are: Is the SQL Server CPU too high; for example, is it greater than 80 percent? Is the disk latency high? In an ideal world, you have enough RAM to have the entire RTC and RTCDYN databases in memory. Then, the only reason the server would be accessing the disk, is to write to the log files and flush to the databases. Tests have shown that 12 GB of RAM is sufficient for 100 thousand user deployments. This is based on the assumption that the RTC and RTCDYN databases size totals less than 12 GB. If your databases are larger than that, then you may need additional memory. You can determine whether your SQL Server requires additional RAM by reviewing the SQL Server Buffer Manager page life expectancy performance counter. A value less than 3,600 indicates memory pressure. You should also see little to no reads on your database drive if you have enough memory since SQL Server should only be writing to the database. There is an additional throttling mechanism in a Lync Server 2010 Front End Server that will start if the server processing time is high. The DBStore latency throttling is only activated if the latency to the SQL Server is high. One example where such throttling is activated is if the front-end server is CPU-bound. If the average processing time (LC:SIP-07-Load Management\SIP-000-Average Holding Time For Incoming Messages) on the server exceeds six seconds, then the server goes into throttling mode and only allows one outstanding transaction per client connection. Once the processing time drops to three seconds, then the server drops out of throttling mode and allows up to 20 outstanding transactions per client connection. Whenever the number of transactions on a specific connection exceeds the threshold above, the connection is marked as flow controlled. The result is the server does not post any receives on it and the LC:SIP-01-Peers\Flow Controlled Connections counter is incremented. If a connection stays in a flow-controlled state for more than one minute, then the server closes it. It does so lazily; when it has a chance to check the connection, it determines if it was throttled for too long and closes it if it has been more than one minute. Operations Management and Monitoring of a Lync Server 2010 Environment
24
These are the two throttling mechanisms and there is one performance counter that summarizes what, if any, throttling the server is performing. LC:SIP-04-Responses\SIP-053-Local 503 Responses/sec The term "Local" in the above counter refers to locally generated responses. The 503 code corresponds to server unavailablewhere you should not see any 503 codes on a healthy server. However, during the period after a server is just brought online, you may see some 503 codes. When all of the users sign back in and the server returns to a stable state, there should no additional 503 codes. LC:SIP-04-Responses\SIP-074-Local 504 Responses/sec This performance counter indicates connectivity issues with other servers and it can indicate failures to connect or delays in connecting. If you are seeing 504 errors then the following performance counter should be checked: LC:SIP-01-Peers\SIP-017-Sends Outstanding This counter indicates the number of outbound queued requests and responses. If this counter is high then the issue is most likely not on the local server. Note that this counter can be high if there are network latency issues. It could also indicate issues with the local network interface card (NIC), but is more likely due to an issue on a remote server. This counter would most likely be high on a Director server when the pool it is attempting to communicate with is overloaded. The key with this counter is to look at the instances, not just the total. 4.3.6 Monitoring Operating System You must monitor the performance of all servers and components in the Lync Server 2010 Server. Obviously, one of the most important components is the operating system itself. Lync Server 2010 supports x64 editions of: Windows Server 2008 Windows Server 2008 R2 The most transparent way to monitor an operating system is by using Windows Server Operating System Management Pack for Operations Manager 2007 R2. It provides the fundamental monitoring basics, such as performance, health, and availability for computers running Windows Server 2008 and Windows Server 2008 R2. By detecting, alerting, and automatically responding to critical events and performance indicators, these management packs reduce resolution times for issues and increase the overall availability and performance of systems running the Windows Server operating systems. Apart from relevant Windows Server management packs for System Center Operations Manager 2007, the following system tools and resources can be used to monitor the health of the operating system (depending on the operating system version): 4.3.6.1.1 Windows Server 2008 R2 Windows Server 2008 R2 includes the following additional features and tools to assist administrators with basic operating system monitoring and management: Windows Resource Monitor is a powerful tool for understanding how system resources are used by processes and services. In addition to monitoring resource usage in real time, a Operations Management and Monitoring of a Lync Server 2010 Environment
25
Resource Monitor can help analyze unresponsive processes, identify which applications are using files, and control processes and services. Reliability Analysis Component is an in-box agent that provides detailed experience information on system usage and reliability. This information is exposed through a WMI interface, making it available for consumption by Portable Readers Systems. By exposing Reliability Analysis Component through a WMI interface, developers can monitor and analyze applications, increasing reliability and performance. Windows Server 2008 R2 uses the built-in Reliability Analysis Component to calculate a reliability index, which provides information about overall system usage and stability over time. The Reliability Analysis Component also keeps track of any important changes to the system that are likely to have an impact on stability, such as Windows updates and application installations. 4.3.6.1.2 Windows Server 2008 Windows Reliability and Performance Monitor Windows Reliability and Performance Monitor is an MMC Snap-in that combines the functionality of previous standalone tools including Performance Logs and Alerts, Server Performance Advisor, and System Monitor. It provides a graphical interface for customizing performance data collection and Event Trace Sessions. It also includes Reliability Monitor, an MMC Snap-in that tracks changes to the system and compares them to changes in system stability, providing a graphical view of their relationship: 4.3.6.2 Windows Reliability and Performance Monitor While Windows Reliability and Performance Monitor combine functionalities of some former standalone tools such as Performance Logs and Alerts, and System Monitor and Server Performance Advisor, it also provides some new functionality to Windows Server 2008 and Windows Server 2008 R2, such as: Data Collector Sets Resource View Reliability Monitor Wizards and templates for creating logs Data Collector Set groups data collectors into reusable elements for use with different performance monitoring scenarios. After a group of data collectors are stored as a Data Collector Set, operations such as scheduling can be applied to the entire set through a single property change. Windows Reliability and Performance Monitor includes default Data Collector Set templates to help system administrators begin immediately collecting performance data specific to a server role or monitoring scenario. The home page of Windows Reliability and Performance Monitor is the new Resource View screen, which provides a real-time graphical overview of CPU, disk, network, and memory usage. By expanding each of these monitored elements, system administrators can identify which processes are using which resources. In previous versions of Windows, this real-time, process-specific data was only available in limited form in Task Manager. Reliability Monitor calculates a System Stability Index that reflects whether unexpected issues reduced the reliability of the system. A graph of the Stability Index over time quickly identifies dates when issues began to occur. The accompanying System Stability Report provides details to help troubleshoot the root cause of reduced reliability. By viewing changes to the system (installation or removal of applications, Operations Management and Monitoring of a Lync Server 2010 Environment
26
updates to the operating system, or addition or modification of drivers) side by side with failures (application failures, operating system crashes, or hardware failures), a strategy for addressing the issues can be developed quickly. Adding counters to log files and scheduling their start, stop, and duration can now be performed through a Wizard interface. In addition, saving this configuration as a template enables system administrators to collect the same log on subsequent computers without repeating the data collector selection and scheduling processes. Performance Logs and Alerts features have been incorporated into the Windows Reliability and Performance Monitor for use with any Data Collector Set. 4.3.7 Central Management Store Replication Status When an administrator makes a change of some kind to Lync Server (for example, when an administrator creates a new voice policy or changes the Address Book Server configuration settings) that change is recorded in the Central Management store. In turn, the change must then be replicated to all the computers running Lync Server services or server roles. In order to replicate data, the Master Replicator (running on the Central Management Server) creates a snapshot of the modified configuration data; a copy of this snapshot is then sent to each computer running Lync Server services or server roles. On those computers, a replication agent receives the snapshot and uploads the modified data; the agent then sends the Master Replicator a message reporting the latest replication status. The Get-CsManagementStoreReplicationStatus cmdlet enables you to verify the replication status for any (or all) of the Lync Server computers in your organization. Who can run this cmdlet? By default, members of the following groups are authorized to run the Get- CsManagementStoreReplicationStatus cmdlet locally: RTCUniversalUserAdmins, RTCUniversalServerAdmins. To return a list of all RBAC roles this cmdlet has been assigned to (including any custom RBAC roles you have created yourself), run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Get- CsManagementStoreReplicationStatus"} 4.3.8 Monitoring Network Performance Lync Server 2010 is a real-time communications technology that relies heavily on the network to enable communication between userseither through instant messaging (IM), voice calls, or video communication. It is thus critical to monitor the network performance on an ongoing basis to help ensure that a users chosen communication modality provides the best possible experience. Network performance can be measured at two levels: 1. Overall network performance This level of performance measurement will allow an organization to create a "big-picture" view of their network and is usually implemented through third-party network monitoring systems. These systems will receive performance and capacity data from remote network devices such as routers and switched throughout the network to allow administrators to determine the health of any given network component at any given time of day. 2. Individual server performance This level of performance measurement is limited to a specific server and will assist administrators with gauging the network performance of a specific server to either assist with troubleshooting a specific performance issue or to gauge the respective servers performance over a given period as part of a capacity planning process. Operations Management and Monitoring of a Lync Server 2010 Environment
27
You can monitor the network by using the tools described in the following sections. 4.3.8.1 Tools for Overall Network Performance Monitoring System Center Operations Manager 2007 R2 System Center Operations Manager provides end-to-end service management that is easy to customize and extend for improved service levels across your IT environment. This enables Operations and IT Management teams to identify, and resolve issues affecting the health of distributed IT services. End-to- end service management is not restricted to Microsoft-based environments. Support for Web Services for Management (WS-Management), Simple Network Management Protocol (SNMP), and partner solutions allow for systems that do not run Microsoft operating systems and hardware to be included in service monitoring within Operations Manager 2007. System Center Operations Manager 2007 R2 and Third-Party Network Management Solutions EMC Smarts EMC Solutions for Operations Manager help you quickly resolve issues affecting service levels throughout your network. With EMC Solutions for Operations Manager, you can manage and monitor your entire IT service chain with one integrated, automated solution. You will easily identify the root causes of performance and availability issues, and resolve them faster reducing impact and costs. Key benefits include: a. Advanced, easy-to-use management Focus on delivering strategic business value rather than manually sorting and filtering confusing alert messages. b. Faster resolution Solve IT issues and respond to mission-critical business needs faster, reducing impact and cost. c. Streamlined operations Avoid IT complexity by consolidating multiple management tools, applications, and terminals. Additional information can be found here: Microsoft System Center Operations Manager Solutions for Microsoft System Center Operations Manager
Third-Party Solutions HP Network Management Center (previously known as HP OpenView) HP Network Management Center provides integrated fault and performance management to improve network availability and performance. Network Management Center is part of the HP automated network management solution that unifies fault, performance, configuration, and change management. Cisco Network Management and Automation products For the Enterprise, Cisco has a number of management products available including CiscoWorks LAN Management Solution as well as Cisco Network Analysis Module, to help improve operational efficiency and reduce network downtime. For additional data on these products, refer to the Cisco website at http://www.cisco.com/en/US/products/sw/netmgtsw/index.html. Simple Network Management Protocol (SNMP) Simple Network Management Protocol (SNMP) is a network management standard that defines a strategy for managing TCP/IP networks. SNMP enables you to capture configuration and status information about the network, and send the information to a designated computer for event monitoring. This standards based network management protocol uses a distributed architecture that includes: Operations Management and Monitoring of a Lync Server 2010 Environment
28
o Multiple managed nodes, each with an SNMP entity called an agent which provides remote access to management instrumentation. o At least one SNMP entity referred to as a manager which runs management applications to monitor and control managed elements. Managed elements are devices such as hosts, routers, and so on; they are monitored and controlled by accessing their management information. o A management protocol, SNMP, is used to convey management information between the management stations and agents. Management information refers to a collection of managed objects that reside in a virtual information store called a Management Information Base (MIB). Note Examples of third-party network monitoring solutions are provided above. This list is not definitive and Microsoft does not favor any specific vendor solution. Consult with your network service provider and or your respective technology provider to determine the best network monitoring solution for your organization. 4.3.8.2 Tools for Monitoring Individual Server Network Performance System Center Operations Manager 2007 R2 System Center Operations Manager 2007 R2 would allow administrators to view network performance of individual servers through the Windows Server 2008 Base Operating System Management Pack: The Windows Server operating system management pack includes a "Performance" management pack that would allow administrators to monitor Network Adapter performance as well as adapter health. Windows Network Monitor Collects, displays, analyzes resource usage on a server, and measures network traffic. Network Monitor exclusively monitors network activity. By capturing and analyzing network data, and using this data with performance logs, you can determine the network usage, identify network issues, and forecast future network needs. For more information about Network Monitor 3.4, and to learn how to install and configure Network Monitor as well as capture and analyze data, review this session: Network Monitor 3.3 Overview. Also, review the Network Monitor blog at: http://blogs.technet.com/b/netmon/. 4.3.9 Scanning for Viruses and Checking Virus Definitions We highly recommend installing an IM-level antivirus product. IM is a well-known source for proliferating both virus and malicious software throughout an organization. Microsoft Forefront Security for Lync Server provides multi-engine scanning with virus, malicious software, file and keyword filter protection as well as seamless integration with Office Communications Server. In addition to Forefront Security for Lync Server, we also highly recommend installing a file-level, antivirus solution to protect the servers file system. Keeping scanner engines and virus definitions updated is crucial. Configuring and monitoring the health of the updates assures that the most current scanning information is being used to protect both Office Communications Server and file-system. Important When using a third-party, file-level antivirus program on a server containing Lync Server 2010 and Forefront Security for Lync Server, make sure that the folders in which Forefront Security for Lync Server and the Lync Server are installed are not scanned, to prevent their corruption. For the full list of exclusions, see http://support.microsoft.com/kb/943620. Operations Management and Monitoring of a Lync Server 2010 Environment
29
4.3.10 Viewing and Analyzing Monitoring Server Reports Monitoring Server reports provides several different measures of voice quality to monitor the QoE that is being delivered to end users. Additionally, Monitoring Server includes several built-in reports that your organization can use to watch usage and media quality trends on your organization's network as well as troubleshoot media quality issues that arise. For a full understanding of Monitoring Server Reports and how Monitoring Server measures voice quality, it is necessary to obtain some background information about mean opinion score (MOS) values at Mean Opinion Score and Metrics. A primary part of keeping Monitoring Server Reports interesting for daily and weekly operations is viewing and analyzing Media Quality Reports, in particular: QoE Summary/Trend Reports QoE Performance Reports 4.3.10.1 Viewing Reports from the Monitoring Server 1. From a web browser. Browse to the location of your servers hosting the SQL reporting services. 2. View the required reports from the browser screen. 3. (Optional) Export a report by selecting the export option and the required output format. 4.3.10.2 Configuring Call Detail Recording (CDR) 1. From a user account that is a member of the RTCUniversalServerAdmins group (or has equivalent user rights), or assigned to the CsServerAdministrator or CsAdministrator role, log on to any computer that is in the network in which you deployed Lync Server 2010. 2. Open a browser window, and then enter the Admin URL to open the Lync Server Control Panel. 3. In the left navigation bar, click Monitoring and Archiving, and then click Call Detail Recording. 4. On the Call Detail Recording page, click the appropriate site in the table, click Edit, and then click Show Details. 5. To turn on purging, select Enable Purging for Monitoring Servers. 6. In Keep call detail recordings for maximum duration (days): select the maximum number of days that call detail recordings should be retained. 7. In Keep error report data for maximum duration (days): select the maximum number of days that error reports should be retained. 8. Click Commit. 4.3.10.3 Configuring QoE 1. Log on to the computer as a member of the RTCUniversalServerAdmins group, or as a member of the CsVoiceAdministrator, CsServerAdministrator, or CsAdministrator role. For details, see Delegate Setup Permissions. 2. Open a browser window, and then enter the Admin URL to open the Lync Server Control Panel. 3. In the left navigation bar, click Monitoring and Archiving, and then click Quality of Experience Data. 4. On the Quality of Experience Data page, click the appropriate site from the table, click Edit, and then click Show Details. Operations Management and Monitoring of a Lync Server 2010 Environment
30
5. To turn on purging, select Enable Purging for Monitoring Servers. 6. In Keep call detail recordings for maximum duration (days): select the maximum number of days that QoE data should be retained. 7. Click Commit. 4.3.10.4 Modifying the Archiving Policy 1. From a user account that is assigned to the CsArchivingAdministrator or CsAdministrator role, log on to any computer in your internal deployment. 2. Open a browser window, and then enter the Admin URL to open the Lync Server Control Panel. 3. In the left navigation bar, click Monitoring and Archiving, and then click Archiving Policy. 4. Click Global in the list of policies, click Edit, and then click Show details. 5. In Edit Archiving Policy - Global, do the following: 6. To enable or disable internal archiving for the deployment, select or clear the Archive internal communications check box. 7. To enable or disable external archiving for the deployment, select or clear the Archive external communications check box. 8. Click Commit. 4.3.10.5 Applying an archiving policy to a User From a user account that is assigned to the CsArchivingAdministrator or CsAdministrator role, log on to any computer in your internal deployment. 1. Open a browser window, and then enter the Admin URL to open the Lync Server Control Panel. 2. In the left navigation bar, click Users, and then search on the user account that you want to configure. 3. In the table that lists the search results, click the user account, click Edit, and then click Show details. 4. In Edit Lync Server User under Archiving policy, select the archiving user policy that you want to apply. 5. Click Commit.
4.3.10.6 QoE Summary/Trend Reports The QoE summary/trends reports are useful for finding the peak usage times of day and examining the media quality during those times to help ensure that your organization's network resources are adequate. Your organization can also use the many filters available in the report to isolate performance numbers for certain locations, client and device types, and servers. QoE summary/trend reports consist of: UC-to-UC Summary/Trend Report PSTN Summary/Trend Report Conference Summary/Trend Report Operations Management and Monitoring of a Lync Server 2010 Environment
31
4.3.10.7 QoE Performance Reports QoE performance reports provide details about the three reports that concentrate on the QoE performance of Mediation Servers, A/V Conferencing Servers, and endpoint locations. 4.3.10.7.1 Mediation Server Performance Report The Mediation Server Performance report lists the metrics achieved by one or more Mediation during the specified time period. The metrics for the unified communications (UC)-to-Mediation Server leg and the Mediation Server-to-Gateway leg of each call are reported separately. Use this report to compare the volume and performance of your organization's various Mediation Servers. For each Mediation Server (and for each call leg), the report displays the following: Number of calls Packet Loss Round Trip Time Jitter Conversational mean opinion score (MOS) Sending MOS Listening MOS Network MOS Network MOS Degradation Echo Return Signal Level 4.3.10.7.2 A/V Conferencing Server Performance Report The A/V Conferencing Server Performance report provides lists of metrics achieved by one or more A/V Conferencing Servers during the specified time period. This report can be used to compare the volume and performance of your organizations various A/V Conferencing Servers. Your organization can also isolate the report to show only the experience for specific client types, including Lync clients or PSTN clients. For each A/V Conferencing Server, the report displays the following: Number of conferences Packet Loss Round Trip Time Jitter Conversational mean opinion score (MOS) Sending MOS Listening MOS Network MOS Network MOS Degradation Operations Management and Monitoring of a Lync Server 2010 Environment
32
Echo Return Signal Level 4.3.10.7.3 Location-Based Performance Report The Location-Based Performance report provides a list of network locations and for each location shows the number of calls in each pre-determined range of quality. The goal of this report is to provide insight into the media quality of the bulk of your organization calls for various locations, so that you can identify poorly performing locations, and see the different grades of media quality in your organizations different locations. When displaying the report, different tables of metrics appearone table for each metric your organization chooses to report on. You can choose from the following metrics for this report: Conversational mean opinion score (MOS) Network MOS Network MOS Degradation Sending MOS Listening MOS Packet Loss Jitter Latency 4.3.11 Validating Voice Number Normalization and Routing Correct number normalization and routing is critical for functional Enterprise Voice environment. Especially during migrations from private branch exchange (PBX) to standalone Lync Server environment, one of the keys to successful migration is to reveal and document all existing dialing rules, and create appropriate normalization rules, voice policies, phone usages and routes. Validating number normalization and routing is important not only during migrations but also during normal, stable operation of the system. We recommend conducting this validation on a daily basis by using the Lync Server Control Panel, starting with developing a robust set of test cases against the current set of normalization rules that have been published in the Lync Server global settings. These test cases should be run on a daily basis to highlight any unwanted changes made and committed to the dial plan. Lync Server Control Panel also helps you visualize, test, modify, archive, and share configuration information about voice routing and in modifying Enterprise Voice number normalization rules, dial plans, voice policy, and routes. It has additional features for doing the following: Exporting and importing or backing up voice routing data between systems. Testing configuration changes before uploading them to a live system. Creating and running configuration test cases to help ensure the usability of routing data after making changes to it, but before committing them the changes to a deployed system. Creating and modifying number normalization rules, location profiles, voice policy, and routing data without writing the necessary regular expressions. Operations Management and Monitoring of a Lync Server 2010 Environment
33
Analyzing a location profile for compatibility with the Lync Server Phone Edition. More information about voice routing tests can be found at http://technet.microsoft.com/en- us/library/gg398915.aspx 4.3.12 Validating Address Book Access Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsAddressBookService cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsAddressBookService"}
Description The Test-CsAddressBookService cmdlet provides a way for you to verify that a user can connect to the Address Book Download Web service. When you run the cmdlet, Test-CsAddressBookService connects to the Address Book Download Web service on the specified pool and requests the location of the Address Book files. If the Address Book Download Web service supplies that location, the test is considered successful. If the request is denied, then the test is considered a failure.
Running the Test The Test-CsAddressBookService cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the fully qualified domain name (FQDN) of the Lync Server pool being tested. For example: Test-CsAddressBookService -TargetFqdn "atl-cs-001.litwareinc.com" To run this check using an actual user account, you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsAddressBookService: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsAddressBookService -TargetFqdn "atl-cs-001.litwareinc.com"- UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsAddressBookService cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
34
If the specified user is able to connect to the Address Book Service you will get back output similar to this, with the Result property marked as Success:
If the specified user is not able to make this connection, the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetUri : TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Diagnosis : ErrorCode=4005,Source=atl-cs-001.litwareinc.com, Reason=Destination URI either not enabled for SIP or does not exist. Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because the specified user (that is, the Destination URI) either does not exist or has not been enabled for Lync Server. You can verify whether or not a user account is valid, and verify that you supplied the correct SIP address, by running a command like this one: Get-CsUser -Identity "sip:kenmyer@litwareinc.com" | Select-Object SipAddress, Enabled
If Test-CsAddressBookService fails then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included Test-CsAddressBookService will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, this sample output shows that Test-CsAddressBookService, at least in this example, was able to download the Address Book file:
Sending Http GET Request. File Path = https://atl-cs-001.litwareinc.com:443/abs/handler/f-1299.lsabs Attempt Number = 1 TimeOut (msec) = 60000 Successfully Downloaded the ABS file https://atl-cs- 001.litwareinc.com:443/abs/handler/f-1299.lsabs
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsAddressBookService might fail: Operations Management and Monitoring of a Lync Server 2010 Environment
35
You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
Operations Management and Monitoring of a Lync Server 2010 Environment
36
4.3.13 Validating Address Book Web Query
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsAddressBookWebQuery cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsAddressBookWebQuery"}
Description The Test-CsAddressBookWebQuery cmdlet provides a way for administrators to verify that users can use the Address Book Web Query service to search for a specific contact. When you run the cmdlet, Test- CsAddressBookWebQuery will first connect to the Web Ticket service in order to be authenticated. If authentication is successful, the cmdlet will then connect to the Address Book Web Query service and search for the specified contact. If that contact is found, the cmdlet will attempt to return that information to the local computer. The test will be marked a success only if all of those steps can be completed.
Running the Test The Test-CsAddressBookWebQuery cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool as well as the SIP address of the user acting as the target of the search. For example: Test-CsAddressBookWebQuery -TargetFqdn "atl-cs-001.litwareinc.com" TargetSipAddress "sip:davidlongmire@litwareinc.com" To run this check using an actual user account you must create a Windows PowerShell credentials object that contains a valid user name and password. You must then include that credentials object as well as the SIP address assigned to the account when calling Test-CsAddressBookWebQuery: $credential = Get-Credential "litwareinc\kenmyer" Test-CsAddressBookWebQuery -TargetFqdn "atl-cs-001.litwareinc.com" TargetSipAddress "sip:davidlongmire@litwareinc.com" UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsAddressBookWebQuery cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
37
If the specified user is able to connect to the Address Book Service and retrieve the targeted user address, you will get back output similar to this with the Result property marked as Success:
If the specified user is not able to connect or if the target user address cannot be retrieved, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetUri : https://atl-cs-001.litwareinc.com:443/groupexpansion/service.svc TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : Address Book Web service request has failed with response code NoEntryFound. Diagnosis :
The preceding output states that the test failed because the target user could not be found. You can verify whether or not a valid SIP address was passed to Test-CsAddressBookWebQuery by running a command similar to the following: Get-CsUser | Where-Object {$_.SipAddress eq "sip:davidlongmire@litwareinc.com"
If Test-CsAddressBookWebQuery fails, then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsAddressBookWebQuery will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, this output indicates that Test-CsAddressBookWebQuery was able to connect to the Address Book Service, but was not able to locate the target SIP address: 'QueryAddressBookWebService' activity started. Calling Address Book Web Query Service. ABWS URL = https://atl-cs-001.litwareinc.com:443/groupexpansion/service.svc Address book query exception : Address Book Web service request has failed with response code NoEntryFound.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsAddressBookWebQuery might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this: Operations Management and Monitoring of a Lync Server 2010 Environment
38
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
The target user might not be in the Address Book.
The Address Book might not have fully updated and replicated. You can force an update of all the Address Book Servers in your organization by running the following command:
Update-CsAddressBook
4.3.14 Viewing Status of Pools For each Enterprise Pool and the pool of each Standard Edition Server, you can view information about pool status as well as the status of other components used by the pool through the Lync Server 2010 Control Panel. We recommended implementing Operations Manager 2007 with the Lync Server 2010 management pack to assist administrators in viewing the heath of pools within their environment. Using the Lync Server Control Panel 1. Open the Lync Server 2010 Control Panel. 2. Click Topology, and check status of all servers in the Topology.
4.3.15 Monitoring Back End Lync Server 2010 Storage Performance The Lync Server 2010 back-end databases are a critical part of the Lync Server 2010 deployment. We recommend constantly monitoring the databases and respective transaction logs to help ensure that the Lync Server 2010 back end is performing optimally. The following table identifies performance counters that should be monitored to get information about Storage Performance. The baseline values for these counters need to be determined first (when system is at its normal, expected load) to understand the performance changes when system is stressed. Performance counters to be monitored Performance Counter Thresholds Transactions/sec (RTC)
Transactions/sec (rtcdyn)
Operations Management and Monitoring of a Lync Server 2010 Environment
4.3.16 Monitoring Group Chat We highly recommend running the most recent Cumulative Server Update Installer Available on Microsoft Download Center update for performance improvements. Assuming you are running latest cumulative update, use the following stress test table for metrics to understand if your Group Chat Servers are running at optimal health. Test environment and user model Test Environment and User Model Three Group Chat Servers in a Group Chat pool, each with 8 GB memory and 8 processors. Two Lync Server 2010 Front Ends in Enterprise Edition. 60,000 concurrent users across three Group Chat Servers. 25,000 channels hosted by Group Chat Pool. Channel Size: Small Channel Size: 30 Medium Channel Size: 150 Large Channel Size: 2500 Channel Count: Number small channels: 24,000 Number medium channels 800 Number large channels 24 Operations Management and Monitoring of a Lync Server 2010 Environment
40
Test Environment and User Model Total Channels 24,824 Invite channels: Half the channels were invite channels Number of channels a user joins: Small: 12 Medium: 2 Large: 1 Join rate: 10 total/second, 3.33/second per server Logout rate: 10 total/second, 3.33/second per server Chat rate: 20 total/second, 6.66/second per server Important The following performance counter numbers will likely vary when different hardware specifications and/or user profiles are used. Performance counter to be monitored Performance Counter Thresholds Process(ChannelService)->%Processor Time Min: 0 4.3.17 Validate Domain Name System (DNS) Settings for DNS Load Balancing To support the FQDN used by DNS load balancing, you must provision DNS to resolve the pool FQDN (such as pool01.contoso.com) to the IP addresses of all the servers in the pool (for example, 192.168.1.1, 192.168.1.2, and so on). You should include only the IP addresses of servers that are currently deployed. Additionally if you are using DNS load balancing for the Edge pools the following DNS entries are required: For the Lync Server Access Edge service, you need one entry for each server in the pool. Each entry must resolve the FQDN of the Lync Server Access Edge service (for example, sip.contoso.com) to the IP address of the Lync Server Access Edge service on one of the Edge Servers in the pool. For the Lync Server Web Conferencing Edge service, you need one entry for each server in the pool. Each entry must resolve the FQDN of the Lync Server Web Conferencing Edge service (for example, webconf.contoso.com) to the IP address of the Lync Server Web Conferencing Edge service on one of the Edge Servers in the pool. For the Lync Server Audio/Video Edge service, you need one entry for each server in the pool. Each entry must resolve the FQDN of the Lync Server Audio/Video Edge service (for example, Operations Management and Monitoring of a Lync Server 2010 Environment
41
av.contoso.com) to the IP address of the Lync Server Audio/Video Edge service on one of the Edge Servers in the pool. If you wish to utilize DNS load balancing on the internal interface of the Edge pool, you must add one DNS record, which resolves the internal FQDN of the Edge pool to the IP address of each server in the pool. To verify that DNS is returning the correct values for DNS load balancing you should use the nslookup tool. To return all values for a DNS record with nslookup you should run the command: nslookup <FQDN > You would run this command for every FQDN used in DNS load balancing configuration to verify that every record set for DNS load balancing returned all of the correct entries. 4.3.18 Running Synthetic Transactions Synthetic transactions are typically conducted in two different ways. You can use the CsHealthMonitoringConfiguration cmdlets to set up test users for each of their Registrar pools. These test users are a pair of users who have been preconfigured for use with synthetic transactions. (Typically these are test accounts and not accounts that belong to actual users.) With test users configured for a pool, you can run a synthetic transaction against that pool without having to specify the identities of (and supply the credentials for) the user accounts involved in the test. Alternatively, you can run a synthetic transaction by using actual user accounts. For example, if two users are unable to exchange instant messages, you could run a synthetic transaction using those two user accounts (as opposed to a pair of test accounts), and then try to diagnose and resolve the issue. If you decide to conduct a synthetic transaction using actual user accounts, you will need to supply the logon names and passwords for each user. 4.3.18.1 Validating Audio/Video Conferences
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsAVConference cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsAVConference"}
Description The Test-CsAVConference cmdlet checks to see if two test users are able to participate in an audio/video (A/V) conference. When the cmdlet runs, the two users are logged on to the system. After they face successfully logged on, the first user creates an A/V conference, and then waits for the second user to Operations Management and Monitoring of a Lync Server 2010 Environment
42
join that conference. After a brief exchange of data, the conference is deleted and the two tests users are logged off. Note that Test-CsAVConference does not conduct an actual A/V conference between the two test users. Instead, the cmdlet simply verifies that the two users can make all the connections necessary to carry out such a conference.
Running the Test The Test-CsAVConference cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsAVConference -TargetFqdn "atl-cs-001.litwareinc.com" To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test-CsAVConference: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire"
Test-CsAVConference -TargetFqdn "atl-cs-001.litwareinc.com" -SenderSipAddress "sip:kenmyer@litwareinc.com" -SenderCredential $credential1 - ReceiverSipAddress "sip:davidlongmire@litwareinc.com" -ReceiverCredential $credential2 For more information, see the help documentation for the Test-CsAVConference cmdlet.
Determining Success or Failure If the specified users are able to successfully complete an A/V conference, you will get back output similar to this, with the Result property marked as Success:
If the users are not able to complete the conference, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 404, Not Found Diagnosis : ErrorCode=4005,Source=atl-cs-001.litwareinc.com, Reason=Destination URI either not enabled for SIP or does not exist. Microsoft.Rtc.Signaling.DiagnosticHeader Operations Management and Monitoring of a Lync Server 2010 Environment
43
For example, the preceding output states that the test failed because at least one of the two user accounts was invalid, either because the account does not exist or because the account has not been enabled for Lync Server. You can verify the existence of the two test accounts, and whether or not they have been enabled for Lync Server, by running a command similar to the following:
When the Verbose parameter is included Test-CsAVConference will return a step-by-step account of each action it attempted when checking the ability of the specified users to participate in an AV conference. For example, suppose your test fails and you get back the following Diagnosis: ErrorCode=1008,Source=accessproxy.litwareinc.com,Reason=Unable to resolve DNS SRV record
If you rerun the test using the Verbose parameter, the step-by-step information returned will include output similar to this: VERBOSE: 'Register' activity started. Sending Registration request: Target Fqdn = atl-cs-001.litwareinc.com User Sip Address = sip:davidlongmire@litwareinc.com Registrar Port = 5061. Auth Type 'Trusted' is selected. 'Register' activity started. Sending Registration request: Target Fqdn = atl-cs-001.litwareinc.com User Sip Address = sip:kenmyer@litwareinc.com Registrar Port = 5061. Auth Type 'Trusted' is selected. An exception 'The endpoint was unable to register. See the ErrorCode for specific reason.' occurred during Workflow The last line in that output indicates that the user sip:kenmyer@litwareinc.com was unable to register with Lync Server. That means that you should verify that the SIP address sip:kenmyer@litwareinc.com is valid, and that the associated user has been enabled for Lync Server.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsAVConference might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
Operations Management and Monitoring of a Lync Server 2010 Environment
44
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
4.3.18.2 Testing Lync Client Authentication
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsClientAuth cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsClientAuth"}
Description The Test-CsClientAuth cmdlet enables you to determine whether or not a user can log on to the Lync Server by using a client certificate, you can run the Test-CsClientAuth cmdlet. After calling Test- CsClientAuth, the cmdlet will contact the certificate provisioning service and download a copy of any client certificates for the specified user. If a client certificate can be found and downloaded, Test- CsClientAuth will then attempt to log on using that certificate. If logon succeeds, Test-CsClientAuth will log off and report that the test succeeded. If a certificate cannot be found or downloaded, or if the cmdlet is unable to logon using that certificate, then Test-CsClientAuth will report that the test failed.
Running the Test The Test-CsClientAuth cmdlet is run by using the account of any user who has been enabled for Lync Server. To run this check using an actual user account you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsClientAuth: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsClientAuth -TargetFqdn "atl-cs-001.litwareinc.com"-UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsClientAuth cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
45
Determining Success or Failure If the specified user is able to log on to Lync Server by using a client certificate you will get back output similar to this, with the Result property marked as Success:
If the specified user is not able to log on, the Result will be shown as Failure and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:03.3645259 Error : Could not download a CS Certificate for the given user. Check if provided uri and credentials are correct. Diagnosis :
For example, the preceding output states that the test failed because a valid client certificate could not be located for the specified user. You can return a list of the client certificates issued to a user by running a command like this:
When the Verbose parameter is included, Test-CsClientAuth will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example:
Trying to download a CS certificate for User : kenmyer@litwareinc.com endpoint : STEpid Web Service url : https://atl-cs- 001.litwareinc.com:443/CertProv/CertprovisioningService.svc Could not download a CS certificate from web service. CHECK: - Web service url is valid and the web services are functional - If using PhoneNo\\Pin to authenticate, make sure they match the user uri Operations Management and Monitoring of a Lync Server 2010 Environment
46
- If using NTLM\Kerberos auth, make sure you provided valid credentials
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsClientAuth might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
The test user might not have a valid client certificate. You can return information about the client certificates assigned to a user by using a command similar to this:
Operations Management and Monitoring of a Lync Server 2010 Environment
47
4.3.18.3 Testing Lync Server Services
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsComputer cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsComputer"}
Description Test-CsComputer verifies the status of all the Lync Server 2010 services running on the local computer. (Test-CsComputer can only be run locally; it cannot be run from a remote instance of Windows PowerShell.) The cmdlet also checks to see if the appropriate firewall ports have been opened on the computer, and determines whether or not the Active Directory groups created when you installed Lync Server 2010 have been added to the corresponding local groups. For example, Test-CsComputer will verify that the Active Directory group RTCUniversalUserAdmins has been added to the local Administrators group.
Running the Test The Test-CsComputer cmdlet can only be run on the local computer; you cannot call Test-CsComputer from a remote instance of Windows PowerShell. By default, Test-CsComputer displays very little output on-screen; instead, information returned by the cmdlet is written to an HTML file. Because of that, it is recommended that you include the Verbose parameter and the Report parameter any time you run Test- CsComputer. The Verbose parameter will provide slightly more detailed output on-screen while the cmdlet runs; the Report parameter allows you to specify a file path and file name for the HTML file generated by Test-CsComputer. If you do not include the Report parameter the HTML file will automatically be saved to your Users folder and be given a name similar to this: ce84964a-c4da-4622- ad34-c54ff3ed361f.html. The following sample command runs Test-CsComputer and saves the output to a file named C:\Logs\ComputerTest.html:
For more information, see the help documentation for the Test-CsComputer cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
48
Because of the number of verification checks that it performs, Test-CsComputer does not report back a simple Yes, the test succeeded or No, the test did not succeed. Instead, you will need to view the generated HTML file by using Internet Explorer in order to determine the success or failure of each individual test.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsComputer might fail: The test computer might not be enabled for use with Lync Server. This can happen if the Lync Server services or server roles on the computer have changed and the Enable-CsComputer cmdlet was not run. To fix this issue, run the following command:
Enable-CsComputer
Replication might not be up to date on the test computer. You can check the current replication status for a computer by running the Get-CsManagementStoreReplicationStatus cmdlet:
Operations Management and Monitoring of a Lync Server 2010 Environment
49
The topology might need to be enabled. If you make changes to the Lync Server topology (changes that might affect the local computer), then you must enable the new topology. You can enable the topology at any time by running this command:
Enable-CsTopology
Operations Management and Monitoring of a Lync Server 2010 Environment
50
4.3.18.4 Testing Ability to Connect to a Federated Domain
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsFederatedPartner cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsFederatedPartner"}
Description Test-CsFederatedPartner verifies your ability to connect to the domain of a federated partner. In order to verify the connectivity to a domain, that domain must be listed in the collection of allowed (federated) domains. You can retrieve a list of the domains on your allowed domains list by using this command: Get-CsAllowedDomain
Running the Test The Test-FederatedPartner cmdlet requires two pieces of information: the FQDN of your Edge Server and the FQDN of the federated partner. For example, this command tests the ability to connect to the domain contoso.com: Test-CsFederatedPartner -TargetFqdn "atl-edge-001.litwareinc.com" Domain "contoso.com" This command enables you to test the connections to all the domains currently on your allowed domains list: Get-CsAllowedDomain | ForEach-Object {Test-CsFederatedPartner -TargetFqdn "atl-edge-001.litwareinc.com" Domain $_.Identity} For more information, see the help documentation for the Test-CsFederatedPartner cmdlet.
Determining Success or Failure If the specified domain can be contacted, you will get back output similar to this with the Result property marked as Success:
TargetFqdn : atl-cs-001.litwareinc.com Result : Success Latency : 00:00:00 Error : Diagnosis : Operations Management and Monitoring of a Lync Server 2010 Environment
51
If the specified domain cannot be contacted, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 504, Server time-out Diagnosis : ErrorCode=2, Source=atl-cs-001.litwareinc.com,Reason=See response code and reason phrase. Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because of a server timeout error. This typically indicates either network connectivity problems or problems contacting the Edge Server.
If Test-CsFederatedPartner fails, then you might want to rerun the test, this time including the Verbose parameter:
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsFederatedPartner might fail: The Edge Server might not be available. You can the FQDNs of your Edge Servers by using this command:
Get-CsService EdgeServer | Select-Object PoolFqdn
You can then ping each Edge Server to verify that it is accessible over the network. For example:
ping atl-edge-001.litwareinc.com
The specified domain might not be listed on the allowed domains list. To verify the domains that have been added to the allowed domains list, use this command:
Get-CsAllowedDomain
If youd like to see a list of domains that your users have been blocked from communicating with, then use this command:
Get-CsBlockedDomain
Operations Management and Monitoring of a Lync Server 2010 Environment
52
4.3.18.5 Testing Ability to Employ Group Expansion
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsGroupExpansion cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsGroupExpansion"}
Description The Test-CsGroupExpansion lets you verify whether or not group expansion is working within your organization. When group expansion is enabled, users configure distribution groups as a contact. That means that those users can then send the same instant message to all the group members simply by addressing the message to the group rather than each individual member of that group. Group expansion enables you to quickly and easily view all the group members and their current status. With the Test-CsGroupExpansion cmdlet, you specify an Active Directory distribution group by using the groups email address. Test-CsGroupExpansion then uses group expansion to retrieve the group membership and compare the retrieved list with the membership of the group email address that you supplied. If the two lists match, then group expansion is working correctly. Note that you can test group expansion in two different ways: by testing the service itself or by testing the associated web service.
Running the Test The Test-CsGroupExpansion cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool being tested as well as the email address for a valid distribution group. For example: Test-CsGroupExpansion -TargetFqdn "atl-cs-001.litwareinc.com" GroupEmailAddress "Sales@litwareinc.com" To run this check using an actual user account, you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsGroupExpansion: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsGroupExpansion -TargetFqdn "atl-cs-001.litwareinc.com" GroupEmailAddress "Sales@litwareinc.com" UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsGroupExpansion cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
53
Determining Success or Failure If the specified user is able to use group expansion, you will get back output similar to this with the Result property marked as Success:
If the specified user is not able to use group expansion, then the Result will be shown as Failure and additional information will be recorded in the Error and Diagnosis properties:
Test-CsGroupExpansion : The endpoint was unable to register. See the ErrorCode for specific reason.
The preceding output states that the test failed because the specified user was unable to register with Lync Server; this will typically occur if the test account does not exist or has not enabled for Lync Server. You can verify the existence of an account, and determine whether or not the account has been enabled for Lync Server by running a command similar to the following:
When the Verbose parameter is included Test-CsGroupExpansion will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, this output indicates that the specified distribution group could not be found: Trying to get web ticket. Web Service url : https://atl-cs- 001.litwareinc.com:443/WebTicket/WebTicketService.svc Using NTLM/Kerb auth. GetWebTicketActivity completed. Operations Management and Monitoring of a Lync Server 2010 Environment
54
'VerifyDistributionList' activity started. DLX Web Service Response Status is: NotFound. 'VerifyDistributionList' activity completed in '0.2597923' secs.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsGroupExpansion might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
Group expansion might be disabled. It is possible to turn off group expansion; if group expansion has been disabled then the Test-CsGroupExpansion cmdlet will fail. To determine whether or not group expansion is enabled, use a command similar to this:
Operations Management and Monitoring of a Lync Server 2010 Environment
55
4.3.18.6 Testing Ability to Do Group IM
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsGroupIM cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsGroupIM"}
Description The Test-CsGroupIM cmdlet verifies that users in your organization can conduct group instant messaging sessions. When you run Test-CsGroupIM, the cmdlet attempts to sign on a pair of test users to Lync Server. If successful, Test-CsGroupIM creates a new conference using the first test user, then invites the second user to join the conference. After an exchange of messages, both users are then disconnected from the system. Note that all of this happens without any user interaction, and without affecting any actual users. For example, suppose the test account sip:kenmyer@litwareinc.com corresponds to a real user with a real Lync Server account. In that case, the test will be conducted without any disruption to the real Ken Myer. For example, even when the Ken Myer test account logs off from the system, Ken Myer the person will remain logged on. Likewise, the real Ken Myer will not receive an invitation to join the conference. That invitation will be sent to, and accepted by, the test account.
Running the Test The Test-CsGroupIM cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsGroupIM -TargetFqdn "atl-cs-001.litwareinc.com"
To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test-CsGroupIM: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire"
Test-CsGroupIm -TargetFqdn "atl-cs-001.litwareinc.com" -SenderSipAddress "sip:kenmyer@litwareinc.com" -SenderCredential $credential1 - ReceiverSipAddress "sip:davidlongmire@litwareinc.com" -ReceiverCredential $credential2 Operations Management and Monitoring of a Lync Server 2010 Environment
56
For more information, see the help documentation for the Test-CsGroupIM cmdlet.
Determining Success or Failure If the two users are able to complete a group instant messaging session, you will get back output similar to this with the Result property marked as Success:
If the two users are not able to complete the instant messaging session, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 404, Not Found Diagnosis : ErrorCode=4005,Source=atl-cs-001.litwareinc.com, Reason=Destination URI either not enabled for SIP or does not exist. Microsoft.Rtc.Signaling.DiagnosticHeader
The preceding output states that the test failed because at least one of the test accounts was not valid, either because the account does not exist or because the user has not been enabled for Lync Server. You can verify the existence of an account, and whether or not the account has been enabled for Lync Server by running a command similar to this:
When the Verbose parameter is included, Test-CsGroupIM will return a step-by-step account of each action it attempted when checking the ability of the specified users to participate in a group instant messaging sessions. For example, if your test fails and you are told that one or more of the user accounts is not valid, you can rerun the test using the Verbose parameter and determine which user account is invalid: Sending Registration request: Target Fqdn = atl-cs-001.litwareinc.com User SIP Address = sip:kenmyer@litwareinc.com Register Port = 5061 Auth type 'IWA' is selected. An exception 'The log on was denied. Check that the proper credentials are being used and the account is active' Operations Management and Monitoring of a Lync Server 2010 Environment
57
As you can see, in this example the user with the SIP address sip:kenmyer@litwareinc.com was not able to log on.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsGroupIM might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
The instant messaging service might not be available. With Lync Server, you can configure the system so that instant messaging is not available if the archiving database cannot be accessed. You can verify that by running a command similar to the following:
If BlockOnArchiveFailure is set to True, then you should verify whether or not the archiving database is available. You can return the locations of your archiving databases by using the following command:
Get-CsService ArchivingDatabase
The Archiving Server might not be available. You can retrieve the FQDN of your Archiving Servers by using this command:
Get-CsService ArchivingServer
You can then ping the appropriate server to verify that it is available. For example:
ping atl-archiving-001.litwareinc.com
Operations Management and Monitoring of a Lync Server 2010 Environment
58
4.3.18.7 Testing Ability to IM between Two Users
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsIM cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsIM"}
Description The Test-CsIM cmdlet verifies that a pair of test users is able to exchange instant messages. When called, the Test-CsIM cmdlet starts off by trying to log on a pair of test users to Lync Server. Assuming the two logons are successful, the cmdlet then initiates an IM session between the two test users. (User 1 invites User 2 to an IM session, and User 2 accepts the invitation.) After verifying that messages can be exchanged between the two users, Test-CsIM then ends the IM session and logs both users off of the system.
Running the Test The Test-CsIM cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsIM -TargetFqdn "atl-cs-001.litwareinc.com" To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test-CsIM: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire"
Test-CsIm -TargetFqdn "atl-cs-001.litwareinc.com" -SenderSipAddress "sip:kenmyer@litwareinc.com" -SenderCredential $credential1 - ReceiverSipAddress "sip:davidlongmire@litwareinc.com" -ReceiverCredential $credential2 For more information, see the help documentation for the Test-CsIM cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
59
If the two users are able to complete an instant messaging session you will get back output similar to this, with the Result property marked as Success:
If the test users are not able to complete the session, the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 504, Server time-out Diagnosis : ErrorCode=2, Source=atl-cs-001.litwareinc.com,Reason=See response code and reason phrase. Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because the specified user could not be found. You can verify whether or not a SIP address is valid (and whether or not the user assigned that SIP address has been enabled for Lync Server) by running this command:
When the Verbose parameter is included, Test-CsIM will return a step-by-step account of each action it attempted when checking the ability of the two test users to take part in an IM session. For example, heres sample output that occurs when an invalid set of user credentials (in this case, an incorrect password) is supplied to Test-CsIM:
Sending Registration request : Target Fqdn = atl-cs-011.litwareinc.com User Sip Address = sip:kenmyer@litwareinc.com Registrar Port = 5061 Auth Type 'IWA' is selected. Registration hit against sip/atl-cs-001.litwareinc.com 'Register' activity completed in '0.0601795' secs. An exception 'The log on was denied. Check that the proper credentials are being used and the account is active.' occurred during the Workflow.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsIM might fail: Operations Management and Monitoring of a Lync Server 2010 Environment
60
You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
The instant messaging service might not be available. With Lync Server, you can configure the system so that IM is not available if the archiving database cannot be accessed. You can verify that by running a command similar to the following:
If BlockOnArchiveFailure is set to True, then you should verify whether or not the archiving database is available. You can return the locations of your archiving databases by using the following command:
Get-CsService ArchivingDatabase
The Archiving server might not be available. You can retrieve the FQDN of your Archiving servers by using this command:
Get-CsService ArchivingServer
You can then ping the appropriate server to verify that it is available. For example:
ping atl-archiving-001.litwareinc.com
Operations Management and Monitoring of a Lync Server 2010 Environment
61
4.3.18.8 Testing Configuration of the Kerberos Account Assigned to a Site
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsKerberosAccountAssignment cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsKerberosAccountAssignment"}
Description The Test-CsKerberosAccountAssignment cmdlet provides a way for you to verify that a Kerberos account has been associated with a given site, that this account has been configured correctly, and that the account is working as expected. Kerberos accounts are computer accounts that can serve as the authentication principal for all the computers in a site that are running Internet Information Server (IIS). Because these accounts use the Kerberos authentication protocol, the accounts are referred to as Kerberos accounts, and the new authentication process is known as Kerberos web authentication. This enables you to manage all your IIS servers by using a single account.
Running the Test By default, Test-CsKerberosAccountAssignment displays very little output on-screen; instead, information returned by the cmdlet is written to an HTML file. Because of that, it is recommended that you include the Verbose parameter and the Report parameter any time you run Test-CsKerberosAccountAssignment. The Verbose parameter will provide slightly more detailed output on-screen while the cmdlet runs; the Report parameter allows you to specify a file path and file name for the HTML file generated by Test- CsKerberosAccountAssignment. If you do not include the Report parameter the HTML file will automatically be saved to your Users folder and be given a name similar to this: ce84964a-c4da-4622- ad34-c54ff3ed361f.html. You must also specify a site Identity when running Test-CsKerberosAccountAssignment. Kerberos accounts are assigned at the site scope. The following command runs Test-CsKerberosAccountAssignment and saves the output to a file named C:\Logs\KerberosTest.html:
Test-CsKerberosAccountAssignment Identity "site:Redmond" Report "C:\Logs\KerberosTest.html" -Verbose For more information, see the help documentation for the Test-CsKerberosAccountAssignment cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
62
Determining Success or Failure The Test-CsKerberosAccountAssignment cmdlet does not return a simple indication of success or failure. Instead, you must view the generated HTML file by using Internet Explorer:
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsKerberosAccountAssignment might fail: You might have specified an invalid site Identity. To return a list of valid site Identity, use this command:
Get-CsSite | Select-Identity Identity
A site Identity typically looks like this:
site:Redmond
The specified site might not have a Kerberos account assigned to it. You can verify whether or not a Kerberos account has been assigned to a site by running a command similar to this:
Your Kerberos account might have an invalid password. If you see the following error message in report, you will probably need to reset the Kerberos account password:
Operations Management and Monitoring of a Lync Server 2010 Environment
63
InvalidKerberosConfiguration: The Kerberos configuration is invalid.
InvalidKerberosConfiguration: The Kerberos configuration on atl- cs001.litwareinc.com is invalid. The expected assigned account is litwareinc\kerberostest. Ensure that the account has not expired, and the configured password on the machine matches the Active Directory password of the account.
You can set the password using the Set-CsKerberosAccountPassword cmdlet.
Operations Management and Monitoring of a Lync Server 2010 Environment
64
4.3.18.9 Testing Peer to Peer Audio/Video Call
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsP2PAV cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsP2PAV"}
Description Test-CsP2PAV is used to determine whether a pair of test users is able to participate in a peer-to-peer A/V conversation. To test this scenario, the cmdlet starts off by logging on the two users to Lync Server. Assuming that the two logons succeed, the first user then invites the second user to join an A/V call. The second user accepts the call, the connection between the two users is tested, and then the call is ended and the test users are logged off from the system. Test-CsP2PAV does not actually conduct an A/V call; multimedia information is not exchanged between the test users. Instead, the cmdlet simply verifies that the appropriate connections can be made and that the two users are capable of conducting such a call.
Running the Test The Test-CsP2PAV cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsP2PAV -TargetFqdn "atl-cs-001.litwareinc.com"
To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test-CsP2PAV: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire"
Test-CsP2PAV -TargetFqdn "atl-cs-001.litwareinc.com" -SenderSipAddress "sip:kenmyer@litwareinc.com" -SenderCredential $credential1 - ReceiverSipAddress "sip:davidlongmire@litwareinc.com" -ReceiverCredential $credential2 For more information, see the help documentation for the Test-CsP2PAV cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
65
Determining Success or Failure If the two test users are able to complete a peer-to-peer A/V call, then you will get back output similar to this with the Result property marked as Success:
If the test users are not able to complete the call, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 480, Temporarily Unavailable Diagnosis : ErrorCode=15030,Source=atl-cs-001.litwareinc.com,Reason=Failed to route to Exchange Server Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because the Exchange Server could not be contacted. This error message typically indicates a problem the configuration of Exchange Unified Messaging.
If Test-CsP2PAV fails then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsP2PAV will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, suppose your test fails with the following Diagnosis: ErrorCode=6003,Source=atl-cs-001.litwareinc.com,Reason=Unsupported out of dialog request If you rerun Test-CsP2PAV and include the Verbose parameter, you will get output similar to this: VERBOSE: 'Register' activity started. Sending Registration request: Target Fqdn = atl-cs-011.litwareinc.com User Sip Address = sip:kenmyer@litwareinc.com Registrar Port = 5062. Auth Type 'IWA' is selected. An exception 'The endpoint was unable to register. See the ErrorCode for specific reason.' occurred during workflow Microsoft.Rtc.SyntheticTransactions.Workflows.STP2PAVWorkflow execution. Operations Management and Monitoring of a Lync Server 2010 Environment
66
Although it might not be immediately obvious, if you examine the output carefully youll see that an invalid Registrar port (port 5062) was specified. In turn, that caused the test to fail.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsP2PAV might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
Operations Management and Monitoring of a Lync Server 2010 Environment
67
4.3.18.10 Testing the Ability of a User to Log On to Lync Server
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsRegistration cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsRegistration"}
Description The Test-CsRegistration cmdlet enables you to verify that users in your organization can log on to Lync Server. When you run Test-CsRegistration, the cmdlet attempts to sign on a test user to Lync Server and then, if successful, disconnects that test user from the system. All of this happens without any user interaction, and without affecting any actual users. For example, suppose the test account sip:kenmyer@litwareinc.com corresponds to a real user with a real Lync Server account. In that case, the test will be conducted without any disruption to the real Ken Myer. When the Ken Myer test account logs off from the system, Ken Myer the person will remain logged on.
Running the Test The Test-CsRegistration cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server Registrar pool being tested. For example: Test-CsRegistration -TargetFqdn "atl-cs-001.litwareinc.com" To run this check using an actual user account, you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsRegistration: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsRegistration -TargetFqdn "atl-cs-001.litwareinc.com"-UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsRegistration cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
68
If the specified user is able to log on to (and then log off from) Lync Server, you will get back output similar to this with the Result property marked as Success:
If the specified user is not able to log on or log off, the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 404, Not Found Diagnosis : ErrorCode=1003,source=atl-cs-001.litwareinc.com,Reason=User does not exist Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because the specified user could not be found. You can verify whether or not a SIP address is valid (and whether or not the user assigned that SIP address has been enabled for Lync Server) by running this command:
Get-CsUser "sip:kenmyer@litwareinc.com"
If Test-CsRegistration fails, then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsRegistration will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example: VERBOSE: 'Register' activity started. Sending Registration request: Target Fqdn = atl-cs-011.litwareinc.com User Sip Address = sip:kenmyer@litwareinc.com Registrar Port = 5061. Auth Type 'Trusted' is selected.
An exception 'The endpoint is unable to register. See the ErrorCode for specific reason' occurred during Workflow Microsoft.Rtc.SyntheticTransactions.Workflow.STRegistrerWorkflow execution. Exception Call Stack: at Microsoft.Rtc.Signaling.SipAsyncResult'1.ThrowIfFailed()
Reasons Why the Test Might Have Failed Operations Management and Monitoring of a Lync Server 2010 Environment
69
Here are some common reasons why Test-CsRegistration might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
You specified an invalid Registrar pool. You can return the FQDNs of your Registrar pools by using this command:
Get-CsService Registrar | Select-Object PoolFqdn
The Registrar pool is not currently available. Try pinging the pool to see if it responds:
ping atl-cs-001.litwareinc.com
Operations Management and Monitoring of a Lync Server 2010 Environment
70
4.3.18.11 Testing User Presence Publishing and Subscribing
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsPresence cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsPresence"}
Description Test-CsPresence is used to determine whether a pair of test users can log on to Lync Server and then exchange presence information. To do this, the cmdlet first logs the two users on to the system. If both logons succeed, the first test user then asks to receive presence information from the second user. The second user publishes this information, and Test-CsPresence verifies that the information was successfully transmitted to the first user. After the exchange of presence information, the two test users are then logged off from Lync Server.
Running the Test The Test-CsPresence cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsPresence -TargetFqdn "atl-cs-001.litwareinc.com"
To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test-CsPresence: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire"
Test-CsPresence -TargetFqdn "atl-cs-001.litwareinc.com" -PublisherSipAddress "sip:kenmyer@litwareinc.com" -PublisherCredential $credential1 - SubscriberSipAddress "sip:davidlongmire@litwareinc.com" -SubscriberCredential $credential2 For more information, see the help documentation for the Test-CsPresence cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
71
If the specified users are able to exchange presence information, then you will get back output similar to this, with the Result property marked as Success:
If the two users are not able to exchange presence information, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 404, Not Found Diagnosis : ErrorCode=4005,Source=atl-cs-001.litwareinc.com, Reason=Destination URI either not enabled for SIP or does not exist. Microsoft.Rtc.Signaling.DiagnosticHeader
For example, the preceding output states that the test failed because at least one of the two user accounts is not valid: either the account does not exist or it has not been enabled for Lync Server. You can verify the existence of the accounts, and determine whether or not they have been enabled for Lync Server, by running a command similar to this:
When the Verbose parameter is included, Test-CsPresence will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example:
Registration Request hit against Unknown 'Register' activity completed in '0.0345791' secs. 'SelfSubscribeActivity' activity started. 'SelfSubscribeActivity' activity completed in '0.0041174' secs. 'SubscribePresence' activity started. 'SubscribePresence' activity completed in '0.0038764' secs. 'PublishPresence' activity started. An exception 'Presence notification is not received within 25 secs.' occurred ruing Workflow Microsoft.Rtc.SyntheticTransactions.Workflows.STPresenceWorkflow execution. The fact that the presence notification was not received within 25 seconds might indicate that network issues are preventing information from being exchanged.
Operations Management and Monitoring of a Lync Server 2010 Environment
72
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsPresence might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
Operations Management and Monitoring of a Lync Server 2010 Environment
73
4.3.18.12 Testing Service Activation and Group Permissions
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsRegistration cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsTopology"}
Description The Test-CsTopology cmdlet provides a way for you to verify that Lync Server 2010 is functioning correctly at a global scope. By default, the cmdlet checks your entire Lync Server infrastructure, verifying that the required services are running and that the appropriate permissions have been set for these services and for the universal security groups created when you install Lync Server. In addition to verifying the validity of Lync Server as a whole, Test-CsTopology also lets you check the validity of a specific service. For example, this command checks the state of the A/V Conferencing Server on the pool atl-cs-001.litwareinc.com: Test-CsTopology Service "ConferencingServer:atl-cs-001.litwareinc.com"
Running the Test By default, Test-CsTopology displays very little output on-screen; instead, information returned by the cmdlet is written to an HTML file. The Report parameter allows you to specify a file path and file name for the HTML file generated by Test-CsTopology. If you do not include the Report parameter the HTML file will automatically be saved to your Users folder and be given a name similar to this: ce84964a-c4da- 4622-ad34-c54ff3ed361f.html. The following sample command runs Test-CsTopology and saves the output to a file named C:\Logs\ComputerTest.html: Test-CsTopology -Report "C:\Logs\ComputerTest.html" -Verbose For more information, see the help documentation for the Test-CsTopology cmdlet.
Determining Success or Failure Unlike most of the test cmdlets, Test-CsTopology does report back Success or Failure; in part, thats due to the large number of verification checks that the cmdlet must make each time it runs. Instead, data is saved to an HTML report that can then be viewed by using Internet Explorer:
Operations Management and Monitoring of a Lync Server 2010 Environment
74
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsTopology might fail: Replication might not be up-to-date on the test computer. You can check the current replication status for a computer by running the Get-CsManagementStoreReplicationStatus cmdlet:
The topology might need to be enabled. If you make changes to the Lync Server topology (changes that might affect the local computer), then you must enable the new topology. You can enable the topology at any time by running this command:
Enable-CsTopology
Operations Management and Monitoring of a Lync Server 2010 Environment
75
4.3.18.13 Testing Dial-in Conferencing Session
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsDialInConferencing cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsDialInConferencing"}
Description The Test-CsDialInConferencing cmdlet verifies whether or not a user can participate in a dial-in conference. Test-CsDialInConferencing works by attempting to log a test user onto the system. If the logon succeeds, the cmdlet will then use the users credentials and permissions to try all of the available dial-in conferencing access numbers. The success or failure of each dial-in attempt will be noted, then the test user will be logged off from Lync Server. Test-CsDialInConferencing only verifies that the appropriate connections can be made. The cmdlet does not actually make any phone calls or create any dial-in conferences that other users can join.
Running the Test The Test-CsDialInConferencing cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool being tested. For example:
To run this check using an actual user account, you must create a Windows PowerShell credentials object containing the account name and password. You must then include that credentials object and the account SIP address the calling Test-CsDialInConferencing: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsDialInConferencing -TargetFqdn atl-cs-001.litwareinc.com" - UserSipAddress "sip:kenmyer@litwareinc.com" -UserCredential $credential For more information, see the help documentation for the Test-CsDialInConferencing cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
76
If the specified user is able to log on to Lync Server and then make a connection using one of the available dial-in conferencing access numbers, then you will get back output similar to this, with the Result property marked as Success:
If the specified user is unable to make this connection, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : The log on was denied. Check that the proper credentials are being used and the account is active. Inner Exception:NegotiateSecurityAssociation failed, error: - 2146893044 Diagnosis :
The preceding output indicates that the test user was denied access to Lync Server itself; this typically means that the user credentials passed to Test-CsDialInConferencing were not valid. In turn, you should recreate the Windows PowerShell credentials object. Although you can retrieve the password for the user account, you can verify the SIP address by using a command similar to this:
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsDialInConferencing might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
You might have an invalid dial-in conferencing access number. You can return your currently- configured list of dial-in conferencing access numbers by using this command:
Operations Management and Monitoring of a Lync Server 2010 Environment
77
Get-CsDialConferencingAccessNumber
Operations Management and Monitoring of a Lync Server 2010 Environment
78
4.3.18.14 Testing the Dial Plan
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsDialPlan cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsDialPlan"}
Description The Test-CsDialPlan cmdlet enables you to see the results of applying a dial plan to a given telephone number. Dial plans provide information, including how normalization rules are applied, required to enable Enterprise Voice users to make telephone calls. Given a dialed number and a dial plan, this cmdlet will verify which normalization rule within the dial plan will be applied and what the translated number will be. You can use this cmdlet for troubleshooting issues with number translations, or for verifying the application of rules against certain numbers.
Running the Test The Test-CsDialPlan cmdlet requires you to do two things. First, you must obtain an instance of the dial plan being tested; that can be done by using the Get-CsDialPlan cmdlet. Second, you must specify the phone number that needs to be normalized. The format used for the phone number should match the number as dialed/entered by a user. For example, this command retrieves an instance of the dial plan, RedmondDialPlan, and checks to see if the phone number 12065551219 can be normalized:
If you have a normalization rule that automatically adds the country code (in this example, 1) and the area code (206), then you might want to check the phone number 5551219, like so: Get-CsDialPlan -Identity "RedmondDialPlan" | Test-CsDialPlan -DialedNumber "5551219" | Format-List For more information, see the help documentation for the Test-CsDialPlan cmdlet.
Determining Success or Failure Test-CsDialPlan differs from many of the Lync Server test cmdlets because it only indirectly indicates whether a test succeeded or failed. With Test-CsDialPlan you do not get back output similar to this with the Result clearly labeled: Operations Management and Monitoring of a Lync Server 2010 Environment
79
TargetFqdn : atl-cs-001.litwareinc.com Result : Success Latency : 00:00:06.8630376 Error : Diagnosis : Instead, if Test-CsDialPlan succeeds, then you will get back information about the normalization rule that was able to successfully translate and employ the specified phone number:
If Test-CsDialPlan fails (that is, if the dial plan does not have a normalization rule that can translate the specified phone number), you will simply get back empty output like this:
TranslatedNumber : MatchingRule :
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsDialPlan might fail: You might have used an invalid format when specifying the phone number. Dial plans include normalization rules that enable Lync Server to translate the phone numbers dialed or entered by a user. As such, your dial plan should have normalization rules that match the numbers users are likely to dial. For example, if users might dial the country code, area code, and then the phone number itself, that means your dial plan should have a normalization rule to handle phone numbers such as this:
12065551219
However, if you enter an invalid phone number (for example, leaving off the final digit), then Test- CsDialPlan will fail. However, thats not because the dial plan is faulty but because you have entered a phone number than cannot be interpreted.
Operations Management and Monitoring of a Lync Server 2010 Environment
80
4.3.18.15 Testing Civic Addresses Against the Master Street Address Guide
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsRegistration cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsLisCivicAddress"}
Description The Test-CsLisCivicAddress cmdlet is used to verify locations that have been added to your Location Information service (LIS) database. The cmdlet works by comparing locations against the locations found in the Master Street Address Guide (MSAG) belonging to your E9-1-1 Network Routing Provider. If you do not have a network routing provider or if the provider cannot be reached, then your tests will fail. If you add the optional switch parameter UpdateValidationStatus to your command, then the corresponding MSAGValid database property will be set to True for each address passing the test.
Running the Test The Test-CsLisCivicAddress cmdlet can be used to test individual addresses or to test multiple addresses. For example, this command tests a single address located in Redmond, WA: Test-CsLisCivicAddress -HouseNumber 1234 -HouseNumberSuffix "" - PreDirectional "" -StreetName Main -StreetSuffix St -PostDirectional "" -City Redmond -State WA -PostalCode 98052 -Country US UpdateValidationStatus
By comparison, this command tests all the addresses currently in your LIS database: Get-CsLisCivicAddress | Test-CsLisCivicAddress -UpdateValidationStatus For more information, see the help documentation for the Test-CsLisCivicAddress cmdlet.
Determining Success or Failure Test-CsLisCivicAddress will report back Success or Failure for each of the supplied addresses. An address test will fail if the address cannot be found or if the service provider cannot be contacted.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsLisCivicAddress might fail: The LIS service provider might not be available. You can retrieve the URL of your LIS service provider by running the Get-CsLisConfiguration cmdlet: Operations Management and Monitoring of a Lync Server 2010 Environment
81
Get-CsLisConfiguration
You can then ping that URL to verify that the service provider is available.
Operations Management and Monitoring of a Lync Server 2010 Environment
82
4.3.18.16 Testing LIS Server Configuration
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsLisConfiguration cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsLisConfiguration"}
Description The Test-CsLisConfiguration cmdlet verifies your ability to contact the LIS web service. If the web service can be contacted, then the test will be considered a success, regardless of whether or not any specific locations can be found.
Running the Test The Test-CsLisConfguration cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsLisConfiguration -TargetFqdn "atl-cs-001.litwareinc.com"
To run this check using an actual user account you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsLisConfiguration: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsLisConfiguration -TargetFqdn "atl-cs-001.litwareinc.com"- UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsLisConfiguration cmdlet.
Determining Success or Failure If the LIS is correctly configured, you will get back output similar to this, with the Result property marked as Success:
TargetUri : https://atl-cs-001.litwareinc.com:443/locationinformation/ liservice.svc TargetFqdn : atl-cs-001.litwareinc.com Operations Management and Monitoring of a Lync Server 2010 Environment
83
Result : Success Latency : 00:00:06.1616913 Error : Diagnosis :
If the specified user is not able to log on or log off, the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetUri : TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 11004, The requested name is valid but no data of the requested type was found Diagnosis :
Test-CsLisConfiguration : No matching cluster found in topology.
For example, the preceding output includes the note No matching cluster found in topology. That typically indicates a problem with the Edge Server: the LIS using the Edge Server to connect to the service provider and validate addresses.
If Test-CsLisConfiguration fails then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsLisConfiguration will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example:
Calling Location Information Service. Service Path = https://atl-cs- 001.litwareinc.com:443/locationinformation/liservice.svc Subnet = BssId = 5 ChassisId = PortId = PortIdSubType = Undefined Type Mac An exception 'Location Information Web Service request has failed with a response code Item400.' occurred during Workflow Microsoft.Rtc.SyntheticTrsnactions.Workflows.STLisConfigurationWorkflow execution.
If you examine the preceding output closely, youll see that the cmdlet failed after trying to call the Location Information Service. One of the parameters used in that call was this: BssId = 5 Thats an invalid value for the Basic Service Set Identifier (BssID). Instead, a BssID should look like this: 12-34-56-78-90-ab Operations Management and Monitoring of a Lync Server 2010 Environment
84
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsLisConfiguration might fail: An incorrect parameter value was supplied. As shown in the previous example, the optional parameters must be configured correctly or the test will fail. Rerun the command without the optional parameters and see if that succeeds.
Operations Management and Monitoring of a Lync Server 2010 Environment
85
4.3.18.17 Testing Location Policy
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsLocationPolicy cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsLocationPolicy"}
Description The Test-CsLocationPolicy cmdlet verifies that a location policy has been assigned to a user. The location policy is used to apply settings that relate to E9-1-1 functionality and client location. The location policy determines whether a user is enabled for E9-1-1, and, if so, what the behavior is of an emergency call. For example, you can use the location policy to define what number constitutes an emergency call (911 in the United States), whether corporate security should be automatically notified, and how the call should be routed. You can test location policies on users or on network subnets. If you run the test against a subnet (by specifying a value for the Subnet parameter), the cmdlet will attempt to resolve the location policy for that subnet. If no location policy is assigned to the subnet, the location policy for the configured user will be retrieved. If the subnet policy is retrieved successfully, the output will include a LocationPolicyTagID value beginning with subnet-tagid. If a location policy for the subnet was not found, the LocationPolicyTagID will begin with user-tagid.
Running the Test The Test-CsLocationPolicy cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool being tested. For example: Test-CsLocationPolicy -TargetFqdn "atl-cs-001.litwareinc.com" To run this check using an actual user account, you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsLocationPolicy: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsLocationPolicy -TargetFqdn "atl-cs-001.litwareinc.com"-UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsLocationPolicy cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
86
Determining Success or Failure If the specified user has a valid location policy, then you will get back output similar to this, with the Result property marked as Success:
If a valid location policy cannot be found for the specified user, then Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:00 Error : 404, Not Found Diagnosis : ErrorCode=4005,Source=atl-cs-001.litwareinc.com, Reason=Destination URI either not enabled for SIP or does not exist. Microsoft.Rtc.Signaling.DiagnosticHeader
The preceding output states that the test failed because the specified user is not valid: either the account does not exist or the user has not been enabled for Lync Server. You can verify whether the validity of an account, and determine whether or not that account has been enabled for Lync Server, by running a command similar to this:
When the Verbose parameter is included, Test-CsLocationPolicy will return a step-by-step account of each action it attempted when verifying the location policy. For example, this output indicates that Lync Server could not log on the test user, probably because an invalid password was supplied:
Sending Registration request : Target Fqdn = atl-cs-011.litwareinc.com User Sip Address = sip:kenmyer@litwareinc.com Registrar Port = 5061 Auth Type 'IWA' is selected. Registration hit against sip/atl-cs-001.litwareinc.com Operations Management and Monitoring of a Lync Server 2010 Environment
87
'Register' activity completed in '0.0601795' secs. An exception 'The log on was denied. Check that the proper credentials are being used and the account is active.' occurred during the Workflow.
Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsLocationPolicy might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
Operations Management and Monitoring of a Lync Server 2010 Environment
88
4.3.18.18 Testing Lync Phone Edition Login
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsPhoneBootstrap cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsPhoneBootstrap"}
Description The Test-CsPhoneBootstrap cmdlet enables administrators to verify that a given userusing the phone number and PIN assigned to him or heris able to log on to the system from a Lync 2010 Phone Edition- compatible device. (No device is actually needed in order to run the test.) In order for Test-CsPhoneBootstrap to make its check, the Registrar pool that hosts the user account being tested must be discoverable using DHCP. To determine whether a Registrar is discoverable in this manner, use the cmdlet Get-CsRegistrarConfiguration and check the value of the EnableDHCPServer property. If this property is set to False, you will need to use Set-CsRegistrarConfiguration to set the property value to True and make the Registrar discoverable using DHCP. This can also be done by using Enterprise DHCP Server and configuring the Lync Server-specific options.
Running the Test To run the Test-CsPhoneBootstrap cmdlet, you must, at a minimum, supply the phone number and client PIN number for a valid Lync Server user. For example, this command tests the logon ability for the user with the phone number 12065551219 and the PIN number 0712:
For a more comprehensive check, you can also include the users SIP address. In that case, the phone number, client PIN number, and SIP address must all be valid for the test to succeed: Test-CsPhoneBootstrap -PhoneOrExt "+12065551219" -Pin "0712" UserSipAddress "sip:kenmyer@litwareinc.com"
For more information, see the help documentation for the Test-CsPhoneBootstrap cmdlet.
Determining Success or Failure Operations Management and Monitoring of a Lync Server 2010 Environment
89
If the specified user was able to connect to Lync Server, you will get back output similar to this, with the Result property marked as Success:
If the specified user was not able to make a connection, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:04.1993845 Error : ERROR - No response received for Web-Ticket service. Diagnosis :
The preceding output states that the test failed because the Web Ticket service failed to respond. This could be due to a problem with the service itself, or it could be due to the SIP address, phone number, and/or client PIN passed to Test-CsPhoneBootstrap. You can verify the users SIP address and phone number by using a command similar to this one:
When the Verbose parameter is included, Test-CsPhoneBootstrap will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, here is a portion of the output for an unsuccessful logon, a session in which an invalid PIN number was included:
Using PIN auth with Phone\Ext : 12065551219 Pin : 0712 Could not get web ticket CHECK : - Web service url is valid and the web services are functional - If using PhoneNo\PIN to authenticate, make sure they match the user uri - If using NTLM\Kerberos auth, make sure you provided valid credentials
Reasons Why the Test Might Have Failed Operations Management and Monitoring of a Lync Server 2010 Environment
90
Here are some common reasons why Test-CsPhoneBootstrap might fail: You might have specified an invalid SIP address. You can verify that a SIP address is correct by using a command like this one:
Get-CsUser Identity "sip:kenmyer@litwareinc.com"
You might have specified an invalid PIN number. Although you cannot retrieve the users PIN number, you can verify that the user at least has a PIN number by using a command similar to this:
The Registrar pool is not DHCP-enabled. To determine whether your Registrar pool has been enabled for DHCP, run the Get-CsRegistrarConfiguration cmdlet and check the value of the EnableDHCPServer property. For example:
Get-CsRegistrarConfiguration Identity "global"
Operations Management and Monitoring of a Lync Server 2010 Environment
91
4.3.18.19 Testing PSTN Phone Call
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test-CsRegistration cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsPstnOutboundCall"}
Description The Test-CsPstnOutboundCall cmdlet tests the ability of a user to make a call to a phone number located on the PSTN. When you run Test-CsPstnOutboundCall, the cmdlet first attempts to log the test user on to Lync Server. If the logon succeeds, the cmdlet will then try to make a phone call across the PSTN gateway. This phone call will be placed using the dial plan, voice policy, and other policies and settings assigned to the test account. When the call is answered, the cmdlet sends dual-tone multifrequency (DTMF) codes across the network in order to verify media connectivity. When conducting its test, Test-CsPstnOutboundCall will make an actual phone call: the target phone will ring and must be answered for the test to succeed. This call must also be manually ended by the administrator.
Running the Test The Test-CsPstnOutboundCall cmdlet can be run using either a preconfigured test account (see Setting Up Test Accounts for Running Lync Server Tests) or the account of any user who has been enabled for Lync Server. To run this check using a test account, all you need to do is specify the FQDN of the Lync Server pool being tested as well as the PSTN phone number being called. For example: Test-CsPstnOutboundCall -TargetFqdn "atl-cs-001.litwareinc.com" TargetPstnPhoneNumber "+12065551219" To run this check using an actual user account, you must first create a Windows PowerShell credentials object that contains the account name and password. You must then include that credentials object and the SIP address assigned to the account when calling Test-CsPstnOutboundCall: $credential = Get-Credential "litwareinc\kenmyer"
Test-CsPstnOutboundCall -TargetFqdn "atl-cs-001.litwareinc.com" TargetPstnPhoneNumber "+12065551219" -UserSipAddress "sip:kenmyer@litwareinc.com" UserCredential $credential For more information, see the help documentation for the Test-CsPstnOutboundCall cmdlet. Operations Management and Monitoring of a Lync Server 2010 Environment
92
Determining Success or Failure If the specified user is able to make the call, and if the call is answered, you will get back output similar to this, with the Result property marked as Success:
If the specified user is not able to make the call or if the call is not answered, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:0987365 Error : 403, Forbidden Diagnosis : ErrorCode=12001,Source=atl-cs-001.litwareinc.com,Reason=User Policy does not contain phone route usage
The preceding output states that the test failed because the voice policy assigned to the specified user does not include a phone usage. (Phone usages tie voice policies to voice routes. Without both a voice policy and a corresponding voice route you cannot make calls over the PSTN.)
If Test-CsPstnOutboundCall fails then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsPstnOutboundCall will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, this output indicates that network problems are preventing a connection with the PSTN:
Establishing Audio Video call to 'sip:+12065551219@litwareinc.com;user=phone'. An exception 'A 404 (Not Found) response was received from the network and the operation failed. Reasons Why the Test Might Have Failed Here are some common reasons why Test-CsPstnOutboundCall might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
Operations Management and Monitoring of a Lync Server 2010 Environment
93
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False that means that the user is not currently enabled for Lync Server.
The voice policy assigned to the specified user does not have a valid PSTN usage. You can determine the voice policy that has been assigned to a user by using a command similar to this:
And then you can determine the PSTN usages (if any) that have been assigned to that policy by using a command similar to the following, which retrieves information about the per-user voice policy RedmondVoicePolicy:
Get-CsVoicePolicy Identity "RedmondVoicePolicy"
Operations Management and Monitoring of a Lync Server 2010 Environment
94
4.3.18.20 Testing PSTN Peer to Peer Call
Verification schedule Daily Testing tool Windows PowerShell Permissions required When run locally using the Lync Server Management Shell, users must be members of the RTCUniversalServerAdmins security group. When run using a remote instance of Windows PowerShell, users must be assigned an RBAC role that has permission to run the Test- CsPstnPeerToPeerCall cmdlet. To see a list of all RBAC roles that can use this cmdlet, run the following command from the Windows PowerShell prompt: Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsPstnPeerRoPeerCall"}
Description The Test-CsPstnPeerToPeerCall cmdlet verifies the ability a pair of users has to conduct a peer-to-peer call over the public switched telephone network (PSTN) gateway. When you call Test- CsPstnPeerToPeerCall, the cmdlet will first attempt to log on two test users to Lync Server. Assuming that the logons succeed, the cmdlet will then have user 1 attempt to call user 2 over the PSTN gateway; Test-CsPstnPeerToPeerCall will make this call using the dial plan, voice policy, and other policy and configuration settings assigned to the test user. If the test goes as planned, the cmdlet will verify that user 2 was able to answer the call, and then log off both test accounts from the system. Test-CsPstnPeerToPeerCall makes an actual phone call, one that verifies that a connection can be made and that also transmits DTMF codes across the network to determine whether media can be sent over the connection. However, the call is answered by the cmdlet itself, and no manual termination of the call is necessary. (That is, no one needs to answer and then hang up the phone that was called.)
Running the Test The Test-CsPstnPeerToPeerCall cmdlet can be run using either a pair of preconfigured test accounts (see Setting Up Test Accounts for Running Lync Server Tests) or the accounts of any two users who have been enabled for Lync Server. To run this check using test accounts, all you need to do is specify the FQDN of the Lync Server pool being tested. For example:
To run this check using actual user accounts, you must create two Windows PowerShell credentials objects (objects containing the account name and password) for each account. You must then include those credentials objects and the SIP addresses of the two accounts when calling Test- CsPstnPeerToPeerCall: $credential1 = Get-Credential "litwareinc\kenmyer" $credential2 = Get-Credential "litwareinc\davidlongmire" Operations Management and Monitoring of a Lync Server 2010 Environment
95
Test-CsPstnPeerToPeerCall -TargetFqdn "atl-cs-001.litwareinc.com" - SenderSipAddress "sip:kenmyer@litwareinc.com" -SenderCredential $credential1 -ReceiverSipAddress "sip:davidlongmire@litwareinc.com" -ReceiverCredential $credential2 For more information, see the help documentation for the Test-CsPstnPeerToPeerCall cmdlet.
Determining Success or Failure If the specified users are able to complete a peer-to-peer call, you will get back output similar to this, with the Result property marked as Success:
If the specified users are not able to complete a peer-to-peer call, then the Result will be shown as Failure, and additional information will be recorded in the Error and Diagnosis properties:
TargetFqdn : atl-cs-001.litwareinc.com Result : Failure Latency : 00:00:0182361 Error : 403, Forbidden Diagnosis : ErrorCode=12001,Source=atl-cs-001.litwareinc.com, Reason=User Policy does not contain phone route usage
The preceding output states that the test failed because the voice policy assigned to at least one of the specified users does not include a phone usage. (Phone usages tie voice policies to voice routes. Without both a voice policy and a corresponding voice route, you cannot make calls over the PSTN.)
If Test-CsPstnPeerToPeerCall fails, then you might want to rerun the test, this time including the Verbose parameter:
When the Verbose parameter is included, Test-CsPstnPeerToPeerCall will return a step-by-step account of each action it attempted when checking the ability of the specified user to log on to Lync Server. For example, this output indicates that network problems are preventing a connection with the PSTN:
Establishing Audio Video call to 'sip:+12065551219@litwareinc.com;user=phone'. An exception 'A 404 (Not Found) response was received from the network and the operation failed.
Reasons Why the Test Might Have Failed Operations Management and Monitoring of a Lync Server 2010 Environment
96
Here are some common reasons why Test-CsPstnPeerToPeerCall might fail: You specified an invalid user account. You can verify that a user account exists by running a command similar to this:
Get-CsUser "sip:kenmyer@litwareinc.com"
The user account is valid, but the account is not currently enabled for Lync Server. To verify that a user account has been enabled for Lync Server, run a command similar to the following:
If the Enabled property is set to False, that means that the user is not currently enabled for Lync Server.
The voice policy assigned to the specified user does not have a valid PSTN usage. You can determine the voice policy that has been assigned to a user by using a command similar to this:
And then you can determine the PSTN usages (if any) that have been assigned to that policy by using a command similar to the following, which retrieves information about the per-user voice policy RedmondVoicePolicy:
Get-CsVoicePolicy Identity "RedmondVoicePolicy"
4.4 Weekly Tasks 4.4.1 Archive Event Logs If event logs are not configured to overwrite events as required, they must be regularly archived and deleted. This action is especially important for security logs, which may be required when investigating attempted security breaches. Your organization will need to define policies and procedures for log archiving. 4.4.2 Create Reports Create status reports to help with capacity planning, SLA reviews, and performance analysis. Use daily data from event log and System Monitor to create reports on disk, memory, and CPU usage. Use System Center Operations Manager to generate uptime and availability reports. 4.4.3 Incident Reports Perform a weekly audit of your organizations incident reports relating to Lync Server. This audit should include: The top generated, resolved, and pending incidents. Solutions for unresolved incidents. Operations Management and Monitoring of a Lync Server 2010 Environment
97
Updating reports to include new trouble tickets. Updating a document repository for troubleshooting guides and post mortems about outages. Since your organizations incident tracking system is a choice independent of Lync Server, specific instructions or pointers are not available. Consult the documentation for the system your organization chose. 4.4.4 Check IIS Logs and Performance Perform a weekly review of IIS logs and performance. For more information about monitoring IIS logs and performance, see Windows Server 2003 Internet Information Services (IIS) Event Logging Overview. The review should include: Web Service Cache counters to monitor the WWW service cache. FTP Service counters to monitor the File Transfer Protocol (FTP) service. Active Server Pages (ASPs) counters to monitor applications that run as ASPs. For more information about monitoring IIS logs and performance, see Windows Server 2003 Internet Information Services (IIS) Event Logging Overview. 4.4.5 Generate Database Reports To generate reports on the SQL Database: 1. Open Lync Server 2010. 2. In the console tree, expand the forest node, expand Enterprise pools, and click the pool for which you want to generate a database report. 3. In the details pane, click the Database tab. 4. On the Database tab, do the following: a. To view the name of the database, expand General Settings, and view the database name. b. To retrieve current user summary statistics for the pool, expand User Summary Reports, click Go, and view the results. c. To retrieve current per-user data for an individual user of the pool, expand Per-User Reports, type the users SIP URI, click Go, and view the results. To retrieve current conference summary statistics for the pool, expand Conference Summary Reports, click Go, and view the results. 4.4.6 Check for Security and Lync Server Updates Identify any new service packs, hotfixes, or updates. If appropriate, test these in a test lab, and use the change control procedures to arrange for deployment to the production servers. Also, Lync Server component updates are now available as part of Windows update. All Lync Server component updates must be updated at the same time on all of the servers running Lync Server for which the updates are applicable. Operations Management and Monitoring of a Lync Server 2010 Environment
98
4.4.7 Run the Lync Server 2010 Best Practice Analyzer The Lync Server 2010 Best Practices Analyzer Tool is a diagnostic tool that gathers configuration information and determines whether the configuration is set according to Microsoft best practices. Documentation for this tool is at http://technet.microsoft.com/en-us/library/jj204694.aspx. The tool compares your deployments configuration data against a set of pre-defined rules for Lync Server, and reports potential issues. For every issue reported, the tool provides the current configuration in the Lync Server environment, and the recommended configuration. With the proper network access, the tool can examine your AD DS and servers running Lync Server 2010 to do the following: Proactively perform health checks, verifying that the configuration is set according to recommended best practices Generate a list of issues, such as suboptimal configuration settings or unsupported or not recommended options Judge the general health of a system Help troubleshoot specific issues Prompt you to download updates if they are available Provide online and local documentation about reported issues, including troubleshooting tips Generate configuration information that can be captured for later review Make sure that the RTCBPA.msi is installed on all Lync Server 2010 servers, and generate a weekly Health Check Report. Note the results and remediate, if required. 4.4.8 Review SLA Performance Figures Check the key performance data for the previous week. Review performance against the requirements of the SLA. Identify trends and items that have not met their targets. 4.4.9 Review System Center Operations Manager Management Pack and Quality of Experience Reports Obtain and review Lync Server 2010 Management Pack and Quality of Experience reports. 4.4.10 Generating and Viewing Database Reports for Enterprise Pools To generate pool reports: 1. Open Lync Server 2010. 2. In the console tree, expand the forest node, expand Enterprise pools, and click the pool for which you want to generate a database report. 3. In the details pane, click the Database tab. 4. On the Database tab, do the following: a. To view the name of the database, expand General Settings, and view the database name. b. To retrieve current user summary statistics for the pool, expand User Summary Reports, click Go, and view the results. Operations Management and Monitoring of a Lync Server 2010 Environment
99
c. To retrieve current per-user data for an individual user of the pool, expand Per-User Reports, type the users SIP URI, click Go, and view the results. To retrieve current conference summary statistics for the pool, expand Conference Summary Reports, click Go, and view the results. For each Enterprise Pool, administrators can use the Database tab to view the database name as well as retrieve and view reports from the database. Database Reports and Descriptions Section Description User Summary Reports Dbanalyze /v /report:diag [/sqlserver:value] This section displays aggregate information about users in a pool, such as the number of enabled users, the average number of contacts per user, and the number of users for specific features. When using these reports, the following information may be helpful: An enabled user is a user who has been enabled for Lync Server 2010 by using the Active Directory Users and Computers Snap-in. An active user is a user who has logged on or registered. The summary reports also offer a set of statistical information regarding contacts. These statistics are only valid for the population of users who have logged on at least once, and who have at least one contact. Consequently, you will typically not see a minimum number of contacts of 0. Because of this behavior, if a user has no contacts (but is active, in that the user has registered), you may see: <empty> for some of the statistics fields. Per-User Reports Dbanalyze /v /report:disk [/sqlserver:value] Unlike the summary reports, which are calculated over a user population, these are reports about a specific user. Conference Summary Reports Dbanalyze /v /report:conf [/sqlserver:value] This section displays aggregate information about conference summary statistics for the pool, such as the number of active conferences and total number of participants.
4.4.11 Running Bandwidth Utilization Analyzer Bandwidth Utilization Analyzer is a tool that creates reports about various views of bandwidth consumption by the UC endpoints across WAN links in the enterprise network. These reports can be used to understand the current bandwidth consumption pattern and to aid in bandwidth capacity planning. It also iterates on the bandwidth capacity that is assigned to various links. This tool does the following: Generates specific reports for audio utilization across the network Helps with more effective capacity planning and iteration on the bandwidth capacity that is assigned to various links Operations Management and Monitoring of a Lync Server 2010 Environment
100
Bandwidth Utilization Analyzer can generate graphical plots of bandwidth capacity and utilization reports; they are as follows: All the WAN links in the enterprise network Filtered by selected WAN links that have been chosen Filtered by WAN links that have exceeded link capacity Filtered by WAN links that have been under-utilizing the provisioned bandwidth Filter by WAN links that have been reaching critical levels (a bandwidth utilization that is greater than 90 percent of bandwidth capacity of the WAN link) Filtered by WAN link typenetwork-site links, interregional links, and links within a site Filtered by network region Documentation for this tool is available at http://technet.microsoft.com/en-us/library/jj945604.aspx. 4.5 Monthly Tasks 4.5.1 Viewing Status of Global Settings for a Forest Administrators should review the global settings for a Lync Server 2010 deployment on a monthly basis. The objective would be to review implemented settings against a set of known configurationsa baseline configuration to help ensure that settings are valid and to determine whether the baseline documentation should be updated. Changes to global setting should be implemented through a Change Control process which should include documenting the new settings. Global settings that should be reviewed include: General settings, including the supported Session Initiation Protocol (SIP) domains for Microsoft Lync Server 2010. To review the SIP domains authorized for use in your organization, open the Lync Server Topology Builder, import your current topology, and then click Lync Server 2010. Logically enough, SIP domain information will be shown under the heading SIP domain. Operations Management and Monitoring of a Lync Server 2010 Environment
101
SIP domain information can also be returned by using Windows PowerShell and the Get- CsSipDomain cmdlet. To return this information, run the Get-CsSipDomain Windows PowerShell command.
Get-CsSipDomain will return information similar to this for all your authorized SIP domains: Identity Name IsDefault -------- ---- --------- fabrikam.com fabrikam.com True na.fabrikam.com na.fabrikam.com False If the IsDefault property is set to True, that means that the corresponding domain is your default SIP domain. You can use the Set-CsSipDomain cmdlet to change the default SIP domain for your organization. However, you cannot simply delete the default SIP domain because that would leave you without a default domain. If you wanted to delete the fabrikam.com domain (as shown in the preceding example), you would first need to configure na.fabrikam.com to be your default domain. Meeting settings, including meeting policy definitions and support for participation of anonymous users in meetings. Meeting configuration settings can be viewed in the Lync Server Control Panel by clicking Conferencing, clicking Meeting Configuration, and then clicking the desired collection of meeting configuration settings. For example, the following graphic shows the global meeting Operations Management and Monitoring of a Lync Server 2010 Environment
102
configuration settings. Note that the last item, Admit anonymous users by default, enables or disables the ability of anonymous users to participate in meetings:
Meeting configuration settings can also be retrieved by using Windows PowerShell and the Get- CsMeetingConfiguration cmdlet. For example, this command returns information about the global meeting configuration settings: Get-CsMeetingConfiguration Identity "Global" Meeting configuration settings can also be configured at the site scope. Because of that, you might want to use the following command, which returns information about all your meeting configuration settings: Get-CsMeetingConfiguration
The Get-CsMeetingConfiguration cmdlet returns information similar to the following: Identity : Global PstnCallersBypassLobby : True EnableAssignedConferenceType : True DesignateAsPresenter : Company AssignedConferenceTypeByDefault : True AdmitAnonymousUsersByDefault : True
Again, the final item in the list, AdmitAnonymousUsersByDefault, enables or disables the ability of anonymous users to participate in meetings.
Operations Management and Monitoring of a Lync Server 2010 Environment
103
When checking meeting configuration settings, you might find it useful to compare the current settings against the default equivalents. You can view the default meeting configuration settings by running the following command: New-CsMeetingConfiguration Identity "Global" InMemory The preceding command creates an in-memory-only instance of the global meeting configuration settings, an instance that uses the default value for each and every property. No actual meeting configuration settings are created when you run the command; however, all the default property values will be displayed on-screen. Edge Servers and their settings. Configuration information for your Edge Servers can be viewed by using the Lync Server Control. In Control Panel, click Topology, click Status, and then double-click the name of one of your Edge Servers. That will bring up a detailed view of the Lync Server services running on the selected computer:
From there, double click the EdgeServer service to view configuration information for the selected Edge Server: Operations Management and Monitoring of a Lync Server 2010 Environment
104
Edge Server information can also be retrieved by using Windows PowerShell. This command returns information about all the Edge Servers configured for use in your organization: Get-CsService EdgeServer The returned information includes all of the FQDN and port settings for each Edge Server: Identity : EdgeServer: dc.fabrikam.com Registrar : Registrar: LYNC-SE.fabrikam.com AccessEdgeInternalSipPort : 5061 AccessEdgeExternalSipPort : 5061 AccessEdgeClientPort : 443 DataPsomServerPort : 8057 DataPsomClientPort : 444 MediaRelayAuthEdgePort : 5062 MediaRelayAuthInternalTurnTcpPort : 443 MediaRelayAuthExternalTurnTcpPort : 445 MediaRelayAuthInternalTurnUdpPort : 3478 MediaRelayAuthExternalTurnUdpPort : 3478 MediaCommunicationPortStart : 50000 MediaComunicationPortCount : 10000 AccessEdgeExternalFqdn : dc.fabrikam.com DataEdgeExternalFqdn : dc.fabrikam.com AVEdgeExternalFqdn : InternalInterfaceFqdn : ExternalMrasFqdn : dc.fabrikam.com DependentServiceList : {Registrar:LYNC-SE.fabrikam.com, ConferencingServer:LYNC-SE.fabrikam Operations Management and Monitoring of a Lync Server 2010 Environment
105
com, MediationServer:LYNC-SE. fabrikam.com} ServiceId : fabrikam.com-EdgeServer-2 SiteId : site:fabrikam.com PoolFqdn : dc.fabrikam.com Version : 5 Role : EdgeServer
Federation settings, including whether it is configured and, if so, the FQDN and port. Federation is enabled and disabled by using the global collection of Access Edge configuration settings; among other things, these means that federation is configured on an all-or-nothing basis: either federation is enabled for the entire organization or federation is disabled for the entire organization. In Lync Server Control Panel, you can view by federation settings by clicking External User Access, and then clicking Access Edge Configuration:
If there is a check mark beside Enable federation that means that federation has been enabled for your organization.
Your Access Edge configuration settings can also be returned by using Windows PowerShell. To do that, run the following Windows PowerShell command: Get-CsAccessEdgeConfiguration In turn, that command will return data similar to this: Identity : Global AllowAnonymousUsers : False AllowFederatedUsers : False Operations Management and Monitoring of a Lync Server 2010 Environment
If the AllowFederatedUsers property is set to True, that means that federation has been enabled for your organization. (Setting AllowFederatedUsers to True also means that, in a split domain scenario, your on-premises users will be able to communicate seamlessly with your in-the-cloud users.)
To retrieve the FQDN and port settings for your Edge Server, see the preceding task (Edge Servers and their settings).
Enabling federation at the global scope only means that users can potentially communicate with federated users. To determine whether any individual users can actually communicate with federated users requires you to examine the external user access policy assigned to that user. External user access policies can be view in the Control Panel by clicking External Access, and then clicking External User Access:
If there is a check mark under the heading Federated user access, then users managed by that policy can communicate with federated users.
External user access information can also be returned by using Windows PowerShell. For example, this command returns information for the global external user access policy: Operations Management and Monitoring of a Lync Server 2010 Environment
107
Get-CsExternalAccessPolicy Identity "Global" And this command returns information for all your external user access policies: Get-CsExternalAccessPolicy The returned information will look similar to this: Identity : False Description : EnableFederationAccess : False EnablePublicCloudAccess : False EnablePublicCloudAccessAudioVideoAccess : False EnableOutsideAccess : False If EnableFederationAccess is set to True, then users managed by the given policy are allowed to communicate with federated users. Archiving settings for internal and federated communications. Before verifying the settings for internal and external archiving, you should make sure that archiving has been enabled in the first place. You can determine whether or not archiving has been enabled by opening the Lync Server Control Panel, clicking Monitoring and Archiving, and then clicking Archiving Configuration:
If Archiving is set to Disable archiving, that means that nothing is currently being archived. To enable archiving, select the desired collection of archiving configuration settings, click Edit, and then set archiving to either Archive IM sessions or Archive IM and web conferencing sessions: Operations Management and Monitoring of a Lync Server 2010 Environment
108
Archiving configuration settings can also be verified by using Windows PowerShell and the Get- CsArchivingConfiguration cmdlet: Get-CsArchivingConfiguration Identity "Global" Note that archiving settings can also be configured at the site scope. To return information about all your archiving settings, use this command: Get-CsArchivingConfiguration The Get-CsArchivingConfiguration cmdlet returns data similar to this: Identity : Global EnableArchiving : False EnablePurging : False PurgeExportedArchivesOnly : False BlockOnArchiveFailure : False KeepArchivingDataForDays : 14 PurgeHourOfDay : 2 ArchiveDuplicateMessages : True CachePurgingInterval : 24 If the EnableArchiving property is set to False, that means that no communication sessions will be archived. If you want to archive instant messaging sessions only, use a command like the following to enable the archiving of IM sessions: Set-CsArchivingConfiguration Identity "Global" EnableArchiving "IMOnly" To archive conferencing sessions as well as instant messaging sessions, use this command: Set-CsArchivingConfiguration Identity "Global" EnableArchiving "IMOnly" If youd like to compare your current archiving settings with the default settings, run the following Windows PowerShell command: New-CsArchivingConfiguration Identity Global" -InMemory That command creates an in-memory-only instance of the global archiving configuration settings. This is not a real collection of settings that gets used by Lync Server; however, it does display the default values for all the archiving configuration properties.
You can also use this command to return the FQDN of your Archiving servers: Get-CsService -ArchivingServer After you have verified that archiving has been enabled, you can then view your archiving policies to determine whether internal and/or external communication sessions are being archived. In Operations Management and Monitoring of a Lync Server 2010 Environment
109
Lync Server Control Panel, you can view archiving policies by clicking Monitoring and Archiving, and then clicking Archiving Policy:
If there is a check under the heading Archive internal, that means that the communication session involving internal users (authenticated users with accounts in your Active Directory directory service) will be archived. If there is a check under the heading Archive external, that means communications that include at least one external user will be archived. An external user is an unauthenticated user who does not have an account in your Active Directory directory service.
Archiving policy information can also be retrieved by using the Get-CsArchivingPolicy cmdlet. For example, this command returns information about the global archiving policy: Get-CsArchivingPolicy Identity "Global" Because archiving policies can also be configured at the site and the per-user scope, you might also want to use this command, which returns information about all your archiving policies: Get-CsArchivingPolicy The information you get back from Get-CsArchivingPolicy will look similar to this: Identity : Global Description : ArchiveInternal : False ArchiveExternal : False
Note that, by default, both internal and external archiving are disabled in an archiving policy.
Call Detail Record (CDR) settings for peer-to-peer, conferencing, and Voice call detail recording. Operations Management and Monitoring of a Lync Server 2010 Environment
110
Both your CDR and QoE settings can be verified by using the Lync Server Control Panel. In Control Panel, click Monitoring and Archiving, and then click Call Detail Recording to review your CDR settings:
If there is a check mark under CDR, it means that Call Detail Recording has been enabled.
To check the settings for QoE, click Quality of Experience Data. Operations Management and Monitoring of a Lync Server 2010 Environment
111
Again, a check mark under the QoE heading indicates that QoE monitoring has been enabled.
Detailed information about your CDR settings can also be returned by using the Get- CsCdrConfiguration cmdlet. For example, this command returns information about the global collection of CDR configuration settings: Get-CsCdrConfiguration Identity "Global" Because CDR can also be configured at the site scope, you might also want to run this command, which returns information about all your CDR configuration settings: Get-CsCdrConfiguration The Get-CsCdrConfiguration cmdlet returns information similar to this for each collection of CDR configuration settings: Identity : Global EnableCDR : True EnablePurging : True KeepCallDetailForDays : 60 KeepErrorReportForDays : 60 PurgeHourOfDay : 2 Similar information can be returned for QoE monitoring by using the Get-CsQoEConfiguration cmdlet. For example, this command returns information about the global collection of QoE configuration settings: Get-QoEConfiguration Identity "Global" That information will look similar to this: Identity : Global ExternalConsumerIssuedCertId : Operations Management and Monitoring of a Lync Server 2010 Environment
112
EnablePurging : True KeepQoEDataForDays : 60 PurgeHourOfDay : 1 EnableExternalConsumer : False ExternalConsumerName : ExternalConsumerURL : EnableQoE : True If you want to compare your current CDR settings with the default CDR settings, the default values can be reviewed by running this command: New-CsCdrConfiguration Identity "Global" InMemory Likewise, the default values for QoE monitoring can be retrieved by using this command: New-CsQoEConfiguration Identity "Global" InMemory You can also return the FQDN of your Monitoring servers by running this command: Get-CsService -MonitoringServer
Voice settings. The voice settings typically of interest to administrators are contained in voice policies and voice routes: voice policies contain the settings that determine the capabilities exposed to individual users (such as the ability to forward or transfer calls), while voice routes determine how (and if) calls are routed across the PSTN.
Voice policy information can be viewed in Lync Server Control Panel by clicking Voice Routing, clicking Voice Policy, and then double-clicking the voice policy of interest:
Operations Management and Monitoring of a Lync Server 2010 Environment
113
Voice policy information can also be retrieved by using Windows PowerShell. For example, this command returns information about the global voice policy: Get-CsVoicePolicy Identity "Global" And this command returns information about all the voice policies configured for use in the organization: Get-CsVoicePolicy The information returned by the Get-CsVoicePolicy cmdlet looks similar to the following: Identity : Global PstnUsages : {} Description : AllowSimulRing : True AllowCallForwarding : True AllowPSTNReRouting : True Name : DefaultPolicy EnableDelegation : True EnableTeamCall : True EnableCallTransfer : True EnableCallPark : False EnableMaliciousCallTracing : False EnableBWPolicyOverride : False PreventPSTNTollBypass : False Keep in mind that you can also create queries that returned a subset of your voice policies. For example, this command returns all the voice policies that allow call forwarding: Get-CsVoicePolicy | Where-Object {$_.AllowCallForwarding eq $True} And this command returns all the voice policies that dont allow call forwarding: Get-CsVoicePolicy | Where-Object {$_.AllowCallForwarding eq $False} Voice route information can be viewed in Control Panel by clicking Voice Routing, and then clicking Route: Operations Management and Monitoring of a Lync Server 2010 Environment
114
In Windows PowerShell, use the Get-CsVoiceRouting cmdlet to return information about your voice routes: Get-CsVoiceRoute That command returns information like this for all your voice routes: Identity : LocalRoute Priority : 0 Description : NumberPattern : ^(\+1[0-9]{10})$ PstnUsages : {} PstnGatewayList : {} Name : LocalRoute SuppressCallerId : AlternateCallerId : Lync Server allows you to create voice routes that do not have a PSTN usage and/or do not specify a PSTN gateway; however, you cannot actually route calls over a voice route that does not have these two property values configured. Because of that, you might find it useful to periodically run this command, which returns the identity of any voice route that does not have a PSTN usage: Get-CsVoiceRoute | Where-Object {$_.PstnUsages eq $Null} | Select- Object Identity Similarly, this command returns the identity of any voice route that has not been configured with a PSTN gateway: Get-CsVoiceRoute | Where-Object {$_.PstnGatewayList eq $Null}} | Select-Object Identity
Operations Management and Monitoring of a Lync Server 2010 Environment
115
Conferencing Attendant settings for PSTN dial-in conferencing. Conferencing Attendant settings can only be retrieved by using the Get- CsDialInConferencingConfiguration cmdlet; these settings are not available in the Lync Server Control Panel. To view your Conferencing Attendant settings, use a Windows PowerShell command similar to the following, which returns the global collection of Conferencing Attendant settings: Get-CsDialInConferencingConfiguration Identity "Global" Note that Conferencing Attendant settings can also be configured at the site scope. To return information about all your Conferencing Attendant settings, use this command instead: Get-CsDialInConferencingConfiguration The Get-CsDialInConferencingConfiguration cmdlet returns data similar to this: Identity : Global EntryExitAnnouncementsType : UseNames EnableNameRecording : True EntryExitAnnouncementsEnabledByDefault : False If EntryExitAnnouncementsEnabledByDefault is set to False, that means the conferencing announcements are disabled. To enable entry and exit announcements, run a Windows PowerShell command similar to this: Set-CsDialInConferencingConfiguration Identity "Global" EntryExitAnnouncementsEnabledByDefault $True
4.5.2 Viewing Edge Server Settings General Edge Server configurations should be reviewed against the data in the configuration management databaseto help ensure that all changes were documented as per the defined change control procedures. Additional checks could include: 1. Verifying the SIP URI "Allow" and "Block" list for Federated domainsto determine whether listed namespaces are still valid. You can use Lync Server Control Panel to view the domains currently found on both your Allowed Domains and your Blocked Domains lists. To view this information by using Lync Server Control Panel, click External User Access, and then click Federated Domains: Operations Management and Monitoring of a Lync Server 2010 Environment
116
You can also use Windows PowerShell to view the allowed and blocked lists. To review the domains on the Allowed Domains list, run the following Windows PowerShell command Get-CsAllowedDomain That command returns information similar to this for each of the domains on the Allowed Domains list: Identity : contoso.com Domain : contoso.com ProxyFqdn : Comment : MarkForMonitoring : False Comment : To review the domains on the blocked domains list, use this command: Get-CsBlockedDomain In turn, you will get back information like this for each blocked domain: Identity : tailspintoys.com Domain : tailspintoys.com Windows PowerShell also enables you to verify that you can connection to the domains on your Allowed Domains list. For example, this command verifies the connection between your Edge Server (the TargetFqdn) and the federated domain contoso.com: Test-CsFederatedPartner TargetFqdn "atl-edge-001.litwareinc.com" Domain "contoso.com" And this command verifies the connection between your Edge Server and all of the domains found on your Allowed Domains list: Operations Management and Monitoring of a Lync Server 2010 Environment
117
Get-CsAllowedDomain | ForEach-Object {Test-CsFederatedPartner TargetFqdn "atl-edge-001.litwareinc.com" Domain $_.Domain} 2. If multiple Edge Servers are deployed in a load balanced array, we recommend verifying that all Edge Servers in the array are configured in the same way. You can view settings for Edge Servers in the details pane of the Lync Server 2010 extension for the Computer Management snap-in. 4.5.3 Check Lync Server 2010 Server Certificates Run the CsCertificateConfiguration cmdlets to retrieve the certificate information against each Lync Server, which requires a certificate. By default, privately issued certificates expire after 12 months. We recommend recording all server certificate expiration dates in a central document repository, such as a SharePoint task list, with the ability to send alerts to responsible IT administrators when expiration dates approach. This repository should also be used to manage public certificates that are installed on the Lync Server 2010 Edge Server infrastructureto avoid service interruption when public certs expire. Lync Server 2010 will write an event to the Windows Event log when a certificate is about to expire. Administrators can use standard tools such as Windows Event log to search for these type of events or if System Center Operations Manager 2007 is implemented then System Center Operations Manager includes a rule that will alert administrators of the certificate expiry. 4.5.4 Check Trunk Configuration against a Phone Number You can use this synthetic transaction cmdlets (CsTrunkConfiguration) to verify that a trunking configuration performs as expected against a dialed phone number. Each configuration contains specific settings defining the relationship and capabilities between the Mediation Server and the PSTN gateway, IP-PBX or Session Border Controller (SBC) at the service provider. These settings configure such things as whether media bypass is enabled on this trunk, whether real-time transport control protocol (RTCP) packets are sent under certain conditions, and whether to require secure real-time protocol (SRTP) encryption. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsTrunkConfiguration "}
4.5.5 Check voice normalization rule You can use the CsVoiceNormalizationRule cmdlet to see the results of applying a voice normalization rule to a given telephone number. Voice normalization rules are a required part of phone authorization and call routing. They define the requirements for convertingor translatingnumbers from a format typically entered by users to a standard (E.164) format. Use this cmdlet to troubleshoot dialing issues or to verify that rules will work as expected against given numbers. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsVoiceNormalizationRule"} Operations Management and Monitoring of a Lync Server 2010 Environment
118
4.5.6 Test Telephone Number against a Voice Policy Voice policies are tied to voice routes through PSTN usages. A call from a user who has been assigned a particular voice policy can only be sent through a route that has a PSTN usage matching a usage on the policy and also a number pattern that matches the number dialed. Call the Test-CsVoicePolicy cmdlet to determine which (if any) route will be used to route a call from a user with a particular voice policy, and also what phone usage ties the policy to the route. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsVoicePolicy "} 4.5.7 Test telephone Number Against a Voice Route A voice route includes a regular expression that identifies which phone numbers will be routed through a given voice route: phone numbers matching the regular expression will be routed through this route. This cmdlet (CsVoiceRoute) verifies whether or not a given telephone number will be routed through a specified voice route based on the number pattern of the route (the NumberPattern property). You can use this cmdlet to troubleshoot routing issues or to simply try out phone numbers against specific routes to help ensure that the results are what you expect. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsVoiceRoute "} 4.5.8 Test Voice Configuration This cmdlet (CsVoiceTestConfiguration) tests a phone number against a voice route, usage, dial plan, and voice policy to enable you to verify intended outcomes or to compare the actual outcome to the expected outcome. The voice configurations to be tested can be defined by entering the appropriate parameters individually, or by using the New-CsVoiceTestConfiguration cmdlet. If you enter values for DialedNumber, DialPlan, and VoicePolicy, the output will include the translated number, the normalization rule used to create that translation, the route that was used, and the PSTN usage. If instead you enter a value for the TestCaseInputObject parameter, you can also retrieve status on whether the results matched expected results that you supplied to the test object when you created it with the New-CsVoiceTestConfiguration cmdlet. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsVoiceTestConfiguration "} 4.5.9 Test Voice Rules, Routes, and Policies When a user makes a phone call, the route the call takes to reach its destination depends on the policies and dial plans assigned to that user. Given a users SIP address and a phone number, this cmdlet (CsVoiceUser) returns the number translated to E.164 format (based on the users dial plan), the normalization rule that supplied that translation, the first route (based on route Priority) with a number pattern matching that phone number, and the phone usage that would link the voice policy of that user to the voice route. This cmdlet can be used to determine whether a specific phone number will route and translate as expect based on user settings, and can help troubleshoot issues experienced by individual users. To see a list of all RBAC roles that can run this cmdlet, run command Get-CsAdminRole | Where-Object {$_.Cmdlets match "Test- CsVoiceUser"} Operations Management and Monitoring of a Lync Server 2010 Environment
119
4.5.10 Security Checks Perform regular audits of security, including firewall rules, user rights, group membership, delegate rights, and so on. 4.5.11 Capacity Planning Review capacity figures for the previous month, and produce a plan for any upgrades that may be required in the coming months to keep the system operating within limits specified by your organization's SLAs. 4.5.12 Disaster Recovery Test Perform a system recovery for a Lync Server 2010 pool server to test your documented disaster recovery process. This test will simulate a complete hardware failure for one server, and will help to ensure that the resources, plans, and data are available for recovery. Try to rotate the focus of the test each month, so that your organization tests the failure of a different server or other piece of equipment every time. Note that the schedule by which organizations perform Disaster Recovery testing will vary. It is, however, critical that disaster recovery testing not ignored or neglected.
Exports your Lync Server 2010 topology, policies, and configuration settings to a file. Among other things, this file can then be used to restore this information to the Central Management store after an upgrade, a hardware failure, or some other issue has resulted in data loss. Imports your Lync Server 2010 topology, policies, and configuration settings to either the Central Management store or to the local computer. Import-CsConfiguration -ByteInput <Byte[]> [-Force <SwitchParameter>] [-LocalStore <SwitchParameter>]
Import-CsConfiguration -FileName <String> [-Force <SwitchParameter>] [-LocalStore <SwitchParameter>] Back up production Lync Server 2010 data: Back up the RTC and LCSLog databases by using the standard SQL Server 2005 backup process to dump the database to a file or tape dump device. Use third-party backup application to back up the data to file or to tape. Use the DBIMPEXP.exe utility to create an XML export of the entire RTC database. Use the file system backup or third-party to back up meeting content and compliance logs. Use the Export-CsConfiguration command-line tool to back up Lync Server 2010 settings. The first step in the failover procedure includes a forced move of users from the production pool to the Disaster Recovery pool. This will be a forced move since the production pool will not be available to accept the user relocation. Lync Server 2010 move user process is effectively a change to an attribute on the user account object as well as a record update on the RTC SQL database. Operations Management and Monitoring of a Lync Server 2010 Environment
120
This data can be restored through the following two processes: RTC database can be restored from the original backup dump device from the production SQL Server using the standard SQL 2005 Server restore process, or using a third-party backup/restore utility. User contact data can be restored with the DBIMPEXP.exe utility using the XML file created from the production SQL Server export. After this data is restored, users can effectively connect to the Disaster Recovery Lync Server 2010 pool, and operate normally. To enable users to connect to the Disaster Recovery Lync Server 2010 pool, a DNS record change will be required. The production Lync Server 2010 pool will be referenced by clients using the auto-configuration and DNS SRV records of: SRV: _sip._tls.<domain> /CNAME: SIP.<domain> CNAME: SIP.<domain> /cvc-pool-1.<domain> To facilitate the failover, this CNAME record must be updated to reference the DROCSPool FQDN CNAME: SIP.<domain> /DROCSPool.<domain> Sip.<domain> AV.<domain> webconf.<domain> OCSServices.<domain>
Important For detailed administration and management procedures, see the Microsoft Lync Server 2010 Administration Guide. 4.6 As-Needed Tasks Perform the following tasks as necessary; however, they are frequently also covered by standard procedures: Full Security Auditing You can perform this audit regularly, in response to an upgrade or redesign of the messaging system, or in response to an attempted (or successful) security breach. The procedure may involve port scans on servers and firewalls, audits of security fixes, and third-party penetration tests. Replace Certificates about to Expire Checking Lync Server Certificates is one of regular weekly tasks, and as part of the procedure an administrator should have a record of all certificates expiry dates. This record enables an administrator to create an alert/notification when a particular certificate is about to be expired and replaced as needed. Updating Performance Baselines Update performance baselines after an upgrade or configuration change. Your organization can use baselines to measure performance changes and to detect issues that affect system performance. Managing Enterprise Pool Initial configuration of Enterprise pools, Standard Edition servers, and any other servers in your organization's environment should have been accomplished during Operations Management and Monitoring of a Lync Server 2010 Environment
121
deployment of the individual servers. Post-deployment management of servers and pools for Standard Edition servers and Enterprise pools includes the following tasks: o Managing Front End Servers o Managing Web Conferencing o Managing Conferencing o Changing Service Account Credentials o Managing Databases o Starting and Stopping Services and Deactivating Server Roles o Removing Servers and Server Roles, Removing Pools, and Decommissioning Servers and Pools Managing Usage You can configure Lync Server 2010 to provide the features and functionality that are most appropriate for your organization. This includes the following: o Managing Support for On-Premise Web Conferencing Meetings o Managing the Use of Distribution Groups to Send Instant Messages o Managing Contacts, Presence, and Queries o Configuring Client Version Filtering o Configuring Intelligent IM Filtering o Configuring Archiving, Call Detail Recording, and Meeting Compliance Managing Edge Server Connectivity Ongoing management of the servers and settings required to provide external connectivity includes the following: o Managing Connectivity between Internal Servers and Edge Servers o Configuring Internal and External Interfaces and Certificates for Edge Servers o Managing Federated Partner Access Administering the Address Book Administering Address Book Servers includes the following: o Configuring Address Book Server phone normalization o Managing the Address Book Server from the command line Managing User Accounts Management of user accounts includes the following: o Enabling User Accounts for Lync Server o Configuring Lync Server Users using the Wizard o Configuring Individual Lync Server User Account Properties o Searching for Lync Server Users o Moving Lync Server Users o Deleting Lync Server Users Analyzing Lync Server 2010 Log Files One very helpful tool, generally used for troubleshooting purposes, is the Lync Server 2010 Logging Tool described in detail in Using Lync Server 2010 Logging Tool. Because the Logging Tool generates log files (on a per-server basis), these log files can be viewed and analyzed by using the Snooper tool, if the Microsoft Office Server 2007 R2 Resource Kit Tools is installed Operations Management and Monitoring of a Lync Server 2010 Environment
122
on the computer. Otherwise, logs can also be analyzed by using a text editor, which is substantially less transparent and more complex than using the Snooper utility. To View and Analyze Protocol Messages In the Logging Tool, when you have ended the debug session, click Analyze Log Files to view the log files by using the Snooper tool. You can analyze protocol logs for the following components: Lync Server SipStack (SIP) Lync Server S4 (SIP) Lync Server Conferencing signaling traffic (C3P), including MCU Infra C3P and Focus C3P Lync Server Web conferencing traffic (PSOM) Lync Server Unified Communications Client Platform client (UCCP) Error reports from the archiving database To help organize the performance of as-needed tasks, see As-Needed Operations Checklist. Important For detailed administration and management procedures, see the Microsoft Lync Server 2010 Administration Guide. 4.6.1 Backup (and Restore) Policies or Configuration Settings Lync Server 2010 makes it possible for you to back up and restore the entire system. However, if you would like to back up (and then maybe someday restore) a single policy or a single collection of configuration settings, retrieve the appropriate policy, and then pipe that object to the Export-Clixml cmdlet, which saves the policy information as an XML file: Get-CsClientPolicy -Identity "RedmondClientPolicy" | Export-Clixml -Path C:\Backup\RedmondClientPolicy.xml You may now experiment with RedmondClientPolicy and change a bunch of the settings. If you decide instead to simply restore the old policy, enter: $x = Import-Clixml -Path C:\Backup\RedmondClientPolicy.xml Set-CsClientPolicy Instance $x Note that this simple little approach will work for most policies and settings; however, it won't work with some of the more complicated itemsitems that contain multiple sub-objects (like routing configuration settings, which contain a bunch of separate voice routes). But we'll see if we can find an easy way to handle those more complicated objects as well. 5 Operations Checklists The Lync Server 2010 operations checklists provide guidelines for the your Lync Server team to perform the daily, weekly, and monthly maintenance tasks required to keep the Lync Server 2010 servers performing optimally. The following checklists are adapted to suit your specific needs: Daily Operations Checklist Weekly Operations Checklist Monthly Operations Checklist
Operations Management and Monitoring of a Lync Server 2010 Environment
123
5.1 Daily Operations Checklist Complete these checklists to record daily operations. Task Completed Environmental Verify that environmental conditions are tracked and maintained. Note any known variations in temperature out of the desired range.
Verify that physical security measures such as locks, dongles, and access codes have not been breached, and that they function correctly.
Make sure that the physical network and related hardware such as routers, switches, hubs, physical cables, and connectors are operational.
Backups Make sure that the recommended daily online data backup is completed. Make sure that the recommended daily online configuration file backup is completed. Backup the Lync Server topology to an XML file using Lync Server Topology Builder. Backup all data shares housing content and metadata. Backup all SQL back-end databases. Follow the established procedure for long-term storage rotation, labeling, and archiving. Verify that the transaction logs were successfully purged (if the backup type purges). Make sure that backups meet SLA standards. Disk Usage Create a list of all drives and label them in three categories: Drives with transaction logs Drives with queues Other drives
Operations Management and Monitoring of a Lync Server 2010 Environment
124
Check disks with transaction log files.
Check other disks.
Use server monitors to check free disk space.
Check performance on disks.
Drive Letter Designation (drives with transaction logs, drives with queues, and other drives) Available space (MB) Available % free <data here> <data here> <data here> Drive Letter Designation (drives with transaction logs, drives with queues, and other drives) Available space (MB) Available % free <data here> <data here> <data here> Drive Letter Designation (drives with transaction logs, drives with queues, and other drives) Available space (MB) Available % free <data here> <data here> <data here>
Operations Management and Monitoring of a Lync Server 2010 Environment
125
Event Logs Filter application and system logs on the Lync Server 2010 server to see all errors. Filter application and system logs on the Lync Server 2010 server to see all warnings. Note repetitive warnings and errors in the logs. Respond to discovered failures and issues. Use Analyze Log Files to view the log files In the Lync Server 2010 Logging Tool.
Lync Server Performance
Results of Processor\% Processor Time: __________________________
Results of the following SQL Server performance counters:
Results of: LC:SIP - 07 - Load Management\SIP - 000 - Average Holding Time For Incoming Messages __________________
Results of the SQL Back End Server counter SQL Server Buffer Manager\Page life expectancy __________________
Results of the front-end server counters: LC:USrv-00-DBStore\Usrv-002-Queue Latency (msec) _________________________ LC:USrv-00-DBStore\Usrv-004-Sproc Latency (msec) _________________________
Was further investigation needed? Y / N
Operating System Use Windows Reliability and Performance Monitor to check Windows Server 2008 or 2012 operating system.
Use Performance Console Tools (System Monitor, Performance Logs and Alerts and Task Manager) to check Windows Server 2003/2003 R2 operating systems.
Operations Management and Monitoring of a Lync Server 2010 Environment
126
Central Management Store Replication Status Verify that the Get-CsManagementStoreReplicationStatus cmdlet indicates that the replication status for the servers is up to date.
Network Performance Consult with network administrators to determine whether there are or were network outages such as WAN link failures, equipment failure, recently identified performance bottlenecks, or scheduled outages due to hardware or software updates.
Consult the network administrators to obtain network usage reports for SIP, A/V, and Web Conferencing traffic.
Use System Center Operations Manager to view the network performance of individual Lync Server 2010 servers such as servers hosting MCUs to determine whether the network adaptor is a performance bottleneck.
Use Network Monitor to capture network statistics where required. Viruses and Virus Definitions Scan for viruses and check virus definition updates. When scanning for viruses, exclude drives that are specifically for Lync Server 2010.
Monitoring Server Reports Verify that following reports are viewed and checked: QoE Summary/Trend Reports o UC-to-UC Summary/Trend Report o PSTN Summary/Trend Report o Conference Summary/Trend Report QoE Performance Reports o Mediation Server Performance Report o A/V Conferencing Server Performance Report o Location based Performance Report
Voice Number Normalization and Routing Use Lync Server Control Panel to run voice routing tests. Address Book Verify a user can access the server that hosts the Address Book Download Web service by running the Test-CsAddressBookService cmdlet.
Operations Management and Monitoring of a Lync Server 2010 Environment
127
Verify a user can search for, and return, information from the Address Book by running the Test-CsAddressBookWebQuery cmdlet.
Pool Status Using the Lync Control panel, check status of all servers in the Topology. Lync Server 2010 Back End Storage Checks Performance Counter Normal Operating Range Value Transactions/sec (RTC)
Transactions/sec (rtcdyn)
Transactions/sec (tempdb)
Log Flushes/sec (RTC)
Log Flushes/sec (rtcdyn)
Log Flushes/sec (tempdb)
Disk Transfers/sec (read+write) - RTC db
Disk Transfers/sec - RTC log
Disk Transfers/sec - rtcdyn db
Disk Transfers/sec - rtcdyn log
Group Chat Use the stress test table metrics to verify your Group Chat Servers are running at optimal health.
DNS Settings/ DNS Load Balancing Verify that DNS is returning the correct values for DNS load balancing.
Synthetic Transactions Operations Management and Monitoring of a Lync Server 2010 Environment
128
Validate Audio Conferences by running Test-CsAVConference Validate Lync client Authentication by running Test-CsClientAuth Validate Lync Server Services by running Test-CsComputer Validate ability to connect to a federated domain by running Test-CsFederatedPartner Validate Ability to employ group expansion by running Test-CsGroupExapnsion Validate ability to do group instant messaging by running Test-CsGroupIM Validate instant messaging between two users by running Test-CsIM Validate configuration of Kerberos Account assigned to a site by running Test- CsKerberosAccountAssignment
Validate peer to peer audio/video call by running Test-CsP2PAV Validate user logon ability by running Test-CsRegistration Validate user presence publishing and subscribing by running Test-CsPresence Validate Lync service activation and group permissions by running Test-CsTopology Validate Dial-In-Conferencing session by running Test-CsDialInConferencing Validate Dial Plan by running Test-CsDialPlan Validate Civic Address against master street address by running Test-CsLisCivicAddress Validate your ability to contact the Location Information Server (LIS) web service by running Test-CsLisConfguration
Validate that a location policy has been assigned to a user by running Test- CsLocationPolicy
Validate Lync Phone Edition Login by running Test-CsPhoneBootstrap Validate PSTN phone call by running Test-CsPstnOutboundCall Operations Management and Monitoring of a Lync Server 2010 Environment
129
Validate PSTN peer to peer call by running Test-CsPstnPeerToPeerCall
Operations Management and Monitoring of a Lync Server 2010 Environment
130
5.2 Weekly Operations Checklist Complete these checklists to record weekly operations. You can modify these checklists based on the organization's requirements. Prepared by: Date:
Task Completed Archive Event Logs
Archive logs per your organizations policies.
Create Reports
Use daily data from event log and System Monitor to create reports.
Report on disk usage. Create reports on memory and CPU usage. Using System Center Operations Manager Generate uptime and availability reports. Incident Reports List the top generated, resolved, and pending incidents. Create solutions for unresolved incidents. Update reports to include new trouble tickets. Create a document depository for troubleshooting guides and post mortems about outages.
IIS Logs Examine event log and filter. IIS logs provide information about the changes. Examine System Monitor for IIS performance to examine the output of performance counters: Web Service counters to monitor the World Wide Web Publishing Service (WWW service).
Operations Management and Monitoring of a Lync Server 2010 Environment
131
Web Service Cache counters to monitor the WWW service cache. FTP Service counters to monitor the File Transfer Protocol (FTP) service. Active Server Pages (ASPs) counters to monitor applications that run as ASPs. Database Reports Generate and review the Database reports, and remediate as needed.
Updates Maintain a list of applied hot fixes, service packs, update rollups, and security updates. Determine if there are new hot fixes for Windows Server. Determine if there are service packs for Windows Server. Determine if there are updates to complementary services such as IIS, AD DS, and DNS server.
Apply updates uniformly across servers and workstations in the organization. Perform critical security updates as soon as possible, based on organization policy. Best Practice Analyzer Make sure that the RTCBPA.msi is installed on all Lync Server 2010 servers, and generate a weekly Health Check Report. Note the results and remediate, if required.
SCOM QoE reports Obtain and review Lync Server 2010 MP and QoE reports. Bandwidth Utilization Reports Obtain and review the reports generated by running the Bandwidth Utilization Analyzer tool.
Status Meeting Agenda Hold a staff meeting to review the following areas:
Server and network status for the overall organization and segments.
Operations Management and Monitoring of a Lync Server 2010 Environment
132
Organizational performance and availability. Overview reports and incidents. Risk analysis and evaluation including upcoming changes. Capacity, availability, and performance reviews. SLA performance, and review items that have not met target objectives.
5.3 Monthly Operations Checklist Use these checklists to record monthly operations. You can modify these checklists based on your own requirements. Prepared by: Date:
Task Completed Global Settings
Review global settings.
Edge Server settings
Review Edge Server settings.
Certificate Status
Use the synthetic transaction CscertificateConfiguration to retrieve and verify certificate information on each Lync Server that requires a certificate.
Verify Trunk Configuration against a Phone Number
Use the synthetic transaction CsTrunkConfiguration to verify that a trunking configuration performs as expected against a dialed phone number
Verify Normalization Rules
Use the synthetic transaction CsVoiceNormalizationRule to see the results of applying a voice normalization rule to a given telephone number.
Verify Telephone Number against a Voice Policy
Operations Management and Monitoring of a Lync Server 2010 Environment
133
Use the synthetic transaction CsVoicePolicy to determine which (if any) route will be used to route a call from a user with a particular voice policy, and also what phone usage ties the policy to the route. The results should match your architectural expectations.
Verify Telephone Number against a Voice Route
Use the synthetic transaction CsVoiceRoute to determine whether or not a given telephone number will be routed through a specified voice route based on the number pattern of the route (the NumberPattern property) The results should match your architectural expectations.
Verify Voice Configuration
Use the synthetic transaction CsVoiceTestConfiguration to test a phone number against a voice route, usage, dial plan, and voice policy and enable you to verify intended outcomes or to compare the actual outcome to the expected outcome.
Verify Voice Rules, Routes and Policies
Use the synthetic transaction CsVoiceUser to determine whether a specific phone number will route and translate as expected based on user settings.
Security checks
Audit the following security considerations: firewall rules user rights group membership delegate right Other access privileges your organization deems necessary
Capacity Planning
Check capacity and performance against SLA requirements.
Review SLA requirements and capacity figures from previous month.
Produce and implement upgrade path based on projected growth from previous growth data.
Disaster Recovery Testing
Verify whether the Disaster Plan succeeds or fails.
Operations Management and Monitoring of a Lync Server 2010 Environment
134
Operations Management and Monitoring of a Lync Server 2010 Environment
135
6 Monitoring Lync Server 2010 with System Center Operations Manager 6.1 Introduction to the Microsoft Lync Server Management Pack The Microsoft Lync Server Management Pack (MP) is your monitoring solution of choice for monitoring any Lync Server deployment. The MP implements traditional Event Log and Performance counter-based instrumentation is utilized as well as enabling newly available instrumentation in Lync Server, such as pair events (failure/success) for several Key Health Indicators as well as fully utilizing the new Synthetic Transactions (Test-Cs* Windows PowerShell cmdlets). 6.1.1 Getting the Latest Management Pack and Documentation You can find the Lync Server 2010 Management Pack in the System Center Operations Manager 2007 Catalog (http://go.microsoft.com/fwlink/?LinkId=82105). This is recommended if you are running System Center Operations Manager 2007. You can find the Lync Server 2013 Management Pack at http://www.microsoft.com/en- us/download/confirmation.aspx?id=35842. This is recommended if you are running System Center Operations Manager 2012. Lync Server 2010 can be managed from either SCOM release, given the appropriate management packs. 7 Operational Dependencies The Reference Architecture discussed in this document will help to ensure that you have a Lync Server 2010 deployment that not only scales to the organizations requirements but is architected as per Microsofts best practices. Be that as it may the Lync Server 2010 implementation is a dynamic service and like any other service in the enterprise still requires adequate monitoring and proactive management in order to maintain high level of service availability and service quality to the business. As Lync Server 2010 becomes deeply ingrained in the organizations everyday business it is important that the service be managed by accurate and tangible service level management. The Lync system the architecture can become complex and very integrated and in order to maintain effective service level management and establish SLAs for Lync Server 2010 it becomes critical to understand the systems dependencies on other platforms and servers. Equally important is to note which business services, such as voice and UC integrated applications, become dependent on Lync. At very least a service map for Lync Server 2010 must be establish noting all the said dependencies. The service map will allow you to formulate a SLA between Lync and its dependent service and provide a starting place for SLA negotiation. The following table lists the typical dependency services for a Lync infrastructure. Each of these technologies should have its own proactive monitoring. SLAs must be established between Lync and the following support silos: Dependency Service Dependency Service Operations Management and Monitoring of a Lync Server 2010 Environment
136
Operating systems Server Hardware Active Directory Public Key Infrastructure Domain Naming Service Database Services Storage Services System Management Monitoring and distribution
It is assumed that the organization is operationally mature in exercising basic service level management functions such as change, incident and release management as prescribed by the MOF. The Lync solution should be adopted by these functions and become subject to the same operational management processes. Building on the information obtained above we now have a greater understanding of what can impact the Lync service in the enterprise. To help ensure Lync Server 2010 service availability and quality, the following monitoring tools must accompany the reference architecture deployment: Component Description Applicable Site Lync Server 2010 Monitoring Server Deploy at least one Lync Server 2010 Monitoring Server role per Central site and configure Quality of Experience (QoE) Reporting Pack. Refer to the Microsoft Lync Server 2010 Monitoring Deployment Guide for further details: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=8ad1bdf1- c43a-4dd4-af07-52027a4bcaeb Central sites Tether each pool to its nearest instance of the Monitoring Server role. Central sites Branch sites System Center Operations System Center Operations Manager 2007 with the Microsoft Lync Server 2010 Management Pack (MP) imported. Central sites Operations Management and Monitoring of a Lync Server 2010 Environment
137
Component Description Applicable Site Manager 2007 The Management Pack implements traditional Event Log and Performance counter based instrumentation is utilized as well as enabling newly available instrumentation in Lync Server 2010, such as pair events (failure/success) for several Key Health Indicators as well as fully utilizing the new Synthetic Transactions (Test-Cs* Windows PowerShell cmdlets). System Center Operations Manager Lync Management Pack has the following features: Central Discovery; Reduced Alert Noise; Prioritized Alerts with resolution; Stateful Alerting; Transient Handling; Synthetics Transactions; Call Reliability Monitoring; Media Quality Monitoring; Port Monitoring; Simple URL Monitoring. Refer to the MonitoringCS_withSCOM.docx document from UCTAP connect documentation. Make sure that Central Discovery to discovery of roles and components that need to be monitored are automatically completed based on a central discovery script that reads the topology document published in Central Management Database. Central Site Branch Site Edge Site Deploy System Centre Operations Manager 2007 agents to all deployed servers running Lync Server. Central Site Branch Site Edge Site Make Prioritized Alerts are configured for notification: High Priority Alerts Medium Priority Alerts Other Alerts. Central Site Branch Site Edge Site
Configure Port monitoring for your deployment. Central Site Branch Site Edge Site
Configure URL monitoring for your deployment Central Site
Lync and System Center Operations Manager integration Deploy Call Reliability and Media Quality Monitoring Call reliability and media quality monitoring use the monitoring server machine as their watcher node to monitor call reliability and media quality of Microsoft Lync Server. Both of these features query the monitoring server databases to do analysis. Central Site Branch Site
Make sure Media Quality alert thresholds are accurately configured. The Central Site Operations Management and Monitoring of a Lync Server 2010 Environment
138
Component Description Applicable Site following table indicates the maximum Network Mean Opinion Scores by Codec. In production these scores should be monitored for a set period and acceptable thresholds must be established base on the organization specific NMOS scores. Branch Site
Lync Server 2010 Synthetic Transaction Watcher Deploy a dedicated Lync Server to be a synthetic transaction watcher. Synthetic transactions are Lync Server 2010 Windows PowerShell cmdlets that are automatically triggered by the management pack on a preset interval. These are executed on a synthetic transaction watcher node which is an administrator designated server responsible for discovery and execution of STs for each pool. We do not recommend using an existing Microsoft Lync Server 2010 server as a synthetic transaction watcher node. This is due to the high CPU/memory utilization requirements for running STs. Use a new server machine (or a virtual machine) for the synthetic transaction watcher node.
Central Site
Deploying synthetic transactions watcher node. Refer to the MonitoringCS_withSCOM.docx document from UCTAP connect documentation. Central Site
Scenario Codec Max NMOS UC-UC call RTAudio WB 4.10 UC-UC call RTAudio NB 2.95 Conference call Siren 3.72 UC-PSTN call RTAudio NB 2.95 UC-PSTN call G-711 3.61 Maximum Network MOS scores per codec Over and above the above pro-active monitoring activities, maintenance tasks should be executed for Central, Edge and Branch sites on a recurring daily, weekly and monthly basis as defined in the Lync RA Operations Guide.
8 Troubleshooting To meet the Reference Architecture SLAs and to ensure a smooth transition to our support teams, a common troubleshooting approach must be defined along with a requisite set of troubleshooting tools and approaches as defined in the Lync Server Networking Guide at http://go.microsoft.com/fwlink/p/?LinkID=390677.
9 Key Health Indicators (KHIs) We strongly recommend that SCOM be used to monitor the health of the Lync Server 2010 system. Also, refer to the discussion of KHIs in the Lync Server 2013 Networking guide as well as the Excel spreadsheet for use with Lync 2010 provided at http://go.microsoft.com/fwlink/p/?LinkID=390677.