U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

LEGAL INFORMATION
By accepting this certain document of ZTE CORPORATION you agree to the following terms. If you do not agree to the following terms, please notice that you are not allowed to use this document. Copyright 2014 ZTE CORPORATION. Any rights not expressly granted herein are reserved. This document contains proprietary information of ZTE CORPORATION. Any reproduction, transfer, distribution, use or disclosure of this document or any portion of this document, in any form by any means, without the prior written consent of ZTE CORPORATION is prohibited. and are registered trademarks of ZTE CORPORATION. ZTEs company name, logo and product names referenced herein are either trademarks or registered trademarks of ZTE CORPORATION. Other product and company names mentioned herein may be trademarks or trade names of their respective owners. Without the prior written consent of ZTE CORPORATION or the third party owner thereof, anyones access to this document should not be construed a s granting, by implication, estopped or otherwise, any license or right to use any marks appearing in the document. The design of this product complies with requirements of environmental protection and personal security. This product shall be stored, used or discarded in accordance with product manual, relevant contract or laws and regulations in relevant country (countries). This document is provided as is and as available. Information contained in this document is subject to continuous update without further notice due to improvement and update of ZTE CORPORATIONs products and technologies.

ZTE CORPORATION
Address: Address: NO. 55 Hi-tech Road South ShenZhen P.R.China 518057
http://dms.zte.com.cn TSM.Aftersales@zte.com.cn

Website: Email:

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

Revision History
Product Version Document Version R1.0 Serial Number Reason for Revision First published

Author
Date 2012-10-10 Document Version R1.0 Prepared by Chen Yanhao, Cai Wei Reviewed by Chen Taiming Approved by Chen Taiming

Intended audience: RNS commissioning engineer

II

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

About This Document

Summary
Chapter 1 Background Background RCB CPU Load Monitoring Solution & High Load Handling Solution RUB CPU Load Monitoring Solution & High Load Handling Solution Summary 2 RCB CPU Load Monitoring Solution & High Load Handling Solution 3 RUB CPU Load Monitoring Solution & High Load Handling Solution 4 Summary Description

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

III

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

TABLE OF CONTENTS
1 2 2.1 2.2 2.3 2.3.1 2.3.2 2.3.3 2.3.4 3 3.1 3.2 3.3 3.3.1 3.3.2 3.3.3 4 Background..................................................................................................... 1 RCB CPU Load Monitoring Solution & High Load Handling Solution ........ 2 Principle & Calculation of RCB CPU Load Increase .......................................... 2 RCB CPU Load Monitoring Solution ................................................................. 3 Solution to High Load of RCB CPU ................................................................... 4 Adjusting Number of Sites Among RCP Modules ............................................. 4 RCB Board Expansion ...................................................................................... 4 Inter-RNC Sites Migration ................................................................................. 4 Others............................................................................................................... 4 RUB CPU Load Monitoring Solution & High Load Handling Solution ........ 6 Basic Principles of RUBCPU Load Control Policy ............................................. 6 RUB CPU Load Monitoring Solution ................................................................. 6 Solution to High Load of RUB CPU ................................................................... 6 Balancing CPU Load Among RUBs of the Same Resource Shelf ..................... 6 Balancing CPU Load Among RUBs of Different Resource Shelves .................. 7 Performing RUB & Resource Shelf Capacity Expansion ................................... 7 Summary ......................................................................................................... 9

IV

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

TABLES
Table 2-1 Control-Plane Service Model ................................................................................ 2 Table 2-2 RCB CPU Load Monitoring KPIs & Threshold ...................................................... 3 Table 3-1 Counters of Discarded Messages......................................................................... 6

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

Background
RCB, i.e., RNC control-plane processing board, consists of two CPU modules. In V3.11, RCB can fulfill different functions (i.e., CMP and DMP) by running different versions. CMP processes the common messages of the control plane, while DMP processes the dedicated messages of the control plane. RUB consists of one CPU and 15 DSPs. CPU exchanges messages with other boards and manages the DSPs. With the increase in users and service applications, the signaling traffic and RCB load are increased. Besides, with the increase in signaling traffic, the messages exchanged among the CPU of RUB, other boards, and the DSP of the current board increase continuously, so does the CPU load of the RUB. When the load reaches the threshold, network congestion will occur and network KPIs will become worse. This will affect end users service experience There was the situation that call setup failure occurs due to high CPU load of RCB and RUB. If the CPU load of RCB is too high, there will be system error, which will lead to more serious problems.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

RCB CPU Load Monitoring Solution & High Load Handling Solution
Principle & Calculation of RCB CPU Load Increase
The increase in RCP CPU load is directly related to BHCA. The relation between RCP CPU load increase and BHCA increase is similar to linear relation. Therefore, whether the current CPU load increase is reasonable can be ascertained by comparing BHCA load curve with RCP CPU load curve. CPU utilization ratio = Expected equivalent BHCA / BHCA supported by RCP

2.1

Table 2-1

Control-Plane Service Model Value (Example) Unit Numb er of times/ busy hour Weight Counter No. (U9.3) C310090252+C310090274 +C310090275+C31009027 6+C310090277+C3100902 80+C310090293+C310090 309 C310322216+C310322217 +C310322218+C31032223 2+C310322233+C3103222 34+C310332558+C310332 514+C310332536+C31033 2547+C310332503+C3103 32525+C310342761+C310 342780+C310342807+C31 0342826+C310342837+C3 10342838+C310342839+C 310342840+C310342841+ C310342842+C310342843 +C310342849+C31034285 0+C310342851+C3103428 52+C310342853+C310342 854+C310342855+C31034 2861+C310342862+C3103 42863+C310342864+C310 342865+C310342866+C31 0342867+C310342873+C3 10342874+C310342875+C 310342876+C310342877+ C310342878+C310342879 +C310352945+C31035295 2+C310352959+C3103528 85+C310352886+C310352

Control-Plane Service Model Number of RAB assignments for busy-hour CS/PC service per user

100%

Number of busy-hour CS/PC service handovers per user

20

Numb er of times/ busy hour

35%

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

Control-Plane Service Model

Value (Example)

Unit

Weight

Counter No. (U9.3) 887+C310352888+C31035 2889+C310352890+C3103 52891+C310352897+C310 352898+C310352899+C31 0352900+C310352901+C3 10352902+C310352903+C 310575251+C310575252+ C310575255

Number of DRBC times for busy-hour PS service per user

0.17

Numb er of times/ busy hour

35%

C310404201+C310404203

Number of TCH migrations for busy-hour PS service per user

Numb er of times/ busy hour

31%

C310414096+C310414097 +C310414098+C31041409 9+C310414100+C3104141 01+C310414102+C310414 103+C310414104+C31041 4105+C310414106+C3104 14107+C310414130+C310 414131+C310414132+C31 0414133+C310414134+C3 10414135+C310414136+C 310414137+C310414138+ C310414139+C310414140 +C310414141+C31041690 1+C310416902+C3104169 03+C310416904+C310416 909+C310416910+C31041 6911+C310416912+C3105 75256+C310575257 C310080005+C310080006 +C310080007+C31008001 2+C310080013+C3100800 14+C310080015+C310080 016+C310080017+C31008 0018+C310080019+C3100 80020

Number of times of busy-hour RRC per user without RAB assignment Number of times for processing ALCAP

Numb er of times/ busy hour Numb er of times/ busy hour

44.7%

15

15%

C310930000

In Table 2-1, the figures in red are examples of services occurring in busy hours. They can be modified according to the operators requirements or the statistics in the existing network. Equivalent BHCA requirements = 31 + 200.35 + 0.170.35 + 40.31 + 30.447 +150.15 = 14.89. In the above formula, the figures in red are corresponding to those in Table 2-1, and they can be modified according to the actual situation.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

CPU utilization ratio = Equivalent BHCA / BHCA supported by RCP (14.89/250k for the above example) Note: The calculation in V3.11 is changed and will be updated later.

2.2

RCB CPU Load Monitoring Solution

Monitor RCB CPU overload alarms and RCB CPU load KPIs. Monitor the C311340001 counter (%) and collect statistics by the granularity of 15 minutes. Monitor average RCB CPU load when it reaches 60%.

Table 2-2

RCB CPU Load Monitoring KPIs & Threshold Object

Every CPU Every CPU

Monitoring KPIs
Average CPU load Peak CPU load C311340001

Granularity
15 minutes 15 minutes

Alarm Threshold
50% 70%

High Load Threshold

60% 80%

C311340000

When the average CPU load of RCB reaches 60%, it is necessary to perform load balance or capacity expansion to the RCB board. When it reaches 70%, which is a really dangerous, this situation needs to be handled immediately.

2.3
2.3.1

Solution to High Load of RCB CPU

Adjusting Number of Sites Among RCP Modules
Usually, the load of different RCPs is different. The CPU of some busy RCPs usually reaches the alarm threshold, at this time the CPU load of other RCPs may still stay normal. Therefore, in the short term, if the load of some RCPs is surging, we may move some cells on this RCP to an idle RCP to gain some time for capacity expansion. Operation Procedure: 1. 2. Export the CPU utilization ratio KPIs of all RCPs. Confirm the CPU load of all RCPs and find the RCPs with high load and low load respectively. Select the sites under the high-load RCP and modify the home module to the low-load RCP. This can be performed in batch through quick configuration template.

3.

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

4.

Observe the busy-hour KPIs and RCP CPU load after the modification and see whether the load among RCPs is balanced.

Note: After the home module of the sites is modified to the RCP module, it is necessary to observe whether this modification causes imbalance to other resources, e.g., number of selective preference shelf subsystems.

2.3.2

RCB Board Expansion

If the CPU load of all RCBs of the RNC is balanced and exceeds 80% of the expansion threshold, the CPU utilization ratio of high-load RCB cannot be modified by data modification. This can only be achieved by RCB capacity expansion. Operation Procedure 1. If there are idle RCB slots in the control shelf, plug in the RCB directly. Otherwise, it is necessary to add control shelf or rack to add slots for RCB. After RCB board is added, balance the number of sites among modules with the method in Section 2.3.1 to achieve load balance.

2.

Note: After the RCB board is added, the number of sites should be balanced among selective preference shelf subsystems, so as to avoid service inconsistency among resource shelves.

2.3.3

Inter-RNC Sites Migration

When RCB or control shelf cannot be added on RNC, inter-RNC site migration may be used, i.e., migrate some sites at the border between two RNCs from the high-load RNC to the adjacent RNC, so as to control the load Operation Procedure: 1. Analyze the RCP CPU load of the RNC through KPI analysis and export the RCP table of the sites. Analyze the data acquired at Step 1, ascertain the sites to be migrated, and migrate the target RCB of these sites. Perform cutover to switch the sites to be migrated to the RCP module of the target RNC. Observe the network KPIs and RNC load after the migration.

2.

3.

4.

Note: The neighbor cells need also to be adjusted in the migration process. After the migration, the number of sites should be balanced among the selective preference shelf subsystems in the target RNC.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

2.3.4

Others
1. If the current RNC version is V3.09, it can be upgraded to V3.11 to reduce RCB load. In the case of emergency, such as sudden high-load and grand festivals, master/slave RCB can be configured as non-master/slave mode, so that the number of available RCP modules can be increased to reduce the load. For any problems about the above operations, please contact the chief engineers in the headquarters.

2.

3.

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

RUB CPU Load Monitoring Solution & High Load Handling Solution
Basic Principles of RUBCPU Load Control Policy
There are two thresholds on RNC, i.e., RupCongestThr and RupResumeThr (RupResumeThr < RupCongestThr). When RUP detects that the average CPU seizure rate is greater than RupCongestThr for RupJudgeTime seconds, this RUP will become CPU load control state. After an RUP becomes load control state, the RUP will receive the CCCH signaling (mainly RRC connection requests) of the RUP by RupAcceptRate. If the load of the current RUP within RupJudgeTime seconds is still not reduced to RupResumeThr, this RUP will access service by RupAcceptRateRupAcceptRate (powers of N), and so on. If the average load is reduced to RupResumeThr, this RUP becomes normal operation state. The default value of RupCongestTh is 95%. When RupCongestThr is configured as 100%, the RNC will not start this load control function. After the load control policy is started, the discarded CCCH signaling is mainly RRC signaling. In this case, users cannot access the network. The discarded messages can be observed through the following RNC counters.

3.1

Table 3-1

Counters of Discarded Messages Measurement Type Cell transmission quality statistics Cell transmission quality statistics Measurement Subtype CCCH signaling transmission quality statistics CCCH signaling transmission quality statistics Counter No. C31053599 4 Counter Name Number of uplink CCCH signaling discarded Number of uplink CCCH signaling

Measurement Object RNC cell type

RNC cell type

C31053599 5

3.2

RUB CPU Load Monitoring Solution

Monitor RUB CPU overload alarms and RUB CPU load KPIs.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

Monitor the C311340001 counter (%) and collect statistics by the granularity of 15 minutes. Monitor average RUB CPU load when it reaches 60%. When the average RUB CPU load reaches 80%, perform load balance or capacity expansion to the RUB board.

3.3
3.3.1

Solution to High Load of RUB CPU

Balancing CPU Load Among RUBs of the Same Resource Shelf
In the same resource shelf, the CPU load of some RUB boards is high. It is found that this problem is mainly caused by imbalanced number of cells among these RUB boards, resulted from power-on time inconsistency of these boards. This can be solved by blocking and unblocking the cells in the preference shelf subsystems. Troubleshooting Procedure: Telnet to the RUB board and collect the ShowHpmuInfo information. Check whether the number of cells is balanced among different boards and whether the DSP state is normal. The description of information collection is as shown below: "Status refers to DSP status. MasterUnBlock is the normal status. Other status is abnormal. CellNum is the number of cells established under the DSP. UciuNum is the number of services under the DSP.

The Board is : MASTER, ---------------------------------------SlaveNo 1 2 3 4 5 6 7 8 9 10 Status CellNum 30 30 30 9 29 30 30 27 1 30 UciuNum 24 43 37 9 25 37 33 14 24 34

MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

11 12 13 14 15

MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock MasterUnBlock

30 9 29 0 21

23 17 27 37 45

Status: If the DSP status is abnormal, check whether the DSP is blocked and whether DSP service IP is configured. CellNum: If the number of cells is imbalanced, block/unblock all cells under the preference shelf subsystem in batch. If the number of services is low on some DSPs, contact the chief engineers in the headquarters. After the above operations, collect the statistics of ShowHpmuInfo and observe whether the DSP status is normal, whether the number of cells is balanced, and whether the CPU load of the RUBs in the shelf is balanced.

3.3.2

Balancing CPU Load Among RUBs of Different Resource Shelves

V3.09: In V3.09, each CMP module is configured with one "selective preference shelf subsystem, and each site is allocated to only one CMP module. The selective preference shelf subsystem of the site can be obtained from the above configuration. According to the configuration relation, the cells and services of the site can be established on the RUB in the resource shelf. If the CPU load of the inter-shelf RUBs is imbalanced, modify the home module of the site to reduce the RUB load. Block/unblock the cells of the modified site after the modification. Moreover, in the process of modifying the home module of the site, it is necessary to monitor the CPU load of the RCB. Observe whether the CPU load of the RUBs among different shelves has been balanced, and whether the load of RCBs is balanced. V3.11: In V3.11, one site is configured with one selective preference shelf subsystem. According to the configuration relation, the cells and services of the site can be established on the RUB in the resource shelf.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

If the inter-shelf RUB CPU load is imbalanced, modify the selective preference shelf subsystem of the site from high-load subsystem to low-load subsystem. Block/unblock the cells of the modified site after the modification. Observe whether the CPU load of the RUBs among different shelves has been balanced,

3.3.3

Performing RUB & Resource Shelf Capacity Expansion

If the CPU load of all RUBs of the RNC is balanced and exceeds 80% of the expansion threshold, the CPU utilization ratio of high-load RCB cannot be modified by data modification. This can only be achieved by RUB capacity expansion. If there are idle slots in the resource shelf, add RUB at Slots 1-8 and 11-16. If there is more than one resource shelf and only some resource shelves have idle slots, perform the operations according to Sections 3.3.1 and 3.3.2 after RUB expansion. If there is no idle slot in the resource shelf, expand the capacity of the resource shelf first and then expand the capacity of RUB. Refer to Section 3.3.2 for the operations after RUB expansion. For any problems about the above operations, please contact the chief engineers in the headquarters.

10

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

U_ST_RNC RCB/RUB Load Monitoring and Capacity Expansion Guide

Summary
CPU load monitoring of RNC is a long process. With the increase of service volume, it needs to be done continuously. It is necessary to keep monitoring KPIs and analyzing system load to locate and adjust the RNC close to the overload threshold, so as to avoid network KPI decrease, the influence upon users experience, and system error.

ZTE Confidential Proprietary

2014 ZTE CORPORATION. All rights reserved.

11