MGCP must:

http://smbitsolutions.wordpress.com/2012/01/26/troubleshooting-callmanager-issues-mgcp-h323/

Troubleshooting Voice: MGCP Part 1

Over the coming weeks I will be running a new series here on Troubleshooting Voice. I often have students in class that report to me
that one of the most difficult parts of their CCIE Voice exam experience was having to deal with the inner workings of some of the
protocols and how to read and decipher them accurately. I have also begun to see this more and more across the various mailing lists
and forums, and so I decided it was time to start an entire series on these not-to-be-feared topics. Since these protocols are covered
quite in-depth in the CCNP Voice course (most specifically in the CVOICE portion), I highly encourage people starting out in Unified
Communications, not to skip the lower level courses, and to really dig in at that CCNA Voice and then CCNP Voice level, before going
into the CCIE Voice. At each level something is presented that is not explained at the next level, so it really is crucial to go through
each progression of the track in a sequential and systematic order. This goes especially for those who might already have a CCIE, and
think they understand what the CCIE is all about. They probably understand very well what the exam itself is all about, however the
underlying Voice technologies are quite vastly different than the data world they may be used to. In fact, I hear this quite a lot from
people making the jump from a R&S IE to the Voice side of the realm Man, this Voice stuff is totally different!.
To begin with, we will start out a bit easy, and go over the basics of everyones favorite client/server gateway protocol MGCP or
Media Gateway Control Protocol.
MGCP has a series of commands that are exchanged between the client and the server. In the basic Cisco UC world (basic meaning
enterprise side of things rather than the carrier side), the client (gateway) is almost always an IOS voice-enabled router, and the
server (call agent) is always the Unified Communications Manager (UCM).
Here are the basic commands that are used to exchange messages between call-agent and gateway:
Connection Commands

CRCX = CReate Connection

DLCX = DeLete Connection

MDCX = MoDify Connection

Audit Commands

AUEP = AUdit EndPoint

AUCX = Audit Connection

Request Command

RQNT = Request for Notification

Endpoint Command

EPCF = EndPoint ConFiguration

Notify Command

NTFY = Notify

Restart Command

RSIP = ReStart In Progress

Now, lets look at each command in a bit more detail.

Connection Commands
Three messages are used by a call-agent to manage an RTP connection on a media gateway

CRCX = CReate Connection

In this message the call-agent instructs the gateway to establish a connection with an endpoint. The parameters in the
CReateConnection provide the what is necessary to build an understanding of each connection. The parameters include the
codec, packetization period, QoS marking, usage of echo cancellation, silence suppression, gain control, RTP security, and
resource reservations.

DLCX = DeLete Connection

This message informs the recipient to delete a connection. The call agent or the gateway can issue the command. The gateway
or the call agent issues the command to advise that it no longer has the resources to sustain the call. As a side effect, the call
agent collects statistics on the execution of the connection. The statistics include number of packets sent, received, and lost,
interarrival jitter, and average transmission delay.
MDCX = MoDify Connection
This message instructs the gateway to update its connection parameters for a previously established connection. The call agent
issues the command. The parameters used are the same as in the CreateConnection command, with the addition of a
ConnectionID that identifies the connection within the endpoint.
Audit Commands
Two messages are used by a call-agent to query the status of a media gateway
AUEP = AUdit EndPoint
This message requests the status of an endpoint. Information that can be audited with this command includes RequestedEvents,
DigitMap, SignalRequests, RequestIdentifier, QuarantineHandling, NotifiedEntity, ConnectionIdentifiers, DetectEvents,
ObservedEvents, EventStates, BearerInformation, RestartMethod, RestartDelay, ReasonCode, PackageList,
MaxMGCPDatagram, and Capabilities. The response will include information about each of the items for which auditing info
was requested.
AUCX = Audit Connection

This message simply requests the status of a connection.

Request Command
One message is used by a call-agent to request a notification of events on a media gateway
RQNT = Request for Notification

This message instructs the media gateway to watch for events on an endpoint and the action to take when they occur, such as
fax tones. The call-agent may then decide to specify use of a different type of encoding method should be used and instruct the
gateway to change with a ModifyConnection command.

Endpoint Command
One message is used by a call-agent to manage a media gateway
EPCF = EndPoint ConFiguration

The EndpointConfiguration command can be used to specify the encoding of the signals that will be received by the endpoint,
such as A-law or mu-law. The call-agent can use the EndpointConfiguration command to pass this information to the media
gateway.

Notify Command
One message is used by the media gateway to notify the call-agent about an event which the call-agent requested notification about
NTFY = Notify

This message informs the call-agent of an event for which a notification was requested. A single notification message may
carry an entire list of events that the gateway detected and accumulated.

Restart Command
One message is used by the media gateway to tell the call-agent that it is in the process of restarting
RSIP = ReStart In Progress

This message tells the call-agent that the gateway and its endpoints are removed from service or are being placed back in
service. The message carries an EndPointId to identify the endpoint(s) that are put in-service or out-of-service. The
RestartMethod parameter specifies the type of restart.
The various RestartMethods are defined as:
o

Graceful restart method indicates that the specified endpoints will be taken out-of- service after the specified delay.

Forced restart method indicates that the specified endpoints are taken abruptly out- of-service. The established
connections, if any, are lost.

Restart method indicates that service will be restored on the endpoints after the specified restart delay,that is, the
endpoints will be in-service. The endpoints are in their clean default state and there are no connections that are
currently established on the endpoints.

Disconnected method indicates that the endpoint has become disconnected and is now trying to establish
connectivity. The restart delay specifies the number of seconds the endpoint has been disconnected. Established
connections are not affected.

Cancel-graceful method indicates that a gateway is canceling a previously issued graceful restart command. The
endpoints are still in-service.

Here is a brief visual aid I quickly put together to help put some of these commands into a bit of perspective as to when each is sent
and what function it serves:

In our next post, we will explore how these messages interact in a much more in-depth fashion, between the call-agent and the
gateway, with the aid looking at debug output from a live UCM and IOS MGCP gateway.Throughout this series, we will be taking a
look at virtually every protocol that is used in the Cisco UC network, so be sure to check back regularly for the complete set.

Troubleshooting Voice: MGCP Part 2

I just finished up 2 weeks of a really great CCNP Voice bootcamp, covering all 5 of the latest 8.0 version exams from Cisco (CVOICE,
CIPT1, CIPT2, CAPPS, & TVOICE). All in all we ended up with 62 completely brand-new hours of informative video that we are
sure you will be excited to watch when they are posted to our streaming and download sites here in probably just about a week. We
went fairly in-depth on most every topic, one of them being MGCP during our TVOICE section of class.
BTW, with this new 62 hours of CCNP Voice video, this brings INE to 320 hours of total CCNA-to-CCIE Voice video-on-demand
training. Far, far more than any other vendor. And it is all up-to-date and taught by me, not by subcontracted instructors.
You may recall that in my last post related to MGCP Troubleshooting, we took a basic look at the MGCP commands that a Call-Agent
(server) instructs a Gateway (client) to preform something the RFC refers to as verbs.
In this post, we are going to take a look at the output of the debug mgcp packets command for a single call, and then break down
each section of the output into transactions (i.e. Command and Response).
So to begin with, here is the complete output a of single call from debug mgcp packet:

ISDN Se0/0/0:23 Q931: RX <- SETUP pd = 8 callref = 0x00FF

Bearer Capability i = 0x8090A2
Standard = CCITT
Transfer Capability = Speech
Transfer Mode = Circuit
Transfer Rate = 64 kbit/s
Channel ID i = 0xA18381
Preferred, Channel 1
Progress Ind i = 0x8583 - Origination address is non-ISDN
Display i = 'Seattle US Phone'
Calling Party Number i = 0x4180, '2065015111'
Plan:ISDN, Type:Subscriber(local)
Called Party Number i = 0xC1, '2065011002'
Plan:ISDN, Type:Subscriber(local)

MGCP Packet received from 177.1.10.12:2427--->

CRCX 256 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
X: 3
L: p:20, a:G.729, s:off, t:b8, fxr/fx:t38
M: recvonly
R: D/[0-9ABCD*#]
Q: process,loop
<--MGCP Packet sent to 177.1.10.12:2427--->
200 256 OK
I: 31
v=0
o=- 49 0 IN IP4 177.1.254.1
s=Cisco SDP 0
c=IN IP4 177.1.254.1
t=0 0
m=audio 18714 RTP/AVP 18 100
a=rtpmap:18 G.729/8000
a=fmtp:18 annexb=no
a=rtpmap:100 X-NSE/8000
a=fmtp:100 200-202
a=X-sqn:0
a=X-cap: 1 audio RTP/AVP 100
a=X-cpar: a=rtpmap:100 X-NSE/8000
a=X-cpar: a=fmtp:100 200-202
a=X-cap: 2 image udptl t38
<--ISDN Se0/0/0:23 Q931: TX -> CALL_PROC pd = 8
Channel ID i = 0xA98381
Exclusive, Channel 1

callref = 0x80FF

ISDN Se0/0/0:23 Q931: TX -> ALERTING pd = 8 callref = 0x80FF

Progress Ind i = 0x8088 - In-band info or appropriate now available
MGCP Packet received from 177.1.10.10:2427--->
RQNT 257 S0/SU0/DS1-0/1@Branch2 MGCP 0.1

X: 1
R: D/[0-9ABCD*#]
S: G/rt
Q: process,loop
<--MGCP Packet sent to 177.1.10.10:2427--->
200 257 OK
<--ISDN Se0/0/0:23 Q931: TX -> CONNECT pd = 8 callref = 0x80FF
ISDN Se0/0/0:23 Q931: RX <- CONNECT_ACK pd = 8 callref = 0x00FF
MGCP Packet received from 177.1.10.12:2427--->
MDCX 258 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
L: p:20, a:PCMU, s:off, t:b8, fxr/fx:t38
M: sendrecv
R: D/[0-9ABCD*#], FXR/t38
S:
Q: process,loop
v=0
o=- 49 0 IN EPN S0/SU0/DS1-0/3@corphq.voice.ine.com
s=Cisco SDP 0
t=0 0
m=audio 18214 RTP/AVP 0
c=IN IP4 177.1.11.26
a=X-sqn:0
a=X-cap:1 image udptl t38
<--MGCP Packet sent to 177.1.10.12:2427--->
200 258 OK
<--ISDN Se0/0/0:23 Q931: RX <- DISCONNECT pd = 8 callref = 0x00FF
Cause i = 0x8290 - Normal call clearing

MGCP Packet received from 177.1.10.12:2427--->

MDCX 259 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
M: recvonly
R: D/[0-9ABCD*#]
Q: process,loop
<--MGCP Packet sent to 177.1.10.12:2427--->
200 259 OK
<--MGCP Packet received from 177.1.10.12:2427--->
DLCX 260 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
S:
<--MGCP Packet sent to 177.1.10.12:2427--->
250 260 OK
P: PS=974, OS=155840, PR=981, OR=156960, PL=1, JI=3, LA=0
<---

So now just looking at the first transaction, we see an Inbound call (thanks to the inclusion of the 'debug isdn q931' output) and that
the call-agent (UCM) instructs the gateway to "CRCX" or "CreateConnection", and the Gateway responds with very simple "200 OK",
while also including the SDP (IETF Session Description Protocol) pertaining to such things as audio codec and DTMF. Notice how
after each call-agent instructed verb (command) and gateway response there is a 3 digit number that corresponds. This is the
transaction number, and use of it essentially ensures that the response is specific to the command (as there may be many commands
and responses being issued on a heavily populated MGCP gateway). We can clearly see in this initial command that the call-agent is
instructing the use of the "a:G.729" audio codec on the "L" line, and the gateway is obliging with the SDP response that will use the
RTP/AVP (Audio Video Profile) codec number 18, G.729 without Annex B. More on this line reveals other things about the call such
as "p" being the packet or sampling size of 20ms, "s" being VAD, "t" being the RFC 2474 ToS/DSCP bits (b8 is hex for 10111000 or

DSCP EF), and fax capabilities. You also might note that the gateway was instructed to "M: recvonly" or as far as the
"connectionMode" is concerned, that the gateway should only Receive packets, not send them. The "C" line is the global CallID and
the "X" line is essentially the local callID. The "R" line is the supported DTMF.
MGCP Packet received from 177.1.10.12:2427--->
CRCX 256 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
X: 3
L: p:20, a:G.729, s:off, t:b8, fxr/fx:t38
M: recvonly
R: D/[0-9ABCD*#]
Q: process,loop
<--MGCP Packet sent to 177.1.10.12:2427--->
200 256 OK
I: 31
v=0
o=- 49 0 IN IP4 177.1.254.1
s=Cisco SDP 0
c=IN IP4 177.1.254.1
t=0 0
m=audio 18714 RTP/AVP 18 100
a=rtpmap:18 G.729/8000
a=fmtp:18 annexb=no
a=rtpmap:100 X-NSE/8000
a=fmtp:100 200-202
a=X-sqn:0
a=X-cap: 1 audio RTP/AVP 100
a=X-cpar: a=rtpmap:100 X-NSE/8000
a=X-cpar: a=fmtp:100 200-202
a=X-cap: 2 image udptl t38
<---

The next transaction shows us the ISDN is telling us the phone out on the PSTN is now alerting (ringing) and the MGCP Request
Notify tells the gateway with the "S" line to play the "rt" or ringback tone.
ISDN Se0/0/0:23 Q931: TX -> ALERTING pd = 8 callref = 0x80FF
Progress Ind i = 0x8088 - In-band info or appropriate now available
MGCP Packet received from 177.1.10.10:2427--->
RQNT 257 S0/SU0/DS1-0/1@Branch2 MGCP 0.1
X: 1
R: D/[0-9ABCD*#]
S: G/rt
Q: process,loop
<--MGCP Packet sent to 177.1.10.10:2427--->
200 257 OK
<---

In this next transaction we see the ISDN output showing that the call is now connecting and the MGCP output that the call-agent us
informing the gateway to "MDCX" or "ModifyConnection" again, this time to change its RTP connectionMode to send and receive
packets "M: sendrecv". This is where the call was answered, and audio now commences.
ISDN Se0/0/0:23 Q931: TX -> CONNECT pd = 8 callref = 0x80FF
ISDN Se0/0/0:23 Q931: RX <- CONNECT_ACK pd = 8 callref = 0x00FF
MGCP Packet received from 177.1.10.12:2427--->
MDCX 258 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
L: p:20, a:PCMU, s:off, t:b8, fxr/fx:t38
M: sendrecv
R: D/[0-9ABCD*#], FXR/t38
S:
Q: process,loop

v=0
o=- 49 0 IN EPN S0/SU0/DS1-0/3@corphq.voice.ine.com
s=Cisco SDP 0
t=0 0
m=audio 18214 RTP/AVP 0
c=IN IP4 177.1.11.26
a=X-sqn:0
a=X-cap:1 image udptl t38
<--MGCP Packet sent to 177.1.10.12:2427--->
200 258 OK

In this next transaction we see from the ISDN output that the call is being disconnected and the call-agent is informing the gateway to
"MDCX" or "ModifyConnection" again, this time to change its RTP connectionMode back to receive only. This is the call-agent
prepping the gateway that the call is being torn down.
ISDN Se0/0/0:23 Q931: RX <- DISCONNECT pd = 8 callref = 0x00FF
Cause i = 0x8290 - Normal call clearing
MGCP Packet received from 177.1.10.12:2427--->
MDCX 259 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
M: recvonly
R: D/[0-9ABCD*#]
Q: process,loop
<--MGCP Packet sent to 177.1.10.12:2427--->
200 259 OK
<---

In this final (and immediately following the previous) section, we see the call-agent instructing the gateway to "DLCX" or
"DeleteConnection". The gateway obliges, and also provides some useful statistics (ConnectionParameters to be specific) about the
call, namely "P: PS=974, OS=155840, PR=981, OR=156960, PL=1, JI=3, LA=0". (PS=PacketsSent, OS=OctetsSent,
PR=PacketsReceived, OR=OctetsReceived, PL=PacketsLost, JI=Jitter, LA=Latency). BTW, if we had anything such as no audio or a
one-way audio issue, we could clearly see that by either both PS & PR or in the case of only one-way audio only PS or PR having a
value of 0 (no packets sent and/or received).
MGCP Packet received from 177.1.10.12:2427--->
DLCX 260 S0/SU0/DS1-0/3@corphq.voice.ine.com MGCP 0.1
C: D0000000028182cf000000F500000015
I: 31
X: 3
S:
<--MGCP Packet sent to 177.1.10.12:2427--->
250 260 OK
P: PS=974, OS=155840, PR=981, OR=156960, PL=1, JI=3, LA=0
<---

Seeing as we have just covered SDP, it will only make sense to move on to looking as SIP, as it uses SDP for audio information as
well.

Troubleshooting MGCP Registration : understanding the language of machines.

A while back it occurred to me that the best Unified Communications Engineers are the ones that best understand the language of
machines . The truth is, when a device develops a problem, it will try to tell you in its own way using its own language. Just as in any
other form of interaction; success is rare in the absence of being able to give expression to ones wishes and also being able to listen.
An Engineer speaks the language when he issues commands to a device using a graphical user interface or a command line
interface . An Engineer interprets/understands the language of machines when he is able to decipher trace and error logs.
So, I made up my mind that where possible, I would try to listen first: I decided to look for the root causes of a problem by first
taking a look at the traces and logs before double checking the configurations. The aim of this is simply geared towards improving
my ability to understand what these devices are trying to tell me in logs and traces when something goes wrong. practice makes
perfect!
I suspect someone out there shares my view that understanding logs and traces is what completes an Engineer so I am going to share
this little episode that occurred in my home lab recently.
So a few days ago I took my personal router and switch with me to a data-center where I was carrying out a server rebuild. I had
modified the configurations on my devices to replicate the clients network but for some reason, when I got home to do some practice
labs, I was almost surprised that my MGCP gateway would not register to my Call Manager.
It was time to explore machine language so I logged unto the voice gateway and issued the following commands:

1) Debug mgcp packets

2) Terminal monitor
3) On the call-manager, I enabled detailed traces for voice gateways and MGCP.
4) Lastly, from the voice gateways global configuration interface, I issued the commands no mgcp following by mgcp . By so
doing, I was essentially restarting the MGCP process.
Here is what I saw happening on the gateway:
*Jul 6 11:23:29.324: MGCP Packet sent to 192.168.0.55:2427>
RSIP 205478228 *@Router.homelab.com MGCP 0.1
RM: restart
<
*Jul 6 11:23:29.332: MGCP Packet received from 192.168.0.55:2427>
200 205478228
<
*Jul 6 11:23:29.332: MGCP Packet received from 192.168.0.55:2427>
AUEP 31 S0/SU3/DS1-0/1@Router.homelab.com MGCP 0.1
F: X, A, I
<
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
In the above, we see that the process started of by the gateway (10.10.10.3) sending a RSIP Restart in progress message to the callmanager(192.168.0.55).
Next, we see a 200 acknowledgment message from the Cisco call manager basically saying: Im ok with that; thanks for letting me
know .

Next we see an Audit end point message (AUEP) from the call-manager asking the gateway to provide it with a report of the
current status of the ISDN trunk. The call manager is very specific because it was asking for an Audit of interface S0/SU3/DS10/1@Router.homelab.com. In human language, that means: give me a report of the first channel (DS1-0/1) of interface 0/3/0 at a
router which has a qualified domain name of Router.homelab.com.
So far, this is normal behavior. You can double check with the Cisco Link below which shows the registration process in graphical
format.
http://www.cisco.com/en/US/tech/tk1077/technologies_tech_note09186a00801da84e.shtml#t6
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
*Jul 6 11:23:29.336: MGCP Packet sent to 192.168.0.55:2427>
500 31 Endpt Unknown
<
*Jul 6 11:23:29.336: MGCP Packet received from 192.168.0.55:2427>
AUEP 32 S0/SU3/DS1-0/2@Router.homelab.com MGCP 0.1
F: X, A, I
<
*Jul 6 11:23:29.336: MGCP Packet sent to 192.168.0.55:2427>
500 32 Endpt Unknown
<
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
In the first line above, we see that the voice gateway responded to the call-managers Audit endpoint request with a 500 endpoint
unknown response.
In the next two lines, we see that the call-manager tries to ask the gateway the same question about the second channel (DS1-0/2)on
of the ISDN trunk but once again, the gateway tells the call-manager that it does not know what its talking about.

In case you have not understand how to spot the channel that an audit is being requeted for, take a look at the output below. I think it
will make everything sink in:
Router# sh mgcp endpoint
Interface E1 0/3/0
ENDPOINT-NAME V-PORT SIG-TYPE ADMIN
S0/SU3/ds1-0/1@Router 0/3/0:15 none up
S0/SU3/ds1-0/2@Router 0/3/0:15 none up
S0/SU3/ds1-0/3@Router 0/3/0:15 none up
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
So at this point I already knew why my gateway was not registering but I was curious about seeing how the call-manager would try to
express the problem to me so I collected call-manager traces and this is what I saw:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|MGCPHandler received msg from: 10.10.10.3
RSIP 205478228 *@Router.homelab.com MGCP 0.1
RM: restart
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
From the above we see that the call-manager received a restart in progress(RSIP) message from the voice-gateway
Router.homelab.com which is at IP Adress 10.10.10.3.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
In the logs below, we see the MGCP manager'; which is running on the call-manager, scrambling to figure out who device
Router.homelab.com is by querying the call-manager database. In the second line below, we see that call-manager believes that the
gateway is using a protocol of PRI Euro which runs on E1.

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|MGCPManager::loadDbMGCPDeviceDigitalAccessPriMatchFunc - found GwName(S0/SU3/DS1-0@Router.homelab.com
MGCPpn9d Started. Device = S0/SU3/DS1-0@Router.homelab.com, Protocol=PRI EURO, Side = USER.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
In the first and second line below, we see the traces confirming that the ISDN layer three Backhaul link from the gateway to the callmanager is opened.
In the fourth line, we see that the call-manager determines that this is an MGCP trunk because the model type is 121 (MODEL TYPE
= 121)
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|MGCPBhHandler 10.10.10.3 - TCP opened and BH registered for the device
MGCPpn9d - Backhaul link OPEN for Devicename S0/SU3/DS1-0@Router.homelab.com
MGCPHandler send msg SUCCESSFULLY to: 10.10.10.3
MGCPpn9d: dn_discovery_DbDnRes DEVICE NAME = S0/SU3/DS1-0@Router.homelab.com MODEL TYPE = 121
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Since the call-manager believes that this is an MGCP trunk (MODEL TYPE = 121)running on protocol type PRI EURO, the callmanager will not request for an audit of a 24 channel T1 trunk would it? it will request for an audit of a 31 channel E1 trunk that is
probably running in Europe or somewhere else that uses the E1 standard.
So naturally, in the second and fourth lines below, we see call-manager was requesting for an audit (AUED) of the 1st channel (DS10/1) and 31st (DS1-0/31) channel respectively. I did not added the audit request for channel 2 to 30 because it was not needed, but as

can be seen in the last line of trace, the gateways response to every audit request was: sorry, I dont know what you are talking
about (500 31 Endpt Unknown)
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|MGCPHandler send msg SUCCESSFULLY to: 10.10.10.3
AUEP 31 S0/SU3/DS1-0/1@Router.homelab.com MGCP 0.1
F: X, A, I
|MGCPHandler send msg SUCCESSFULLY to: 10.10.10.3
AUEP 60 S0/SU3/DS1-0/31@Router.homelab.com MGCP 0.1
F: X, A, I
|MGCPHandler received msg from: 10.10.10.3
500 31 Endpt Unknown
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
If you recall from the beginning of this post I said that I already knew what was wrong when the voice-gateway responded with the
endpoint unknown message to the audit endpoint request from the call-manager. we are about to see the reason why.
The next thing I did was to turn off all debugging on the voice-gateway by using the command: u all at configuration mode.
However, I still had terminal monitor on.
I then logged unto the Cisco Communications Manager webpage and reset my gateway from there. I went back to the voice-gateway
and this is the shortened version of what I saw:
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
*Jul 6 14:23:32.884: %SYS-5-CONFIG_I: Configured from console by console

controller E1 0/3/0
^
% Invalid input detected at ^ marker.
pri-group timeslots 1-31 service mgcp
^
% Invalid input detected at ^ marker.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Do you know whats happening above? Basically, because I had reset the voice-gateway on call-manager, a signal was sent to the
voice-gateway that it should reboot the trunk and download a new set of configurations.
The gateway clearly download the new configuration because the gateway was actually trying to configure the E1 port as can be seen
in the second line of the trace output above. However, we see that the router return an error of invalid input detected. This means
that this command is not allowed. But the question is why? . . . lets find out.
So on the voice gateway I issued the commands do sh run | in controller and do sh run | in card :
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Router(config)#do sh run | in controller
controller T1 0/3/0
Router(config)#do sh run | in card type
card type t1 0 3
voice-card 0
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
So from the above we can clearly see what the problem was. The Call-manager was trying to configure an E1 interface but there was
no E1 interfaces on the gateway just T1. So all along, the call manager was trying to take audit for an E1 interface that did not exist

and the gateway could not register its MGCP ports because they were not configured as T1 on the call-manager. Here is what I did to
resolve it:
Router(config)#no card type t1 0 3
Router(config)# card type E1 0 3
Router(config)# reload
Then I reset the gateway on call-manager and everything worked perfectly. I really hope someone has found this informative because
my original thought was that this case is not blog-worthy. However, since the main focus was on understanding traces and becoming
more familiar with them, I hope a greater understanding of traces has been delivered to the readers of this post.

MGCP Components
These sections discuss the two attributes of MGCP that allow it to function. Endpoints are references to specific voice ports on a
gateway while call-agents are the control devices that administer the gateways.
Endpoints

Endpoints are any of the voice ports on the designated gateway. These voice ports provide connectivity to both analog ports (such as
Foreign Exchange Office (FXO)/Foreign Exchange Station (FXS)) and digital trunks (such as a T1 or E1) to the PSTN. Ports on
gateways are identified by endpoints in very specific ways. It is important to note that gateways can have multiple endpoints
dependent on the number of ports it contains, and that the endpoints are case insensitive. This is a sample endpoint and an analysis of
each portion of it:

AALNAnalog Access Line eNdpoints. This name is used to designate that the type of endpoint is analog. This means that
either FXO or FXS voice interface cards (VICs) are in use. This value changes dependent on what type of endpoint is in use.
For example, if a DS3 interface is used, this value would be "ds3". More on the digital endpoint specification is given later in
this document.
S1Slot 1. This is the slot number on the chassis that holds the voice network module.

SU0Subunit 0. This is the slot number on the voice network module that holds VICs and voice/WAN interface cards
VWICs.

0This is the voice port number on a specific VIC or the VWIC.

av-vg200-1.cisco.comThis is the hostname of the sample endpoint. If the gateway has been configured with a domain name,
it is appended to the hostname as seen in this example.

In this endpoint, the voice port 1/0/0 on a gateway with a hostname of av-vg200-1 and a domain name of cisco.com is described.
AALN describes this to be an analog port, S1 describes that the network module is in slot 1, and SU0/0 indicates the interface card and
port number on the network module.
Here is an example of an MGCP endpoint identifier for T1 PRI. Note the only difference is the trunk type and the B-channel. The
trunk type designates what type of trunk the endpoint describes. Some examples of valid trunk types are ds1, ds3, e1, and e3. The Bchannel specifies which B-channel on the trunk this endpoint is associated with.

Call Agents

Call agents are external control devices in a voice system. Cisco CallManager is the call agent referenced in this document. In MGCP,
the call agent is the device that has complete control of the gateway. This is a very efficient system as all the administration is
performed by the call agent. There is very little setup required on the end of the gateway, as all route patterns and dial-plans are
configured on the Cisco CallManager.

MGCP Commands
MGCP is implemented by a set of commands and responses between the call agent and the gateway transmitted in plain text. Because
plain text is used, it can be very useful to understand these commands to troubleshoot problems related to MGCP. These commands
are transmitted and received across UDP port 2427. There are eight different types of MGCP commands. This table defines them:
Command

Message Name

Sent By

Description

AUEP

AUCX

CRCX

DLCX

MDCX

AuditEndpoint

AuditConnection

CreateConnection

DeleteConnection

ModifyConnection

CallManager

Determines the
status of a
given endpoint.

CallManager

Retrieves all the

parameters
associated with
a connection.

CallManager

Creates a
connection
between two
endpoints.

Both

From
CallManager:
Terminates a
current
connection.
From Gateway:
Indicates that a
connection can
no longer be
sustained.

CallManager

Changes the
parameters
associated with
an established
connection.

RQNT

NTFY

RSIP

NotificationRequest

Notify

RestartInProgress

CallManager

Instructs the
gateway to
watch for
special events
such as hooks
or DTMF tones.
It is also used to
instruct the
gateway to
provide a signal
to the endpoint
(for example,
dial tone and
busy tone).

Gateway

Informs the
Cisco
CallManager
when requested
events occur.

Gateway

Informs the
Cisco
CallManager
that an
endpoint or
group of
endpoints are
taken out or
placed back
into service.

Parameters are sent along with commands to specify exactly what is required or what information is given. Refer to Sample of Debug
MGCP Packets for detailed explanations on parameters. This information is beyond the scope of this document.
It is important to remember that this protocol is used for control purposes only. No voice data is transmitted through the MGCP
protocol itself. All the voice data transfer occurs directly between the phone and the gateway. This diagram explains these
relationships:

The Cisco 7960 IP phones in this example use the Skinny Call Control Protocol (SCCP) to communicate with the Cisco CallManager.
The actual voice data is transferred through Real-time Transport Protocol (RTP) directly between the two devices. MGCP is used by
the Cisco CallManager only to control the gateway.

Cisco CallManager Implementation and Call Flows

Cisco CallManager's implementation of MGCP uses specific command sequences to perform a variety of tasks. These are several
examples of how calls are made and how the gateways are registered. The concept of PRI Backhauling is also covered in this section.
Registration and Endpoint Initialization

This diagram describes how Cisco CallManager registers voice gateways in its database with use of MGCP. The acknowledgment
(ACK) commands are standard TCP acknowledgements of the received command:

Sample FXS Call Flow

This diagram shows a sample FXS call flow (dialing and connection):

Note: In Cisco IOS Software Release 12.3(8)XY and later, the pre-package keyword is supported for the mgcp package-capability
command. The mgcp package-capability pre-package command can be configured in the gateway to solve issues like outbound call
failures in a T1 CAS gateway. Refer to Configuring MGCP Gateway Support for Cisco CallManager for more information.
PRI Backhauling

The one thing that distinguishes a PRI from other interfaces is the fact that the data that is received from the PSTN on the D-channel
and needs to be carried in its raw form back to the Cisco CallManager to be processed. The gateway does not process or change this
signaling data, it simply passes it onto the Cisco CallManager through TCP port 2428. The gateway is still responsible for the
termination of the Layer 2 data. That means that all the Q.921 data-link layer connection protocols are terminated on the gateway, but
everything above that (Q.931 network layer data and beyond) is passed onto the Cisco CallManager. This also means that the gateway
does not bring up the D-channel unless it can communicate with Cisco CallManager to backhaul the Q.931 messages contained in the
D-channel. This figure illustrates these relationships:

For more information on these topics, the Cisco Press book Troubleshooting Cisco IP Telephony
MGCP and its interactions with Cisco CallManager.

provides an in-depth overview on