Logging and Debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR - Karneliuk

5/12/2017 Logging and debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR | karneliuk.
com
Logging and debugging in Nokia (Alcatel-Lucent) SR OS and Cisco

IOS XR
JULY 19, 2016
Hello my friend,
Logs are one of the most valuable (and therefore important) things, which we
have in troubleshooting of our network devices and services. The brief
overview of logs gives us understanding of what and when has happened. This
paradigm is applicable both for production and lab environment, thats why we
proceed further with configuration of logging at Nokia (Alcatel-Lucent) SR OS
and Cisco IOS XR.
Where can we find logs?

I will start the article with the explanation, what I wont do. I wont describe
the configuration of remote logging (to syslog server) and will focus only on
local logging.
Though local logging isnt academic term, it describes the core idea of that
process. The outputs of logging process (actually messages from different
protocols and other parts of operation system) are stored locally to our router,
http://karneliuk.com/2016/07/logging-and-debugging-in-nokia-alcatel-lucent-sr-os-and-cisco-ios-xr/ 1/12
5/12/2017 Logging and debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR | karneliuk.com
regardless speaking about Nokia (Alcatel-Lucent) VSR (or SR 7750, for

example) or Cisco IOS XRv. There are four main destinations, where we can
send logs locally. Lets review them on both platforms:
Nokia (ALU)
Cisco IOS XR Explanation
SR OS
Logs are sent to CLI (command
line interface), where you can
console console observe them in real-time. In
this case console line itself
(serial or USB console).
Logs are sent to CLI, which
session monitor now relates to ssh/telnet
connections.
Logs are stored in routers
memory buffered RAM. They are lost after
reboot of Cisco router.
Logs are stored in file on
file-id archive flash/HDD. They are resistant
to reboot of the device.
For all other Cisco OS (IOS, IOS XE, NX-OS, ASA

OS) there are the same types of logs destinations.
Lets review how we can configure all these types of local logging, as well as
how we can see their content. But first of all lets see the default configuration.
Probably we dont have to configure anything. How do you think?
Default configuration of logging in Nokia (Alcatel-Lucent)

SR OS
The good news is that logging is configured by default both at Cisco IOS XR
and Nokia (Alcatel-Lucent) SR OS. Another question that the default logs
configuration usually doesnt satisfy the needs of troubleshooting or learning.
For example at Nokia (Alcatel-Lucent) we have the following loggers
configured by default:
A:SR3# show log log-id

=============================================================================
Event Logs
=============================================================================
Log Source Filter Admin Oper Logged Dropped Dest Dest Size
Id Id State State Type Id

99 M none up up 36 0 memory 500
100 M 1001 up up 14 21 memory 500
=============================================================================
There are two logs configured by default. The 99th log shows all events that
have taken place in the router since the last reboot (you can figure it out from
dest type that is memory). The 100th log shows only major ones. At this pane
you see the summary information of all configured log. The detailed content
you see be adding the log number to the command above:
A:SR3# show log log-id 99

=============================================================================
Event Log 99
=============================================================================
Description : Default System Log
Memory Log contents [size=500 next event=37 (not wrapped)]36 2016/07/13
14:44:58.99 UTC MAJOR: SYSTEM #2002 Base Configuration Save Succeeds
Configuration file saved to: cf3:\config.cfg35 2016/07/13 14:09:18.89 UTC
MAJOR: SYSTEM #2002 Base Configuration Save Succeeds
Configuration file saved to: cf3:\config.cfgPress any key to continue (Q to
quit)
In Nokia (Alcatel-Lucent) SR OS the latest logs are

shown first, whereas in Cisco IOS XR the oldest logs
are shown first. You will see that later.
Ive said previously that 100th log has only information about major events.
How have I found it out? In Nokia (Alcatel-Lucent) SR OS (VSR or SR 7750)
we have concept of log filters, which gives you possibility to selectively filter
only interesting information. You can see that Filter ID for 100th log has
value 1001. Lets take a look on that filter:
A:SR3# show log filter-id 1001

=============================================================================
Log Filter
=============================================================================
Filter-id : 1001 Applied : yes Default Action: drop
Description : Collect events for Serious Errors Log

Log Filter Match Criteria

Entry-id : 10 Action : forward
Application : Operator : off
Event Number : 0 Operator : off
Severity : major Operator : greaterThanOrEqual
Subject : Operator : off
Match Type : exact string :
Message :
Match Type : sub string Operator : off
Router : Operator : off
Match Type : exact string Operator : off
Description : Collect only events of major severity or higher

=============================================================================
Log-filters have very rich set of parameters, which can be used for log filtering.
In this filter (which is built-in by default) the parameter Severity together
with operator greaterThanOrEqual and action forward, what together mean
that all the logs with severity major or higher will be forwarded to the
destination (memory for log 100). All other logs will be dropped, which you
can figure out from field default action with value drop.
There is one more important thing, before well switch to Cisco. In Nokia
(Alcatel-Lucent) SR OS there also four types of logs sources in addition to
different destinations. These inputs are specific for SR OS, there is no such
analog in Cisco, thats why I want to spend some time describing it. There four
types are as follows:
Main relates to all usual events that happens in Nokia (Alcatel-

Lucent) SR operation (i.e. establishing IGP adjacency, booting of card,
flapping of ports, etc).
Change means the changes done in the configuration of router.
Security relates to security breaching events.
Debug-trace covers all the low level information (BGP update
packets, ARP request/reply, etc)
The details of their configuration well provide further in configuration
sections.
Default configuration of logging in Cisco IOS XR

The logging structure in Cisco IOS XR (actually all types of IOS) is much
simpler. You have to enter just one command:
RP/0/0/CPU0:XR1#show logging
Fri Jun 17 00:55:50.143 UTC
Syslog logging: enabled (0 messages dropped, 0
flushes, 0 overruns)
Console logging: level warnings, 3093 messages
logged
Monitor logging: level debugging, 4643 messages
logged
Trap logging: level informational, 0 messages
logged
Buffer logging: level debugging, 5496 messages
loggedLog Buffer (2097152
bytes):RP/0/0/CPU0:Jun 16 23:27:29.097 :
config[65695]: %MGBL-CONFIG-6-DB_COMMIT :
Configuration committed by user cisco. Use
show configuration commit changes 1000000147
to view the changes.
RP/0/0/CPU0:Jun 16 23:30:13.155 :
config[65695]: %MGBL-SYS-5-CONFIG_I :
Configured from console by cisco
RP/0/0/CPU0:Jun 16 23:34:27.438 :
In the beginning there are configuration parameters for logging. Here provided
the default levels, which you have at the just new booted router.
For those of you, who are new to Cisco IOS XR, by

default debugging isnt logged to console comparing to
Cisco IOS, where it was done.
Lets speak about differences between Nokia (Alcatel-Lucent) SR OS and

Cisco IOS XR:
As Ive already mentioned, the information is provided in such a way

that at the beginning you see the oldest logs.
Cisco IOS XR has no option for choosing source of the logs, so all the
information from the system including debug is collected together
Cisco IOS XR has no option for filtering what is logged besides severity.
Nokia (Alcatel-Lucent) SR OS provides possibility to filter what is
logged based on protocol, text string and so on.
Cisco IOS XR has no concept of separation of log streams. The
command show logging shows you all the logs saved
For filtering saved logs Cisco IOS XR provides possibility to filter them
only based on regexp. Nokia (Alcatel-Lucent) SR OS has such
possibility as well. On the other hand, if you have applied the necessary
filter at the beginning, its either not needed or provide additional level
of filtering granularity.
Probably after that comparison you can think that I like Nokia (Alcatel-Lucent)
and hate Cisco. No, I dont. On the one hand Im CCIE, so I cant hate Cisco
On the other hand Im Nokias employee and have possibility to lab
extensively our IP/MPLS portfolio and I find it really awesome. I respect
greatly both vendors and by that comparison I just outline the different
approaches, which both vendors implement. Nokia (Alcatel-Lucent) SR OS
provides greater control on what, how and where to log, whereas Cisco
provides very easy configuration of logging and access to all the logged
information simultaneously.
Logging to the console

The first thing that I want to configure with is logging to the console. Whereas
in Cisco IOS XR its configured by default for messages with severity level 5
and higher, in Nokia (Alcatel-Lucent) SR OS such logging isnt configured at
all. So, lets start with the latter then:
*A:SR3>config>log# info
-
log-id 1
description DEBUG_TO_CONSOLE
time-format local
from debug-trace
to console
no shutdown
exit
-
This configuration of log stream will send all low level information (and only
it) to the console, when well activate debug at our Nokia (Alcatel-Lucent)
VSR (or SR 7750). By default all debugging is turned off, just in the same way
as it done at Cisco. What is also interesting is that you can switch off this
stream without deconfiguring it just by typing:
*A:SR3# configure log log-id 1 shutdown

This is very convenient, because you can have some sort of preconfigured
debugging streams with certain filters, which are activated only in emergency
cases.
Well, the provided configuration sends only debug information to console. If

you want to send also information about other events (main, change or
security), you have to configure separate log streams. Lets say I want to
send to the console also information about ports flapping (going up or down). I
need the following configuration to accomplish this goal:
-
filter 1
default-action drop
entry 1
action forward
match
message eq pattern Interface
exit
exit
exit
log-id 2
filter 1
from main
to session
no shutdown
exit
-
Actually you can add many different entries to the logs filter. Lets test,
whether this configuration works:
A:SR3# configure router

A:SR3>config>router# interface toXR1
A:SR3>config>router>if# shutdown
1 2016/07/13 21:35:31.39 UTC WARNING: SNMP
#2004 Base toXR1
Interface toXR1 is not
operational*A:SR3>config>router>if# no
shutdown
2 2016/07/13 21:36:07.60 UTC WARNING: SNMP
#2005 Base toXR1
Interface toXR1 is operational
Overall logic for Nokia (Alcatel-Lucent) SR OS log

stream is: one source, one destination and one filter per
stream.
Now its time to configure the same logging level in Cisco IOS XR. Its just
one command:
RP/0/0/CPU0:XR1(config)#show conf
logging console debugging
end
After the numerous lines of configuration in Nokia (Alcatel-Lucent) SR OS,
such simplification looks incredible. But in SR OS, Ive configured exactly
what I want: all active debug traces and interface flapping. All other logs arent
displayed in my CLI. In Ciscos configuration Ill see everything, what is going

on in the router (all information of all severity levels + active debug traces).
RP/0/0/CPU0:XR1(config)#int gig 0/0/0/0

RP/0/0/CPU0:XR1(config-if)#shut
RP/0/0/CPU0:XR1(config-if)#commit
Fri Jun 17 04:08:52.010 UTC
RP/0/0/CPU0:Jun 17 04:08:52.040 : ifmgr[227]:
%PKT_INFRA-LINK-5-CHANGED : Interface
GigabitEthernet0/0/0/0, changed state to
Administratively Down
RP/0/0/CPU0:Jun 17 04:08:52.140 :
RP/0/0/CPU0:XR1(config-if)#no shut
RP/0/0/CPU0:XR1(config-if)#commit
Fri Jun 17 04:08:59.400 UTC
RP/0/0/CPU0:Jun 17 04:08:59.410 : ifmgr[227]:
%PKT_INFRA-LINK-3-UPDOWN : Interface
GigabitEthernet0/0/0/0, changed state to Down
RP/0/0/CPU0:Jun 17 04:08:59.410 : ifmgr[227]:
%PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol
on Interface GigabitEthernet0/0/0/0, changed
state to Down
RP/0/0/CPU0:XR1(config-if)#RP/0/0/CPU0:Jun 17
04:08:59.440 : ifmgr[227]: %PKT_INFRA-LINK-3-
UPDOWN : Interface GigabitEthernet0/0/0/0,
changed state to Up
RP/0/0/CPU0:Jun 17 04:08:59.440 : ifmgr[227]:
%PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol
on Interface GigabitEthernet0/0/0/0, changed
state to Up
RP/0/0/CPU0:Jun 17 04:08:59.510 :
It worth to say that in Cisco IOS XR its possible to active/deactivate logging

for certain protocol under this protocol configuration or for certain interface
under its configuration. For instance log adjacency changes disable under
router ospf xyz configuration will effectively disable all information about
OSPF logged to all destinations (console, buffer and all others).
Logging to ssh/telnet CLI

The previously described logging destination (console) is applicable to you
only, if you are physically attached to the console port on the device. In real
life such case is usual for installation of new device or lab activities, where you
have direct access to the devices. In all other cases, like day-to-day operation
or troubleshooting you will be connected to the devices remotely via ssh or
telnet, what results in another CLI. This makes us to create separate
configuration for logging.
In general you can be connected remotely to the

physical console by KVM device, which you access via
ssh/telnet. In that case to you connection will be treated
by router (both Cisco and Nokia (Alcatel-Lucent)) as
serial, so logging is configured according to previous
chapter.
Comparing to the configuration of logging to console, you just have to change

the destination. Lets create the rule at Nokia (Alcatel-Lucent) VSR (SR 7750),
which will send to telnet console information about OSPF process:
-
filter 11
default-action drop
description OSPF
entry 1
action forward
match
application eq ospf
exit
exit
exit
log-id 3
description OSPF_TO_TERMINAL
filter 11
from main
to session
no shutdown
exit
-
We cant test it right now as we dont have OSPF configured. But you can
apply it to the configuration from my second article, where Ive briefly
configured OSPF at Nokia (Alcatel-Lucent) VSR (SR 7750) and Cisco IOS
XR.
As you may guess, in Cisco IOS XR the configuration of such logging is

impossible, so well just configure the output of logs to ssh/telnet session,
which is called in Cisco vty lines, with severity level of 6 and above. As you
can see above, by default Cisco IOS XR sends even debug to console, what can
effectively overload you line and prevent any commands to be entered, if
debug is improper activated.
logging monitor notifications
end
To see the output of logs in the vty line, when you are connected to Cisco IOS
XR router, you should activate such function:
RP/0/0/CPU0:XR1#terminal monitor
Logging to buffer
Obviously its impossible to be connected to the device 24/7. On the other hand
very often different issues happen, when we are away from console, so we
need to have a possibility to read about them (from associated logs) afterwards.
Besides logging to syslog server, which isnt covered in this article, the most
common way is to store logs locally in RAM. These logs are stored until the
reboot and completely deleted after it. Usually it isnt problem, because for
long-term purposes the logs are stored either at local disc or to syslog server.
At Nokia (Alcatel-Lucent) VSR (SR 7750) the logging to the memory (RAM)
is activated by default for main issues stream, so we dont have to configure it.
Lets configure the same for the debugging.
-
log-id 4
description DEBUG_TO_MEMORY
from debug-trace
to memory
no shutdown
exit
-
Now you can to see the logs from main stream using rule configured by default
show log log-id 99 and to see debugs you will use show log log-id 4.
Cisco IOS XR has configured by default all logs of all severity levels to be
logged to the console. You can change it by:
logging buffered notifications
end
Logging to local storage

The last but not least topic regarding local storage of logs is writing them to the
file, which is stored at local HDD/flash. You will want to do it, if you dont
have central syslog server, but you want the logs to be saved after reboot.
At Nokia (Alcatel-Lucent) VSR (SR 7750) you can do it as follows:
-
file-id 1
location cf3:
exit
log-id 10
from main
to file 1
no shutdown
exit
-
Now you can check that this log stream is operational:
A:SR3# show log log-id

=============================================================================
Event Logs
=============================================================================
Log Source Filter Admin Oper Logged Dropped Dest Dest Size
Id Id State State Type Id

10 M none up up 1 0 file 1 N/A
=============================================================================
To see the content of this log, you dont need to do anything special. Just look
it through as any other log stream.
A:SR3# show log log-id 10

=============================================================================
Event Log 10
=============================================================================
Description : (Not Specified)
26 2016/07/15 10:58:06.96 UTC MAJOR: SYSTEM #2002 Base Configuration Save
Succeeds
Configuration file saved to: cf3:\config.cfg
Now lets take a look at Cisco IOS XR. The configuration of such type of
logging is a much longer comparing to what we have already done there:
logging archive
device harddisk
severity warnings
file-size 10
frequency daily
!
end
Though it saves logs to the HDD, you should be careful with it. The main idea
here that it doesnt save logs to the HDD in real time, but rather it archives
them once a day or week depending on the configuration. If your Cisco IOS
XR router was rebooted before such replication of logs to HDD, you will lose
them.
Debugging
The last topic for this article is debugging. It goes hand in hand with logging,
because its output fully depends on configuration of logging. At this point I
assume that all the previous configuration is implemented, including sending of
debug to the console. Actually the activation and deactivation of debugging is
the same in Nokia (Alcatel-Lucent) VSR (SR 7750) and Cisco IOS XR:
Nokia (Alcatel-Lucent) SR
Cisco IOS XR
OS
A:SR3# debug router ip
RP/0/0/CPU0:XR1#debug icmp ipv4
icmp
Lets test how it works by issuing ping from Cisco IOS XR to Nokia (Alcatel-
Lucent) VSR (SR 7750). Cisco side:
RP/0/0/CPU0:XR1#ping 10.0.0.1 count 1

Fri Jul 15 12:25:58.561 UTC
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 10.0.0.1,
timeout is 2 seconds:
RP/0/0/CPU0:Jul 15 12:25:58.791 : ipv4_io[266]:
IPv4 ICMP: Sending icmp echo request to
10.0.0.1
!
Success rate is 100 percent (1/1), round-trip
min/avg/max = 1/1/1 ms
RP/0/0/CPU0:XR1#RP/0/0/CPU0:Jul 15 12:25:58.791
: ipv4_io[266]: ip_icmp_lib_ipv4_receive:
Receiving pak(0xb0c08113) tid: 5
RP/0/0/CPU0:Jul 15 12:25:58.791 : ipv4_io[266]:
IPV4 ICMP: GigabitEthernet0/0/0/0: Received
echo reply from 10.0.0.1
RP/0/0/CPU0:Jul 15 12:25:58.791 : ipv4_io[266]:
ip_icmp_lib_ipv4_receive: sending
pak(0xb0c08113) to transport: 1, tid: 5
Nokias side:
A:SR3#
21 2016/07/15 11:58:58.47 UTC MINOR: DEBUG
#2001 Base PIP
PIP: ICMP
instance 1 (Base), interface index 4 (toXR1),
ICMP ingressing on toXR1:
10.0.0.2 -> 10.0.0.1
type: Echo (8) code: No Code (0)

22 2016/07/15 11:58:58.47 UTC MINOR: DEBUG
#2001 Base PIP
PIP: ICMP
instance 1 (Base), interface index 4 (toXR1),
ICMP egressing on toXR1:
10.0.0.1 -> 10.0.0.2
type: Echo Reply (0) code: No Code (0)

My personal opinion is that in Nokia (Alcatel-Lucent) SR OS the output is

much more readable and informative than in Cisco, though it uses more space
for its output. In Nokia (Alcatel-Lucent) SR OS output you see full information
regarding source/destination IP addresses in ICMP packets, as well as
type/code of ICMP messages. From that point of view Cisco IOS XR show
only destination of ping (source for reply) and in general in text form, what
type of the ICMP packet is sent or received.
If you want to disable debugs, you can do it either selectively for certain
technology or in overall all logging:
Nokia (Alcatel-Lucent)
Cisco IOS XR
SR OS
A:SR3# debug RP/0/0/CPU0:XR1#no debug
router ip no icmp icmp ipv4
! !
A:SR3# no debug RP/0/0/CPU0:XR1#undeb all
Lessons learned
Probably Ive spend too much time on such topic, because my aim is not to
provide command to command parity between Nokia (Alcatel-Lucent) SR OS
and Cisco IOS XR. But what Ive faced during my labs is that absence of
default logging to the console of VSR (SR 7750) confuses me a lot, because
after Cisco Ive expectation to see everything what is going on in real time.
Conclusion
In general the configuration of logging in Nokia (Alctel-Lucent) SR OS can be
more complicated, because it provides you much more options for selectively
logs filtering than Cisco IOS XR. In reality such filtering significantly eases
the troubleshooting, because you can in advance predefine all the necessary
filters (for BGP, ISIS, LDP and so on) and during troubleshooting you will just
per technology check its status. Moreover the structure, where each log stream
is stored separately provides possibility to have older logs per used filter, than
you could have, if all logs are stored together. On the other hand you have
possibility to configure almost the same simple all-inclusive log, as we have
in Cisco. So its totally up to you how you will use this possibility.
BR,
Anton Karneliuk

Logging and Debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR - Karneliuk

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Logging and Debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR - Karneliuk

Uploaded by

Copyright:

Available Formats

5/12/2017 Logging and debugging in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR | karneliuk.

Logging and debugging in Nokia (Alcatel-Lucent) SR OS and Cisco

Where can we find logs?

regardless speaking about Nokia (Alcatel-Lucent) VSR (or SR 7750, for

For all other Cisco OS (IOS, IOS XE, NX-OS, ASA

Default configuration of logging in Nokia (Alcatel-Lucent)

A:SR3# show log log-id

A:SR3# show log log-id 99

In Nokia (Alcatel-Lucent) SR OS the latest logs are

A:SR3# show log filter-id 1001

Main relates to all usual events that happens in Nokia (Alcatel-

Default configuration of logging in Cisco IOS XR

For those of you, who are new to Cisco IOS XR, by

Lets speak about differences between Nokia (Alcatel-Lucent) SR OS and

As Ive already mentioned, the information is provided in such a way

Logging to the console

*A:SR3# configure log log-id 1 shutdown

Well, the provided configuration sends only debug information to console. If

A:SR3# configure router

Overall logic for Nokia (Alcatel-Lucent) SR OS log

displayed in my CLI. In Ciscos configuration Ill see everything, what is going

RP/0/0/CPU0:XR1(config)#int gig 0/0/0/0

It worth to say that in Cisco IOS XR its possible to active/deactivate logging

Logging to ssh/telnet CLI

In general you can be connected remotely to the

Comparing to the configuration of logging to console, you just have to change

As you may guess, in Cisco IOS XR the configuration of such logging is

Logging to local storage

At Nokia (Alcatel-Lucent) VSR (SR 7750) you can do it as follows:

Now you can check that this log stream is operational:

A:SR3# show log log-id

A:SR3# show log log-id 10

RP/0/0/CPU0:XR1#ping 10.0.0.1 count 1

My personal opinion is that in Nokia (Alcatel-Lucent) SR OS the output is

You might also like