IBP1545_12 MANAGEMENT OF THE AUTOMATION AND CONTROL SYSTEM OF TGP.

Jorge Roca Mitma1, Neptali Mayorga 2

Copyright 2012, Brazilian Petroleum, Gas and Biofuels Institute - IBP

This Technical Paper was prepared for presentation at the Rio Oi & Gas Expo and Conference 2012, held between September, 1720, 2012, in Rio de Janeiro. This Technical Paper was selected for presentation by the Technical Committee of the event according to the information contained in the final paper submitted by the author(s). The organizers are not supposed to translate or correct the submitted papers. The material as it is presented, does not necessarily represent Brazilian Petroleum, Gas and Biofuels Institute opinion, or that of its Members or Representatives. Authors consent to the publication of this Technical Paper in the Rio Oil & Gas Expo and Conference 2012 Proceedings.

Abstract
TGP pipeline transportation system comprises two gas pipelines, one for natural gas of 714 km long and the other one for natural gas liquids of 540 km long, both pipelines are parallel and start in the Cuscos Amazon Rainforest passing through the Andes and then reach the Peruvian Coast in two points, the NGL pipeline ends in Pisco and the NG pipeline continues until reaching the City Gate of Lurn in the city of Lima. To carry out the transportation of NGL from Cusco to Pisco, there are four pumping stations, three pressure regulating stations and 19 block valves installed through the whole pipeline; likewise, for the NG transportation from Cusco-Malvinas to Lurn-Lima, there is a gas compression plant, two pressure regulating plants and 27 block valves. To control of each operation location of the polyduct, there is a distributed and autonomous instrumentation and process control platform per each operating station. Each of these distributed systems has redundancy within its architecture in the local HMI, industrial networks, data servers and controllers. This redundancy in the common equipment is a means to obtain a high availability and reliability of the system. On the other hand, the design has been made in such a way that the increase of components proper of the redundancy in the systems do not turn into a negative aspect, since with the increase of equipment, the points of failure shall also increase. The operation of each station may be local or remote. The local control of the plant is made through the redundant HMI, allowing field operators to manage the processes of the station. With regard to the remote operation, there is a central Control Room located in Lima, and all operating decisions of all STD of NG and NGL are taken from there. To make the centralized control of the whole transportation system possible, there is a high-performance central SCADA system, in which converge the supervision and control signals of all the operation stations and valves of the pipeline. The communication of the central SCADA system is carried out through an exclusive network and the connection to each remote station is carried out according to the MODBUS TCP/IP protocol. The maintenance, technical support and management of the automation and control system are conducted by the SCADA&Control department of the IT management. For the maintenance of these systems, there is a team of professional specialists, who executes the preventive maintenance according to an annual maintenance plan. Moreover, this team of technicians and engineers, who are permanently trained, provide technical support and also execute the corrective maintenance effectively and with a Minimum Response Time. On the other hand, one of the tasks of this department is to analyze, raise and execute continuous improvements on the automation and control systems, either for existing operation stations or for expansion projects, always in accordance with the TGP quality management system.

______________________________ 1 Electronic Engineer - Transportadora de Gas del Per - TGP 2 IT Manager, Electronic Engineer - Transportadora de Gas del Per - TGP

Rio Oil & Gas Expo and Conference 2012

1. Introduction
The Automation and Control of processes within a Gas Pipeline Transportation System is a key factor for the correct operation of all the line. Despite having installed a good technology in the Control System, both in Hardware and Software, it is necessary to understand the importance of the Maintenance of the system components. If there is not a good maintenance plan and good technical personnel trained to solve the problems that may arise during the operation, it is very likely that substantial economic losses occur. Within the NG and NGL pipeline Transportation System, the management and handling of Preventive and Corrective Maintenance activities of the Automation and Control Systems have been developed; this has allowed achieving a high efficiency in its operation. This document allows detailing the tools and principles applied to obtain these results.

2. Management of the Maintenance of the Automation and Control Systems

The maintenance of these systems are based on an annual maintenance plan programmed to prevent anomalies in the hardware and industrial communication networks, these tasks are conducted according to the Integrated Management System SGI of TGP and in accordance with the safety and occupational health standards ISO 9001, ISO 14001 and OHSAS 18001. TGP has a work team specialized in the management of the control platform installed. On the other hand, for the management of the whole control and supervision system of the facilities under operation, there is an asset management system aimed at the administration and handling of all the information of the control equipment. This tool allows the people responsible for these systems to manage configurations, implement support systems, track the users actions, configure field instruments and receive alerts in case of potential problems. Modules compatible with the HART protocol have been integrated within the control systems for the remote configuration of the field instruments. These modules provide a complete analogical capacity and the HART protocol advantage in E/S modules that may be used locally or installed remotely, with no need of external HART multiplexers or additional wiring, providing access to more data of field devices, this is an advantage, since users can view information on configuration and diagnostics of HART devices on a centralized basis. This is possible since the asset manager accepts the FDT/DTM open technology (IEC-62453 e ISA103). Furthermore, there is a central supervision system that uses OPC technology to exclusively monitor the performance and status of the equipment of the control system of the whole polyduct. This system generated alerts that are sent by e-mail and text messages, alerting the people responsible for these systems in case of any anomaly of the system, which allows having an early and timely response.

Figure 1. Hart Protocol

2.1. Architecture of Automation and Control System System of control and data acquisition of NG and NGL block valves data: Each block valve has a RTU intended for the data acquisition of the operation signals of the pipeline and status signals of the block valves. Independently from the control of each RTU, each valve has safety mechanical devices that close the valve in case of emergency. For the supervision and local control of each valve, local operators panels are placed in the RTUs, over a redundant controlnet network. In addition, each valve is integrated to the central SCADA by means of a gateway module, integrated in the same RTU backplane, which gives all the analogical and digital signals to the central SCADA 2

Rio Oil & Gas Expo and Conference 2012 by means of the Modbus TCP/IP protocol. This enables to obtaine in real time the field information of the block valves integrated to the operation of the polyduct.

Figure 2. Architecture of the Block Valves Control System of control and data acquisition of NG and NGL regulation stations: TGP has two NG and three NGL regulation stations, the regulation stations have the function of regulating the NGL or NG pressure according to the pressure profile of the operation, for this purpose, the station has an automatic control system based on a PLC of processes (PLC-STN), a safety PCL (PLC-ESD) and a local operator panel integrated over a redundant ControlNet industrial network. The function of the PLC-STN is the automatic control for the pressure regulation of the station, as well as the control and handle of each valve and installation equipment to increase the control system availability, the architecture of this system has redundancy in the processor and energy source. The function of the PLC-ESD is to act in cases that affect the integrity of the installation, producing the total closure of the station, this safety system is independent from the process control system and has a SIL2 certificated safety level with failure tolerance. Both systems have a gateway to send all the operation and safety information from the station to the central SCADA, by means of the Modbus protocol TCP/IP. System of control and data acquisition of the NGL pumping stations: TGP has four NGL pumping stations, which function is to raise NGL pressure and go through the Peruvian Andes, for this purpose, each pumping station has an automatic control system based on a PLC of processes (PLC-STN) and a safety PLC (PLC-ESD). The function of the PLC-STN is to control the entire pumping station, which comprises four motor pumps operating in parallel; all equipment has an independent control system based on PLC of the same family of the PCL station. This allows the optimal integration of the equipment into the operation process of the station governed by the PLC-STN. This integration is carried out by means of a redundant ControlNet industrial network; to increase the availability of the stations control system, the architecture of this system is redundant in the processor and energy source. The function of the PLC-ESD is to act in cases that affect the integrity of the installation, producing the total closure of the station. This 3

Rio Oil & Gas Expo and Conference 2012 safety system is independent from the process control system; it also manages redundancy in the processor and energy source with a SIL2 safety level with failure tolerance. Both systems have a gateway to send all the operation and safety information from the station to the central SCADA through the Modbus protocol TCP/IP.

Figure 3. HMI of the Pumping Station 1 System of control and data acquisition of the Chiquintirca NG compression plant: The Chiquintirca TGP compression plant is the only soundproof plant in Latin America, with the purpose of eliminating the generation of noise. In addition, it is the highest power plant installed at more than 3000 meters high: 72 000 horsepower and it consists of 4 turbo-compressors of 18000 horsepower each. It is designed to be extended with other two turbocompressors similar to those already existing turbo-compressors, with a total installed power of 108 000 horsepower.

Figure 4. Chiquintirca Compression Station All the equipment-package of this plant is based on a same PLC family, thus allowing the direct integration to the central control system, governed by an operation PLC (PLC-STN) that integrates each equipment- package through 4

Rio Oil & Gas Expo and Conference 2012 an industrial network based on the redundant controlnet protocol on a fiber optic physical medium, in addition all the field instrumentation supports the HART protocol, being integrated to the control system by means of this protocol. It also has a DeviceNet field network for the handling of the motor control center. The plant safety system is governed by a safety PLC (PLC-ESD), the function of the PLC-ESD is to act in cases that affect the integrity and safety of the installation, producing the total closure of the station, this safety system is independent from the process control system and is integrated with all the safety systems of each equipment-package that comprise the plant. Both PLC-STN and PLC-ESD systems manage redundancy in the processor and energy source with a SIL2 safety level with failure tolerance, it has a gateway to send all the operation and safety information from the station to the central SCADA through the Modbus TCP/IP Modbus. Locally, there is a local supervision system based on the Ethernet/IP protocol, with a group of redundant HMIs that locally monitor and control the station and each turbo -compressor.

Figure 5. Control Architecture of a Compression Plant Furthermore, in the NG and NGL delivery and reception points, there are LACT units installed with an autonomous system for the physical measure. Each unit of measurement has a standard technology that allows integration through the Modbus TCP/IP protocol to the central SCADA.

Rio Oil & Gas Expo and Conference 2012 2.2. Management of Preventive Maintenance Within the management and follow-up structure of the operations of the automation and control system of the Camisea pipeline, some procedures and indicators are used to measure the efficiency in the maintenance of the equipment. As part of the preventive maintenance, we currently use as a baseline the following parameters, based on the experience and good practices: The permanent follow-up of the progress of the annually planned activities, which is shown in figure 6, is a very important parameter for its compliance, because it allows maintaining a high availability in the systems and preventing sudden stops of any operation station, which may lead to a total stop of the NG or NGL transportation, as the case may be. The proper management of the preventive maintenances guarantees the operability of these systems and equipment over time.

Figure 6. Control System Maintenance Plan For a proper follow-up of the activities, the ICMR indicator was developed, which is calculated as follows:

ICMR

Number of visits carried out % Number of scheduled visits

(1)

The visits to the field location are referred to the preventive maintenance activities scheduled. This percentage allows us making a follow-up to the execution of activities. Within the management systems for the control of the annually planned activities, the weekly planning is carried out, which allow us prioritizing and managing the activities that may arise out of the planned activities. Together with the monthly maintenance program, this allows us assigning the group work activities and executing the tasks properly. Additionally, a record of the daily activities carried out by a work team is kept (Report of Daily Activities). The objective of this document is to keep a record of the daily activities executed and typified according to the events described in the planning made per effective hours, hours of on-site assistance, types of maintenance, technical location, Technician assigned to the activity, Client, etc. For the management, follow-up and control of the planned activities, the following indicators have been developed: 6

Rio Oil & Gas Expo and Conference 2012 Indicator of hours worked per type of maintenance (IHTTM): The objective of this indicator is to record the statistics of the hours spent in each type of maintenance. This indicator does not allow determining the variations in it, having the annual maintenance plan as a basis. It is computed as follows:

IHTTM Hours worked per type of maintenance

(2)

Indicator of effective hours worked in each base (IHTCB): The objective of this indicator is to keep the statistics of the effective hours worked per each technician and support engineer and observe the effectiveness of the activities executed in comparison to the planned activities, optimizing the maintenance. It is computed as follows:

ICHET Effective hours per group

(3)

On the other hand, as part of the compliance with the safety standards in the execution of the works, there is the STOP card system, which consists of the preventive observation for the safety at work, allowing the engineers and technicians to prevent the unsafe acts in the activities, recording them in the system. To measure the commitment to the work team safety, the following indicator was developed: STOP Indicator (ISTOP): The purpose of this indicator is keeping the statistics of the unsafe acts in the on-site and office activities to monitor and correct these deviations.

ISTOP Stop Number

(4)

Indicator of Types of Failures (ITA): The objective of this indicator is to keep a record of the more frequent failures recorded during the month and typify the more common failures to take actions on the same. The following measurement parameters for the typing of failures: Mean Time to Repair (MTTR):

MTTR

Total time of corrective repairs # of corrective repairs

(5)

Mean Time to Failure (MTTF):

MTTF

No. of operation hours No. of failures

(6)

Mean Time Between Failures (MTBF):

MTBF

Sum of operation hours Sum of corrective stops

(7)

2.3. Corrective Maintenance and Technical Support Since the operability of the automation and control systems is a critical process within the gas transportation, it requires minimum response times to maintain the operation and not affect the Gas transportation. For this purpose, the development and attention of the events that require a corrective maintenance and/or technical support are well structured within the SCADA & Control department as follows: the reports of failures and technical support requests are notified by means of e-mails, tickets of the Web Help Desk system (WHD) of TGP, telephone calls or alerts by text messages to cellular phones. The automation and control technicians and engineers verify the failure reported depending 7

Rio Oil & Gas Expo and Conference 2012 on the criticality, in order to identify the root cause of the failure. Then, depending on the affected system and its criticality, the pertinent corrective measures are taken. All the actions of corrective maintenance or technical support, as the case may be, are recorded in the Web Help Desk and/or in the Intranet of TGP. In case they are recorded in the WHD System, a new ticket number is generated and the activity is assigned to a responsible person. In this way, the WHD system is a very helpful tool to follow-up all cases until they are closed. The corrective maintenances and technical support provided by the work group may be remote or local, depending of the type and criticality of failure (See Figure 9). For this purpose, the personnel should be qualified and well trained and capable of adding or removing components to the system, without interrupting or affecting the operation of the gas transportation. After correcting the failure, it shall be analyzed if the Automation system and/or equipment need changes, for such purpose, the change control procedure shall be executed; otherwise, the corrective maintenance cycle is closed (ticket closing in WDH). 2.4. Centralized Management tools for the Automation and Control Systems The equipment platform of the Automation and Control systems, considered to be of high criticality for the NG and NGL Transportation business of TGP, have been implemented and standardized in a single product line, which allows centralizing all the information from these systems efficiently distributed with a single tool (see Figure 7). This usefulness is aimed at managing the assets of all the operation plants and block valves that are part of the Pipeline Transportation System (STD). Moreover, this platform is exclusively devoted to handle and manage the control equipment (Controllers, distributed I/O, HMI, Operators panels), being able to perform the following functions: Audit Log, The Audit Log displays database information regarding actions users perform in the automation and control system, such as making edits to a controller project or checking out a file from the application database. Audits can originate from any piece of control system software, and include the time the action was logged, what product it was logged from, what computer it was logged from, and what user performed the action. An example of an audit would be a user going online with a controller processor and making online changes. In this example, the asset control server would receive audits for the user going online with the controller, as well as each edit made (e.g., creating a tag, deleting a rung, etc.). Event Log, The Event Log displays database information regarding events that occur in the automation and control system. Events are generally system initiated, for example an upload occurred or a scheduled task completed. The Event Log appears by default whenever you display the Logs tab. Event records can originate from any piece of asset control software, and include the time the action was logged, what product it was logged from, and what computer it was logged from. An example of an event would be a piece of control system software checking out an activation from an Activation server, and would include the specific information on the activation name, serial number, and type checked out. Audit & Event Caching, if a situation occurs where the Asset control server is offline for a period of time; audit and event messages are not lost. The asset control client will cache all log messages from the local computer, then push the messages up to the server when a connection is re-established. The asset control client does not need to be active (i.e., open) for the cached logs to be sent to the server Audit & Event Log Historical Reporting, while the asset control client offers searching capabilities natively, broader queries can be accomplished using SQL Server. A SQL Server script can be used to query and potentially export audit, event, and/or network health and diagnostics logs. Audit & Event Log Management, through the course of normal operation, the Audit and Event logs of the asset control system can grow quite large. For this reason the administrators of the asset control system to back up and delete older log messages.

Rio Oil & Gas Expo and Conference 2012

Figure 7. Asset Management of the Control System Archive Management, The asset software client can be used to run the Archive Cleanup Wizard, which will assist the user in deleting any old/unnecessary file versions stored in the asset software archive. Version numbers will not be edited for newer file versions, and a record of the deleted version(s) will be kept, but the space used in the MSSQL Database will be freed up. Version Incrementing, The asset software Archive uses a checksum compare to distinguish between two different files. In order for a newly checked-in file to receive an incremented version number, it must return a different checksum value than the latest file already checked in to the archive. Due to the way in which the checksum value is calculated, only changes to the file itself will alter the file enough for asset software to recognize a difference. An example of a file alteration which would not return a different checksum value would be a change to the date/time stamp of the file. Archive Historical Reporting, It may be desirable to perform historical reporting on an asset software archive, outside of the capabilities present in the asset software client. While the client offers information on all archived files, it is possible to perform broader analysis of the archive contents using SQL Server, including historical archive activity and archive asset types. Archive Backup & Compare, this disaster recovery function provides two different basic operations: Backup and Backup & Compare. The Backup operation will upload the assets configuration file and check it into the archive without doing any comparisons, regardless of whether or not the file is different from the latest version already checked. In contrast, the Backup & Compare will upload the assets configuration file and compare it to a designated version already checked in to the archive. To use this function to also check the file into the asset control software, the Promote New Upload to Master property of the device within the schedule must be set to true. This will cause the uploaded file to be checked into the asset control software when differences are found between it and the latest version already checked in, and will automatically promote the uploaded version as the master version of the file. Additionally, for the programming, configuration and diagnosis of the control systems, there is a license server that manages all the management software of the PLC, RTUs, HMI and ControlNet, DeviceNet, Modbus and Ethernet industrial networks. It is very important to highlight that with this tool, the technicians and control engineers may have access to these systems from any operation station. On the other hand, the control of the access to these systems is restricted by safety levels only to specialists and personnel responsible for the respective maintenance.

Rio Oil & Gas Expo and Conference 2012

Figure 8. Server and Client Software maintenance for control equipment connection 2.5. Training Plan To operate the automation and control system is necessary that the process of diagnosis, modification and repair may be developed in time and form during a critical situation. In other words, the system shall have maintenance capacity. The Information Technology department of the TGP considers that it is very important the expertise of the technicians and engineers in charge of maintaining and provide support to the automation and control systems of the Pipeline Transportation System of TGP. For such reason, the training and permanent updating is part of the provision of the best tools in their professional learning, thus ensuring that the systems have a high availability and reliability. This has allowed our professionals to apply all the technology available in order to maintain our stations functioning, as well as having a very fast response time in the process of detecting, diagnosing and solving problems in the system. To make the task of training our technicians and engineers possible, an annual specialization program is provided by the manufacturers of the control and automation equipment, with whom there is a direct interaction. Moreover, all the control platform of the different operation plants is standardized, which makes possible that the personnel responsible for this equipment can continuously and properly specialize in the key points of the specialization. An important factor in the system training process is the equipped automation and control laboratory, capable of simulating the operation from a block valve to a complete Gas Compression plant, being possible to carry out evaluations and diagnosis of specific cases that may occur during the operation of the Pipeline Transportation System (STD). The failure simulation in the control system implemented in the laboratory generates continuous improvement actions, which are essential in the operation, because it achieves faster response times when there are on-site failures. Furthermore, having a technical laboratory allows performing pilot tests with new equipment before implementing it, as well as developing new applications that generate a higher availability and reliability of the control equipment of the plants. In figure 9, the performance of tests in the technical Automation and Control laboratory can be observed.

10

Rio Oil & Gas Expo and Conference 2012

Figure 9. Execution of Maintenance and Technical Support

3. Quotations and Bibliography

JON STENERSON. Programmable Logic Controllers with ControlLogix, 2009. FactoryTalk AssetCentre Design Considerations, for ROCKWELL AUTOMATION HART Communication Foundation. http://www.hartcomm.org/ FDT Group. http://www.fdtgroup.org/ MODBUS. http://www.modbus.org/

11