Introduction To Scada
Introduction To Scada
Jeff Dagle, PE Pacific Northwest National Laboratory Grainger Lecture Series for the University of Illinois at Urbana-Champaign September 15, 2005
DATA
Critical Operational Data Performance Metering Events and Alarms
SCADA is used extensively in the electricity sector. Other SCADA applications include gas and oil pipelines, water utilities, transportation networks, and applications requiring remote monitoring and control. Similar to real-time process controls found in buildings and factory automation.
Communication Methods Directly wired Power line carrier Microwave Radio (spread spectrum) Fiber optic
Control Center Provides network status, enables remote control, optimizes system performance, facilitates emergency operations, dispatching repair crews and coordination with other utilities.
Control Strategy
Control Center
Supervisory control and data acquisition Balance generation and demand (dispatching) Monitor flows and observe system limits Coordinate maintenance activities, emergency response functions
Key Priorities:
1. Safety 2. Protect equipment from damage 3. Reliability 4. Economics
Control Areas
Reliability Overview
Balance generation and demand Balance reactive power supply and demand Monitor flows and observe thermal limits Observe power and voltage stability limits Operate for unplanned contingencies Plan, design and maintain a reliable system Prepare for emergencies Reliably operate the system you have!
SCADA Functions
Supervisory Control Data Acquisition Real Time Database Graphical Operator Interface Alarm Processing Data Historian/Strip Chart Trending Mapboard Interface
Communications
Directly wired (typical for shorter distances) Power line carrier (less common) Microwave (very frequently used) Radio (VHF, spread spectrum) Fiber optic (gaining popularity)
Analysis
State Estimation/Contingency Analysis Economic Dispatch Short Term Load Forecasting
One-Line Diagram
Alarm Processor
Frequency Control
Frequency Control
Actual Frequency Scheduled Frequency Actual Net Interchange Scheduled Net Interchange
Frequency Regulation
64 Equipment Damage 63 62 Overfrequency Generation Trip Time Correction 61 Frequency Governor Response 60 59 58 57 Equipment Damage 56 Governor Response 59.95 Nominal Frequency Normal Frequency Deviation and AGC Corrective Action Range 60.02 Frequency 60.01 60.00 59.99 59.98 59.97 59.96 Governor Response 60.05 60.04 60.03
IED
PLC
IED
SCADA Trends
Open Protocols
Open industry standard protocols are replacing vendor-specific proprietary communication protocols
Market Systems
Interconnection into market systems is relatively new Some disagree this should be done Few agree on how it should be done securely
Protocol Background
International Standards Organization Open System Interconnection Reference Model ISO OSI Reference Model (protocol stack)
Provides interface to application services Data representation Starts, maintains, and ends each logical session End-to-end reliable communications stream Routing and segmentation/reassembly of packets Transmit chunks of information across a link Transmit unstructured bits across a link
Device #1
REPEATER
BRIDGE
ROUTER
3 Application
Data Link
Physical
2 starting octets of the header 1 octet count of USER DATA in the header and body 1 octet Frame Control 2 octet destination address 2 octet source address 2 octet Cyclic Redundancy Check Each block following the header has 16 octets of User defined data
FIN FIR
SEQUENCE
FIN
FIR
SEQUENCE Number between 0 and 63 to ensure frames are being received in sequence
Roosevelt Dam
As reported by the Washington Post June 27, 2002: Bureau of Reclamation facility in Arizona SCADA system controlling dam floodgates accessed by a 12-year old hacker in 1998 Hacker had complete command of the SCADA system controlling the dams massive floodgates Motivation: exploring on a lark
Queensland, Australia
April 23, 2000: Vitek Boden, 48, caught with stolen computer, radio transmitter. During his 46th successful intrusion. Until then, utility managers didnt know why the system was leaking hundreds of thousands of gallons of sewage into parks, rivers, and the Hyatt Regency hotel grounds. Attack method: software on his laptop identified itself as pumping station 4. He then suppressed alarms, and became the central control system with unlimited command of 300 SCADA nodes. Disgruntled former employee convicted and sentenced to two years in prison.
Bellingham, Washington
June 10, 1999: 237,000 gallons of gasoline leak from 16 pipeline, ignited 1.5 hours later. Three deaths, 8 injuries, extensive property damage. Immediately prior to and during the incident, the SCADA system exhibited poor performance that inhibited the pipeline controllers from seeing and reacting to the development of an abnormal pipeline operation. Warning issued to other pipeline operators by the Office of Pipeline Safety July 1999 NTSB report issued October 2002 Key recommendation:
Utilize an off-line development system for implementing and testing changes to the SCADA database
Olympic Pipe Line Co. filed for Chapter 11 Bankruptcy March 27, 2003
Computer Failures that Occurred in the Electric Power Industry August 14, 2003
First Energy
2:14 pm alarm function fails
No audible or visual indications of failures are presented to power system operators
While technicians were working the problem, control room operators were not fully aware of the failures By 2:54 pm, other failures of the energy management system caused both primary and backup servers to stop functioning Warm reboot completed at 3:08 pm, technicians believed that they corrected the problem
Alarm processor still in failed condition
From 3:05 to 4:05 pm, several key 345kV transmission lines in Northern Ohio trip due to contact with trees in their right of way. This eventually initiates a cascading overloads of additional 345 kV and 138 kV lines, leading to an uncontrolled cascading failure of the grid.
By 4:10 pm Northern Ohio & eastern Michigan are collapsing, many units have tripped, only connection remaining is with Ontario.
7 seconds later, the Northeast portion of the interconnected power system separates, which then breaks into multiple islands. 61,800 MW load is lost as 508 generating units at 265 power plants trip.
Investigation Process
Joint U.S. - Canada Power System Outage Task Force Electric System Working Group Nuclear Working Group Security Working Group
Phase I
Investigate the outage to determine its causes and why it was not contained Interim report released 11/19/03
Phase II
Develop recommendations to reduce the possibility of future outages and minimize the scope of any that occur Final report released 4/5/04 Report available at: http://electricity.doe.gov (Blackout link)
Sequence of Events
Transmission System Performance, Protection, Control Maintenance & Damage Generator Performance, Protection, Controls Maintenance & Damage
Frequency/ACE
NPCC
Substation instrumentation
Digital fault recorders, digital protective relays, synchronized phasor measurement units Some instrument clocks were not synchronized to an established time standard Data format issues
Data entry and tracking procedures SQL-server data base developed to provide inventory and querying capabilities
Challenge: Database developed in parallel with the investigation process
The Blackout of August 14, 2003 Root causes associated with SCADA
Failure of the alarm processor in First Energys SCADA system prevented operators from having adequate situational awareness of critical operational changes to the electrical grid Effective reliability oversight was prevented when the state estimator at the Midwest Independent System Operator failed due to incomplete information on topology changes
Prevented contingency analysis
Post-September 11 Activities
Top 50 Energy Assets Identified (subsequently grew to 100+ assets) Quick assessment surveys initiated on priority ranking, encompassing a range of assessment options
Electric power
Oil
Outreach Activities
Industry engagement (NERC, EPRI, etc.) Federal agency liaison (NIST, NSA, etc.)
Natural Gas
Structured adversaries
Terrorists, hactivists Organized crime Foreign nations
Insiders
Witting Unwitting Half-witting
Anecdotes
2003 CSI/FBI Computer Crime and Security Survey
Overall financial losses from 530 survey respondents totaled $201,797,340. This is down significantly from 503 respondents reporting $455,848,000 last year. (75 percent of organizations acknowledged financial loss, though only 47% could quantify them.) The overall number of significant incidents remained roughly the same as last year, despite the drop in financial losses. Losses reported for financial fraud were drastically lower, at $9,171,400. This compares to nearly $116 million reported last year. As in prior years, theft of proprietary information caused the greatest financial loss ($70,195,900 was lost, with the average reported loss being approximately $2.7 million). In a shift from previous years, the second-most expensive computer crime among survey respondents was denial of service, with a cost of $65,643,300--up 250 percent from last year's losses of $18,370,500.
Anecdotes (Contd)
Carnegie-Mellon CERT
120000 100000 Incidents 80000 60000 40000 20000 0
88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 19 19 19 19 19 19 19 19 19 19 19 19 20 20 20 20
*Q1-Q3
Vulnerability Concerns
Confidentiality
Protecting information from unauthorized access Important for deregulation, competitive intelligence
Integrity
Assuring valid data and control actions Most critical for real-time control applications
Availability
Continuity of operations Important for real-time control applications Historically addressed with redundancy
Vulnerable Points
SCADA IED Modem Remote Control IED Local Control Substation Controller LAN 1 Network Interface Network Interface Remote Access Remote Access
Sources of Vulnerabilities
Exploits at the application, platform, or network level Remote trusted access by
Other enterprise elements (e.g., front office, support functions, etc.) Vendors, customers, business partners Security coordinators, neighboring control areas, etc.
Vulnerability Trends
Much more interconnectivity
Internal and external networks merging Functional, organization interconnection
Increased standardization
Open protocols, common operating systems and platforms
Industry in transition
Deregulation, mergers, new systems and procedures Driven to do more with less
FERC disclosure requirements (market sensitive information) Public franchise issues Public-access web sites
Generally vulnerabilities are greater in:
Redundant systems may compensate for SCADA system failures Consequences are a function of
Expected SCADA system restoration time When the failure occurs (impact to power schedules) System stress at time of failure Whether or not redundant controls exist
Substation
Connected to Network as Node 10
RTU TEST SET
Modem
Intruder Spare Spare Node 10 Node 6 Node 7
Node 2
Node 3
Node 4
Node 5
PLC
PLC
Protective Relay
Protective Relay
May 1998 - Presidential Decision Directive 63: Policy on Critical Infrastructure Protection
Certain national infrastructures are so vital that their incapacitation or destruction would have a debilitating impact on the defense or economic security of the United States
October 2001 - Executive Order 13231 Critical Infrastructure Protection in the Information Age November 2002 Homeland Security Act of 2002: Formation of Dept Homeland Security
National Strategy for the Physical Protection of Critical Infrastructures and Key Assets
February 2003
Lead Agency
Department of Agriculture Environmental Protection Agency Health and Human Services Department of Energy Department of Treasury Department of Interior Department of Defense
Information Technology Multi-State Public Transit Research and Education Network Surface Transportation Telecommunications Water
Analyze information for trends, crosssector dependencies, specific targets Coordinate with other ISACs
http://www.esisac.com
North American Electric Reliability Council (NERC) Critical Infrastructure Protection Committee (CIPC) Physical Security CIPC Cyber Security
Executive Committee
Manage policy matters and provide support to SCs, WGs
Operations Policy
ESISAC Subcommittee
Develop & maintain ISAC capability to respond to security threats & incidents Outreach WG Reporting Technologies WG Indications, Analysis, Warnings WG Grid Monitoring System TF IDS Pilot TF
Mitigation Strategies
Security through obscurity
Poor defense against structured adversary
Isolated network
Unrealistic given todays business demands
Communication encryption
Concerns over latency, reliability, interoperability Vendors waiting for customer demand
Signal authentication
May provide good defense without the concerns associated with full signal encryption
Good configuration management Effective policies and procedures Security awareness, training, and management control
Implement properly configured firewalls and intrusion detection systems Have a defined Enterprise-level computer network security policy
Ref: Concerns About Intrusion into Remotely Accessible Substation Controllers and SCADA Systems, Schweitzer Engineering Laboratories, www.selinc.com
Other Issues
Early detection is critical
Actively look for signs of malicious activity Carefully evaluate trends, patterns Notify appropriate authorities if malicious activity is detected Actively participate in NERC Indications & Warnings program Ensure effective mechanisms are in place to follow-through
Infrastructure Analysis
Physical and control domain Simulation and analysis Optimization, best practices
System-Level Security
Fault-tolerant architecture Defense in depth security strategy Inherent redundancy, resiliency
Cyber Security
Confidentiality integrity and availability Operational and enterprise systems Technologies, policies, procedures
Risk Management
Consequence based analysis Quantitative and qualitative Prioritized mitigation strategies