Professional Documents
Culture Documents
1, FEBRUARY 2018 69
Abstract—The Internet of Things (IoT) extends network into power system infrastructures, which creates an Internet
connectivity and computing capability to physical devices. of Energy [2].
However, data from IoT devices may increase the risk of pri- A key element of the smart grid is the advanced metering
vacy violations. In this paper, we consider smart meters as a
prominent early instance of the IoT, and we investigate their infrastructure (AMI), which relies on smart meters for bi-
privacy protection solutions at customer premises. In particu- directional power flow and communication capabilities. The
lar, we design a load hiding approach that obscures household mass rollout of smart meters provides significant benefits
consumption with the help of energy storage units. For this pur- for managing energy supply and demand for power utilities
pose, we leverage the opportunistic use of existing household and customers alike. However, this industrial IoT solution
energy storage units to render load hiding less costly. We pro-
pose combining the use of electric vehicles (EVs) and heating, increases the risk of privacy violations. For example, by using
ventilating, and air conditioning (HVAC) systems to reduce or nonintrusive load monitoring data analysis techniques, the
eliminate the reliance on local rechargeable batteries for load fine-grained smart meter readings can be disaggregated to
hiding. To this end, we formulate a Markov decision process to reveal customer usage patterns, personal routines, and behav-
account for the stochastic nature of customer demand and use a ioral preferences [3]–[6]. In general, privacy concerns are
Q-learning algorithm to adapt the control policies for the energy
storage units. We also provide an idealized benchmark system by amplified by the presence of IoT devices, which expand
formulating a deterministic optimization problem and deriving the reach of tracking, monitoring, and surveillance. In this
its equivalent convex form. We evaluate the performance of our paper, we focus on privacy-preserving mechanisms for user
approach for different combinations of storage units and with consumption data reported by smart meters.
different benchmark methods. Our results show that the oppor- Data protection techniques, such as data anonymization and
tunistic joint use of EV and HVAC units can reduce the need of
dedicated large-capacity or fast-charging-cycle batteries for load data aggregation, can be applied to mitigate the potential pri-
hiding. vacy leakage of smart meters. Data anonymization removes
any attribute information from the meter readings to obscure
Index Terms—Electric vehicle (EV), Internet of Things (IoT),
Markov decision process (MDP), privacy, Q-learning, smart their relationship with the customers. It relies on an escrow
metering. company as an intermediary to pseudonymize meter measure-
ments [7], [8]. However, those pseudonymized traces can still
be associated with the households that produced them [9]. Data
I. I NTRODUCTION aggregation, on the other hand, aims to reduce the amount
HE CONTINUOUS evolution of pervasive computation, of sensitive information that can be leaked. The privacy-
T communication, and control is creating a new world
populated by intelligent connectivity on physical devices.
preserving aggregation relies on the homomorphic features
of cryptographic computation [10]–[13]. These methods are
The emerging Internet of Things (IoT) provides technology- limited by their required computation or communication com-
enabled solutions for physical assets, which covers a broad plexity, and their performance depends on specific model
spectrum of use cases that are driven by the ability to con- design with respect to privacy protection [14]. Furthermore,
nect, monitor, exchange information, and take autonomous aggregation is primarily designed to tackle the computation
actions [1]. An important and rapidly growing realization of overhead and resource management issues caused by storing
IoT is the smart grid. By connecting different grid devices and analyzing the huge volumes of measurement data.
to a communication network for real-time monitoring and The above solutions are designed from the utility compa-
surveillance, the smart grid can extend computing intelligence nies’ perspective. In general, these solutions can be applied to
other IoT applications that collect private information, e.g.,
Manuscript received March 17, 2017; revised September 20, 2017; accepted e-healthcare data management. However, IoT privacy chal-
October 31, 2017. Date of publication November 8, 2017; date of cur- lenges go beyond these conventional methods in that different
rent version February 9, 2018. This work was supported by the Natural participants (e.g., utility companies and customers) in the
Sciences and Engineering Research Council of Canada. (Corresponding
author: Lutz Lampe.) IoT marketplace may have unaligned interests in collecting
The authors are with the Department of Electrical and Computer and using the data. Therefore, customer-oriented solutions are
Engineering, University of British Columbia, Vancouver, BC required to accommodate individual privacy preferences.
V6T 1Z4, Canada (e-mail: ynsun@ece.ubc.ca; lampe@ece.ubc.ca;
vincentw@ece.ubc.ca). One approach to achieve customer-oriented privacy protec-
Digital Object Identifier 10.1109/JIOT.2017.2771370 tion is through data perturbation. For example, smart meter
2327-4662 c 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
70 IEEE INTERNET OF THINGS JOURNAL, VOL. 5, NO. 1, FEBRUARY 2018
measurements can be perturbed by injecting random noise or replace local dedicated large-capacity or fast-charging-cycle
applying data compression techniques [15]–[17]. The tradeoff batteries for load hiding was discussed in [38], where the
between the measurement precision and perturbation noise is stochastic nature of both EV charging process and house-
analyzed in [18]. However, tampering with smart meter read- hold energy demand are captured by the Markov decision
ings before transmission to the utility company may reduce process (MDP). The use of cascaded rechargeable batteries to
their relevance for billing purposes. An alternative approach alleviate the privacy leakage from smart meter measurement
applies data obfuscation to alter the actual energy consump- was discussed in [39]. In this paper, we extend the approach
tion that is measured and reported by the smart meter. The from [37] and [38] to the combined use of existing house-
customer energy profile can be distorted by integrating an hold energy storage units (e.g., controllable loads and EVs)
alternative energy source such as a renewable energy gener- to design more cost-effective privacy-preserving solutions for
ation unit [19]–[22] or with the help of energy storage units customers. Our proposed solution is different from [37]–[39]
at the customer premises. In this paper, we pursue a privacy- in that we aim to exploit the existing household energy storage
preserving approach that builds on the use of household energy units for load hiding. In particular, we consider their use cases
storage units, which we refer to as household load hiding. as both assistive and alternative energy storage solution to a
dedicated battery to achieve hiding by leveraging the combi-
nation of BLH and LLH. Therefore, our proposed solution can
A. Related Work address smart meter privacy concerns and accommodate indi-
Prior load hiding schemes obfuscate the smart meter mea- vidual privacy preference completely at the customer premises
surements by using either a local rechargeable battery or a in a cost-effective manner.
user controllable load. The former and latter are commonly
referred to as battery-based load hiding (BLH) and load-based
load hiding (LLH). B. Contributions
BLH methods hide the energy consumption variations Achieving our proposed combined load hiding is a chal-
through controlling the battery charging and discharging pro- lenging problem, mainly due to the limited predictability of
cess. The variances of household load profile can be offset household demand, physical charging and discharging con-
to maintain a constant output load with the presence of a straints of energy storage units, and the fact that thermal
rechargeable battery [23]–[25]. The physical constraints of the appliances first and foremost need to satisfy the customer’s
battery, however, make it often impractical to flatten out the comfort requirements. This is aggravated by the interde-
actual power variations. BLH schemes can also achieve hiding pendencies of scheduling decisions, i.e., current scheduling
by randomizing the load profile to preserve a certain differen- decisions affect the availability of energy storage and demand
tial privacy [26], or using stochastic battery control strategy in the future. Our main contributions can be summarized as
to minimize information leakage [27]. The tradeoff between follows.
the smart meter privacy and electricity cost in the context of 1) We formulate an MDP problem to capture the uncertain-
home energy management and demand response by using a ties in household demand and customer behavior. The
rechargeable battery is addressed in [28]–[32]. Considering MDP addresses the difficulties in how to schedule the
the battery purchase expenses, however, BLH methods can energy storage units considering the charging and dis-
be costly to implement. Different from these BLH methods, charging constraints as well as the limited predictability
this paper aims to exploit alternative energy storage that has of customer demand.
already been in place in the household to reduce the cost 2) We propose a model-free learning method that can adapt
incurred by batteries, and thus to render load hiding less costly to the optimal control policies for scheduling the energy
in the implementation phase. storage units, which addresses the difficulty added by
LLH methods achieve hiding by shifting the energy demand the interdependencies of scheduling decisions. We also
of user controllable loads. The proposed schemes either flatten derive a benchmark system by formulating a determinis-
the energy consumption or inject artificial power signatures tic optimization problem, which can be used to evaluate
to hide the load profile and prevent attack cases such as the effectiveness of our learning method.
occupancy detection [33]–[36]. Considering the restrictions of 3) We evaluate our approach by simulation using different
using appliances that are interruptible and can store energy, combinations of energy storage with a local dedicated
the available options are limited for LLH schemes. Different battery, an EV, and a heating, ventilating, and air con-
from these LLH schemes, this paper aims to combine BLH and ditioning (HVAC) unit. Our results show that the latter
LLH by exploiting the potential use of household controllable can be effectively used as assistive batteries, trading off
loads with local dedicated batteries, and thus to render load the level of privacy achieved through load hiding and
hiding more practical in the implementation phase. the aggregate cost for energy consumption. By compar-
The idea of using assistive battery to achieve hiding was ing with different benchmark methods, we also validate
first proposed in [37] and [38] through the integration of our idea that the combination of BLH and LLH as well
electric vehicles (EVs). A convex optimization problem was as the opportunistic use of typical household energy
formulated to exploit the combined use of EVs with local storage units (e.g., EV and HVAC system) can greatly
dedicated rechargeable batteries to disguise the household alleviate the need for local dedicated large-capacity or
load profile [37]. The use of EVs as assistive batteries to fast-charging-cycle batteries.
SUN et al.: SMART METER PRIVACY: EXPLOITING POTENTIAL OF HOUSEHOLD ENERGY STORAGE UNITS 71
T ≤ ∞. We now introduce the models for thermal and chem- Moreover, customers require a certain SOC level when the
req
ical energy storage units, which are characterized by different EV departs, denoted by SOCEV , and thus
system dynamics and physical constraints. req
SOCEV,td = SOCEV . (10)
A. Thermal Energy Storage Unit Model Notice that the EV can have multiple arrival and departure
Thermal energy storage units include thermostatically con- events during one scheduling period and it will follow the
trollable loads such as HVAC systems and electric water tanks, above constraints whenever such an event occurs.
which we refer to as thermal appliances in this paper and
denote their set by . Thermal appliances can convert electric- C. Combining Energy Storage Units for Load Hiding
ity to heat. Their storage and load-shifting behavior is affected We consider three specific combinations of different energy
by thermal dynamics and customer activities. storage units for load hiding purposes.
in and T amb denote the temperature inside and outside
Let Ti,t 1) A Dedicated Local Rechargeable Battery With Thermal
i,t
the space of appliance i ∈ at time t ∈ T , respectively. Appliances: The thermal appliances demand can be
Given its heat rated power qi,t , we further introduce θi and shifted to alleviate the need for a large-capacity or
σi,t to specify the thermal coefficient (rate-of-heat flow) of fast-charging-cycle battery. Customers can choose to
appliance i and the heat transfer between appliance i and its use those loads during off-peak hours instead of peak
72 IEEE INTERNET OF THINGS JOURNAL, VOL. 5, NO. 1, FEBRUARY 2018
hours based on their comfort requirements. Customers dedicated battery and the EV are given by φB,t = SOCB,t and
may also be given an incentive by the utility company φEV,t = (SOCEV,t , aEV,t , dEV,t ), respectively.
to apply such load shifting through demand response At each decision epoch, the LCU needs to choose an action
program, unrelated to privacy protection. to schedule the energy storage units. Let ut denote the control
2) A Dedicated Local Rechargeable Battery With an EV: action vector at time t, where ut = (q1,t , . . . , qW,t ). Due to
The use of an EV can also reduce the reliance on a the constraints of the energy storage units, not all the actions
dedicated large-capacity or fast-charging-cycle battery. can be chosen at a given state. We thus introduce U (s) to
Although the EV can only be scheduled when parked denote the feasible set of all possible actions given state s.
at home, its plugged-in time often overlaps with resi- U (s) satisfies (1)–(3) for thermal appliances, and (4)–(10) for
dential peak demand periods. Therefore, the EV can be batteries.
exploited as an assistive battery which is beneficial in
both load hiding and peak reduction. B. System Dynamics
3) An EV and Thermal Appliances: An EV and thermal Given the system state st and control action ut , the evolution
appliances can jointly be used for load hiding. The EV of the MDP can be described by the system state transition
serves as the battery when plugged-in at home, while probability P(st+1 | st , ut ), for some st , st+1 ∈ S, ut ∈ U (st ),
the thermal appliances are assistive energy storage units and t ∈ T . We assume the household base load evolves
when the EV departs. By using those existing energy according to a Markov chain, with its transition probabilities
storage units, no extra expenses for battery purchase and dependent only on occupancy patterns, where the evolution of
installation will be incurred. the household base load is not affected by the control actions
of the energy storage units. Therefore, the system transition
III. C OMBINING E NERGY S TORAGE U NITS : probabilities between the composite states st and st+1 ∈ S
O PTIMIZATION F RAMEWORK when action ut is taken can be expressed as
In this section, we propose an optimization framework to P(st+1 | st , ut ) = P(lt+1 | lt )P(φt+1 | φt , ut ). (11)
combine household energy storage units to achieve load hid-
Since the state of a specific energy storage unit evolves with
ing. Our primary objective is to protect customer privacy in
transition probabilities independent of other energy storage
a cost-effective manner. Specifically, we are motivated by the
units, we have
fact that household electricity consumption essentially comes
from the inhabitants’ activities. This indicates that the house-
W
hold occupancy patterns can be associated with the finite P(φt+1 | φt , ut ) = P(φi,t+1 | φi,t , qi,t ). (12)
set of on-off household appliance loads. Markovian models i=1
have been shown to be effective in simulating active occu- We now describe how to determine P(φi,t+1 | φi,t , qi,t ).
pancy patterns [43]. To this end, we formulate an MDP For thermal appliances, we assume that heat transfer
problem that captures the uncertainties of household demand between appliance i and its surrounding environment again
and energy storage availability as well as the interdependencies evolves according to a Markov chain, independent of temper-
of scheduling decisions. ature as well as the control action. To simplify the discussion,
we also assume Ti,t amb to be known, which can be obtained
that we have p = 1. When the EV is parking at goal is to minimize the total costs of the episode, which is a
k =0 t,k
home, its arrival and remaining parking duration states will random variable dependent on the state transition probabilities.
move to (aEV,t+2 , dEV,t+2 ) = (1, dEV,t+1 − 1) with proba- Thus, we define the objective function as the expected total
bility p = 1 in the next time slot. The process is repeated cost and can be given as
until the remaining parking duration becomes zero and the T
EV departs. Therefore, the evolution of the EV plugged-in π
Js1 (π ) = Es1 c(st , ut ) | s1 . (19)
status and remaining plugged-in time can be expressed as t=1
P aEV,t+1 = j , dEV,t+1 = k | aEV,t = j, dEV,t = k The objective of the LCU is to find an optimal policy π ∗ that
⎧
⎪
⎪ 1, j = j = 1, k = k − 1 minimizes the expected total cost (19).
⎪
⎪ j = k = 0, j = k = 1
⎨ 1,
= pt,0 , j = j = 0, k = k = 0 (15) IV. S OLUTION M ETHODOLOGY
⎪
⎪ ,
= 1, j = k = 0, k ∈ {1, . . . , N}
⎪
⎪ p j In this section, we present the solution for the MDP
⎩ t,k
0, otherwise. problem. We devise a model-free learning method to mini-
For the local rechargeable battery, we have mize the objective function (19). In addition, we present a
P(φB,t+1 | φB,t , qB,t ) = 1 if (5) and (9) hold, and benchmark for the performance achieved with this practical
P(φB,t+1 | φB,t , qB,t ) = 0 otherwise. stochastic problem framework by formulating a deterministic
problem that considers household demand and the availability
C. Objective Function of storage as perfectly known for the duration of a scheduling
period. We show how to solve the deterministic problem by
We consider both privacy leakage and electricity cost in
transforming it into an equivalent convex form.
the objective of the proposed load hiding scheme, since the
customers may only be willing to tolerate certain additional
consumption costs for the sake of privacy. Intuitively, a con- A. Model-Free Learning Using the Q-learning Algorithm
stant reported electricity usage can eliminate the possibility of To determine the optimal policy of our MDP problem,
inferring individual appliances usages. We thus aim to main- we need to know the state transition probabilities P(lt+1 | lt )
tain a constant load and quantify the reduction in privacy and P(φt+1 | φt , ut ). Unfortunately, the nonstationary transi-
leakage as to what extent the scheduled operations can reduce tion probabilities of household base load and energy storage
the energy fluctuations. Specifically, denoting the constant load are difficult to approximate. The usage of energy storage units
we want to maintain (i.e., the average consumption over a largely depends on customer preferences, and the household
scheduling period) by lc , the privacy leakage f (st , ut ) given base load can vary drastically from peak to off-peak hours.
system state st and action taken ut can be expressed as Therefore, we use a model-free learning method. Specifically,
we adopt the Q-learning algorithm for its simplicity [44]. The
l c − lt − W
i=1 qi,t t
f (st , ut ) =
lc , t∈T. (16) algorithm learns the action-value function that captures the
2 expected cost associated with a state-action pair (s, u), which
The household purchases and sells electricity at prices hct and is defined as their Q value, by repeatedly choosing an action,
hdt at time t, respectively, which are assumed to be perfectly yielding some costs, and obtaining information about outcome
known, as the utility companies generally use fixed price plans states. The optimal scheduling policy can then be constructed
as the TOU prices at the residential sector. The electricity cost by selecting the action with the least cost in each state once
given system state st and action taken ut can be written as the action-value function is learned.
W + W + The Q-learning algorithm uses an exploration and exploita-
tion policy to choose actions given the current state. In
g(st , ut ) = hct qi,t t + lt − hdt − qi,t t − lt
particular, an η-greedy policy is used, which explores a ran-
i=1 i=1
dom action with probability η and exploits a greedy action
(17)
with probability 1 − η. At each time t, given the current state
where t ∈ T and [x]+ = max (x, 0). st , it chooses the greedy action u∗ which minimizes the current
74 IEEE INTERNET OF THINGS JOURNAL, VOL. 5, NO. 1, FEBRUARY 2018
action value Q(st , u) most of the time, but with probability η, Algorithm 1 Q-learning Algorithm
it instead selects an action randomly. Thus, the policy π(st , u), 1: Initialize Q function value, learning parameters α, η and weight-
∀u ∈ U (st ) can be given as [44, p. 122] ing parameter λ.
2: Repeat (for each episode):
η
1 − η + |U (s , if u = u∗ 3: Obtain electricity pricing information hct and hdt , t ∈ T .
π(st , u) = t )| (20) Set desired household constant load output lc .
η
|U (st )| , if u
= u∗ . 4: Initialize energy storage units parameters.
5: Set the initial state: initialize energy storage units status, and
After taking action ut , the system incurs an instantaneous observe the initial household base load.
cost c(st , ut ) and the system state transits to st+1 . The update 6: t: = 1.
rule can be given by [44, p. 148] 7: Repeat (for each decision epoch of episode):
8: Observe the current state st .
Q(st , ut ) ← Q(st , ut ) + α(c(st , ut ) 9: u∗ = arg min Q(st , u).
u ∈ U (st )
+ minu ∈ U (st+1 ) Q(st+1 , u) − Q(st , ut )) (21) 10: Determine the action ut from (20).
11: Take the action ut and calculate the cost from (18).
where α ∈ [0, 1] is the learning rate. It allows us to interpolate 12: Observe the next state st+1 : observe the energy storage
between the old information (current Q value of the state- units status update φt+1 and the household base load lt+1 .
13: Update the Q function from (21).
action pair (st , ut )) and the new information (the observed state
14: t: = t + 1.
st+1 and the one-step cost c(st , ut )). The true expected value 15: Until the end of the episode.
of the Q function will eventually be learned as long as this 16: Until the end of the learning phase.
process continues, as we will observe the possible succeed-
ing states that could have occurred and aggregate over their
outcomes, even though we never directly learn the transition respectively. Thus, the privacy-cost minimization problem is
probabilities. Thus, the learned Q function directly approx- given by
imates the optimal action-value function Q∗ independent of
the policy being followed thereafter. minimize (1 − λ)fT + λgT (22a)
qi,t , i∈∪{B,EV}, t∈T
A description of the Q-learning algorithm is presented in
subject to constraints (1)–(10). (22b)
Algorithm 1. It is executed by the LCU to learn the optimal
policy based on historical household consumption data until Problem (22) is nonconvex due to the definition of g(st , ut ),
the Q function converges. To start learning, the LCU initializes which is the difference between two convex functions.
the Q function and sets the learning and weighting parameter However, we can transform the original problem into its equiv-
(step 1). During the learning phase, it follows the η-greedy alent convex form as follows. We assume that hct ≥ hdt , as the
policy and keeps updating the Q function over all the decision utility companies will sell electricity to the customers at a
epochs of multiple episodes. At the beginning of each episode, higher price than purchasing back in order to make profit. By
the LCU receives the pricing information of the entire schedul- introducing two non-negative auxiliary variables κt+ and κt− ,
ing period from the utility company and sets the desired we can rewrite g(st , ut ) as
constant load (step 3). It sets all the parameters for the energy
g(st , ut ) = hct κt+ − hdt κt− . (23)
storage units (step 4) and the initial state (step 5). For each
decision epoch, the LCU observes the current state (step 8) Substituting (23) into problem (22), the equivalent convex
and determines the control action (step 10). It then updates form can be given by
the Q function (step 13) after taking the action (step 11) and
observing the outcome state (step 12). Steps 8–14 are repeated minimize (1 − λ)fT + λgT (24a)
qi,t ,κt+ ,κt− , i∈∪{B,EV}, t∈T
until the end of the episode. Once the LCU has finished learn-
ing, it determines the optimal action u∗ ∈ U (s) based on the
W
subject to κt+ − κt− = qi,t t + lt , t ∈ T
optimal policy π ∗ by observing the current system state s ∈ S
i=1
for all decision epochs t ∈ T over the entire scheduling period. (24b)
κt+ , κt−
≥ 0, t∈T (24c)
B. Benchmark Problem Formulation
constraints (1)–(10). (24d)
We now formulate a deterministic problem for which we
assume future household demand and customer behavior are Problem (24) can be solved using solvers such as CVX [45].
known, using the same privacy and cost measures defined
in (16)–(18). This idealized formulation serves as a benchmark V. P ERFORMANCE E VALUATION
for the practical scenario considered in the previous section. In this section, we evaluate the performance of our proposed
Since the future household demand and customer behavior household load hiding method. We consider a 24-h schedul-
are assumed to be known, the privacy leakage index and the ing period in our simulation. The AMI currently deployed
electricity cost can be expressed as in, for example, Ontario, Canada supports smart meter data
T
T transmission periodically every 5–60 min [46]. The next gen-
fT = f (st , ut ) and gT = g(st , ut ) eration of smart meters can provide AMI with up to 1-min
t=1 t=1 interval measurement data [47]. In our simulation, we set the
SUN et al.: SMART METER PRIVACY: EXPLOITING POTENTIAL OF HOUSEHOLD ENERGY STORAGE UNITS 75
system with its thermal settings from [41] and [50] as Battery-
HVAC 1 and Battery-HVAC 2, respectively. Similarly, the
combination of the EV and the HVAC system with its thermal
settings from [41] and [50] are referred to as EV-HVAC 1 and
EV-HVAC 2, respectively.
In Fig. 3(a), we can observe an almost flattened load curve
using Battery-HVAC 1 for load hiding. Since the HVAC
system contributes to a large portion of the household load
peaks, shifting its demand effectively eliminates load varia-
tions. Fig. 3(b) shows the hiding results of Battery-HVAC 2.
We still observe a similar trend in the flattened load curve,
but to a lesser extent as the HVAC system in this setting is
less efficient in terms of converting electricity to heat. Overall,
the results validate our approach of using HVAC systems to
assist a local rechargeable battery for load hiding, which can Fig. 4. Cost versus mutual information for Q-learning method using Battery-
HVAC 1, EV-HVAC 1, Battery-EV, and Battery-only combinations for hiding.
be generalized to other existing energy-intensive household Also included are the results for the deterministic benchmark formulation for
thermal appliances (e.g., electric water tanks). Fig. 3(c) shows Battery-HVAC 1 case.
that using Battery-EV for load hiding produces a similarly flat-
tened load curve, except for the period after 7 A . M . when the energy storage unit combinations are used for hiding. As more
EV has departed. This means that the desired output cannot and more households are equipped with high energy effi-
always be maintained due to the EV availability and the phys- cient HVAC systems, we choose the HVAC system from [41]
ical constraints for the battery. We observe from Fig. 3(d) that for performance evaluation here. The dedicated battery has a
using an EV and an HVAC system (without the extra battery) storage capacity of 10 kWh. The results are obtained using
produces a piecewise flattened load curve. However, if an EV different parameters λ ∈ [0, 1], enabling a tradeoff between
and a less efficient HVAC system are used for hiding, notable the cost of energy consumption and privacy leakage (smaller
load variations remain, as shown in Fig. 3(e). In this case, the mutual information corresponds to better privacy protection).
flexibility of using the HVAC system for load hiding is lim- We observe that the Battery-HVAC 1 combination for load
ited by the need to first accommodate the customer’s comfort hiding achieves the best cost-privacy performance, followed
requirements. by the Battery-EV, Battery-only, and the EV-HVAC 1 combi-
The results in Fig. 3 support the idea of using EV and nation. The results substantiate that EV and HVAC can both
HVAC as opportunistic energy storage units for load hiding. be exploited as an alternative or an assistive energy storage
However, their hiding performance is limited by the EV avail- solution to a dedicated battery for load hiding. The battery-
ability, requirements on the EV SOC level at departure, the assisted use case is more effective in terms of electricity cost
thermal efficiency of the HVAC system, and restrictions on though, mainly due to the comfort and EV charging require-
the flexibility of the HVAC use. We further observe from the ments of the customer. We further note that the proposed
figures that the load curves of the Q-learning method exhibit Q-learning method provides a cost-effective tradeoff close to
small fluctuations around the flat lines obtained with the ideal- that obtained for the idealized benchmark case, which assumes
ized benchmark optimization. However, these fluctuations do perfect knowledge of future household consumption and EV
not allow energy usage information to be inferred about the arrival and departure times. We show the Battery-HVAC 1 case
customer. Hence, we conclude that the learning-based method in Fig. 4, but the same trend has been observed for the other
can successfully schedule the energy storage units to dis- storage unit combinations.
guise the household load, using only the available system state We next compare the performance of the Q-learning
information. approach with that of the online control algorithm from [29].
Since this method is only applicable to a dedicated battery
case, Fig. 5 shows the cost-privacy tradeoff when both methods
B. Cost and Privacy Performance Evaluation make use of a 10-kWh battery for load hiding. We observe that
We now quantitatively evaluate the privacy preserving and both methods achieve fairly similar cost-privacy performances
cost reduction performance of our Q-learning method and of for this scenario. This corroborates the effectiveness of our
the benchmark solution. We adopt the mutual information learning framework, as it does not incur a performance penalty
between the original household load and the masked load as for the battery-only case, while its actual utility lies in the
the privacy protection measure [25]. For both masked load applicability to load hiding with a combination of dedicated
time series as well as original load time series, we com- and opportunistic storage units.
pute the mutual information based on [25, eq. (1)] using Finally, we evaluate the possible reduction of required
the load differences at each consecutive time slot pair (i.e., capacity for the dedicated battery when assisted by EV
t, t + 1 ∈ T ) during one scheduling period, applying the and HVAC units. For this, we consider the cost-privacy
quantization interval m = 100 (see [25] for more details). performance for different combinations of storage units with
Fig. 4 shows the electricity cost of the customer per schedul- batteries of different sizes in Fig. 6. For concreteness, we focus
ing period as a function of mutual information when different on the case of maximal privacy (i.e., λ = 0). We observe that
SUN et al.: SMART METER PRIVACY: EXPLOITING POTENTIAL OF HOUSEHOLD ENERGY STORAGE UNITS 77
R EFERENCES
[1] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A survey,”
Comput. Netw., vol. 54, no. 15, pp. 2787–2805, Oct. 2010.
[2] N. Bui, A. P. Castellani, P. Casari, and M. Zorzi, “The Internet of
energy: A Web-enabled smart grid system,” IEEE Netw., vol. 26, no. 4,
pp. 39–45, Jul./Aug. 2012.
[3] G. W. Hart, “Nonintrusive appliance load monitoring,” Proc. IEEE,
vol. 80, no. 12, pp. 1870–1891, Dec. 1992.
[4] M. Zeifman and K. Roth, “Nonintrusive appliance load monitoring:
Review and outlook,” IEEE Trans. Consum. Electron., vol. 57, no. 1,
pp. 76–84, Feb. 2011.
Fig. 5. Cost versus mutual information for proposed Q-learning method and [5] J. Kelly and W. Knottenbelt, “Neural NILM: Deep neural networks
the online control algorithm from [29] using different tradeoff parameters. In applied to energy disaggregation,” in Proc. ACM Int. Conf. Embedded
both cases, a dedicated rechargeable battery is used for load hiding. Syst. Energy Efficient Built Environ., Seoul, South Korea, Nov. 2015,
pp. 55–64.
[6] M. Baker, G. Hicks, S. Rodriguez, and M. Fuller, “A test of
commercially available products for estimating end uses from
smart meter data,” in Proc. Int. Workshop Non Intrusive Load
Monitor., Vancouver, BC, Canada, May 2016. [Online]. Available:
http://nilmworkshop.org/2016/proceedings/Paper_ID22.pdf
[7] C. Efthymiou and G. Kalogridis, “Smart grid privacy via anonymization
of smart metering data,” in Proc. IEEE SmartGridComm, Gaithersburg,
MD, USA, Oct. 2010, pp. 238–243.
[8] C. Rottondi, G. Mauri, and G. Verticale, “A data pseudonymization pro-
tocol for smart grids,” in Proc. IEEE GreenCom, Piscataway, NJ, USA,
Sep. 2012, pp. 68–73.
[9] M. Jawurek, M. Johns, and K. Rieck, “Smart metering de-
pseudonymization,” in Proc. Comput. Security Appl. Conf., Orlando, FL,
USA, Dec. 2011, pp. 227–236.
[10] S. Ruj and A. Nayak, “A decentralized security framework for data
aggregation and access control in smart grids,” IEEE Trans. Smart Grid,
vol. 4, no. 1, pp. 196–205, Mar. 2013.
[11] C. Rottondi, G. Verticale, and C. Krauss, “Distributed privacy-preserving
aggregation of metering data in smart grids,” IEEE J. Sel. Areas
Commun., vol. 31, no. 7, pp. 1342–1354, Jul. 2013.
Fig. 6. Cost and mutual information for different battery sizes with and [12] C.-I. Fan, S.-Y. Huang, and Y.-L. Lai, “Privacy-enhanced data aggrega-
without the use of EV or HVAC systems for load hiding. tion scheme against internal attackers in smart grid,” IEEE Trans. Ind.
Informat., vol. 10, no. 1, pp. 666–675, Feb. 2014.
[13] H. Shen, M. Zhang, and J. Shen, “Efficient privacy-preserving cube-
using a single assistive storage unit can reduce the battery size data aggregation scheme for smart grids,” IEEE Trans. Inf. Forensics
by 50%. Furthermore, the joint use of EV and HVAC achieves Security, vol. 12, no. 6, pp. 1369–1381, Jun. 2017.
a better privacy protection at about the same energy consump- [14] Z. Erkin, J. R. Troncoso-Pastoriza, R. L. Lagendijk, and
F. Perez-Gonzalez, “Privacy-preserving data aggregation in smart
tion cost but with a battery of only one quarter the capacity metering systems: An overview,” IEEE Signal Process. Mag., vol. 30,
of the battery-only case. no. 2, pp. 75–86, Mar. 2013.
[15] J.-M. Bohli, C. Sorge, and O. Ugus, “A privacy model for smart meter-
ing,” in Proc. IEEE ICC Workshops, Capetown, South Africa, May 2010,
VI. C ONCLUSION pp. 1–5.
[16] S. Wang et al., “A randomized response model for privacy preserving
In this paper, we have investigated the potential of existing smart metering,” IEEE Trans. Smart Grid, vol. 3, no. 3, pp. 1317–1324,
household energy storage units to tackle the potential pri- Sep. 2012.
vacy leakage from smart meter readings. We have proposed a [17] L. Sankar, S. R. Rajagopalan, S. Mohajer, and S. Mohajer, “Smart meter
privacy: A theoretical framework,” IEEE Trans. Smart Grid, vol. 4, no. 2,
cost-effective privacy preserving solution from the customers’ pp. 837–846, Jun. 2013.
perspective by leveraging the combined use of existing ther- [18] M. Savi, C. Rottondi, and G. Verticale, “Evaluation of the precision-
privacy tradeoff of data perturbation for smart metering,” IEEE Trans.
mal appliances and energy storage units for household load Smart Grid, vol. 6, no. 5, pp. 2409–2416, Sep. 2015.
hiding. To accomplish this, we have formulated an MDP [19] D. Günüz and J. Gómez-Vilardebó, “Smart meter privacy in the presence
problem that captures the uncertainties in both household of an alternative energy source,” in Proc. IEEE ICC, Budapest, Hungary,
Jun. 2013, pp. 2027–2031.
power demand and customer usage behavior. A model-free [20] O. Tan, D. Günüz, and H. V. Poor, “Increasing smart meter privacy
learning framework has been designed to solve the MDP through energy harvesting and storage devices,” IEEE J. Sel. Areas
problem without prior information on that state transition prob- Commun., vol. 31, no. 7, pp. 1331–1341, Jul. 2013.
[21] G. Giaconi, D. Günüz, and H. V. Poor, “Smart meter privacy with an
abilities. We have also provided a deterministic optimization energy harvesting device and instantaneous power constraints,” in Proc.
problem, whose solution can be considered as a benchmark IEEE ICC, London, U.K., Jun. 2015, pp. 7216–7221.
for solution obtained with the proposed method. Simulation [22] G. Giaconi, D. Günüz, and H. V. Poor, “Smart meter pri-
vacy with renewable energy and an energy storage device,” IEEE
results validated our approach and showed its effectiveness in Trans. Inf. Forensics Security, to be published. [Online]. Available:
achieving a favorable privacy-cost tradeoff with a relatively http://ieeexplore.ieee.org/document/8016368/
78 IEEE INTERNET OF THINGS JOURNAL, VOL. 5, NO. 1, FEBRUARY 2018
[23] G. Kalogridis, C. Efthymiou, S. Z. Denic, T. A. Lewis, and R. Cepeda, [49] Summary of Travel Trends, 2009 National Household Travel Survey,
“Privacy for smart meters: Towards undetectable appliance load sig- U.S. Dept. Transp., Washington, DC, USA, 2009.
natures,” in Proc. IEEE SmartGridComm, Gaithersburg, MD, USA, [50] J. Black, “Integrating demand into the U.S. electric power system:
Oct. 2010, pp. 232–237. Technical, economic, and regulatory frameworks for responsive load,”
[24] S. McLaughlin, P. McDaniel, and W. Aiello, “Protecting consumer Ph.D. dissertation, Eng. Syst. Div., Massachusetts Inst. Technol.,
privacy from electric load monitoring,” in Proc. ACM Conf. Comput. Cambridge, MA, USA, 2005.
Commun. Security, Chicago, IL, USA, Oct. 2011, pp. 87–98. [51] I. Richardson, M. Thomson, D. Infield, and C. Clifford, “Domestic elec-
[25] W. Yang et al., “Minimizing private data disclosures in the smart grid,” tricity use: A high-resolution energy demand model,” Energy Build.,
in Proc. ACM Conf. Comput. Commun. Security, Raleigh, NC, USA, vol. 42, no. 10, pp. 1878–1887, Oct. 2010.
Oct. 2012, pp. 415–427. [52] Customer Generation Price Plan. Accessed: Nov. 29, 2016. [Online].
[26] J. Zhao, T. Jung, Y. Wang, and X. Li, “Achieving differential privacy of Available: http://www.srpnet.com/prices/home/customergenerated.aspx
data disclosure in the smart grid,” in Proc. IEEE INFOCOM, Toronto,
ON, Canada, Apr. 2014, pp. 504–512.
[27] S. Li, A. Khisti, and A. Mahajan, “Privacy-optimal strategies for smart Yanan Sun (S’14) received the B.S. and M.S.
metering systems with a rechargeable battery,” in Proc. Amer. Control degrees from Xi’an Jiaotong University, Xi’an,
Conf. (ACC), Boston, MA, USA, Jul. 2016, pp. 2080–2085. China, in 2011 and 2014, respectively. She is cur-
[28] J. Yao and P. Venkitasubramaniam, “On the privacy-cost tradeoff of an rently pursuing the Ph.D. degree at the Department
in-home power storage mechanism,” in Proc. Allerton Conf. Commun. of Electrical and Computer Engineering, University
Control Comput., Monticello, IL, USA, Oct. 2013, pp. 115–122. of British Columbia, Vancouver, BC, Canada.
[29] L. Yang, X. Chen, J. Zhang, and H. V. Poor, “Cost-effective and privacy- Her current research interests include cyber-
preserving energy management for smart meters,” IEEE Trans. Smart physical system, smart meter privacy, and energy
Grid, vol. 6, no. 1, pp. 486–495, Jan. 2015. storage applications in smart grid.
[30] O. Tan, J. Gómez-Vilardebó, and D. Gündüz, “Privacy-cost trade-offs
in demand-side management with storage,” IEEE Trans. Inf. Forensics
Security, vol. 12, no. 6, pp. 1458–1469, Jun. 2017.
[31] J. Koo, X. Lin, and S. Bagchi, “RL-BLH: Learning-based battery control
for cost savings and privacy preservation for smart meters,” in Proc. Lutz Lampe (M’02–SM’08) received the Dipl.-Ing.
IEEE/IFIP Int. Conf. Depend. Syst. Netw. (DSN), Denver, CO, USA, and Dr.-Ing. degrees in electrical engineering from
Jun. 2017, pp. 519–530. the University of Erlangen-Nuremberg, Erlangen,
[32] Z. Zhang, Z. Qin, L. Zhu, J. Weng, and K. Ren, “Cost-friendly differ- Germany, in 1998 and 2002, respectively.
ential privacy for smart meters: Exploiting the dual roles of the noise,” Since 2003, he has been with the Department
IEEE Trans. Smart Grid, vol. 8, no. 2, pp. 619–626, Mar. 2017. of Electrical and Computer Engineering, University
[33] D. Egarter, C. Prokop, and W. Elmenreich, “Load hiding of house- of British Columbia, Vancouver, BC, Canada,
hold’s power demand,” in Proc. IEEE SmartGridComm, Venice, Italy, where he is a Full Professor. He co-edited Power
Nov. 2014, pp. 854–859. Line Communications: Principles, Standards and
[34] D. Chen, D. Irwin, P. Shenoy, and J. Albrecht, “Combined heat and Applications From Multimedia to Smart Grid (Wiley,
privacy: Preventing occupancy detection from smart meters,” in Proc. 2nd ed., 2016). His current research interests include
IEEE PerCom, Budapest, Hungary, Mar. 2014, pp. 208–215. theory and application of wireless, power line, optical wireless, and optical
[35] D. Chen, S. Kalra, D. Irwin, P. Shenoy, and J. Albrecht, “Preventing fiber communications.
occupancy detection from smart meters,” IEEE Trans. Smart Grid, vol. 6, Dr. Lampe was a co-recipient of a number of Best Paper Awards, includ-
no. 5, pp. 2426–2434, Sep. 2015. ing awards of the 2006 IEEE International Conference on Ultra-Wideband
[36] A. Reinhardt, D. Egarter, G. Konstantinou, and D. Christin, “Worried
(ICUWB), the 2010 IEEE International Communications Conference, and
about privacy? Let your PV converter cover your electricity consump-
the 2011 and 2017 IEEE International Symposium on Power Line
tion fingerprints,” in Proc. IEEE SmartGridComm, Miami, FL, USA,
Communications and its Applications (ISPLC). He is currently an Associate
Nov. 2015, pp. 25–30.
Editor of the IEEE T RANSACTIONS ON C OMMUNICATIONS, IEEE
[37] Y. Sun, L. Lampe, and V. W. S. Wong, “Combining electric vehicle
and rechargeable battery for household load hiding,” in Proc. IEEE C OMMUNICATIONS L ETTERS, and IEEE C OMMUNICATIONS S URVEYS AND
SmartGridComm, Miami, FL, USA, Nov. 2015, pp. 611–616. T UTORIALS. He was the General (Co)-Chair of 2005 IEEE ISPLC, 2009
[38] Y. Sun, L. Lampe, and V. W. S. Wong, “EV-assisted battery load hiding: IEEE ICUWB, and 2013 IEEE International Conference on Smart Grid
A Markov decision process approach,” in Proc. IEEE SmartGridComm, Communications.
Sydney, NSW, Australia, Nov. 2016, pp. 160–166.
[39] Y. H. Liu, S.-H. Lee, and A. Khisti, “Information-theoretic privacy in Vincent W. S. Wong (S’94–M’00–SM’07–F’16)
smart metering systems using cascaded rechargeable batteries,” IEEE received the B.Sc. degree from the University of
Signal Process. Lett., vol. 24, no. 3, pp. 314–318, Mar. 2017.
Manitoba, Winnipeg, MB, Canada, in 1994, the
[40] B. Ramanathan and V. Vittal, “A framework for evaluation of advanced
M.A.Sc. degree from the University of Waterloo,
direct load control with minimum disruption,” IEEE Trans. Power Syst.,
Waterloo, ON, Canada, in 1996, and the Ph.D.
vol. 23, no. 4, pp. 1681–1688, Nov. 2008.
degree from the University of British Columbia
[41] M. Ilic, J. W. Black, and J. L. Watz, “Potential benefits of implementing
load control,” in Proc. IEEE Power Eng. Soc. Win. Meeting, New York, (UBC), Vancouver, BC, Canada, in 2000.
NY, USA, Jan. 2002, pp. 177–182. From 2000 to 2001, he was a Systems Engineer
[42] L. Paull, H. Li, and L. Chang, “A novel domestic electric water heater with PMC-Sierra Inc. (now Microsemi), Sunnyvale,
model for a multi-objective demand side management program,” Elect. CA, USA. He joined the Department of Electrical
Power Syst. Res., vol. 80, no. 12, pp. 1446–1451, Dec. 2010. and Computer Engineering, UBC, in 2002, where
[43] I. Richardson, M. Thomson, and D. Infield, “A high-resolution domestic he is currently a Professor. His current research interests include protocol
building occupancy model for energy demand simulations,” Energy Buil., design, optimization, and resource management of communication networks,
vol. 40, no. 8, pp. 1560–1566, 2008. with applications to wireless networks, smart grid, mobile cloud computing,
[44] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. and Internet of Things.
Cambridge, MA, USA: MIT Press, 1998. Dr. Wong was a recipient of the 2014 UBC Killam Faculty
[45] CVX: MATLAB Software for Disciplined Convex Programming, Version Research Fellowship. He is an Editor of the IEEE T RANSACTIONS
2.1, CVX Res. Inc., San Ramon, CA, USA, Mar. 2017. [Online]. ON C OMMUNICATIONS . He has served as a Guest Editor for the
Available: http://cvxr.com/cvx IEEE J OURNAL ON S ELECTED A REAS IN C OMMUNICATIONS and IEEE
[46] Building Privacy Into Ontario’s Smart Meter Data Management System: W IRELESS C OMMUNICATIONS. He has also served on the Editorial Board
A Control Framework, IESO, Southlake, TX, USA, May 2012. of the IEEE T RANSACTIONS ON V EHICULAR T ECHNOLOGY and the
[47] Sensus. FlexNet AMI System. Accessed: Sep. 20, 2017. Journal of Communications and Networks. He was the Technical Program
[Online]. Available: https://sensus.com/solutions/advanced-metering- Co-Chair of the 2014 IEEE International Conference on Smart Grid
infrastructure-ami/ Communications (SmartGridComm), as well as the Symposium Co-Chair
[48] J. Lyle. (Aug. 2007). Latest Chevy Volt Battery Pack and Generator of IEEE SmartGridComm 2013 and 2017 and IEEE Globecom 2013. He
Details and Clarifications. [Online]. Available: http://gm-volt.com/ is the Chair of the IEEE Vancouver Joint Communications Chapter and has
2007/08/29/latest-chevy-volt-battery-pack-and-generator-details-and- served as the Chair of the IEEE Communications Society Emerging Technical
clarifications/ Sub-Committee on Smart Grid Communications.