04 METHOD OF LAGRANGE MULTIPLIERS

CIVL 4440, Fall 2015

Introduction
*
*

Method of Lagrange multipliers is based on calculus.

These points are then compared to determine which of

them is the global optimum.

The method was originally developed for constrained

problems with equality constraints.

If the problem contains inequality constraints, they will

have to be converted to equality constraints first.

It uses calculus to identify stationary points of an objective

function.

saddle point
http://www.math.okstate.edu/~yq
wang/teaching/math4553_spring
09/demo/minmax1.png

maximum

Stationary
Points

inflection

minimum

Problem with Equality Constraints

Max Z = F(X1, X2, X3, , Xn)
s.t.
g1(X1,
g2(X1,
g3(X1,
:
:
gm(X1,

X2, X3, , Xn) = b1

X2, X3, , Xn) = b2
X2, X3, , Xn) = b3

To apply the method of Lagrange

multipliers, the objective function,
F and constraints, g1 to gm should
be continuous and differentiable.
The problem should also be
bounded, i.e., the optimal value
of Z should not be infinity or
negative infinity.
By definition, b1 to bm are
constants.

X2, X3, , Xn) = bm

Why are these conditions
necessary?

The Lagrangian
The first step is to convert the problem from a constrained one
to an unconstrained one.
This is done by multiplying each constraint with a Lagrange
multiplier, i and subtracting the result of that from the objective
function.
The modified objective function is called the Lagrangian. For a
maximization problem:

L = F 1(g1 b1) 2(g2 b2) m(gm bm)

L=F

(gi bi)

The multiplier for constraint i, i can be

thought of as the unit penalty incurred
when that constraint is violated.

Identifying Stationary Points

Stationary points of the Lagrangian, L can now be determined
by setting the following n + m partial derivatives to zero.

dL/dX1 = 0
dL/dX2 = 0
:
:
dL/dXn = 0

dL/d1 = 0
dL/d2 = 0
:
:
dL/dm = 0

X1, X2, , Xn are decision variables. 1, 2, , n are new decisions

variables that are introduced with the construction of the Lagrangian.

Identifying Stationary Points

Solving the partial derivatives as a series of simultaneous
equations gives a set of stationary points.
Each stationary point refers to a unique combination X1,
X2, , Xn and 1, 2, , n values.
One or more of the stationary points may be infeasible, i.e.,
they are in violation of at least one constraint.
Once determined, those that are feasible are compared and
the one giving the maximum objective function value is the
global optimum.

Identifying Stationary Points

For minimization problems, the procedure is the same
except that the Lagrangian, L is defined as:
L = F + 1(g1 b1) + 2(g2 b2) + + m(gm bm)
L=F+

(gi bi)

It is now a + instead of a .

And the global optimum is the stationary point that gives

the least (instead of the maximum) objective function value.
Does it really matter whether we define L with negatives
or positives in front of the i?

Example of a Problem with Equality Constraints

Consider the following constrained optimization problem:

MAX Z = X3 4X + Y2 + 5
s.t.
X+Y=4
X=0
To apply the method of Lagrange multipliers, modify the objective
function to the following to convert the problem from a constrained
problem to an unconstrained problem.

L = X3 4X + Y2 + 5 1 (X + Y 4) 2 X

Example of a Problem with Equality Constraints

To find the stationary points of L:
dL/dX = 3X2 4 1 - 2 = 0

(1)

dL/dY = 2Y 1 = 0

(2)

dL/d1 = X Y + 4 = 0

(3)

dL/d2 = X = 0

(4)

Example of a Problem with Equality Constraints

Substituting (4) into (3) gives:

X = 0, Y = 4
Substituting that into (2) gives:

1 = 8
And that with X and Y into (1) gives:

2 = -12
Because of the equality constraints, it so happens that this
particular problem has just one stationary point. Therefore, there
is no need for a comparison of stationary points to be done, and
the stationary point identified is the optimum.

Example of a Problem with Equality Constraints

Interpretation of Lagrange multipliers, 1 = 8 and 2 = -12
Principle 1:
When the Lagrange multiplier associated with a constraint is non-zero, it means the
constraint is binding. And when the multiplier is zero, it means the constraint is nonbinding. Both 1 and 2 are non-zero. Therefore, both constraints are binding.
Principle 2:

For a maximization problem, an improvement means an

increase, and for a minimization problem, a decrease.

The Lagrange multiplier represents the improvement in the objective function, Z with
respect to a change in the RHS of the corresponding constraint.
1 = 8 means that if the RHS of the first constraint were to be increased by 1 unit,
the objective function value at optimum will increase (since this is a maximization
problem) by 8 units.
2 = -12 means that if the RHS of the second constraint were to be increased by a 1
unit, the objective function value at optimum will decrease by 12 units.

Example of a Problem with Equality Constraints

Interpretation of Lagrange multipliers, 1 = 8 and 2 = -12
Let us test out Principle 2 by solving and resolving the problem for different values of the RHS
of the constraints. To save time, we will use Solver in Excel.
Constraint 1, 1 = 8
RHS of Constraint

Change in RHS

Objective, Z

Change in Z

Z / RHS

Change in RHS

Objective, Z

Change in Z

Z / RHS

Constraint 2, 2 = -12
RHS of Constraint

Example of a Problem with Equality Constraints

Does it make sense now why it
is important how the signs in
front of the Lagrangian
multipliers in the
Lagrangian, L are defined
depending on whether the
problem is a maximization or
minimization problem?

Example of a Problem with Inequality Constraints

Consider the following constrained optimization problem:
MIN Z = X3 4X + Y2 + 5
s.t.
X + Y = 4
X >= 0
Y <= 10

To apply the method of Lagrange multipliers, the constraints are first converted to equality
constraints. To do that, slack and surplus variables are added at appropriate places.
X + Y = 4
X >= 0
---> X S12 = 0
Y <= 10 ---> Y + S22 = 10

The objective function can now be modified to give Lagrangian:

L = X3 4X + Y2 + 5 + 1(X + Y 4) + 2(X S12) + 3(Y + S22 10)

Example of a Problem with Inequality Constraints

To find the stationary points of the Lagrangian:
dL / dX = 3X2 4 + 1 + 2 = 0

(5)

dL / dY = 2Y + 1 + 3 = 0

(6)

dL / d1 = X + Y 4 = 0

(7)

dL / d2 = X S12 = 0

(8)

dL / d3 = Y + S22 10 = 0

(9)

dL / dS1 = 2 2 S1 = 0

--->

2 = 0 or S1 = 0

dL / dS2 = 2 3 S2 = 0

--->

3 = 0 or S2 = 0

Example of a Problem with Inequality Constraints

Possible solution 1: 2 = 0 and 3 = 0
Eqn (6): 1 = 2Y
Eqn (5): 3X2 4 2Y = 0 . (10)
Eqn (10) + 2 Eqn (7):
--->
--->

3X2 + 2X 12 = 0
X = 1.694 or X = 2.361

This gives the first stationary point:

--->
--->
--->
--->

X = 1.694, Y = 2.306
1 = 4.612, 2 = 0, 3 = 0
S12 = 1.694, S22 = 7.694
Z = 8.403

Example of a Problem with Inequality Constraints

Possible solution 2: 2 = 0 and S2 = 0

Eqn (9): Y = 10
Eqn (7): X = 6
Eqn (8): S12 = 6
This gives the second stationary point, but
this point is infeasible since S12 is negative.

Example of a Problem with Inequality Constraints

Possible solution 3: S1 = 0 and 3 = 0

Eqn (8): X = 0
Eqn (7): Y = 4
This gives the third stationary point:
--->
X = 0, Y = 4
--->
1 = 8, 2 = 12, 3 = 0
--->
S12 = 0, S22 = 6
--->
Z = 21

Example of a Problem with Inequality Constraints

Possible solution 4: S1 = 0 and S2 = 0

Eqn (8): X = 0
Eqn (9): Y = 10
This gives the fourth stationary point, but this
point is infeasible since it violates Eqn (7).

Example of a Problem with Inequality Constraints

10

Y <= 1 0

There are four

stationary points.
Before going further,
let us have a graphical
look at where those
stationary points lie.
What pattern do you
observe?

4
8

X >= 0
6

3
4

Z = 20
Z = 15

Z = 8.4

1
0

-6

-4

-2

0
X

By comparing the four

stationary points,
which is the optimum?
Which constraints are
binding or nonbinding? Graphically?
From the Lagrange
multipliers? From the
slack and surplus
variables?

Example of a Problem with Inequality Constraints

Interpretation of Lagrange multipliers, 1 = 4.612, 2 = 0, 3 = 0
As before, let us test the Lagrange multipliers by solving the problem multiple times for
different values of the RHS of the constraints.
RHS of Constraint

Change in RHS

Constraint 1, 1 = 4.612

Constraint 2, 2 = 0

Constraint 3, 3 = 0

Objective, Z

Change in Z

Z / RHS

In-Class Exercise
There are two polluters, the first discharging R1 kg of
waste into the atmosphere everyday and the second R2 kg
of waste. The profits of the polluters increase with R1
and R2:
Profit of polluter 1 (million $) = 10 R12
Profit of polluter 2 (million $) = 5 R22
Use the method of Lagrange multipliers to find the
values of R1 and R2 that give the maximum total profit.
Note that regulations restrict the sum of R1 and R2 to be
no greater than 5 kg.
Based on your results, estimate the increase in profit
you would expect if the restriction on the sum of R1 and
R2 were to be made less strict and increased by 0.1 kg.