Part 8 - Lagrange multipliers

Introduction

In this part we’ll cover the method of Lagrange multipliers. This is a strategy, for finding local minimum/maximums for a function with a constraint.

Let’s define it properly

Setup

Say we have a function, $f(x, y)$ , which we want to find the min/max points. But this function is subject of some constraint of the form, $g(x, y) = k$ .

Recall our rectangular box problem from the last part:

V(x, y, z) = xyz \newline 2zx + 2zy + yx = 12

So if it’s difficult to try to rewrite this using only two variables, we can instead use this method.

Let’s first get a geometrical sense.

Geometrical sense of Lagrange multipliers

If min/max of $f$ , is subject to constraint, $g(x, y) = k$ , which occurs at $(x_0, y_0)$ , then the level curve containing $(x_0, y_0)$ must touch the constraint curve.

If two curves are touching at $(x_0, y_0)$ , in other words they have a common tangent line at $(x_0, y_0)$ .

Which is just equivalent of saying that they have a common perpendicular line at $(x_0, y_0)$ , and what do we know which is orthogonal to the level curve?

The gradient.

Therefore, this means:

\nabla f(x_0, y_0) \|\| \nabla g(x_0, y_0)

If two vectors are parallel, it’s equivalent of saying:

\boxed{\nabla f(x_0, y_0) \|\| \lambda \nabla g(x_0, y_0)}

All of this assuming that $\nabla g \neq 0$

Let’s properly define our method now.

Algorithm (Lagrange multiplier method)

To find minimum/maximum values of function, $f$ , subject to constraint, $g(x, y) = k$ , assuming minimum/maximum points exists and $\nabla g \neq 0$

Find all numbers $x, y$ and $\lambda$ , such that:

\nabla f(x, y) = \lambda \nabla g(x, y) = k

Compute values from step 1, choose the minimum/maximum value(s).

Let’s do an example.

Example 1

Find the extreme values of, $f(x, y) = xy$ on the disc, $x^2 + y^2 \leq 1$ .

Step 1 - Find critical points (as usual):

\nabla f = 0 \newline \langle y, x \rangle = 0

Which means:

\begin{cases} y = 0 \newline x = 0 \end{cases}

So we have a critical point at $(0, 0)$ . Let’s compute the value:

f(0, 0) = \boxed{0}

Step 2 - Find absolute minimum/maximum values on boundary.

Since we have a unit circle, with equation, $x^2 + y^2 = 1$ , we can view this as a constraint.

g(x, y) = k \newline x^2 + y^2 = 1

Let’s first see if all our assumptions hold, since this is a bounded closed set, by the extreme value theorem, absolute values must exists. So we know that they exists.

Let’s check that $\nabla g \neq 0$ :

\nabla g = \langle 2x, 2y \rangle

Which is only $0$ at $(0, 0)$ which isn’t on the boundary of the circle.

So we can try our new method!

\begin{cases} \langle y, x \rangle = \lambda \langle 2x, 2y \rangle \newline x^2 + y^2 = 1 \end{cases}

So we have a vector that must be equal to another vector, this means they share coordinates.

Which means:

\begin{cases} y = 2\lambda x \newline x = 2\lambda y \newline x^2 + y^2 = 1 \end{cases}

substitute and simplify:

\begin{cases} x = 4\lambda^2 x \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases}

\begin{cases} x - 4\lambda^2 x = 0 \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases}

\begin{cases} x(1 - 4\lambda^2) = 0 \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases}

Case 1, $x = 0$ , therefore, $y = 0$ as well, but our constraint is not fulfilled then.

Case 2, $1 - 4\lambda^2 = 0$ :

\lambda = \pm \dfrac{1}{2}

Case 2.1, $\lambda = \dfrac{1}{2}$

\begin{cases} y = x x = y x^2 + y^2 = 1 \end{cases}

2x^2 = 1 \newline x = \pm \dfrac{1}{\sqrt{2}} \newline y = \pm \dfrac{1}{\sqrt{2}} \newline

The constraint holds, so we have found another point, $\left( \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right)$ and $\left( - \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right)$ .

Case 2.2, $\lambda = - \dfrac{1}{2}$

\begin{cases} y = -x x = -y x^2 + y^2 = 1 \end{cases}

2x^2 = 1 \newline x = \pm \dfrac{1}{\sqrt{2}} \newline y = \pm \dfrac{1}{\sqrt{2}} \newline

The constraint holds, $\left( \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right)$ and $\left( - \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right)$ .

Now, we just compute the value of these points:

f\left( \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right) = \dfrac{1}{2} \newline f\left( - \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right) = \dfrac{1}{2} \newline f\left( \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right) = - \dfrac{1}{2} \newline f\left( - \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right) = - \dfrac{1}{2}

So, we have finally found our minimum and maximum values, $- \dfrac{1}{2}$ and $\dfrac{1}{2}$ , respectively.

Remark 1

A little remark, it is important that the constraint has to be of the form $g(x, y) = k$ . For example, our rectangle problem from last time, we can not describe a rectangular box in that form.

So no lagrange multiplier there.

Let’s look at more difficult example.

Example 2

Find the points on the sphere, $x^2 + y^2 + z^2 = 4$ , that are closest and furthest from the point $(3, 1, -1)$ .

So we have to calculate the distance between these points, or in other words:

d(x, y, z) = \sqrt{(x - 3)^2 + (y - 1)^2 + (z + 1)^2}

But this function isn’t really that pleasant, so instead, let’s calculate the distance squared, or:

d(x, y, z) = (x - 3)^2 + (y - 1)^2 + (z + 1)^2

We can work with this function instead, so let’s use our method. Let’s first compute gradients:

\nabla d = \langle 2(x - 3), 2(y - 1), 2(z + 1) \rangle

\nabla g = \langle 2x, 2y, 2z \rangle

So we have:

\begin{cases} \langle 2(x - 3), 2(y - 1), 2(z + 1) \rangle = \lambda \langle 2x, 2y, 2z \rangle \newline x^2 + y^2 + z^2 = 4 \end{cases}

\begin{cases} 2(x - 3) = 2\lambda x \newline 2(y - 1) = 2\lambda y \newline 2(z + 1) = 2\lambda z \newline x^2 + y^2 + z^2 = 4 \end{cases}

\begin{cases} x - 3 = \lambda x \newline y - 1 = \lambda y \newline z + 1 = \lambda z \newline x^2 + y^2 + z^2 = 4 \end{cases}

\begin{cases} x - \lambda x = 3 \newline y - \lambda y = 1 \newline z - \lambda z = -1 \newline x^2 + y^2 + z^2 = 4 \end{cases}

\begin{cases} x(1 - \lambda) = 3 \newline y(1 - \lambda) = 1 \newline z(1 - \lambda) = -1 \newline x^2 + y^2 + z^2 = 4 \end{cases}

\begin{cases} x = \dfrac{3}{1 - \lambda} \newline y = \dfrac{1}{1 - \lambda} \newline z = \dfrac{-1}{1 - \lambda} \newline x^2 + y^2 + z^2 = 4 \end{cases}

Let’s substitute these into our constraint:

\dfrac{9}{(1 - \lambda)^2} + \dfrac{1}{(1 - \lambda)^2} + \dfrac{1}{(1 - \lambda)^2} = 4

\dfrac{11}{(1 - \lambda)^2} = 4

(1 - \lambda)^2 = \dfrac{11}{4}

1 - \lambda = \pm \dfrac{\sqrt{11}}{2}

Compute vales of $x, y, z$ , in case 1, $1 - \lambda = \dfrac{\sqrt{11}}{2}$ :

\begin{cases} x = \dfrac{6}{\sqrt{11}} \newline y = \dfrac{2}{\sqrt{11}} \newline z = -\dfrac{2}{\sqrt{11}} \newline \end{cases}

Compute vales of $x, y, z$ , in case 2, $1 - \lambda = -\dfrac{\sqrt{11}}{2}$ :

\begin{cases} x = -\dfrac{6}{\sqrt{11}} \newline y = -\dfrac{2}{\sqrt{11}} \newline z = \dfrac{2}{\sqrt{11}} \newline \end{cases}

Now, we could plug these values into our $d$ function, however we can intuitively see which point is closest and furthest.

Case 1 is the closest and case 2 is the furthest.