positive definite

A symmetric matrix with real entries is positive definite if

where is a quadratic form. We write

Usage

For unconstrained optimization, the main goal is to find local maxima. In a univariate, twice-differentiable function, the procedure is 1) find critical points by setting the gradient equal to zero and solving, then 2) perform a second derivative test to figure out the concavity of the function and categorize the critical points found.

For a complicated multivariate function that is twice-differentiable, the process is similar. Notice that when we use second-order Taylor polynomial to approximate the function , its quadratic term (last term) tells us about concavity, specifically through the Hessian matrix (a matrix of second derivatives). This quadratic term has the form .

• is the gradient of the function

Link to original

The quadratic term, when multiplying out the form , looks like a quadratic form . The Hessian matrix is the matrix in this form.

Transclude of quadratic-form#theorem

If the Hessian matrix is positive definite at a point , then the multivariate function is convex near  (i.e. has a local minimum near ). A simpler form of this point is second derivative test or second partial derivative test.

Characteristics

• definite matrix is always invertible (NOT THE OTHER WAY AROUND)
• symmetric definite matrix is always diagonalizable (the Spectral Theorem)

How to check for conditions

Assume is real-valued.

• is congruent with a diagonal matrix with positive real entries
• All eigenvalues of square matrix are real and positive
  • real-valued diagonal matrix with positive diagonal entries is positive definite since all eigenvalues are diagonal entries, making them real and positive
• Sylvester’s criterion: All leading principle minors of a symmetric matrix are positive.

We can ALWAYS turn a square matrix into symmetric matrix.

Special case: 2x2 matrix

For a symmetric matrix

the conditions sufficient to guarantee positive definiteness of is and

Proof: Express the quadratic form as a sum of two squares

Rewrite the quadratic form by trying to complete the square:

Now, we aim to express as a perfect square. Add and subtract the square of half the coefficient of

Simplify:

Substitute back:

Assuming and to ensure a valid sum of squares representation:

This is the special case of Sylvester’s criterion, where and

Sylvester Criterion

Sylvester's criterion

The Sylvester’s criterion can be used to check whether a symmetric matrix is positive definite

A quadratic form ( symmetric matrix) is positive definite if the leading principal minors of are positive. Leading principal minors are the determinants of the top-left square submatrices of a given matrix Image credit: Princeton lecture

Code

Sagemath

def is_positive_definite(A):
    """Apply Sylvester's Criterion to check if a matrix is positive definite. 
    
    Inputs:
    - A: A symmetric matrix
    
    Output:
    - True if the matrix is positive definite, False otherwise
    """
    # Ensure matrix is symmetric
    if A.is_symmetric():
	    print(f"Matrix is not symmetric")
	    return False
    
    n = A.nrows()
    
    # Loop through submatrices
    for k in range(1, n+1):
        # Extract the top-left k x k submatrix
        submatrix = A[:k, :k]
        determinant = submatrix.det()
        
        # If any determinant is non-positive, the matrix is not positive definite
        if determinant <= 0:
            print(f"Submatrix {k}x{k}: \n{submatrix}\nDeterminant: {determinant}")
            return False
    
    return True
 
A = Matrix(QQ, [
    [4, 12, -16],
    [12, 37, -43],
    [-16, -43, 98]
])
 
is_positive_definite()

Link to original

Plotting

For a two-variable equation where is the quadratic form written as and is definite matrix, the eigenvectors are orthogonal along the major and minor axis of the ellipse in the implicit plot

• green eigenvector is associated with the smaller eigenvalue
• red eigenvector is associated with the larger eigenvalue

var('x,y')
x_vec = vector([x,y])
A = Matrix([[5, 2], [2, 2]])
vec1 = A.eigenvectors_right()[0][1][0]
vec2 = A.eigenvectors_right()[1][1][0]
 
f1 = x_vec * A * x_vec
a = 2
 
plot1 = implicit_plot(f1==1, (x, -a, a), (y, -a, a)) + arrow((0,0), vec1, color='red') + arrow((0,0), vec2, color='green')
plot1.show(aspect_ratio=1)