Custom Health Checks

Define custom health checks for Kubernetes resources in Argo CD

Overview

Argo CD provides built-in health checks for standard Kubernetes resources. For custom resources or specialized health logic, you can write custom health checks in Lua.

Built-in Health Checks

Argo CD includes health checks for:

• Deployments, ReplicaSets, StatefulSets, DaemonSets
• Services, Ingresses
• Jobs, CronJobs
• PersistentVolumeClaims
• And more…

When to Use Custom Health Checks

Custom Resources

Add health checks for CRDs that Argo CD doesn’t recognize

Fix Known Issues

Override buggy health checks for resources stuck in Progressing state

Complex Logic

Implement sophisticated health determination based on resource conditions

Domain-Specific

Check health based on your application’s specific requirements

Health Status Values

Custom health checks must return one of these statuses:

Healthy

Resource is healthy and operating normally.

Progressing

Resource is not healthy yet but making progress (default for most resources).

Degraded

Resource is degraded and not functioning properly.

Suspended

Resource is suspended and waiting for external action (e.g., paused Deployment).

Configuration Methods

Method 1: ConfigMap Configuration

Define health checks in the argocd-cm ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  resource.customizations.health.cert-manager.io_Certificate: |
    hs = {}
    if obj.status ~= nil then
      if obj.status.conditions ~= nil then
        for i, condition in ipairs(obj.status.conditions) do
          if condition.type == "Ready" and condition.status == "False" then
            hs.status = "Degraded"
            hs.message = condition.message
            return hs
          end
          if condition.type == "Ready" and condition.status == "True" then
            hs.status = "Healthy"
            hs.message = condition.message
            return hs
          end
        end
      end
    end
    
    hs.status = "Progressing"
    hs.message = "Waiting for certificate"
    return hs

The customization key format is resource.customizations.health.<apiGroup>_<kind>.

Method 2: Contribute Built-in Check

Contribute health checks to the Argo CD repository:

argo-cd/
└── resource_customizations/
    └── your.crd.group.io/           # CRD group
        └── MyKind/                   # Resource kind
            ├── health.lua            # Health check script
            ├── health_test.yaml      # Test cases
            └── testdata/             # Test resource definitions

1. Create Directory Structure

   Follow the structure above in your fork of the Argo CD repository.

2. Write health.lua

   Implement your health check logic.

3. Add Tests

   Create health_test.yaml with test cases:

   tests:
   - healthStatus:
       status: Healthy
       message: Certificate is ready
     inputPath: testdata/healthy-cert.yaml
   - healthStatus:
       status: Degraded
       message: Certificate validation failed
     inputPath: testdata/degraded-cert.yaml
   
4. Run Tests

   go test -v ./util/lua/
   
5. Submit PR

   Submit your contribution following the contributing guide.

Writing Health Check Scripts

Basic Example

hs = {}
hs.status = "Progressing"
hs.message = ""

if obj.status ~= nil then
  if obj.status.phase == "Running" then
    hs.status = "Healthy"
    hs.message = "Resource is running"
  elseif obj.status.phase == "Failed" then
    hs.status = "Degraded"
    hs.message = "Resource has failed"
  end
end

return hs

Condition-Based Check

hs = {}
hs.status = "Progressing"
hs.message = "Waiting for resource to be ready"

if obj.status ~= nil and obj.status.conditions ~= nil then
  for i, condition in ipairs(obj.status.conditions) do
    if condition.type == "Ready" then
      if condition.status == "True" then
        hs.status = "Healthy"
        hs.message = "Resource is ready"
      elseif condition.status == "False" then
        hs.status = "Degraded" 
        hs.message = condition.message or "Resource is not ready"
      end
      return hs
    end
  end
end

return hs

Multiple Conditions

hs = {}

if obj.status ~= nil and obj.status.conditions ~= nil then
  local ready = false
  local available = false
  
  for i, condition in ipairs(obj.status.conditions) do
    if condition.type == "Ready" and condition.status == "True" then
      ready = true
    end
    if condition.type == "Available" and condition.status == "True" then
      available = true
    end
  end
  
  if ready and available then
    hs.status = "Healthy"
    hs.message = "All conditions met"
  else
    hs.status = "Progressing"
    hs.message = "Waiting for conditions"
  end
else
  hs.status = "Progressing"
  hs.message = "No status available"
end

return hs

Wildcard Support

ConfigMap Wildcards

Match multiple resources with wildcards:

resource.customizations: |
  ec2.aws.crossplane.io/*:
    health.lua: |
      # Health check for all EC2 resources
      hs = {}
      if obj.status ~= nil and obj.status.conditions ~= nil then
        for i, condition in ipairs(obj.status.conditions) do
          if condition.type == "Ready" and condition.status == "True" then
            hs.status = "Healthy"
            return hs
          end
        end
      end
      hs.status = "Progressing"
      return hs

Wildcards only work with the resource.customizations key format. The resource.customizations.health.<group>_<kind> format doesn’t support wildcards.

Built-in Wildcards

For contributed health checks, use _ as wildcard in directory names:

resource_customizations/
└── _.aws.crossplane.io/    # Matches *.aws.crossplane.io
    └── _/                   # Matches any kind
        └── health.lua

Avoid Massive Scripts

Don’t write one huge script for many resources. Keep scripts resource-specific for maintainability.

Advanced Configuration

Enable Standard Lua Libraries

Standard Lua libraries are disabled by default for security. Enable only for trusted scripts.

data:
  resource.customizations.useOpenLibs.cert-manager.io_Certificate: "true"
  resource.customizations.health.cert-manager.io_Certificate: |
    # Lua standard libraries are enabled for this script
    hs = {}
    # ... your health check logic
    return hs

Restore Application Health Check

The health check for argoproj.io/Application was removed in v1.8. Restore it for app-of-apps patterns:

resource.customizations.health.argoproj.io_Application: |
  hs = {}
  hs.status = "Progressing"
  hs.message = ""
  if obj.status ~= nil then
    if obj.status.health ~= nil then
      hs.status = obj.status.health.status
      if obj.status.health.message ~= nil then
        hs.message = obj.status.health.message
      end
    end
  end
  return hs

Health Inheritance

Important Concept

Resource health is NOT inherited from child resources—it’s calculated only from the resource’s own status.

The Deployment is healthy even though one ReplicaSet is unhealthy, because health is based only on the Deployment’s own status fields.

Application-Level Health

Argo CD Application health is the worst health of its immediate child resources:

Priority (most to least healthy): Healthy → Suspended → Progressing → Missing → Degraded → Unknown

Example: If an App has a Missing resource and a Degraded resource, the App’s health will be Degraded.

Ignoring Resource Health

Exclude specific resources from affecting Application health:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    argocd.argoproj.io/ignore-healthcheck: "true"

Overriding Built-in Checks

You can override Go-based built-in health checks with Lua scripts. This is useful for:

• Customizing behavior for your environment
• Working around bugs in built-in checks
• Adding more sophisticated logic

Built-in health checks exist for:

• PersistentVolumeClaim, Pod, Service
• APIService, DaemonSet, Deployment, ReplicaSet, StatefulSet
• Workflow (Argo Workflows)
• HorizontalPodAutoscaler
• Job, Ingress

Testing Health Checks

Create comprehensive test coverage:

# health_test.yaml
tests:
- healthStatus:
    status: Healthy
    message: Resource is ready
  inputPath: testdata/healthy.yaml
  
- healthStatus:
    status: Progressing
    message: Waiting for certificate
  inputPath: testdata/progressing.yaml
  
- healthStatus:
    status: Degraded
    message: Certificate validation failed
  inputPath: testdata/degraded.yaml
  
- healthStatus:
    status: Suspended
    message: Certificate renewal paused
  inputPath: testdata/suspended.yaml

Best Practices

Return Early

Return as soon as you determine the status—don’t process unnecessarily.

if condition.status == "True" then
  hs.status = "Healthy"
  return hs  -- Exit immediately
end

Provide Meaningful Messages

Include helpful context in the message:

hs.message = condition.message or "Certificate validation in progress"

Handle Missing Fields

Always check if fields exist before accessing:

if obj.status ~= nil and obj.status.conditions ~= nil then
  -- Process conditions
end

Default to Progressing

When in doubt, return Progressing rather than Healthy:

hs.status = "Progressing"  -- Safe default

Example: Crossplane Resource

hs = {}
hs.status = "Progressing"
hs.message = ""

if obj.status ~= nil then
  if obj.status.conditions ~= nil then
    for i, condition in ipairs(obj.status.conditions) do
      if condition.type == "Ready" then
        hs.message = condition.message
        if condition.status == "True" then
          hs.status = "Healthy"
        else
          hs.status = "Degraded"
        end
        return hs
      end
      if condition.type == "Synced" and condition.status == "False" then
        hs.status = "Degraded"
        hs.message = condition.message
        return hs
      end
    end
  end
end

return hs

Next Steps

Resource Actions

Define custom actions for resources

Config Management Plugins

Create custom config management plugins

Example Health Checks

Browse built-in health checks

Lua Documentation

Learn Lua scripting