> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/argoproj/argo-cd/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Common issues, debugging tools, and solutions for Argo CD operations

This guide covers common troubleshooting scenarios and tools for diagnosing and resolving Argo CD issues.

## Troubleshooting Tools

Argo CD provides `argocd admin` subcommands to validate settings and troubleshoot connectivity issues.

### Settings Validation

Validate Argo CD configuration before applying to production:

```bash theme={null}
argocd admin settings validate
```

This command performs basic validation of:

* ConfigMap settings (argocd-cm)
* RBAC policies (argocd-rbac-cm)
* Resource customizations
* Repository credentials

## Common Issues

### Application Sync Failures

<AccordionGroup>
  <Accordion title="Application stuck in 'OutOfSync' state">
    **Symptoms**: Application shows OutOfSync but sync operation fails or doesn't start.

    **Diagnosis**:

    ```bash theme={null}
    # Check application status
    argocd app get <app-name>

    # View detailed sync status
    kubectl get application <app-name> -n argocd -o yaml

    # Check application controller logs
    kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller
    ```

    **Common causes**:

    * Invalid manifests in Git repository
    * Resource quota exceeded in target cluster
    * RBAC permissions preventing resource creation
    * Cluster connectivity issues

    **Solutions**:

    ```bash theme={null}
    # Validate manifests locally
    kubectl apply --dry-run=client -f manifest.yaml

    # Check resource quotas
    kubectl describe resourcequota -n <namespace>

    # Test cluster connectivity
    argocd cluster get <cluster-url>

    # Force refresh and sync
    argocd app sync <app-name> --force
    ```
  </Accordion>

  <Accordion title="Context deadline exceeded errors">
    **Symptoms**: Application reconciliation fails with `Context deadline exceeded`.

    **Root cause**: Manifest generation is taking too long, exceeding the controller timeout.

    **Solutions**:

    <Steps>
      <Step title="Increase repo server timeout">
        ```yaml theme={null}
        containers:
        - name: argocd-application-controller
          command:
          - argocd-application-controller
          - --repo-server-timeout-seconds=300  # Increase from default 60s
        ```
      </Step>

      <Step title="Scale repo server">
        ```bash theme={null}
        kubectl scale deployment argocd-repo-server -n argocd --replicas=3
        ```
      </Step>

      <Step title="Optimize repository">
        * Use shallow clones for large repositories
        * Enable manifest path annotations for monorepos
        * Reduce parallelism limit if resource-constrained
      </Step>
    </Steps>
  </Accordion>

  <Accordion title="Sync operation permissions errors">
    **Symptoms**: Sync fails with "insufficient permissions" or RBAC errors.

    **Diagnosis**:

    ```bash theme={null}
    # Check AppProject permissions
    kubectl get appproject <project-name> -n argocd -o yaml

    # Verify cluster RBAC
    kubectl auth can-i create deployment -n <namespace> \
      --as=system:serviceaccount:argocd:argocd-application-controller
    ```

    **Solution**: Update AppProject to allow resources:

    ```yaml theme={null}
    apiVersion: argoproj.io/v1alpha1
    kind: AppProject
    metadata:
      name: my-project
    spec:
      clusterResourceWhitelist:
      - group: '*'
        kind: '*'
      destinations:
      - namespace: '*'
        server: '*'
    ```
  </Accordion>
</AccordionGroup>

### Git Repository Issues

<AccordionGroup>
  <Accordion title="Failed to fetch repository">
    **Symptoms**: Applications can't connect to Git repositories.

    **Diagnosis**:

    ```bash theme={null}
    # Test repository connection
    argocd repo get <repo-url>

    # Check repo server logs
    kubectl logs -n argocd -l app.kubernetes.io/name=argocd-repo-server
    ```

    **Common causes**:

    * Invalid credentials
    * Network connectivity issues
    * SSH key not configured
    * Certificate validation failures

    **Solutions**:

    For HTTPS repositories:

    ```bash theme={null}
    # Update credentials
    argocd repo add https://github.com/org/repo \
      --username <username> \
      --password <token>

    # Skip TLS verification (not recommended for production)
    argocd repo add https://github.com/org/repo \
      --insecure-skip-server-verification
    ```

    For SSH repositories:

    ```bash theme={null}
    # Add SSH private key
    argocd repo add git@github.com:org/repo.git \
      --ssh-private-key-path ~/.ssh/id_rsa

    # Add SSH known hosts
    kubectl edit configmap argocd-ssh-known-hosts-cm -n argocd
    ```
  </Accordion>

  <Accordion title="Git ls-remote failures">
    **Symptoms**: Intermittent failures resolving Git references (branches, tags).

    **Solution**: Increase retry count for Git operations:

    ```yaml theme={null}
    containers:
    - name: argocd-repo-server
      env:
      - name: ARGOCD_GIT_ATTEMPTS_COUNT
        value: "5"  # Retry failed Git operations
    ```
  </Accordion>
</AccordionGroup>

### Cluster Connectivity Issues

<AccordionGroup>
  <Accordion title="Cluster connection failed">
    **Symptoms**: Managed cluster shows as "Unreachable" or "Unknown" in UI.

    **Diagnosis**:

    <Steps>
      <Step title="SSH into application controller">
        ```bash theme={null}
        kubectl exec -n argocd -it \
          $(kubectl get pods -n argocd \
            -l app.kubernetes.io/name=argocd-application-controller \
            -o jsonpath='{.items[0].metadata.name}') -- bash
        ```
      </Step>

      <Step title="Export kubeconfig from cluster secret">
        ```bash theme={null}
        argocd admin cluster kubeconfig https://<api-server-url> \
          /tmp/kubeconfig --namespace argocd
        ```
      </Step>

      <Step title="Test connectivity">
        ```bash theme={null}
        export KUBECONFIG=/tmp/kubeconfig
        kubectl get pods -v 9
        ```
      </Step>
    </Steps>

    **Common issues**:

    * Expired certificates
    * Invalid bearer tokens
    * Network policies blocking traffic
    * API server URL changed

    **Solution**: Update cluster credentials:

    ```bash theme={null}
    argocd cluster add <context-name> --name <cluster-name> --upsert
    ```
  </Accordion>
</AccordionGroup>

### Resource Customization Issues

<AccordionGroup>
  <Accordion title="Test custom health checks">
    Custom health checks can be tested before applying to production:

    ```bash theme={null}
    argocd admin settings resource-overrides health \
      ./deployment.yaml \
      --argocd-cm-path ./argocd-cm.yaml
    ```

    Example health check (Lua):

    ```yaml theme={null}
    resource.customizations: |
      argoproj.io/Rollout:
        health.lua: |
          hs = {}
          if obj.status ~= nil then
            if obj.status.phase == "Healthy" then
              hs.status = "Healthy"
              hs.message = "Rollout is healthy"
              return hs
            end
          end
          hs.status = "Progressing"
          hs.message = "Waiting for rollout to complete"
          return hs
    ```
  </Accordion>

  <Accordion title="Test diff customizations">
    Test ignore differences configurations:

    ```bash theme={null}
    argocd admin settings resource-overrides ignore-differences \
      ./deployment.yaml \
      --argocd-cm-path ./argocd-cm.yaml
    ```

    Shows which fields will be ignored during diff operations.
  </Accordion>

  <Accordion title="Test resource actions">
    Execute custom resource actions:

    ```bash theme={null}
    # List available actions
    argocd admin settings resource-overrides list-actions \
      /tmp/deployment.yaml \
      --argocd-cm-path /tmp/argocd-cm.yaml

    # Run action
    argocd admin settings resource-overrides run-action \
      /tmp/deployment.yaml restart \
      --argocd-cm-path /tmp/argocd-cm.yaml
    ```
  </Accordion>
</AccordionGroup>

### Performance Issues

<AccordionGroup>
  <Accordion title="Slow reconciliation times">
    **Symptoms**: Applications take a long time to reconcile and sync.

    **Diagnosis**:

    ```promql theme={null}
    # Check reconciliation duration (Prometheus query)
    histogram_quantile(0.95, 
      rate(argocd_app_reconcile_bucket[5m])
    )

    # Check for high K8s API requests
    rate(argocd_app_k8s_request_total[5m])
    ```

    **Solutions**:

    <Steps>
      <Step title="Increase controller processors">
        ```yaml theme={null}
        containers:
        - name: argocd-application-controller
          command:
          - argocd-application-controller
          - --status-processors=50
          - --operation-processors=25
        ```
      </Step>

      <Step title="Enable controller sharding">
        ```yaml theme={null}
        apiVersion: apps/v1
        kind: StatefulSet
        metadata:
          name: argocd-application-controller
        spec:
          replicas: 2
          template:
            spec:
              containers:
              - name: argocd-application-controller
                env:
                - name: ARGOCD_CONTROLLER_REPLICAS
                  value: "2"
        ```
      </Step>

      <Step title="Optimize monorepo performance">
        Use manifest path annotations:

        ```yaml theme={null}
        metadata:
          annotations:
            argocd.argoproj.io/manifest-generate-paths: .
        ```
      </Step>
    </Steps>
  </Accordion>

  <Accordion title="High memory usage">
    **Symptoms**: Argo CD components OOMKilled or using excessive memory.

    **Common causes**:

    * Too many cached resources
    * Large repositories
    * Too many applications per controller

    **Solutions**:

    For repo server:

    ```yaml theme={null}
    spec:
      template:
        spec:
          containers:
          - name: argocd-repo-server
            resources:
              requests:
                memory: 1Gi
              limits:
                memory: 2Gi
            env:
            - name: ARGOCD_EXEC_TIMEOUT
              value: "180s"
    ```

    For application controller:

    ```yaml theme={null}
    env:
    - name: ARGOCD_CONTROLLER_REPLICAS
      value: "3"  # Shard applications across replicas
    ```

    Mount persistent volume for repo server:

    ```yaml theme={null}
    volumeMounts:
    - mountPath: /tmp
      name: tmp-dir
    volumes:
    - name: tmp-dir
      persistentVolumeClaim:
        claimName: argocd-repo-server-pvc
    ```
  </Accordion>

  <Accordion title="Repository contention">
    **Symptoms**: High `argocd_repo_pending_request_total` metric.

    **Cause**: Multiple applications in same repository causing sequential processing.

    **Solutions**:

    * Enable concurrent processing (create `.argocd-allow-concurrency` file)
    * Scale repo server horizontally
    * Split applications into separate repositories
    * Use manifest path annotations
  </Accordion>
</AccordionGroup>

### Application Health Issues

<AccordionGroup>
  <Accordion title="Application showing 'Unknown' health">
    **Cause**: No health check defined for resource type.

    **Solution**: Add custom health check:

    ```yaml theme={null}
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: argocd-cm
      namespace: argocd
    data:
      resource.customizations: |
        my.custom.resource/MyKind:
          health.lua: |
            hs = {}
            if obj.status ~= nil and obj.status.ready then
              hs.status = "Healthy"
            else
              hs.status = "Progressing"
            end
            return hs
    ```
  </Accordion>

  <Accordion title="Application stuck in 'Progressing' state">
    **Diagnosis**:

    ```bash theme={null}
    # Check resource status
    argocd app get <app-name> --show-operation

    # Check individual resource health
    kubectl get <resource> -n <namespace>
    kubectl describe <resource> <name> -n <namespace>
    ```

    **Common causes**:

    * Pods stuck in ImagePullBackOff
    * Insufficient resources (CPU/memory)
    * Failing health checks
    * Init containers not completing
  </Accordion>
</AccordionGroup>

## Debugging Commands

### Log Collection

```bash theme={null}
# Application controller logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller --tail=100

# Repo server logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-repo-server --tail=100

# API server logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-server --tail=100

# Follow logs in real-time
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller -f
```

### Resource Inspection

```bash theme={null}
# Get application details
argocd app get <app-name>

# Get application as YAML
kubectl get application <app-name> -n argocd -o yaml

# Get application history
argocd app history <app-name>

# Get application events
kubectl get events -n argocd --field-selector involvedObject.name=<app-name>

# Get all applications
argocd app list

# Get application resources
argocd app resources <app-name>
```

### Configuration Verification

```bash theme={null}
# Check ConfigMaps
kubectl get configmap -n argocd argocd-cm -o yaml
kubectl get configmap -n argocd argocd-rbac-cm -o yaml
kubectl get configmap -n argocd argocd-cmd-params-cm -o yaml

# Check secrets
kubectl get secret -n argocd argocd-secret -o yaml

# Validate settings
argocd admin settings validate
```

## Getting Help

<CardGroup cols={2}>
  <Card title="GitHub Issues" icon="github">
    Search existing issues or create new ones:
    [argoproj/argo-cd](https://github.com/argoproj/argo-cd/issues)
  </Card>

  <Card title="Slack Community" icon="slack">
    Join the Argo CD community:
    [CNCF Slack #argo-cd](https://cloud-native.slack.com/)
  </Card>

  <Card title="Documentation" icon="book">
    Official Argo CD docs:
    [argo-cd.readthedocs.io](https://argo-cd.readthedocs.io/)
  </Card>

  <Card title="Stack Overflow" icon="stack-overflow">
    Ask questions with the `argocd` tag:
    [stackoverflow.com](https://stackoverflow.com/questions/tagged/argocd)
  </Card>
</CardGroup>

## Related Resources

* [Monitoring and Metrics](/operations/monitoring)
* [High Availability Setup](/operations/high-availability)
* [Disaster Recovery](/operations/disaster-recovery)