Logging operator troubleshooting

Logo

The following tips and commands can help you to troubleshoot your Logging operator installation.

First things to do

  1. Check that the necessary CRDs are installed. Issue the following command: kubectl get crd The output should include the following CRDs:

    clusterflows.logging.banzaicloud.io     2019-12-05T15:11:48Z
    clusteroutputs.logging.banzaicloud.io   2019-12-05T15:11:48Z
    flows.logging.banzaicloud.io            2019-12-05T15:11:48Z
    loggings.logging.banzaicloud.io         2019-12-05T15:11:48Z
    outputs.logging.banzaicloud.io          2019-12-05T15:11:48Z
    
  2. Verify that the Logging operator pod is running. Issue the following command: kubectl get pods |grep logging-operator The output should include the a running pod, for example:

    NAME                                          READY   STATUS      RESTARTS   AGE
    logging-demo-log-generator-6448d45cd9-z7zk8   1/1     Running     0          24m
    
  3. Check the status of your resources. Beginning with Logging Operator 3.8, all custom resources have a Status and a Problems field. In a healthy system, the Problems field of the resources is empty, for example:

    kubectl get clusteroutput -A
    

    Sample output:

    NAMESPACE   NAME      ACTIVE   PROBLEMS
    default     nullout   true
    

    The ACTIVE column indicates that the ClusterOutput has successfully passed the configcheck and presented it in the current fluentd configuration. When no errors are reported the PROBLEMS column is empty.

    Take a look at another example, in which we have an incorrect ClusterFlow.

    kubectl get clusterflow -o wide
    

    Sample output:

    NAME      ACTIVE   PROBLEMS
    all-log   true
    nullout   false    1
    

    You can see that the nullout Clusterflow is inactive and there is 1 problem with the configuration. To display the problem, check the status field of the object, for example:

    kubectl get clusterflow nullout -o=jsonpath='{.status}' | jq
    

    Sample output:

    {
    "active": false,
    "problems": [
        "dangling global output reference: nullout2"
    ],
    "problemsCount": 1
    }
    

After that, check the following sections for further tips.

Getting Support

If you encounter any problems that the documentation does not address, file an issue or talk to us on Discord or Slack.

Commercial support is also available for the Logging operator.

Before asking for help, prepare the following information to make troubleshooting faster:

Do not forget to remove any sensitive information (for example, passwords and private keys) before sharing.