Validating Helm charts using the Datree GitHub Action

In this guide, I will show you how to set up Datree to test your Helm charts against best practices and security misconfigurations using GitHub Actions.

What is Datree

Datree is a tool that helps to prevent misconfigurations of Kubernetes resources by running policy checks against the definition of these objects. It can be used as a CLI tool inside your CI/CD pipelines as well as an admission controller inside your Kubernetes cluster.

It provides you with a cloud backend that can be used to display the history of your Datree executions, manage different policies, show violations inside your cluster, and much more. If you want to learn more about Datree and the different ways it can be used, check out this video by TechWorld with Nana.

In this guide, I will only focus on setting up and using Datree for checking your Helm charts through GitHub Actions.

Assumptions

Datree account and token

Before we start with any work on GitHub, you need to register on app.datree.io and create a token or use the default token. Simply follow this link to go to the token management area of Datree.

GitHub repository structure

I obviously assume that you are using GitHub to store your Helm charts and use the default repository layout, meaning every Helm chart is located in a separate folder under the root folder called charts. The configuration we'll use to check our Helm charts will be stored in .github/config/datree.yaml and the workflow definition inside of .github/workflows/datree.yaml. Your layout should there look like this:

your-github-repository/
├─ .github/
│  ├─ config/
│  │  ├─ datree.yaml
│  ├─ workflows/
│  │  ├─ datree.yaml
├─ charts/
│  ├─ helm-chart-1
│  ├─ helm-chart-2

GitHub Action Secret

Lastly, after setting up your Datree account, obtaining a token, and creating the GitHub repository structure, you need to add the token as an Action secret to the GitHub repository. For this go to Settings --> Security --> Secrets --> Actions or simply append /settings/secrets/actions to the URL of your GitHub repository. Press the New repository secret button, set the name to DATREE_TOKEN , and paste the token from the Datree cloud backend.

How to setup Datree in GitHub

We need two files to make this work. The first defines the policy configuration used to validate the Helm charts and the other defines our GitHub Action workflow. I will call both files datree.yaml and store them in .github/config and .github/workflows.

Policy definition

Let's start with the definition of our policy in .github/config/datree.yaml:

apiVersion: v1
customRules: null
policies:
  - name: helm_charts
    isDefault: true
    rules:
      - identifier: INGRESS_INCORRECT_HOST_VALUE_PERMISSIVE
        messageOnFailure: Incorrect value for key `host` - specify host instead of using a wildcard character ("*")
      - identifier: SERVICE_INCORRECT_TYPE_VALUE_NODEPORT
        messageOnFailure: Incorrect value for key `type` - `NodePort` will open a port on all nodes where it can be reached by the network external to the cluster
      - identifier: CRONJOB_INVALID_SCHEDULE_VALUE
        messageOnFailure: 'Incorrect value for key `schedule` - the (cron) schedule expressions is not valid and, therefore, will not work as expected'
      - identifier: WORKLOAD_INVALID_LABELS_VALUE
        messageOnFailure: Incorrect value for key(s) under `labels` - the vales syntax is not valid so the Kubernetes engine will not accept it
      - identifier: WORKLOAD_INCORRECT_RESTARTPOLICY_VALUE_ALWAYS
        messageOnFailure: Incorrect value for key `restartPolicy` - any other value than `Always` is not supported by this resource
      - identifier: HPA_MISSING_MINREPLICAS_KEY
        messageOnFailure: Missing property object `minReplicas` - the value should be within the accepted boundaries recommended by the organization
      - identifier: CRONJOB_MISSING_STARTINGDEADLINESECOND_KEY
        messageOnFailure: Missing property object `startingDeadlineSeconds` - set a time limit to the cron execution to allow killing it if exceeded
      - identifier: K8S_DEPRECATED_APIVERSION_1.16
        messageOnFailure: Incorrect value for key `apiVersion` - the version you are trying to use is not supported by the Kubernetes cluster version (>=1.16)
      - identifier: K8S_DEPRECATED_APIVERSION_1.17
        messageOnFailure: Incorrect value for key `apiVersion` - the version you are trying to use is not supported by the Kubernetes cluster version (>=1.17)
      - identifier: CONTAINERS_INCORRECT_PRIVILEGED_VALUE_TRUE
        messageOnFailure: Incorrect value for key `privileged` - this mode will allow the container the same access as processes running on the host
      - identifier: CONTAINERS_MISSING_IMAGE_VALUE_DIGEST
        messageOnFailure: 'Incorrect value for key `image` - add a digest tag (starts with `@sha256:`) to represent an immutable version of the image'
      - identifier: CRONJOB_MISSING_CONCURRENCYPOLICY_KEY
        messageOnFailure: Missing property object `concurrencyPolicy` - the behavior will be more deterministic if jobs won't run concurrently
      - identifier: RESOURCE_MISSING_NAME
        messageOnFailure: Missing key `name` or `generateName` - one of them must be set to apply resource to a cluster

This is of course only a small example of the rules you can define. You can find a list of the built-in rules here. The documentation of Datree also shows examples of how to define your own custom rules.

Workflow definition

Next, let's continue with the workflow definition in .github/workflows/datree.yaml:

on:
  pull_request:
    branches: [main]
    paths: ["charts/**"]
  push:
    branches: [main]
    paths: ["charts/**"]
    
env:
  DATREE_TOKEN: ${{ secrets.DATREE_TOKEN }} 

jobs:
  datree:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Update dependencies
        run: find charts/ ! -path charts/ -maxdepth 1 -type d -exec helm dependency update {} \;
      - name: Run Datree Policy Check
        uses: datreeio/action-datree@main
        with:
          path: 'charts'
          isHelmChart: true
          cliArguments: '--only-k8s-files --policy-config .github/config/datree.yaml'

First, we check out the GitHub repository and update the dependencies of our Helm charts. This is required because of a limitation of Datree, described in this issue. Finally, we can run the policy check on the path charts. We add the flag --only-k8s-files to only check K8s manifests, else there would be errors for values.yaml and other files as they are not valid Kubernetes resources. The flag --policy-config is used to specify the path to the policy we want to use.

And that's already it. With every PR or Push on the main branch which changes anything under charts our workflow will be triggered and execute the policy checks.

Available options for Datree

Datree comes with a lot of options to tune the execution of the policy checks. In the next section, I will show some options that may be useful for you.

Defining the Kubernetes schema version

With the flag --schema-version <K8S_VERSION> we are able to define which Kubernetes version should be used for the schema validation of our resources. It makes sense to validate your Helm charts against multiple Kubernetes versions using multiple step definition, like this:

- name: Run Datree Policy Check against K8S v1.23.0
  uses: datreeio/action-datree@main
  with:
    path: 'charts'
    isHelmChart: true
    cliArguments: '--only-k8s-files --policy-config .github/config/datree.yaml --schema-version 1.23.0'
    
- name: Run Datree Policy Check against K8S v1.24.0
  uses: datreeio/action-datree@main
  with:
    path: 'charts'
    isHelmChart: true
    cliArguments: '--only-k8s-files --policy-config .github/config/datree.yaml --schema-version 1.24.0'

Using a cloud policy

If you want to use a policy that is defined in the Datree cloud backend, simply use the --policy <POLICY_NAME> instead of --policy-config .github/config/datree.yaml.

# Configure Kubernetes schema version
helm datree test <PATH/TO/HELM/CHART> --schema-version <K8S_VERSION>

# Define policy to use from the Datree cloud
helm datree test <PATH/TO/HELM/CHART> --policy <POLICY_NAME>

Don't use the cloud backend

If you don't want the results to end up in the Datree cloud backend, you can add the flag --no-record as a cliArgument to the definition.

- name: Run Datree Policy Check
  uses: datreeio/action-datree@main
  with:
    path: 'charts'
    isHelmChart: true
    cliArguments: '--only-k8s-files --policy-config --no-record'