Advanced installation

In this guide we show how to customize your cluster installation, i.e. if you want to install additional applications, or change the configuration of extisting apps installed by OAS this is the right place. Customizing other parts of your cluster might be possible but not yet covered by this guide. As the name of this guide implies, it is written for users with advanced knowledge of the tools behind Openappstack, most importantly: Kubernetes, Helm, Ansible and Flux.

Warning

Customizing your OAS cluster could break your cluster in a way that it’s not easy to recover. Please be aware of the potential risk when proceeding.

Prerequisites

Customize OAS applications

Apps deployed by OAS are configured using helm values from templates in ansible/roles/apps/templates/settings/APPNAME.yaml. To add new helm values or override existing values you can use APPNAME_extra_values in your CLUSTERDIR/group_vars/all/settings.yaml. Look for the correct APPNAME in flux/<namespace>/APPNAME.yaml.

We use the ansible combine filter with “recursive=True” to merge the APPNAME_extra_values variable into our default settings. A possible pitfall with customizing values from ansible templates is that you need to replace all dynamic expressions which would be templated. We will cover this in the example below.

Example: Customize prometheus data retention

If you want to lower the data retention value (currently 10 days) to lets say 5 days you need to add the following addition to prometheus_extra_values in CLUSTERDIR/group_vars/all/settings.yaml. (For all prometheus default values set by Openappstack see ansible/roles/apps/templates/settings/prometheus.yaml.)

prometheus_extra_values:
  server:
    retention: "5d"

After this, run the Openappstack installation procedure again. Until helm-operator detects changes to configmaps and secrets by itself and updates a helm release you need to manully restart the helm-operator pod in order to apply the config change to prometheus:

kubectl -n oas delete pod -l 'app=helm-operator'

Deploying custom apps with an extra flux instance

Openappstack uses Flux to deploy and auto-update applications. See the Automatic updates section in the design doc for more information abouot Flux.

You can configure an extra Flux instance running on your OAS cluster, which can monitor your own git repo with flux definitions. Be aware that you only need a second Flux instance, the existing helm-operator will deploy the helm releases from both Flux installations.

In order to do that you first need to enable flux-custom in the list of enabled_applications in your settings.yml file. Then configure flux-custom_extra_values to your needs (see the Flux helm chart for all possible values). An example is shown below:

flux-custom_extra_values:
  # https://github.com/fluxcd/flux/tree/master/chart/flux#configuration
  git:
    url: git@my.git.server.org:myself/flux-config.git
    branch: master
    readonly: true
    pollInterval: 1h
  registry:
    excludeImage: '*'
  # If you use ssh instead of https, like in above git repo URL you need to
  # provide the ssh hostkey pringerprint shown with
  #   `ssh-keyscan -t ssh-rsa my.git.server.org`:
  ssh:
    known_hosts: 'my.git.server.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZ2JHLwWuixvUvx630E3fInoJKyABLPzgDH4k4dM6ptQXVfK0Hu53nhIjsCbp/i4u4GLH2B8Tv2umVh8EejvjbqsZpD4fX6PWbM131vA3sPSD4q5qJV1xkZdc4+3STrndRD02py96OtjBl/f6EJW3Upz/xWX/iiL6crp8RZzOsSn9dv/gg4Bn42G3gcNZJVgNJHO8yPAYf1fMVlDYpiKlib6Vuow8EKerqByDnLm1vbnnMHes+F6Pt1JkLzURGE83AwAUfZZTRGyzZEkXVNpcV1Pq3esg1bBAW6dfxeyzF9SJrpYkv0lbUkuBFEz9ExzWpiKe/q/O61W0u9Q6sKwev'
  sync:
    state: secret
  syncGarbageCollection:
    enabled: true

I your Flux definitions in your git repo contain ssh git URLs you also need to configure helm-operator to trust the fingerprints of the servers your git repositories are hosted at. If they are hosted on the same server your Flux repo is at i.e.:

helm_operator:
  extra_opts: "--set git.ssh.known_hosts='my.git.server.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZ2JHLwWuixvUvx630E3fInoJKyABLPzgDH4k4dM6ptQXVfK0Hu53nhIjsCbp/i4u4GLH2B8Tv2umVh8EejvjbqsZpD4fX6PWbM131vA3sPSD4q5qJV1xkZdc4+3STrndRD02py96OtjBl/f6EJW3Upz/xWX/iiL6crp8RZzOsSn9dv/gg4Bn42G3gcNZJVgNJHO8yPAYf1fMVlDYpiKlib6Vuow8EKerqByDnLm1vbnnMHes+F6Pt1JkLzURGE83AwAUfZZTRGyzZEkXVNpcV1Pq3esg1bBAW6dfxeyzF9SJrpYkv0lbUkuBFEz9ExzWpiKe/q/O61W0u9Q6sKwev'

After this, run the Openappstack installation procedure again.

Using a different storage class

Openappstack uses the local path provisioner to create local path persistent volumes on the host. If you want to use a different storage provisioner you need to first set the setting storageClass.defaultClass to false (see ansible/roles/apps/templates/settings/local-path-provisioner.yaml) and can then remove the local-path-provisioner from the list of enabled_applications

After this, run the Openappstack installation procedure again.

Now you can configure your Kubernetes cluster to use a different default storage provisioner which newly created persistent volumes can use.