Bitnami/charts
2
0
Fork 0
mirror of https://github.com/bitnami/charts.git synced 2026-03-14 06:47:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

[bitnami/airflow] feat: add support for custom local settings (#30496)

Browse Source
This commit is contained in:
Juan Ariza Toledano
2024-11-18 16:46:55 +01:00
committed by GitHub
parent 8a54fb8b17
commit 6c3060e024
13 changed files with 60 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
bitnami/airflow/CHANGELOG.md
View File

@@ -1,8 +1,13 @@
# Changelog
## 22.0.1 (2024-11-18)
## 22.1.0 (2024-11-18)
* [bitnami/airflow] Improve Goss tests ([#30492](https://github.com/bitnami/charts/pull/30492))
* [bitnami/airflow] feat: add support for custom local settings ([#30496](https://github.com/bitnami/charts/pull/30496))
## <small>22.0.1 (2024-11-18)</small>
* [bitnami/airflow] Improve Goss tests (#30492) ([c9c33f8](https://github.com/bitnami/charts/commit/c9c33f849aa2b02f0857e65e82bf2e0779413dea)), closes [#30492](https://github.com/bitnami/charts/issues/30492)
* [bitnami/airflow] Metrics based on StatsD (#30459) ([5c88481](https://github.com/bitnami/charts/commit/5c884817bcabe57ff93e91bc2b9ca411f21f615a)), closes [#30459](https://github.com/bitnami/charts/issues/30459)
## 21.9.0 (2024-11-14)

2
bitnami/airflow/Chart.yaml
View File

@@ -39,4 +39,4 @@ maintainers:
name: airflow
sources:
- https://github.com/bitnami/charts/tree/main/bitnami/airflow
version: 22.0.1
version: 22.1.0

3
bitnami/airflow/README.md
View File

@@ -375,7 +375,8 @@ The Bitnami Airflow chart relies on the PostgreSQL chart persistence. This means
| `executor` | Airflow executor. Allowed values: `SequentialExecutor`, `LocalExecutor`, `CeleryExecutor`, `KubernetesExecutor`, `CeleryKubernetesExecutor` and `LocalKubernetesExecutor` | `CeleryExecutor` |
| `loadExamples` | Switch to load some Airflow examples | `false` |
| `configuration` | Specify content for Airflow config file (auto-generated based on other env. vars otherwise) | `""` |
| `existingConfigmap` | Name of an existing ConfigMap with the Airflow config file | `""` |
| `localSettings` | Specify content for Airflow local settings (airflow_local_settings.py) | `""` |
| `existingConfigmap` | Name of an existing ConfigMap with the Airflow config file and, optionally, the local settings file | `""` |
| `dags.enabled` | Enable loading DAGs from a ConfigMap or Git repositories | `false` |
| `dags.existingConfigmap` | Name of an existing ConfigMap with all the DAGs files you want to load in Airflow | `""` |
| `dags.repositories` | Array of repositories from which to download DAG files | `[]` |

11
bitnami/airflow/templates/_init_containers_sidecars.tpl
View File

@@ -27,7 +27,7 @@ Returns an init-container that prepares the Airflow configuration files for main
- |
. /opt/bitnami/scripts/libairflow.sh
mkdir -p /emptydir/app-base-dir
mkdir -p /emptydir/app-base-dir /emptydir/app-conf-dir
# Copy the configuration files to the writable directory
cp /opt/bitnami/airflow/airflow.cfg /emptydir/app-base-dir/airflow.cfg
@@ -53,6 +53,12 @@ Returns an init-container that prepares the Airflow configuration files for main
{{- end }}
info "Airflow configuration ready"
if [[ -f "/opt/bitnami/airflow/config/airflow_local_settings.py" ]]; then
cp /opt/bitnami/airflow/config/airflow_local_settings.py /emptydir/app-conf-dir/airflow_local_settings.py
else
touch /emptydir/app-conf-dir/airflow_local_settings.py
fi
# HACK: When testing the db connection it creates an empty airflow.db file at the
# application root
touch /emptydir/app-base-dir/airflow.db
@@ -101,6 +107,9 @@ Returns an init-container that prepares the Airflow configuration files for main
- name: configuration
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: airflow.cfg
- name: configuration
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: airflow_local_settings.py
{{- end -}}
{{/*

8
bitnami/airflow/templates/config/configmap.yaml
View File

@@ -16,7 +16,7 @@ metadata:
data:
airflow.cfg: |-
{{- if .Values.configuration }}
{{- include "common.tplvalues.render" (dict "value" .Values.configuration "context" .) }}
{{- include "common.tplvalues.render" (dict "value" .Values.configuration "context" .) | nindent 4 }}
{{- else }}
[core]
load_examples={{ ternary "True" "False" .Values.loadExamples }}
@@ -53,7 +53,11 @@ data:
worker_container_tag={{ .Values.image.tag }}
delete_worker_pods=True
delete_worker_pods_on_failure=True
pod_template_file="/opt/bitnami/airflow/pod_template.yaml"
pod_template_file="/opt/bitnami/airflow/config/pod_template.yaml"
{{- end }}
{{- end }}
{{- if .Values.localSettings }}
airflow_local_settings.py: |-
{{- include "common.tplvalues.render" (dict "value" .Values.localSettings "context" .) | nindent 4 }}
{{- end }}
{{- end }}

4
bitnami/airflow/templates/dag-processor/deployment.yaml
View File

@@ -201,6 +201,9 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
{{- if .Values.dags.enabled }}
{{- include "airflow.dags.volumeMounts" . | nindent 12 }}
{{- end }}
@@ -231,6 +234,7 @@ spec:
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
{{- if .Values.dags.enabled }}
{{- include "airflow.dags.volumes" . | nindent 8 }}
{{- end }}

6
bitnami/airflow/templates/scheduler/deployment.yaml
View File

@@ -213,9 +213,12 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
{{- if $kube }}
- name: pod-template
mountPath: /opt/bitnami/airflow/pod_template.yaml
mountPath: /opt/bitnami/airflow/config/pod_template.yaml
subPath: pod_template.yaml
{{- end }}
{{- if and .Values.dags.enabled (not .Values.dagProcessor.enabled) }}
@@ -248,6 +251,7 @@ spec:
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
{{- if $kube }}
- name: pod-template
configMap:

4
bitnami/airflow/templates/setup-db-job.yaml
View File

@@ -113,10 +113,14 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
volumes:
- name: empty-dir
emptyDir: {}
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
{{- end }}

4
bitnami/airflow/templates/triggerer/statefulset.yaml
View File

@@ -205,6 +205,9 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
{{- if .Values.triggerer.persistence.enabled }}
- name: logs
mountPath: /opt/bitnami/airflow/logs
@@ -243,6 +246,7 @@ spec:
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
{{- if .Values.dags.enabled }}
{{- include "airflow.dags.volumes" . | nindent 8 }}
{{- end }}

2
bitnami/airflow/templates/web/configmap.yaml
View File

@@ -31,7 +31,7 @@ data:
AUTH_TYPE = AUTH_LDAP
AUTH_LDAP_SERVER = {{ .Values.ldap.uri | squote }}
AUTH_LDAP_SEARCH = {{ .Values.ldap.basedn }}
AUTH_LDAP_UID_FIELD = {{ .Values.ldap.searchAttribute }}
AUTH_LDAP_UID_FIELD = {{ .Values.ldap.searchAttribute | squote }}
AUTH_LDAP_BIND_USER = {{ .Values.ldap.binddn | squote }}
AUTH_USER_REGISTRATION = {{ .Values.ldap.userRegistration }}
AUTH_USER_REGISTRATION_ROLE = {{ .Values.ldap.userRegistrationRole | squote }}

4
bitnami/airflow/templates/web/deployment.yaml
View File

@@ -215,6 +215,9 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
- name: empty-dir
mountPath: /opt/bitnami/airflow/webserver_config.py
subPath: app-base-dir/webserver_config.py
@@ -252,6 +255,7 @@ spec:
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
- name: webserver-configuration
configMap:
name: {{ include "airflow.web.configMapName" . }}

4
bitnami/airflow/templates/worker/statefulset.yaml
View File

@@ -215,6 +215,9 @@ spec:
- name: empty-dir
mountPath: /opt/bitnami/airflow/airflow.cfg
subPath: app-base-dir/airflow.cfg
- name: empty-dir
mountPath: /opt/bitnami/airflow/config/airflow_local_settings.py
subPath: app-conf-dir/airflow_local_settings.py
- name: empty-dir
mountPath: /opt/bitnami/airflow/webserver_config.py
subPath: app-base-dir/webserver_config.py
@@ -253,6 +256,7 @@ spec:
- name: configuration
configMap:
name: {{ include "airflow.configMapName" . }}
optional: true
- name: webserver-configuration
configMap:
name: {{ include "airflow.web.configMapName" . }}

9
bitnami/airflow/values.yaml
View File

@@ -142,7 +142,14 @@ loadExamples: false
## ...
##
configuration: ""
## @param existingConfigmap Name of an existing ConfigMap with the Airflow config file
## @param localSettings Specify content for Airflow local settings (airflow_local_settings.py)
## ref: https://airflow.apache.org/docs/apache-airflow/stable/howto/set-config.html#configuring-local-settings
## e.g:
## localSettings: |-
## ...
##
localSettings: ""
## @param existingConfigmap Name of an existing ConfigMap with the Airflow config file and, optionally, the local settings file
##
existingConfigmap: ""
## Load custom DAGs files from a ConfigMap or Git repositories