mirror of
https://github.com/bitnami/charts.git
synced 2026-08-10 14:15:55 +08:00
[bitnami/airflow] Add support to run Dag Processor in standalone mode (#30159)
This commit is contained in:
Juan Ariza Toledano
@@ -5,6 +5,8 @@ service:
|
||||
ports:
|
||||
http: 80
|
||||
type: LoadBalancer
|
||||
dagProcessor:
|
||||
enabled: true
|
||||
dags:
|
||||
enabled: true
|
||||
repositories:
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# Changelog
|
||||
|
||||
## 21.0.3 (2024-10-31)
|
||||
## 21.1.0 (2024-11-04)
|
||||
|
||||
* [bitnami/airflow] Release 21.0.3 ([#30161](https://github.com/bitnami/charts/pull/30161))
|
||||
* [bitnami/airflow] Add support to run Dag Processor in standalone mode ([#30159](https://github.com/bitnami/charts/pull/30159))
|
||||
|
||||
## <small>21.0.3 (2024-10-31)</small>
|
||||
|
||||
* [bitnami/airflow] Release 21.0.3 (#30161) ([89e392b](https://github.com/bitnami/charts/commit/89e392b7297346863c80c49ba2325c72321142e3)), closes [#30161](https://github.com/bitnami/charts/issues/30161)
|
||||
|
||||
## <small>21.0.2 (2024-10-31)</small>
|
||||
|
||||
|
||||
@@ -39,4 +39,4 @@ maintainers:
|
||||
name: airflow
|
||||
sources:
|
||||
- https://github.com/bitnami/charts/tree/main/bitnami/airflow
|
||||
version: 21.0.3
|
||||
version: 21.1.0
|
||||
|
||||
@@ -555,6 +555,89 @@ The Bitnami Airflow chart relies on the PostgreSQL chart persistence. This means
|
||||
| `scheduler.networkPolicy.ingressNSMatchLabels` | Labels to match to allow traffic from other namespaces | `{}` |
|
||||
| `scheduler.networkPolicy.ingressNSPodMatchLabels` | Pod labels to match to allow traffic from other namespaces | `{}` |
|
||||
|
||||
### Airflow Dag Processor parameters
|
||||
|
||||
| Name | Description | Value |
|
||||
| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- |
|
||||
| `dagProcessor.enabled` | Run Airflow Dag Processor Manager as a standalone component | `false` |
|
||||
| `dagProcessor.replicaCount` | Number of Airflow Dag Processor replicas | `1` |
|
||||
| `dagProcessor.command` | Override default Airflow Dag Processor cmd | `[]` |
|
||||
| `dagProcessor.args` | Override default Airflow Dag Processor args | `[]` |
|
||||
| `dagProcessor.extraEnvVars` | Add extra environment variables to Airflow Dag Processor containers | `[]` |
|
||||
| `dagProcessor.extraEnvVarsCM` | ConfigMap with extra environment variables | `""` |
|
||||
| `dagProcessor.extraEnvVarsSecret` | Secret with extra environment variables | `""` |
|
||||
| `dagProcessor.livenessProbe.enabled` | Enable livenessProbe on Airflow Dag Processor containers | `true` |
|
||||
| `dagProcessor.livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `180` |
|
||||
| `dagProcessor.livenessProbe.periodSeconds` | Period seconds for livenessProbe | `20` |
|
||||
| `dagProcessor.livenessProbe.timeoutSeconds` | Timeout seconds for livenessProbe | `15` |
|
||||
| `dagProcessor.livenessProbe.failureThreshold` | Failure threshold for livenessProbe | `6` |
|
||||
| `dagProcessor.livenessProbe.successThreshold` | Success threshold for livenessProbe | `1` |
|
||||
| `dagProcessor.readinessProbe.enabled` | Enable readinessProbe on Airflow Dag Processor containers | `true` |
|
||||
| `dagProcessor.readinessProbe.initialDelaySeconds` | Initial delay seconds for readinessProbe | `30` |
|
||||
| `dagProcessor.readinessProbe.periodSeconds` | Period seconds for readinessProbe | `10` |
|
||||
| `dagProcessor.readinessProbe.timeoutSeconds` | Timeout seconds for readinessProbe | `15` |
|
||||
| `dagProcessor.readinessProbe.failureThreshold` | Failure threshold for readinessProbe | `6` |
|
||||
| `dagProcessor.readinessProbe.successThreshold` | Success threshold for readinessProbe | `1` |
|
||||
| `dagProcessor.startupProbe.enabled` | Enable startupProbe on Airflow Dag Processor containers | `false` |
|
||||
| `dagProcessor.startupProbe.initialDelaySeconds` | Initial delay seconds for startupProbe | `60` |
|
||||
| `dagProcessor.startupProbe.periodSeconds` | Period seconds for startupProbe | `10` |
|
||||
| `dagProcessor.startupProbe.timeoutSeconds` | Timeout seconds for startupProbe | `1` |
|
||||
| `dagProcessor.startupProbe.failureThreshold` | Failure threshold for startupProbe | `15` |
|
||||
| `dagProcessor.startupProbe.successThreshold` | Success threshold for startupProbe | `1` |
|
||||
| `dagProcessor.customLivenessProbe` | Custom livenessProbe that overrides the default one | `{}` |
|
||||
| `dagProcessor.customReadinessProbe` | Custom readinessProbe that overrides the default one | `{}` |
|
||||
| `dagProcessor.customStartupProbe` | Custom startupProbe that overrides the default one | `{}` |
|
||||
| `dagProcessor.resourcesPreset` | Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if dagProcessor.resources is set (dagProcessor.resources is recommended for production). | `small` |
|
||||
| `dagProcessor.resources` | Set container requests and limits for different resources like CPU or memory (essential for production workloads) | `{}` |
|
||||
| `dagProcessor.podSecurityContext.enabled` | Enabled Airflow Dag Processor pods' Security Context | `true` |
|
||||
| `dagProcessor.podSecurityContext.fsGroupChangePolicy` | Set filesystem group change policy | `Always` |
|
||||
| `dagProcessor.podSecurityContext.sysctls` | Set kernel settings using the sysctl interface | `[]` |
|
||||
| `dagProcessor.podSecurityContext.supplementalGroups` | Set filesystem extra groups | `[]` |
|
||||
| `dagProcessor.podSecurityContext.fsGroup` | Set Airflow Dag Processor pod's Security Context fsGroup | `1001` |
|
||||
| `dagProcessor.containerSecurityContext.enabled` | Enabled Airflow Dag Processor containers' Security Context | `true` |
|
||||
| `dagProcessor.containerSecurityContext.seLinuxOptions` | Set SELinux options in container | `{}` |
|
||||
| `dagProcessor.containerSecurityContext.runAsUser` | Set Airflow Dag Processor containers' Security Context runAsUser | `1001` |
|
||||
| `dagProcessor.containerSecurityContext.runAsGroup` | Set Airflow Dag Processor containers' Security Context runAsGroup | `1001` |
|
||||
| `dagProcessor.containerSecurityContext.runAsNonRoot` | Set Airflow Dag Processor containers' Security Context runAsNonRoot | `true` |
|
||||
| `dagProcessor.containerSecurityContext.privileged` | Set Airflow Dag Processor container's Security Context privileged | `false` |
|
||||
| `dagProcessor.containerSecurityContext.allowPrivilegeEscalation` | Set Airflow Dag Processor container's Security Context allowPrivilegeEscalation | `false` |
|
||||
| `dagProcessor.containerSecurityContext.readOnlyRootFilesystem` | Set Airflow Dag Processor container's Security Context readOnlyRootFilesystem | `true` |
|
||||
| `dagProcessor.containerSecurityContext.capabilities.drop` | List of capabilities to be dropped | `["ALL"]` |
|
||||
| `dagProcessor.containerSecurityContext.seccompProfile.type` | Set container's Security Context seccomp profile | `RuntimeDefault` |
|
||||
| `dagProcessor.lifecycleHooks` | for the Airflow Dag Processor containers to automate configuration before or after startup | `{}` |
|
||||
| `dagProcessor.automountServiceAccountToken` | Mount Service Account token in pod | `false` |
|
||||
| `dagProcessor.hostAliases` | Deployment pod host aliases | `[]` |
|
||||
| `dagProcessor.podLabels` | Add extra labels to the Airflow Dag Processor pods | `{}` |
|
||||
| `dagProcessor.podAnnotations` | Add extra annotations to the Airflow Dag Processor pods | `{}` |
|
||||
| `dagProcessor.affinity` | Affinity for Airflow Dag Processor pods assignment (evaluated as a template) | `{}` |
|
||||
| `dagProcessor.nodeAffinityPreset.key` | Node label key to match. Ignored if `dagProcessor.affinity` is set. | `""` |
|
||||
| `dagProcessor.nodeAffinityPreset.type` | Node affinity preset type. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard` | `""` |
|
||||
| `dagProcessor.nodeAffinityPreset.values` | Node label values to match. Ignored if `dagProcessor.affinity` is set. | `[]` |
|
||||
| `dagProcessor.nodeSelector` | Node labels for Airflow Dag Processor pods assignment | `{}` |
|
||||
| `dagProcessor.podAffinityPreset` | Pod affinity preset. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard`. | `""` |
|
||||
| `dagProcessor.podAntiAffinityPreset` | Pod anti-affinity preset. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard`. | `soft` |
|
||||
| `dagProcessor.tolerations` | Tolerations for Airflow Dag Processor pods assignment | `[]` |
|
||||
| `dagProcessor.topologySpreadConstraints` | Topology Spread Constraints for pod assignment spread across your cluster among failure-domains. Evaluated as a template | `[]` |
|
||||
| `dagProcessor.priorityClassName` | Priority Class Name | `""` |
|
||||
| `dagProcessor.schedulerName` | Use an alternate K8s scheduler, e.g. "stork". | `""` |
|
||||
| `dagProcessor.terminationGracePeriodSeconds` | Seconds Airflow Dag Processor pod needs to terminate gracefully | `""` |
|
||||
| `dagProcessor.updateStrategy.type` | Airflow Dag Processor deployment strategy type | `RollingUpdate` |
|
||||
| `dagProcessor.updateStrategy.rollingUpdate` | Airflow Dag Processor deployment rolling update configuration parameters | `{}` |
|
||||
| `dagProcessor.sidecars` | Add additional sidecar containers to the Airflow Dag Processor pods | `[]` |
|
||||
| `dagProcessor.initContainers` | Add additional init containers to the Airflow Dag Processor pods | `[]` |
|
||||
| `dagProcessor.extraVolumeMounts` | Optionally specify extra list of additional volumeMounts for the Airflow Dag Processor containers | `[]` |
|
||||
| `dagProcessor.extraVolumes` | Optionally specify extra list of additional volumes for the Airflow Dag Processor pods | `[]` |
|
||||
| `dagProcessor.pdb.create` | Deploy a pdb object for the Airflow Dag Processor pods | `true` |
|
||||
| `dagProcessor.pdb.minAvailable` | Maximum number/percentage of unavailable Airflow Dag Processor replicas | `""` |
|
||||
| `dagProcessor.pdb.maxUnavailable` | Maximum number/percentage of unavailable Airflow Dag Processor replicas | `""` |
|
||||
| `dagProcessor.networkPolicy.enabled` | Specifies whether a NetworkPolicy should be created | `true` |
|
||||
| `dagProcessor.networkPolicy.allowExternal` | Don't require client label for connections | `true` |
|
||||
| `dagProcessor.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` |
|
||||
| `dagProcessor.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` |
|
||||
| `dagProcessor.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy | `[]` |
|
||||
| `dagProcessor.networkPolicy.ingressNSMatchLabels` | Labels to match to allow traffic from other namespaces | `{}` |
|
||||
| `dagProcessor.networkPolicy.ingressNSPodMatchLabels` | Pod labels to match to allow traffic from other namespaces | `{}` |
|
||||
|
||||
### Airflow worker parameters
|
||||
|
||||
| Name | Description | Value |
|
||||
|
||||
@@ -19,6 +19,13 @@ Return the proper Airflow Scheduler fullname
|
||||
{{- printf "%s-scheduler" (include "common.names.fullname" .) | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Return the proper Airflow Dag Processor fullname
|
||||
*/}}
|
||||
{{- define "airflow.dagProcessor.fullname" -}}
|
||||
{{- printf "%s-dag-processor" (include "common.names.fullname" .) | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Return the proper Airflow Worker fullname
|
||||
*/}}
|
||||
@@ -330,6 +337,8 @@ Add environment variables to configure airflow common values
|
||||
key: airflow-secret-key
|
||||
- name: AIRFLOW_LOAD_EXAMPLES
|
||||
value: {{ ternary "yes" "no" .Values.loadExamples | quote }}
|
||||
- name: AIRFLOW_STANDALONE_DAG_PROCESSOR
|
||||
value: {{ ternary "yes" "no" .Values.dagProcessor.enabled | quote }}
|
||||
{{- if not (or .Values.configuration .Values.existingConfigmap) }}
|
||||
- name: AIRFLOW_FORCE_OVERWRITE_CONF_FILE
|
||||
value: "yes"
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
{{- /*
|
||||
Copyright Broadcom, Inc. All Rights Reserved.
|
||||
SPDX-License-Identifier: APACHE-2.0
|
||||
*/}}
|
||||
|
||||
{{- if .Values.dagProcessor.enabled }}
|
||||
apiVersion: {{ include "common.capabilities.deployment.apiVersion" . }}
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ template "airflow.dagProcessor.fullname" . }}
|
||||
namespace: {{ include "common.names.namespace" . | quote }}
|
||||
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
{{- if .Values.commonAnnotations }}
|
||||
annotations: {{- include "common.tplvalues.render" (dict "value" .Values.commonAnnotations "context" $) | trim | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- $podLabels := include "common.tplvalues.merge" ( dict "values" ( list .Values.dagProcessor.podLabels .Values.commonLabels ) "context" . ) }}
|
||||
selector:
|
||||
matchLabels: {{- include "common.labels.matchLabels" ( dict "customLabels" $podLabels "context" $ ) | nindent 6 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
replicas: {{ .Values.dagProcessor.replicaCount }}
|
||||
{{- if .Values.dagProcessor.updateStrategy }}
|
||||
strategy: {{- toYaml .Values.dagProcessor.updateStrategy | nindent 4 }}
|
||||
{{- end }}
|
||||
template:
|
||||
metadata:
|
||||
labels: {{- include "common.labels.standard" ( dict "customLabels" $podLabels "context" $ ) | nindent 8 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
annotations:
|
||||
checksum/configmap: {{ include (print $.Template.BasePath "/config/configmap.yaml") . | sha256sum }}
|
||||
{{- if .Values.dagProcessor.podAnnotations }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.podAnnotations "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- include "airflow.imagePullSecrets" . | nindent 6 }}
|
||||
automountServiceAccountToken: {{ .Values.dagProcessor.automountServiceAccountToken }}
|
||||
{{- if .Values.dagProcessor.hostAliases }}
|
||||
hostAliases: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.hostAliases "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.affinity }}
|
||||
affinity: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.affinity "context" $) | nindent 8 }}
|
||||
{{- else }}
|
||||
affinity:
|
||||
podAffinity: {{- include "common.affinities.pods" (dict "type" .Values.dagProcessor.podAffinityPreset "component" "dag-processor" "customLabels" $podLabels "context" $) | nindent 10 }}
|
||||
podAntiAffinity: {{- include "common.affinities.pods" (dict "type" .Values.dagProcessor.podAntiAffinityPreset "component" "dag-processor" "customLabels" $podLabels "context" $) | nindent 10 }}
|
||||
nodeAffinity: {{- include "common.affinities.nodes" (dict "type" .Values.dagProcessor.nodeAffinityPreset.type "key" .Values.dagProcessor.nodeAffinityPreset.key "values" .Values.dagProcessor.nodeAffinityPreset.values) | nindent 10 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.nodeSelector }}
|
||||
nodeSelector: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.nodeSelector "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.terminationGracePeriodSeconds }}
|
||||
terminationGracePeriodSeconds: {{ .Values.dagProcessor.terminationGracePeriodSeconds }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.tolerations }}
|
||||
tolerations: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.tolerations "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.topologySpreadConstraints }}
|
||||
topologySpreadConstraints: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.topologySpreadConstraints "context" .) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.priorityClassName }}
|
||||
priorityClassName: {{ .Values.dagProcessor.priorityClassName | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.schedulerName }}
|
||||
schedulerName: {{ .Values.dagProcessor.schedulerName }}
|
||||
{{- end }}
|
||||
serviceAccountName: {{ include "airflow.serviceAccountName" . }}
|
||||
{{- if .Values.dagProcessor.podSecurityContext.enabled }}
|
||||
securityContext: {{- include "common.compatibility.renderSecurityContext" (dict "secContext" .Values.dagProcessor.podSecurityContext "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
initContainers:
|
||||
{{- include "airflow.defaultInitContainers.createDefaultConfig" . | nindent 8 }}
|
||||
{{- if .Values.dags.enabled }}
|
||||
{{- include "airflow.defaultInitContainers.loadDAGs" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
{{- include "airflow.defaultInitContainers.loadPlugins" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.initContainers }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.initContainers "context" $) | trim | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.initContainers }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.initContainers "context" $) | trim | nindent 8 }}
|
||||
{{- end }}
|
||||
containers:
|
||||
- name: airflow-dag-processor
|
||||
image: {{ include "airflow.image" . | quote }}
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
|
||||
{{- if .Values.dagProcessor.containerSecurityContext.enabled }}
|
||||
securityContext: {{- include "common.compatibility.renderSecurityContext" (dict "secContext" .Values.dagProcessor.containerSecurityContext "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.diagnosticMode.enabled }}
|
||||
command: {{- include "common.tplvalues.render" (dict "value" .Values.diagnosticMode.command "context" $) | nindent 12 }}
|
||||
{{- else if .Values.dagProcessor.command }}
|
||||
command: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.command "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.diagnosticMode.enabled }}
|
||||
args: {{- include "common.tplvalues.render" (dict "value" .Values.diagnosticMode.args "context" $) | nindent 12 }}
|
||||
{{- else if .Values.dagProcessor.args }}
|
||||
args: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.args "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
env:
|
||||
{{- include "airflow.configure.airflow.common" . | nindent 12 }}
|
||||
{{- include "airflow.configure.database" . | nindent 12 }}
|
||||
{{- include "airflow.configure.redis" . | nindent 12 }}
|
||||
{{- include "airflow.configure.airflow.kubernetesExecutor" . | nindent 12 }}
|
||||
- name: AIRFLOW_COMPONENT_TYPE
|
||||
value: "dag-processor"
|
||||
- name: AIRFLOW_EXECUTOR
|
||||
value: {{ .Values.executor }}
|
||||
- name: AIRFLOW_WEBSERVER_HOST
|
||||
value: {{ include "common.names.fullname" . }}
|
||||
- name: AIRFLOW_WEBSERVER_PORT_NUMBER
|
||||
value: {{ .Values.service.ports.http | quote }}
|
||||
{{- if .Values.extraEnvVars }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.extraEnvVars "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.extraEnvVars }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.extraEnvVars "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if or .Values.dagProcessor.extraEnvVarsCM .Values.dagProcessor.extraEnvVarsSecret .Values.extraEnvVarsCM .Values.extraEnvVarsSecret }}
|
||||
envFrom:
|
||||
{{- if .Values.extraEnvVarsCM }}
|
||||
- configMapRef:
|
||||
name: {{ .Values.extraEnvVarsCM }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraEnvVarsSecret }}
|
||||
- secretRef:
|
||||
name: {{ .Values.extraEnvVarsSecret }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.extraEnvVarsCM }}
|
||||
- configMapRef:
|
||||
name: {{ .Values.dagProcessor.extraEnvVarsCM }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.extraEnvVarsSecret }}
|
||||
- secretRef:
|
||||
name: {{ .Values.dagProcessor.extraEnvVarsSecret }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if not .Values.diagnosticMode.enabled }}
|
||||
{{- if .Values.dagProcessor.customLivenessProbe }}
|
||||
livenessProbe: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.customLivenessProbe "context" $) | trim | nindent 12 }}
|
||||
{{- else if .Values.dagProcessor.livenessProbe.enabled }}
|
||||
livenessProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.dagProcessor.livenessProbe "enabled") "context" $) | nindent 12 }}
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -ec
|
||||
- |
|
||||
CONNECTION_CHECK_MAX_COUNT=0 airflow jobs check --job-type DagProcessorJob --local {{- if not .Values.diagnosticMode.enabled }} 2>/dev/null {{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.customReadinessProbe }}
|
||||
readinessProbe: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.customReadinessProbe "context" $) | trim | nindent 12 }}
|
||||
{{- else if .Values.dagProcessor.readinessProbe.enabled }}
|
||||
readinessProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.dagProcessor.readinessProbe "enabled") "context" $) | nindent 12 }}
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -ec
|
||||
- |
|
||||
airflow jobs check --job-type DagProcessorJob --local {{- if not .Values.diagnosticMode.enabled }} 2>/dev/null {{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.customStartupProbe }}
|
||||
startupProbe: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.customStartupProbe "context" $) | nindent 12 }}
|
||||
{{- else if .Values.dagProcessor.startupProbe.enabled }}
|
||||
startupProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.dagProcessor.startupProbe "enabled") "context" $) | nindent 12 }}
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -ec
|
||||
- |
|
||||
CONNECTION_CHECK_MAX_COUNT=0 airflow jobs check --job-type DagProcessorJob --local {{- if not .Values.diagnosticMode.enabled }} 2>/dev/null {{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.lifecycleHooks }}
|
||||
lifecycle: {{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.lifecycleHooks "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.resources }}
|
||||
resources: {{- toYaml .Values.dagProcessor.resources | nindent 12 }}
|
||||
{{- else if ne .Values.dagProcessor.resourcesPreset "none" }}
|
||||
resources: {{- include "common.resources.preset" (dict "type" .Values.dagProcessor.resourcesPreset) | nindent 12 }}
|
||||
{{- end }}
|
||||
volumeMounts:
|
||||
- name: empty-dir
|
||||
mountPath: /opt/bitnami/airflow/nss-wrapper
|
||||
subPath: app-nss-wrapper-dir
|
||||
- name: empty-dir
|
||||
mountPath: /opt/bitnami/airflow/logs
|
||||
subPath: app-logs-dir
|
||||
- name: empty-dir
|
||||
mountPath: /opt/bitnami/airflow/tmp
|
||||
subPath: app-tmp-dir
|
||||
- name: empty-dir
|
||||
mountPath: /opt/bitnami/airflow/airflow.db
|
||||
subPath: app-default-conf-dir/airflow.db
|
||||
- name: empty-dir
|
||||
mountPath: /tmp
|
||||
subPath: tmp-dir
|
||||
{{- if or .Values.configuration .Values.existingConfigmap }}
|
||||
- name: custom-configuration-file
|
||||
mountPath: /opt/bitnami/airflow/airflow.cfg
|
||||
subPath: airflow.cfg
|
||||
{{- else }}
|
||||
- name: empty-dir
|
||||
mountPath: /opt/bitnami/airflow/airflow.cfg
|
||||
subPath: app-default-conf-dir/airflow.cfg
|
||||
{{- end }}
|
||||
{{- if .Values.dags.enabled }}
|
||||
{{- include "airflow.dags.volumeMounts" . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
{{- include "airflow.plugins.volumeMounts" . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraVolumeMounts }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.extraVolumeMounts "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.extraVolumeMounts }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.extraVolumeMounts "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dags.enabled }}
|
||||
{{- include "airflow.defaultSidecars.syncDAGs" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
{{- include "airflow.defaultSidecars.syncPlugins" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.sidecars }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.sidecars "context" $) | trim | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.sidecars }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.sidecars "context" $) | trim | nindent 8 }}
|
||||
{{- end }}
|
||||
volumes:
|
||||
- name: empty-dir
|
||||
emptyDir: {}
|
||||
{{- if and .Values.dags.enabled .Values.dags.existingConfigmap }}
|
||||
- name: external-dags
|
||||
configMap:
|
||||
name: {{ tpl .Values.dags.existingConfigmap $ }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraVolumes }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.extraVolumes "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.extraVolumes }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.dagProcessor.extraVolumes "context" $) | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,73 @@
|
||||
{{- /*
|
||||
Copyright Broadcom, Inc. All Rights Reserved.
|
||||
SPDX-License-Identifier: APACHE-2.0
|
||||
*/}}
|
||||
|
||||
{{- if and .Values.dagProcessor.enabled .Values.dagProcessor.networkPolicy.enabled }}
|
||||
kind: NetworkPolicy
|
||||
apiVersion: {{ include "common.capabilities.networkPolicy.apiVersion" . }}
|
||||
metadata:
|
||||
name: {{ template "airflow.dagProcessor.fullname" . }}
|
||||
namespace: {{ include "common.names.namespace" . | quote }}
|
||||
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
{{- if .Values.commonAnnotations }}
|
||||
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- $podLabels := include "common.tplvalues.merge" ( dict "values" ( list .Values.dagProcessor.podLabels .Values.commonLabels ) "context" . ) }}
|
||||
podSelector:
|
||||
matchLabels: {{- include "common.labels.matchLabels" ( dict "customLabels" $podLabels "context" $ ) | nindent 6 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
{{- if .Values.dagProcessor.networkPolicy.allowExternalEgress }}
|
||||
egress:
|
||||
- {}
|
||||
{{- else }}
|
||||
egress:
|
||||
# Allow dns resolution
|
||||
- ports:
|
||||
- port: 53
|
||||
protocol: UDP
|
||||
- port: 53
|
||||
protocol: TCP
|
||||
# Allow outbound connections to other cluster pods
|
||||
- ports:
|
||||
- port: {{ .Values.service.ports.http }}
|
||||
to:
|
||||
- podSelector:
|
||||
matchLabels: {{- include "common.labels.matchLabels" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 14 }}
|
||||
{{- if .Values.dagProcessor.networkPolicy.extraEgress }}
|
||||
{{- include "common.tplvalues.render" ( dict "value" .Values.dagProcessor.networkPolicy.extraEgress "context" $ ) | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
ingress:
|
||||
- ports:
|
||||
- port: {{ .Values.service.ports.http }}
|
||||
- port: {{ .Values.worker.containerPorts.http }}
|
||||
{{- if not .Values.dagProcessor.networkPolicy.allowExternal }}
|
||||
from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
{{ printf "%s-dag-processor" (include "common.names.fullname" .) }}-client: "true"
|
||||
{{- if .Values.dagProcessor.networkPolicy.ingressNSMatchLabels }}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
{{- range $key, $value := .Values.dagProcessor.networkPolicy.ingressNSMatchLabels }}
|
||||
{{ $key | quote }}: {{ $value | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.networkPolicy.ingressNSPodMatchLabels }}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- range $key, $value := .Values.dagProcessor.networkPolicy.ingressNSPodMatchLabels }}
|
||||
{{ $key | quote }}: {{ $value | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.dagProcessor.networkPolicy.extraIngress }}
|
||||
{{- include "common.tplvalues.render" ( dict "value" .Values.dagProcessor.networkPolicy.extraIngress "context" $ ) | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,25 @@
|
||||
{{- /*
|
||||
Copyright Broadcom, Inc. All Rights Reserved.
|
||||
SPDX-License-Identifier: APACHE-2.0
|
||||
*/}}
|
||||
|
||||
{{- if and .Values.dagProcessor.enabled .Values.dagProcessor.pdb.create }}
|
||||
kind: PodDisruptionBudget
|
||||
apiVersion: {{ include "common.capabilities.policy.apiVersion" . }}
|
||||
metadata:
|
||||
name: {{ template "airflow.dagProcessor.fullname" . }}
|
||||
namespace: {{ include "common.names.namespace" . | quote }}
|
||||
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
spec:
|
||||
{{- if .Values.dagProcessor.pdb.minAvailable }}
|
||||
minAvailable: {{ .Values.dagProcessor.pdb.minAvailable }}
|
||||
{{- end }}
|
||||
{{- if or .Values.dagProcessor.pdb.maxUnavailable ( not .Values.dagProcessor.pdb.minAvailable ) }}
|
||||
maxUnavailable: {{ .Values.dagProcessor.pdb.maxUnavailable | default 1 }}
|
||||
{{- end }}
|
||||
{{- $podLabels := include "common.tplvalues.merge" ( dict "values" ( list .Values.dagProcessor.podLabels .Values.commonLabels ) "context" . ) }}
|
||||
selector:
|
||||
matchLabels: {{- include "common.labels.matchLabels" ( dict "customLabels" $podLabels "context" $ ) | nindent 6 }}
|
||||
app.kubernetes.io/component: dag-processor
|
||||
{{- end }}
|
||||
@@ -70,7 +70,7 @@ spec:
|
||||
{{- end }}
|
||||
initContainers:
|
||||
{{- include "airflow.defaultInitContainers.createDefaultConfig" . | nindent 8 }}
|
||||
{{- if and .Values.dags.enabled }}
|
||||
{{- if and .Values.dags.enabled (not .Values.dagProcessor.enabled) }}
|
||||
{{- include "airflow.defaultInitContainers.loadDAGs" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
@@ -222,7 +222,7 @@ spec:
|
||||
mountPath: /opt/bitnami/airflow/pod_template.yaml
|
||||
subPath: pod_template.yaml
|
||||
{{- end }}
|
||||
{{- if and .Values.dags.enabled }}
|
||||
{{- if and .Values.dags.enabled (not .Values.dagProcessor.enabled) }}
|
||||
{{- include "airflow.dags.volumeMounts" . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
@@ -234,7 +234,7 @@ spec:
|
||||
{{- if .Values.scheduler.extraVolumeMounts }}
|
||||
{{- include "common.tplvalues.render" (dict "value" .Values.scheduler.extraVolumeMounts "context" $) | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- if and .Values.dags.enabled }}
|
||||
{{- if and .Values.dags.enabled (not .Values.dagProcessor.enabled) }}
|
||||
{{- include "airflow.defaultSidecars.syncDAGs" . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- if .Values.plugins.enabled }}
|
||||
@@ -249,7 +249,7 @@ spec:
|
||||
volumes:
|
||||
- name: empty-dir
|
||||
emptyDir: {}
|
||||
{{- if and .Values.dags.enabled .Values.dags.existingConfigmap }}
|
||||
{{- if and .Values.dags.enabled .Values.dags.existingConfigmap (not .Values.dagProcessor.enabled) }}
|
||||
- name: external-dags
|
||||
configMap:
|
||||
name: {{ tpl .Values.dags.existingConfigmap $ }}
|
||||
|
||||
@@ -1017,6 +1017,313 @@ scheduler:
|
||||
ingressNSMatchLabels: {}
|
||||
ingressNSPodMatchLabels: {}
|
||||
|
||||
## @section Airflow Dag Processor parameters
|
||||
## ref: https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/dagfile-processing.html
|
||||
##
|
||||
dagProcessor:
|
||||
## @param dagProcessor.enabled Run Airflow Dag Processor Manager as a standalone component
|
||||
##
|
||||
enabled: false
|
||||
## @param dagProcessor.replicaCount Number of Airflow Dag Processor replicas
|
||||
##
|
||||
replicaCount: 1
|
||||
## @param dagProcessor.command Override default Airflow Dag Processor cmd
|
||||
##
|
||||
command: []
|
||||
## @param dagProcessor.args Override default Airflow Dag Processor args
|
||||
##
|
||||
args: []
|
||||
## @param dagProcessor.extraEnvVars Add extra environment variables to Airflow Dag Processor containers
|
||||
##
|
||||
extraEnvVars: []
|
||||
## @param dagProcessor.extraEnvVarsCM ConfigMap with extra environment variables
|
||||
##
|
||||
extraEnvVarsCM: ""
|
||||
## @param dagProcessor.extraEnvVarsSecret Secret with extra environment variables
|
||||
##
|
||||
extraEnvVarsSecret: ""
|
||||
## Configure extra options for Airflow Dag Processor containers' liveness, readiness and startup probes
|
||||
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes
|
||||
## @param dagProcessor.livenessProbe.enabled Enable livenessProbe on Airflow Dag Processor containers
|
||||
## @param dagProcessor.livenessProbe.initialDelaySeconds Initial delay seconds for livenessProbe
|
||||
## @param dagProcessor.livenessProbe.periodSeconds Period seconds for livenessProbe
|
||||
## @param dagProcessor.livenessProbe.timeoutSeconds Timeout seconds for livenessProbe
|
||||
## @param dagProcessor.livenessProbe.failureThreshold Failure threshold for livenessProbe
|
||||
## @param dagProcessor.livenessProbe.successThreshold Success threshold for livenessProbe
|
||||
##
|
||||
livenessProbe:
|
||||
enabled: true
|
||||
initialDelaySeconds: 180
|
||||
periodSeconds: 20
|
||||
timeoutSeconds: 15
|
||||
failureThreshold: 6
|
||||
successThreshold: 1
|
||||
## @param dagProcessor.readinessProbe.enabled Enable readinessProbe on Airflow Dag Processor containers
|
||||
## @param dagProcessor.readinessProbe.initialDelaySeconds Initial delay seconds for readinessProbe
|
||||
## @param dagProcessor.readinessProbe.periodSeconds Period seconds for readinessProbe
|
||||
## @param dagProcessor.readinessProbe.timeoutSeconds Timeout seconds for readinessProbe
|
||||
## @param dagProcessor.readinessProbe.failureThreshold Failure threshold for readinessProbe
|
||||
## @param dagProcessor.readinessProbe.successThreshold Success threshold for readinessProbe
|
||||
##
|
||||
readinessProbe:
|
||||
enabled: true
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 15
|
||||
failureThreshold: 6
|
||||
successThreshold: 1
|
||||
## @param dagProcessor.startupProbe.enabled Enable startupProbe on Airflow Dag Processor containers
|
||||
## @param dagProcessor.startupProbe.initialDelaySeconds Initial delay seconds for startupProbe
|
||||
## @param dagProcessor.startupProbe.periodSeconds Period seconds for startupProbe
|
||||
## @param dagProcessor.startupProbe.timeoutSeconds Timeout seconds for startupProbe
|
||||
## @param dagProcessor.startupProbe.failureThreshold Failure threshold for startupProbe
|
||||
## @param dagProcessor.startupProbe.successThreshold Success threshold for startupProbe
|
||||
##
|
||||
startupProbe:
|
||||
enabled: false
|
||||
initialDelaySeconds: 60
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 1
|
||||
failureThreshold: 15
|
||||
successThreshold: 1
|
||||
## @param dagProcessor.customLivenessProbe Custom livenessProbe that overrides the default one
|
||||
##
|
||||
customLivenessProbe: {}
|
||||
## @param dagProcessor.customReadinessProbe Custom readinessProbe that overrides the default one
|
||||
##
|
||||
customReadinessProbe: {}
|
||||
## @param dagProcessor.customStartupProbe Custom startupProbe that overrides the default one
|
||||
##
|
||||
customStartupProbe: {}
|
||||
## Airflow Dag Processor resource requests and limits
|
||||
## ref: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
|
||||
## @param dagProcessor.resourcesPreset Set container resources according to one common preset (allowed values: none, nano, micro, small, medium, large, xlarge, 2xlarge). This is ignored if dagProcessor.resources is set (dagProcessor.resources is recommended for production).
|
||||
## More information: https://github.com/bitnami/charts/blob/main/bitnami/common/templates/_resources.tpl#L15
|
||||
##
|
||||
resourcesPreset: "small"
|
||||
## @param dagProcessor.resources Set container requests and limits for different resources like CPU or memory (essential for production workloads)
|
||||
## Example:
|
||||
## resources:
|
||||
## requests:
|
||||
## cpu: 2
|
||||
## memory: 512Mi
|
||||
## limits:
|
||||
## cpu: 3
|
||||
## memory: 1024Mi
|
||||
##
|
||||
resources: {}
|
||||
## Configure Airflow Dag Processor pods Security Context
|
||||
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod
|
||||
## @param dagProcessor.podSecurityContext.enabled Enabled Airflow Dag Processor pods' Security Context
|
||||
## @param dagProcessor.podSecurityContext.fsGroupChangePolicy Set filesystem group change policy
|
||||
## @param dagProcessor.podSecurityContext.sysctls Set kernel settings using the sysctl interface
|
||||
## @param dagProcessor.podSecurityContext.supplementalGroups Set filesystem extra groups
|
||||
## @param dagProcessor.podSecurityContext.fsGroup Set Airflow Dag Processor pod's Security Context fsGroup
|
||||
##
|
||||
podSecurityContext:
|
||||
enabled: true
|
||||
fsGroupChangePolicy: Always
|
||||
sysctls: []
|
||||
supplementalGroups: []
|
||||
fsGroup: 1001
|
||||
## Configure Airflow Dag Processor containers (only main one) Security Context
|
||||
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container
|
||||
## @param dagProcessor.containerSecurityContext.enabled Enabled Airflow Dag Processor containers' Security Context
|
||||
## @param dagProcessor.containerSecurityContext.seLinuxOptions [object,nullable] Set SELinux options in container
|
||||
## @param dagProcessor.containerSecurityContext.runAsUser Set Airflow Dag Processor containers' Security Context runAsUser
|
||||
## @param dagProcessor.containerSecurityContext.runAsGroup Set Airflow Dag Processor containers' Security Context runAsGroup
|
||||
## @param dagProcessor.containerSecurityContext.runAsNonRoot Set Airflow Dag Processor containers' Security Context runAsNonRoot
|
||||
## @param dagProcessor.containerSecurityContext.privileged Set Airflow Dag Processor container's Security Context privileged
|
||||
## @param dagProcessor.containerSecurityContext.allowPrivilegeEscalation Set Airflow Dag Processor container's Security Context allowPrivilegeEscalation
|
||||
## @param dagProcessor.containerSecurityContext.readOnlyRootFilesystem Set Airflow Dag Processor container's Security Context readOnlyRootFilesystem
|
||||
## @param dagProcessor.containerSecurityContext.capabilities.drop List of capabilities to be dropped
|
||||
## @param dagProcessor.containerSecurityContext.seccompProfile.type Set container's Security Context seccomp profile
|
||||
##
|
||||
containerSecurityContext:
|
||||
enabled: true
|
||||
seLinuxOptions: {}
|
||||
runAsUser: 1001
|
||||
runAsGroup: 1001
|
||||
runAsNonRoot: true
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
seccompProfile:
|
||||
type: "RuntimeDefault"
|
||||
## @param dagProcessor.lifecycleHooks for the Airflow Dag Processor containers to automate configuration before or after startup
|
||||
##
|
||||
lifecycleHooks: {}
|
||||
## @param dagProcessor.automountServiceAccountToken Mount Service Account token in pod
|
||||
##
|
||||
automountServiceAccountToken: false
|
||||
## @param dagProcessor.hostAliases Deployment pod host aliases
|
||||
## https://kubernetes.io/docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases/
|
||||
##
|
||||
hostAliases: []
|
||||
## @param dagProcessor.podLabels Add extra labels to the Airflow Dag Processor pods
|
||||
##
|
||||
podLabels: {}
|
||||
## @param dagProcessor.podAnnotations Add extra annotations to the Airflow Dag Processor pods
|
||||
##
|
||||
podAnnotations: {}
|
||||
## @param dagProcessor.affinity Affinity for Airflow Dag Processor pods assignment (evaluated as a template)
|
||||
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
|
||||
## Note: `dagProcessor.podAffinityPreset`, `dagProcessor.podAntiAffinityPreset`, and `dagProcessor.nodeAffinityPreset` will be ignored when it's set
|
||||
##
|
||||
affinity: {}
|
||||
## Node affinity preset
|
||||
## Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity
|
||||
## @param dagProcessor.nodeAffinityPreset.key Node label key to match. Ignored if `dagProcessor.affinity` is set.
|
||||
## @param dagProcessor.nodeAffinityPreset.type Node affinity preset type. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard`
|
||||
## @param dagProcessor.nodeAffinityPreset.values Node label values to match. Ignored if `dagProcessor.affinity` is set.
|
||||
##
|
||||
nodeAffinityPreset:
|
||||
## e.g:
|
||||
## key: "kubernetes.io/e2e-az-name"
|
||||
##
|
||||
key: ""
|
||||
type: ""
|
||||
## e.g:
|
||||
## values:
|
||||
## - e2e-az1
|
||||
## - e2e-az2
|
||||
##
|
||||
values: []
|
||||
## @param dagProcessor.nodeSelector Node labels for Airflow Dag Processor pods assignment
|
||||
## Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
|
||||
##
|
||||
nodeSelector: {}
|
||||
## @param dagProcessor.podAffinityPreset Pod affinity preset. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard`.
|
||||
## ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
|
||||
##
|
||||
podAffinityPreset: ""
|
||||
## @param dagProcessor.podAntiAffinityPreset Pod anti-affinity preset. Ignored if `dagProcessor.affinity` is set. Allowed values: `soft` or `hard`.
|
||||
## Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
|
||||
##
|
||||
podAntiAffinityPreset: soft
|
||||
## @param dagProcessor.tolerations Tolerations for Airflow Dag Processor pods assignment
|
||||
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
|
||||
##
|
||||
tolerations: []
|
||||
## @param dagProcessor.topologySpreadConstraints Topology Spread Constraints for pod assignment spread across your cluster among failure-domains. Evaluated as a template
|
||||
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/#spread-constraints-for-pods
|
||||
##
|
||||
topologySpreadConstraints: []
|
||||
## @param dagProcessor.priorityClassName Priority Class Name
|
||||
## ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
|
||||
##
|
||||
priorityClassName: ""
|
||||
## @param dagProcessor.schedulerName Use an alternate K8s scheduler, e.g. "stork".
|
||||
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
|
||||
##
|
||||
schedulerName: ""
|
||||
## @param dagProcessor.terminationGracePeriodSeconds Seconds Airflow Dag Processor pod needs to terminate gracefully
|
||||
## ref: https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods
|
||||
##
|
||||
terminationGracePeriodSeconds: ""
|
||||
## @param dagProcessor.updateStrategy.type Airflow Dag Processor deployment strategy type
|
||||
## @param dagProcessor.updateStrategy.rollingUpdate Airflow Dag Processor deployment rolling update configuration parameters
|
||||
## ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy
|
||||
##
|
||||
updateStrategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate: {}
|
||||
## @param dagProcessor.sidecars Add additional sidecar containers to the Airflow Dag Processor pods
|
||||
## Example:
|
||||
## sidecars:
|
||||
## - name: your-image-name
|
||||
## image: your-image
|
||||
## imagePullPolicy: Always
|
||||
## ports:
|
||||
## - name: portname
|
||||
## containerPort: 1234
|
||||
##
|
||||
sidecars: []
|
||||
## @param dagProcessor.initContainers Add additional init containers to the Airflow Dag Processor pods
|
||||
## Example:
|
||||
## initContainers:
|
||||
## - name: your-image-name
|
||||
## image: your-image
|
||||
## imagePullPolicy: Always
|
||||
## ports:
|
||||
## - name: portname
|
||||
## containerPort: 1234
|
||||
##
|
||||
initContainers: []
|
||||
## @param dagProcessor.extraVolumeMounts Optionally specify extra list of additional volumeMounts for the Airflow Dag Processor containers
|
||||
##
|
||||
extraVolumeMounts: []
|
||||
## @param dagProcessor.extraVolumes Optionally specify extra list of additional volumes for the Airflow Dag Processor pods
|
||||
##
|
||||
extraVolumes: []
|
||||
## Airflow Dag Processor Pod Disruption Budget
|
||||
## ref: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
|
||||
## @param dagProcessor.pdb.create Deploy a pdb object for the Airflow Dag Processor pods
|
||||
## @param dagProcessor.pdb.minAvailable Maximum number/percentage of unavailable Airflow Dag Processor replicas
|
||||
## @param dagProcessor.pdb.maxUnavailable Maximum number/percentage of unavailable Airflow Dag Processor replicas
|
||||
##
|
||||
pdb:
|
||||
create: true
|
||||
minAvailable: ""
|
||||
maxUnavailable: ""
|
||||
## Airflow Dag Processor Network Policies
|
||||
## Ref: https://kubernetes.io/docs/concepts/services-networking/network-policies/
|
||||
##
|
||||
networkPolicy:
|
||||
## @param dagProcessor.networkPolicy.enabled Specifies whether a NetworkPolicy should be created
|
||||
##
|
||||
enabled: true
|
||||
## @param dagProcessor.networkPolicy.allowExternal Don't require client label for connections
|
||||
## The Policy model to apply. When set to false, only pods with the correct
|
||||
## client label will have network access to the ports Airflow Dag Processor is listening
|
||||
## on. When true, Airflow Dag Processor will accept connections from any source
|
||||
## (with the correct destination port).
|
||||
##
|
||||
allowExternal: true
|
||||
## @param dagProcessor.networkPolicy.allowExternalEgress Allow the pod to access any range of port and all destinations.
|
||||
##
|
||||
allowExternalEgress: true
|
||||
## @param dagProcessor.networkPolicy.extraIngress [array] Add extra ingress rules to the NetworkPolicy
|
||||
## e.g:
|
||||
## extraIngress:
|
||||
## - ports:
|
||||
## - port: 1234
|
||||
## from:
|
||||
## - podSelector:
|
||||
## - matchLabels:
|
||||
## - role: frontend
|
||||
## - podSelector:
|
||||
## - matchExpressions:
|
||||
## - key: role
|
||||
## operator: In
|
||||
## values:
|
||||
## - frontend
|
||||
extraIngress: []
|
||||
## @param dagProcessor.networkPolicy.extraEgress [array] Add extra ingress rules to the NetworkPolicy
|
||||
## e.g:
|
||||
## extraEgress:
|
||||
## - ports:
|
||||
## - port: 1234
|
||||
## to:
|
||||
## - podSelector:
|
||||
## - matchLabels:
|
||||
## - role: frontend
|
||||
## - podSelector:
|
||||
## - matchExpressions:
|
||||
## - key: role
|
||||
## operator: In
|
||||
## values:
|
||||
## - frontend
|
||||
##
|
||||
extraEgress: []
|
||||
## @param dagProcessor.networkPolicy.ingressNSMatchLabels [object] Labels to match to allow traffic from other namespaces
|
||||
## @param dagProcessor.networkPolicy.ingressNSPodMatchLabels [object] Pod labels to match to allow traffic from other namespaces
|
||||
##
|
||||
ingressNSMatchLabels: {}
|
||||
ingressNSPodMatchLabels: {}
|
||||
|
||||
## @section Airflow worker parameters
|
||||
##
|
||||
worker:
|
||||
|
||||
Reference in New Issue
Block a user