[bitnami/spring-cloud-dataflow] Add support for using different external db credentials for Dataflow Server & Skipper (#24386)

This commit is contained in:
Juan Ariza Toledano
2024-03-13 16:03:44 +01:00
committed by GitHub
parent f7cf9354ed
commit 06783d238d
11 changed files with 138 additions and 64 deletions

View File

@@ -53,4 +53,4 @@ maintainers:
name: spring-cloud-dataflow name: spring-cloud-dataflow
sources: sources:
- https://github.com/bitnami/charts/tree/main/bitnami/spring-cloud-dataflow - https://github.com/bitnami/charts/tree/main/bitnami/spring-cloud-dataflow
version: 26.10.0 version: 26.11.0

View File

@@ -489,36 +489,42 @@ helm uninstall my-release
### Database parameters ### Database parameters
| Name | Description | Value | | Name | Description | Value |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------- | | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| `mariadb.enabled` | Enable/disable MariaDB chart installation | `true` | | `mariadb.enabled` | Enable/disable MariaDB chart installation | `true` |
| `mariadb.jdbcParameter.useMysqlMetadata` | Use MariaDB useMysqlMetadata parameter. | `true` | | `mariadb.jdbcParameter.useMysqlMetadata` | Use MariaDB useMysqlMetadata parameter. | `true` |
| `mariadb.image.registry` | MariaDB image registry | `REGISTRY_NAME` | | `mariadb.image.registry` | MariaDB image registry | `REGISTRY_NAME` |
| `mariadb.image.repository` | MariaDB image repository | `REPOSITORY_NAME/mariadb` | | `mariadb.image.repository` | MariaDB image repository | `REPOSITORY_NAME/mariadb` |
| `mariadb.image.digest` | MariaDB image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag | `""` | | `mariadb.image.digest` | MariaDB image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag | `""` |
| `mariadb.architecture` | MariaDB architecture. Allowed values: `standalone` or `replication` | `standalone` | | `mariadb.architecture` | MariaDB architecture. Allowed values: `standalone` or `replication` | `standalone` |
| `mariadb.auth.rootPassword` | Password for the MariaDB `root` user | `""` | | `mariadb.auth.rootPassword` | Password for the MariaDB `root` user | `""` |
| `mariadb.auth.username` | Username of new user to create | `dataflow` | | `mariadb.auth.username` | Username of new user to create | `dataflow` |
| `mariadb.auth.password` | Password for the new user | `change-me` | | `mariadb.auth.password` | Password for the new user | `change-me` |
| `mariadb.auth.database` | Database name to create | `dataflow` | | `mariadb.auth.database` | Database name to create | `dataflow` |
| `mariadb.auth.forcePassword` | Force users to specify required passwords in the database | `false` | | `mariadb.auth.forcePassword` | Force users to specify required passwords in the database | `false` |
| `mariadb.auth.usePasswordFiles` | Mount credentials as a file instead of using an environment variable | `false` | | `mariadb.auth.usePasswordFiles` | Mount credentials as a file instead of using an environment variable | `false` |
| `mariadb.initdbScripts` | Specify dictionary of scripts to be run at first boot | `{}` | | `mariadb.initdbScripts` | Specify dictionary of scripts to be run at first boot | `{}` |
| `flyway.enabled` | Enable/disable flyway running Dataflow and Skipper Database creation scripts on startup | `true` | | `flyway.enabled` | Enable/disable flyway running Dataflow and Skipper Database creation scripts on startup | `true` |
| `externalDatabase.host` | Host of the external database | `localhost` | | `externalDatabase.host` | Host of the external database | `localhost` |
| `externalDatabase.port` | External database port number | `3306` | | `externalDatabase.port` | External database port number | `3306` |
| `externalDatabase.driver` | The fully qualified name of the JDBC Driver class | `""` | | `externalDatabase.driver` | The fully qualified name of the JDBC Driver class | `""` |
| `externalDatabase.scheme` | The scheme is a vendor-specific or shared protocol string that follows the "jdbc:" of the URL | `""` | | `externalDatabase.scheme` | The scheme is a vendor-specific or shared protocol string that follows the "jdbc:" of the URL | `""` |
| `externalDatabase.password` | Password for the above username | `""` | | `externalDatabase.hibernateDialect` | Hibernate Dialect used by Dataflow/Skipper servers | `""` |
| `externalDatabase.existingPasswordSecret` | Existing secret with database password | `""` | | `externalDatabase.password` | External database password (DEPRECATED: use dataflow.password and skipper.password instead) | `""` |
| `externalDatabase.existingPasswordKey` | Key of the existing secret with database password, defaults to `datasource-password` | `""` | | `externalDatabase.existingPasswordSecret` | Existing secret with database password (DEPRECATED: use dataflow.existingSecret and skipper.existingSecret instead) | `""` |
| `externalDatabase.dataflow.url` | JDBC URL for dataflow server. Overrides external scheme, host, port, database, and jdbc parameters. | `""` | | `externalDatabase.existingPasswordKey` | Key of the existing secret with database password (DEPRECATED: use dataflow.existingSecretPasswordKey and skipper.existingSecretPasswordKey instead) | `""` |
| `externalDatabase.dataflow.database` | Name of the existing database to be used by Dataflow server | `dataflow` | | `externalDatabase.dataflow.url` | JDBC URL for dataflow server. Overrides external scheme, host, port, password, and dataflow.database parameters. | `""` |
| `externalDatabase.dataflow.username` | Existing username in the external db to be used by Dataflow server | `dataflow` | | `externalDatabase.dataflow.database` | Name of the existing database to be used by Dataflow server. Ignored if url is provided | `dataflow` |
| `externalDatabase.skipper.url` | JDBC URL for skipper. Overrides external scheme, host, port, database, and jdbc parameters. | `""` | | `externalDatabase.dataflow.username` | Existing username in the external db to be used by Dataflow server | `dataflow` |
| `externalDatabase.skipper.database` | Name of the existing database to be used by Skipper server | `skipper` | | `externalDatabase.dataflow.password` | Password for the above username. Ignored if existing secret is provided | `""` |
| `externalDatabase.skipper.username` | Existing username in the external db to be used by Skipper server | `skipper` | | `externalDatabase.dataflow.existingSecret` | Name of the existing secret containing database credentials for Dataflow server | `""` |
| `externalDatabase.hibernateDialect` | Hibernate Dialect used by Dataflow/Skipper servers | `""` | | `externalDatabase.dataflow.existingSecretPasswordKey` | Key of the above existing secret with database password, defaults to `datasource-password` | `""` |
| `externalDatabase.skipper.url` | JDBC URL for skipper. Overrides external scheme, host, port, database, and skipper.database parameters. | `""` |
| `externalDatabase.skipper.database` | Name of the existing database to be used by Skipper server. Ignored if url is provided | `skipper` |
| `externalDatabase.skipper.username` | Existing username in the external db to be used by Skipper server | `skipper` |
| `externalDatabase.skipper.password` | Password for the above username. Ignored if existing secret is provided | `""` |
| `externalDatabase.skipper.existingSecret` | Name of the existing secret containing database credentials for Skipper server | `""` |
| `externalDatabase.skipper.existingSecretPasswordKey` | Key of the above existing secret with database password, defaults to `datasource-password` | `""` |
### RabbitMQ chart parameters ### RabbitMQ chart parameters

View File

@@ -74,7 +74,6 @@ To access Spring Cloud Data Flow dashboard from outside the cluster execute the
{{- $passwordErrors := list -}} {{- $passwordErrors := list -}}
{{- $secretNameMariadb := include "scdf.mariadb.fullname" . -}} {{- $secretNameMariadb := include "scdf.mariadb.fullname" . -}}
{{- $secretNameExternalDb := printf "%s-%s" (include "common.names.fullname" .) "externaldb" -}}
{{- $secretNameRabbitmq := include "scdf.rabbitmq.fullname" . -}} {{- $secretNameRabbitmq := include "scdf.rabbitmq.fullname" . -}}
{{- $secretNameExternalRabbitmq := printf "%s-%s" (include "common.names.fullname" .) "externalrabbitmq" -}} {{- $secretNameExternalRabbitmq := printf "%s-%s" (include "common.names.fullname" .) "externalrabbitmq" -}}
@@ -96,12 +95,16 @@ To access Spring Cloud Data Flow dashboard from outside the cluster execute the
{{- $passwordWarnings := list -}} {{- $passwordWarnings := list -}}
{{- if not .Values.mariadb.enabled -}} {{- if not .Values.mariadb.enabled -}}
{{- $requiredExternalDbPassword := dict "valueKey" "externalDatabase.password" "secret" $secretNameExternalDb "field" "mariadb-password" -}} {{- $secretNameServerExternalDb := printf "%s-externaldb" (include "scdf.server.fullname" .) -}}
{{- $passwordWarnings = append $passwordWarnings $requiredExternalDbPassword -}} {{- $requiredServerExternalDbPassword := dict "valueKey" "externalDatabase.dataflow.password" "secret" $secretNameServerExternalDb "field" "datasource-password" -}}
{{- $passwordWarnings = append $passwordWarnings $requiredServerExternalDbPassword -}}
{{- $secretNameSkipperExternalDb := printf "%s-externaldb" (include "scdf.skipper.fullname" .) -}}
{{- $requiredSkipperExternalDbPassword := dict "valueKey" "externalDatabase.skipper.password" "secret" $secretNameSkipperExternalDb "field" "datasource-password" -}}
{{- $passwordWarnings = append $passwordWarnings $requiredSkipperExternalDbPassword -}}
{{- end -}} {{- end -}}
{{- if and (.Values.externalRabbitmq.enabled) (not .Values.rabbitmq.enabled) (not .Values.externalRabbitmq.existingPasswordSecret) -}} {{- if and (.Values.externalRabbitmq.enabled) (not .Values.rabbitmq.enabled) (not .Values.externalRabbitmq.existingPasswordSecret) -}}
{{- $requiredExternalRabbitmqPassword := dict "valueKey" "externalRabbitmq.password" "secret" $secretNameExternalRabbitmq "field" "rabbitmq-password" -}} {{- $requiredExternalRabbitmqPassword := dict "valueKey" "externalRabbitmq.password" "secret" $secretNameExternalRabbitmq "field" "rabbitmq-password" -}}
{{- $passwordWarnings = append $passwordWarnings $requiredExternalRabbitmqPassword -}} {{- $passwordWarnings = append $passwordWarnings $requiredExternalRabbitmqPassword -}}
{{- end -}} {{- end -}}
{{- if not (empty $passwordWarnings) -}} {{- if not (empty $passwordWarnings) -}}

View File

@@ -237,6 +237,20 @@ Return the Data Flow Database User
{{- end -}} {{- end -}}
{{- end -}} {{- end -}}
{{/*
Return the Data Flow Database secret name
*/}}
{{- define "scdf.database.server.secretName" -}}
{{- $secretName := coalesce .Values.externalDatabase.dataflow.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
{{- if $secretName -}}
{{- printf "%s" $secretName -}}
{{- else if .Values.mariadb.enabled }}
{{- printf "%s" (include "scdf.mariadb.fullname" .) -}}
{{- else -}}
{{- printf "%s-externaldb" (include "scdf.server.fullname" .) -}}
{{- end -}}
{{- end -}}
{{/* {{/*
Return the Skipper Database Name Return the Skipper Database Name
*/}} */}}
@@ -260,15 +274,16 @@ Return the Skipper Database User
{{- end -}} {{- end -}}
{{/* {{/*
Return the Database secret name Return the Skipper Database secret name
*/}} */}}
{{- define "scdf.database.secretName" -}} {{- define "scdf.database.skipper.secretName" -}}
{{- if .Values.externalDatabase.existingPasswordSecret -}} {{- $secretName := coalesce .Values.externalDatabase.skipper.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
{{- printf "%s" .Values.externalDatabase.existingPasswordSecret -}} {{- if $secretName -}}
{{- printf "%s" $secretName -}}
{{- else if .Values.mariadb.enabled }} {{- else if .Values.mariadb.enabled }}
{{- printf "%s" (include "scdf.mariadb.fullname" .) -}} {{- printf "%s" (include "scdf.mariadb.fullname" .) -}}
{{- else -}} {{- else -}}
{{- printf "%s-%s" (include "common.names.fullname" .) "externaldb" -}} {{- printf "%s-externaldb" (include "scdf.skipper.fullname" .) -}}
{{- end -}} {{- end -}}
{{- end -}} {{- end -}}

View File

@@ -113,10 +113,14 @@ data:
url: '{{ include "scdf.database.dataflow.url" . }}' url: '{{ include "scdf.database.dataflow.url" . }}'
driverClassName: {{ include "scdf.database.driver" . }} driverClassName: {{ include "scdf.database.driver" . }}
username: {{ include "scdf.database.server.user" . }} username: {{ include "scdf.database.server.user" . }}
{{ if .Values.externalDatabase.existingPasswordSecret }} {{- $secretName := coalesce .Values.externalDatabase.dataflow.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
password: {{ .Values.externalDatabase.existingPasswordKey | default "datasource-password" | printf "${%s}" }} {{- $secretPasswordKey := coalesce .Values.externalDatabase.dataflow.existingSecretPasswordKey .Values.externalDatabase.existingPasswordKey -}}
{{- else -}} {{- if .Values.mariadb.enabled }}
password: ${mariadb-password} password: ${mariadb-password}
{{- else if and $secretName $secretPasswordKey }}
password: {{ printf "${%s}" $secretPasswordKey }}
{{- else }}
password: ${datasource-password}
{{- end }} {{- end }}
testOnBorrow: true testOnBorrow: true
validationQuery: "SELECT 1" validationQuery: "SELECT 1"

View File

@@ -266,7 +266,7 @@ spec:
emptyDir: {} emptyDir: {}
- name: database - name: database
secret: secret:
secretName: {{ include "scdf.database.secretName" . }} secretName: {{ include "scdf.database.server.secretName" . }}
- name: config - name: config
configMap: configMap:
name: {{ include "scdf.server.configmapName" . }} name: {{ include "scdf.server.configmapName" . }}

View File

@@ -0,0 +1,20 @@
{{- /*
Copyright VMware, Inc.
SPDX-License-Identifier: APACHE-2.0
*/}}
{{- $secretName := coalesce .Values.externalDatabase.dataflow.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
{{- if and (not .Values.mariadb.enabled) (not $secretName) }}
apiVersion: v1
kind: Secret
metadata:
name: {{ printf "%s-externaldb" (include "scdf.server.fullname" .) }}
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
namespace: {{ include "common.names.namespace" . | quote }}
{{- if .Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
type: Opaque
data:
datasource-password: {{ coalesce .Values.externalDatabase.dataflow.password .Values.externalDatabase.password | b64enc | quote }}
{{- end }}

View File

@@ -122,10 +122,14 @@ data:
url: '{{ include "scdf.database.skipper.url" . }}' url: '{{ include "scdf.database.skipper.url" . }}'
driverClassName: {{ include "scdf.database.driver" . }} driverClassName: {{ include "scdf.database.driver" . }}
username: {{ include "scdf.database.skipper.user" . }} username: {{ include "scdf.database.skipper.user" . }}
{{ if .Values.externalDatabase.existingPasswordSecret }} {{- $secretName := coalesce .Values.externalDatabase.skipper.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
password: {{ .Values.externalDatabase.existingPasswordKey | default "datasource-password" | printf "${%s}" }} {{- $secretPasswordKey := coalesce .Values.externalDatabase.skipper.existingSecretPasswordKey .Values.externalDatabase.existingPasswordKey -}}
{{- else -}} {{- if .Values.mariadb.enabled }}
password: ${mariadb-password} password: ${mariadb-password}
{{- else if and $secretName $secretPasswordKey }}
password: {{ printf "${%s}" $secretPasswordKey }}
{{- else }}
password: ${datasource-password}
{{- end }} {{- end }}
testOnBorrow: true testOnBorrow: true
validationQuery: "SELECT 1" validationQuery: "SELECT 1"

View File

@@ -232,7 +232,7 @@ spec:
emptyDir: {} emptyDir: {}
- name: database - name: database
secret: secret:
secretName: {{ include "scdf.database.secretName" . }} secretName: {{ include "scdf.database.skipper.secretName" . }}
- name: config - name: config
configMap: configMap:
name: {{ include "scdf.skipper.configmapName" . }} name: {{ include "scdf.skipper.configmapName" . }}

View File

@@ -3,11 +3,12 @@ Copyright VMware, Inc.
SPDX-License-Identifier: APACHE-2.0 SPDX-License-Identifier: APACHE-2.0
*/}} */}}
{{- if and (not .Values.mariadb.enabled) (not .Values.externalDatabase.existingPasswordSecret) }} {{- $secretName := coalesce .Values.externalDatabase.skipper.existingSecret .Values.externalDatabase.existingPasswordSecret -}}
{{- if and (not .Values.mariadb.enabled) (not $secretName) }}
apiVersion: v1 apiVersion: v1
kind: Secret kind: Secret
metadata: metadata:
name: {{ printf "%s-%s" (include "common.names.fullname" .) "externaldb" }} name: {{ printf "%s-externaldb" (include "scdf.skipper.fullname" .) }}
labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }} labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
namespace: {{ include "common.names.namespace" . | quote }} namespace: {{ include "common.names.namespace" . | quote }}
{{- if .Values.commonAnnotations }} {{- if .Values.commonAnnotations }}
@@ -15,5 +16,5 @@ metadata:
{{- end }} {{- end }}
type: Opaque type: Opaque
data: data:
mariadb-password: {{ .Values.externalDatabase.password | b64enc | quote }} datasource-password: {{ coalesce .Values.externalDatabase.skipper.password .Values.externalDatabase.password | b64enc | quote }}
{{- end }} {{- end }}

View File

@@ -1869,49 +1869,70 @@ externalDatabase:
## @param externalDatabase.scheme The scheme is a vendor-specific or shared protocol string that follows the "jdbc:" of the URL ## @param externalDatabase.scheme The scheme is a vendor-specific or shared protocol string that follows the "jdbc:" of the URL
## ##
scheme: "" scheme: ""
## @param externalDatabase.password Password for the above username ## @param externalDatabase.hibernateDialect Hibernate Dialect used by Dataflow/Skipper servers
## e.g: org.hibernate.dialect.MariaDB102Dialect
##
hibernateDialect: ""
## @param externalDatabase.password External database password (DEPRECATED: use dataflow.password and skipper.password instead)
## ##
password: "" password: ""
## @param externalDatabase.existingPasswordSecret Existing secret with database password ## @param externalDatabase.existingPasswordSecret Existing secret with database password (DEPRECATED: use dataflow.existingSecret and skipper.existingSecret instead)
## ##
existingPasswordSecret: "" existingPasswordSecret: ""
## @param externalDatabase.existingPasswordKey Key of the existing secret with database password, defaults to `datasource-password` ## @param externalDatabase.existingPasswordKey Key of the existing secret with database password (DEPRECATED: use dataflow.existingSecretPasswordKey and skipper.existingSecretPasswordKey instead)
## ##
existingPasswordKey: "" existingPasswordKey: ""
## Data Flow user and database ## Data Flow user and database
## ##
dataflow: dataflow:
## @param externalDatabase.dataflow.url JDBC URL for dataflow server. Overrides external scheme, host, port, database, and jdbc parameters. ## @param externalDatabase.dataflow.url JDBC URL for dataflow server. Overrides external scheme, host, port, password, and dataflow.database parameters.
## This provides a mechanism to define a fully customized JDBC URL for the data flow server rather than having it ## This provides a mechanism to define a fully customized JDBC URL for the data flow server rather than having it
## derived from the common, individual attributes. This property, when defined, has precedence over the ## derived from the common, individual attributes. This property, when defined, has precedence over the
## individual attributes (scheme, host, port, database) ## individual attributes (scheme, host, port, dataflow.database)
## Ensure you don't include userspec in the URL and use the username and password fields instead
## ##
url: "" url: ""
## @param externalDatabase.dataflow.database Name of the existing database to be used by Dataflow server ## @param externalDatabase.dataflow.database Name of the existing database to be used by Dataflow server. Ignored if url is provided
## ##
database: dataflow database: dataflow
## @param externalDatabase.dataflow.username Existing username in the external db to be used by Dataflow server ## @param externalDatabase.dataflow.username Existing username in the external db to be used by Dataflow server
## ##
username: dataflow username: dataflow
## @param externalDatabase.dataflow.password Password for the above username. Ignored if existing secret is provided
##
password: ""
## @param externalDatabase.dataflow.existingSecret Name of the existing secret containing database credentials for Dataflow server
##
existingSecret: ""
## @param externalDatabase.dataflow.existingSecretPasswordKey Key of the above existing secret with database password, defaults to `datasource-password`
##
existingSecretPasswordKey: ""
## Skipper and database ## Skipper and database
## ##
skipper: skipper:
## @param externalDatabase.skipper.url JDBC URL for skipper. Overrides external scheme, host, port, database, and jdbc parameters. ## @param externalDatabase.skipper.url JDBC URL for skipper. Overrides external scheme, host, port, database, and skipper.database parameters.
## This provides a mechanism to define a fully customized JDBC URL for skipper rather than having it ## This provides a mechanism to define a fully customized JDBC URL for skipper rather than having it
## derived from the common, individual attributes. This property, when defined, has precedence over the ## derived from the common, individual attributes. This property, when defined, has precedence over the
## individual attributes (scheme, host, port, database) ## individual attributes (scheme, host, port, skipper.database)
## Ensure you don't include userspec in the URL and use the username and password fields instead
## ##
url: "" url: ""
## @param externalDatabase.skipper.database Name of the existing database to be used by Skipper server ## @param externalDatabase.skipper.database Name of the existing database to be used by Skipper server. Ignored if url is provided
## ##
database: skipper database: skipper
## @param externalDatabase.skipper.username Existing username in the external db to be used by Skipper server ## @param externalDatabase.skipper.username Existing username in the external db to be used by Skipper server
## ##
username: skipper username: skipper
## @param externalDatabase.hibernateDialect Hibernate Dialect used by Dataflow/Skipper servers ## @param externalDatabase.skipper.password Password for the above username. Ignored if existing secret is provided
## e.g: org.hibernate.dialect.MariaDB102Dialect ##
## password: ""
hibernateDialect: "" ## @param externalDatabase.skipper.existingSecret Name of the existing secret containing database credentials for Skipper server
##
existingSecret: ""
## @param externalDatabase.skipper.existingSecretPasswordKey Key of the above existing secret with database password, defaults to `datasource-password`
##
existingSecretPasswordKey: ""
## @section RabbitMQ chart parameters ## @section RabbitMQ chart parameters
## RabbitMQ chart configuration ## RabbitMQ chart configuration