mirror of
https://github.com/bitnami/containers.git
synced 2026-03-08 02:37:20 +08:00
Merge remote-tracking branch 'airflow-scheduler/master' into main
This commit is contained in:
Miguel A. Cabrera Minagorri
41
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/bug_report.yml
vendored
Normal file
41
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/bug_report.yml
vendored
Normal file
@@ -0,0 +1,41 @@
|
||||
name: 🐞 Bug
|
||||
description: Create a report to help us improve
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for reporting an issue. Before you open the bug report please review the README file present in the root of this repository.
|
||||
|
||||
Please fill in as much of the following form as you're able.
|
||||
- type: input
|
||||
attributes:
|
||||
label: Name and Version
|
||||
description: Name and version of the affected container image
|
||||
placeholder: bitnami/wordpress:1.2.3
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What steps will reproduce the bug?
|
||||
description: Enter details about your bug.
|
||||
placeholder: |
|
||||
1. In this environment...
|
||||
2. With this config...
|
||||
3. Run '...'
|
||||
4. See error...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What is the expected behavior?
|
||||
description: If possible please provide textual output instead of screenshots.
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What do you see instead?
|
||||
description: If possible please provide textual output instead of screenshots.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional information
|
||||
description: Tell us anything else you think we should know.
|
||||
1
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
1
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
@@ -0,0 +1 @@
|
||||
blank_issues_enabled: false
|
||||
29
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/feature_request.yml
vendored
Normal file
29
bitnami/airflow-scheduler/.github/ISSUE_TEMPLATE/feature_request.yml
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
name: "\U0001F680 Feature request"
|
||||
description: Suggest an idea for this project
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for suggesting an idea to improve this Bitnami container image.
|
||||
Please fill in as much of the following form as you're able.
|
||||
- type: input
|
||||
attributes:
|
||||
label: Name and Version
|
||||
description: Name and version of the affected container image
|
||||
placeholder: bitnami/wordpress:1.2.3
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What is the problem this feature will solve?
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What is the feature you are proposing to solve the problem?
|
||||
description: Describe the requests. If you already have something in mind... PRs are welcome!
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: What alternatives have you considered?
|
||||
32
bitnami/airflow-scheduler/.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
32
bitnami/airflow-scheduler/.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
<!--
|
||||
Before you open the request please review the following guidelines and tips to help it be more easily integrated:
|
||||
|
||||
- Describe the scope of your change - i.e. what the change does.
|
||||
- Describe any known limitations with your change.
|
||||
- Please run any tests or examples that can exercise your modified code.
|
||||
|
||||
Thank you for contributing! We will try to test and integrate the change as soon as we can, but be aware we have many GitHub repositories to manage and can't immediately respond to every request. There is no need to bump or check in on a pull request (it will clutter the discussion of the request).
|
||||
|
||||
Also don't be worried if the request is closed or not integrated sometimes the priorities of Bitnami might not match the priorities of the pull request. Don't fret, the open source community thrives on forks and GitHub makes it easy to keep your changes in a forked repo.
|
||||
-->
|
||||
|
||||
**Description of the change**
|
||||
|
||||
<!-- Describe the scope of your change - i.e. what the change does. -->
|
||||
|
||||
**Benefits**
|
||||
|
||||
<!-- What benefits will be realized by the code change? -->
|
||||
|
||||
**Possible drawbacks**
|
||||
|
||||
<!-- Describe any known limitations with your change -->
|
||||
|
||||
**Applicable issues**
|
||||
|
||||
<!-- Enter any applicable Issues here (You can reference an issue using #) -->
|
||||
|
||||
**Additional information**
|
||||
|
||||
<!-- If there's anything else that's important and relevant to your pull
|
||||
request, mention that information here.-->
|
||||
20
bitnami/airflow-scheduler/.github/workflows/stale.yml
vendored
Normal file
20
bitnami/airflow-scheduler/.github/workflows/stale.yml
vendored
Normal file
@@ -0,0 +1,20 @@
|
||||
name: 'Close stale issues and PRs'
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 1 * * *'
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@v3
|
||||
with:
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
stale-issue-message: 'This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.'
|
||||
stale-pr-message: 'This Pull Request has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thank you for your contribution.'
|
||||
close-issue-message: 'Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.'
|
||||
close-pr-message: 'Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Pull Request. Do not hesitate to reopen it later if necessary.'
|
||||
days-before-stale: 15
|
||||
days-before-close: 5
|
||||
exempt-issue-labels: 'on-hold'
|
||||
exempt-pr-labels: 'on-hold'
|
||||
38
bitnami/airflow-scheduler/2/debian-10/Dockerfile
Normal file
38
bitnami/airflow-scheduler/2/debian-10/Dockerfile
Normal file
@@ -0,0 +1,38 @@
|
||||
FROM docker.io/bitnami/minideb:buster
|
||||
LABEL maintainer "Bitnami <containers@bitnami.com>"
|
||||
|
||||
ENV BITNAMI_PKG_EXTRA_DIRS="/opt/bitnami/airflow/dags" \
|
||||
HOME="/" \
|
||||
OS_ARCH="amd64" \
|
||||
OS_FLAVOUR="debian-10" \
|
||||
OS_NAME="linux"
|
||||
|
||||
COPY prebuildfs /
|
||||
# Install required system packages and dependencies
|
||||
RUN install_packages acl ca-certificates curl gzip libbsd0 libbz2-1.0 libc6 libcom-err2 libcurl4 libedit2 libffi6 libgcc1 libgcrypt20 libgmp10 libgnutls30 libgpg-error0 libgssapi-krb5-2 libhogweed4 libicu63 libidn2-0 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 liblzma5 libmariadb3 libncursesw6 libnettle6 libnghttp2-14 libp11-kit0 libpsl5 libreadline7 librtmp1 libsasl2-2 libsqlite3-0 libssh2-1 libssl1.1 libstdc++6 libtasn1-6 libtinfo6 libunistring2 libuuid1 libxml2 libxslt1.1 netbase procps tar zlib1g
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "wait-for-port" "1.0.1-10" --checksum 35c818ba3f4b5aae905959bc7d3a5e81fc63786e3c662b604612c0aa7fcda8fd
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "python" "3.8.13-2" --checksum 704fe6b545280ae027303b4ec875b495ee6da2256b3d8f877467077b0f031a60
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "postgresql-client" "10.20.0-5" --checksum efc2ccb03c4f0998d86e688d488e45aa534f27e317b253e7fcf037b7c78b7469
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "ini-file" "1.4.1-7" --checksum 7a4f45cbebd2e242278060415dd1a3f0dbe68991fdcaea5bb311e8de54a3e027
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "git" "2.35.1-6" --checksum 75c0294b7af868721549926139e6ddbec9548fbd0e69c7e199d640a4531d858b
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "gosu" "1.14.0-7" --checksum d6280b6f647a62bf6edc74dc8e526bfff63ddd8067dcb8540843f47203d9ccf1
|
||||
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "airflow-scheduler" "2.2.5-0" --checksum d82942ec43230733ea31e587602c6e63ddc6814154439e705685d347af7991de
|
||||
RUN apt-get update && apt-get upgrade -y && \
|
||||
rm -r /var/lib/apt/lists /var/cache/apt/archives
|
||||
RUN chmod g+rwX /opt/bitnami
|
||||
|
||||
COPY rootfs /
|
||||
RUN /opt/bitnami/scripts/airflow-scheduler/postunpack.sh
|
||||
ENV AIRFLOW_HOME="/opt/bitnami/airflow" \
|
||||
APP_VERSION="2.2.5" \
|
||||
BITNAMI_APP_NAME="airflow-scheduler" \
|
||||
LD_LIBRARY_PATH="/opt/bitnami/python/lib/:/opt/bitnami/airflow/venv/lib/python3.8/site-packages/numpy.libs/:$LD_LIBRARY_PATH" \
|
||||
LIBNSS_WRAPPER_PATH="/opt/bitnami/common/lib/libnss_wrapper.so" \
|
||||
LNAME="airflow" \
|
||||
NSS_WRAPPER_GROUP="/opt/bitnami/airflow/nss_group" \
|
||||
NSS_WRAPPER_PASSWD="/opt/bitnami/airflow/nss_passwd" \
|
||||
PATH="/opt/bitnami/common/bin:/opt/bitnami/python/bin:/opt/bitnami/postgresql/bin:/opt/bitnami/git/bin:/opt/bitnami/airflow/venv/bin:$PATH"
|
||||
|
||||
USER 1001
|
||||
ENTRYPOINT [ "/opt/bitnami/scripts/airflow-scheduler/entrypoint.sh" ]
|
||||
CMD [ "/opt/bitnami/scripts/airflow-scheduler/run.sh" ]
|
||||
15
bitnami/airflow-scheduler/2/debian-10/docker-compose.yml
Normal file
15
bitnami/airflow-scheduler/2/debian-10/docker-compose.yml
Normal file
@@ -0,0 +1,15 @@
|
||||
version: '2'
|
||||
services:
|
||||
airflow-scheduler:
|
||||
image: docker.io/bitnami/airflow-scheduler:2
|
||||
environment:
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
volumes:
|
||||
- airflow_scheduler_data:/bitnami
|
||||
command: ["tail", "-f", "/dev/null"]
|
||||
volumes:
|
||||
airflow_scheduler_data:
|
||||
driver: local
|
||||
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"airflow-scheduler": {
|
||||
"arch": "amd64",
|
||||
"digest": "d82942ec43230733ea31e587602c6e63ddc6814154439e705685d347af7991de",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "2.2.5-0"
|
||||
},
|
||||
"git": {
|
||||
"arch": "amd64",
|
||||
"digest": "75c0294b7af868721549926139e6ddbec9548fbd0e69c7e199d640a4531d858b",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "2.35.1-6"
|
||||
},
|
||||
"gosu": {
|
||||
"arch": "amd64",
|
||||
"digest": "d6280b6f647a62bf6edc74dc8e526bfff63ddd8067dcb8540843f47203d9ccf1",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "1.14.0-7"
|
||||
},
|
||||
"ini-file": {
|
||||
"arch": "amd64",
|
||||
"digest": "7a4f45cbebd2e242278060415dd1a3f0dbe68991fdcaea5bb311e8de54a3e027",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "1.4.1-7"
|
||||
},
|
||||
"postgresql-client": {
|
||||
"arch": "amd64",
|
||||
"digest": "efc2ccb03c4f0998d86e688d488e45aa534f27e317b253e7fcf037b7c78b7469",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "10.20.0-5"
|
||||
},
|
||||
"python": {
|
||||
"arch": "amd64",
|
||||
"digest": "704fe6b545280ae027303b4ec875b495ee6da2256b3d8f877467077b0f031a60",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "3.8.13-2"
|
||||
},
|
||||
"wait-for-port": {
|
||||
"arch": "amd64",
|
||||
"digest": "35c818ba3f4b5aae905959bc7d3a5e81fc63786e3c662b604612c0aa7fcda8fd",
|
||||
"distro": "debian-10",
|
||||
"type": "NAMI",
|
||||
"version": "1.0.1-10"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
Bitnami containers ship with software bundles. You can find the licenses under:
|
||||
/opt/bitnami/nami/COPYING
|
||||
/opt/bitnami/[name-of-bundle]/licenses/[bundle-version].txt
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Bitnami custom library
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Constants
|
||||
BOLD='\033[1m'
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Print the welcome page
|
||||
# Globals:
|
||||
# DISABLE_WELCOME_MESSAGE
|
||||
# BITNAMI_APP_NAME
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
print_welcome_page() {
|
||||
if [[ -z "${DISABLE_WELCOME_MESSAGE:-}" ]]; then
|
||||
if [[ -n "$BITNAMI_APP_NAME" ]]; then
|
||||
print_image_welcome_page
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Print the welcome page for a Bitnami Docker image
|
||||
# Globals:
|
||||
# BITNAMI_APP_NAME
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
print_image_welcome_page() {
|
||||
local github_url="https://github.com/bitnami/bitnami-docker-${BITNAMI_APP_NAME}"
|
||||
|
||||
log ""
|
||||
log "${BOLD}Welcome to the Bitnami ${BITNAMI_APP_NAME} container${RESET}"
|
||||
log "Subscribe to project updates by watching ${BOLD}${github_url}${RESET}"
|
||||
log "Submit issues and feature requests at ${BOLD}${github_url}/issues${RESET}"
|
||||
log ""
|
||||
}
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for managing Bitnami components
|
||||
|
||||
# Constants
|
||||
CACHE_ROOT="/tmp/bitnami/pkg/cache"
|
||||
DOWNLOAD_URL="https://downloads.bitnami.com/files/stacksmith"
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Download and unpack a Bitnami package
|
||||
# Globals:
|
||||
# OS_NAME
|
||||
# OS_ARCH
|
||||
# OS_FLAVOUR
|
||||
# Arguments:
|
||||
# $1 - component's name
|
||||
# $2 - component's version
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
component_unpack() {
|
||||
local name="${1:?name is required}"
|
||||
local version="${2:?version is required}"
|
||||
local base_name="${name}-${version}-${OS_NAME}-${OS_ARCH}-${OS_FLAVOUR}"
|
||||
local package_sha256=""
|
||||
local directory="/opt/bitnami"
|
||||
|
||||
# Validate arguments
|
||||
shift 2
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-c|--checksum)
|
||||
shift
|
||||
package_sha256="${1:?missing package checksum}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
echo "Downloading $base_name package"
|
||||
if [ -f "${CACHE_ROOT}/${base_name}.tar.gz" ]; then
|
||||
echo "${CACHE_ROOT}/${base_name}.tar.gz already exists, skipping download."
|
||||
cp "${CACHE_ROOT}/${base_name}.tar.gz" .
|
||||
rm "${CACHE_ROOT}/${base_name}.tar.gz"
|
||||
if [ -f "${CACHE_ROOT}/${base_name}.tar.gz.sha256" ]; then
|
||||
echo "Using the local sha256 from ${CACHE_ROOT}/${base_name}.tar.gz.sha256"
|
||||
package_sha256="$(< "${CACHE_ROOT}/${base_name}.tar.gz.sha256")"
|
||||
rm "${CACHE_ROOT}/${base_name}.tar.gz.sha256"
|
||||
fi
|
||||
else
|
||||
curl --remote-name --silent --show-error --fail "${DOWNLOAD_URL}/${base_name}.tar.gz"
|
||||
fi
|
||||
if [ -n "$package_sha256" ]; then
|
||||
echo "Verifying package integrity"
|
||||
echo "$package_sha256 ${base_name}.tar.gz" | sha256sum --check - || exit "$?"
|
||||
fi
|
||||
tar --directory "${directory}" --extract --gunzip --file "${base_name}.tar.gz" --no-same-owner --strip-components=2
|
||||
rm "${base_name}.tar.gz"
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for managing files
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Replace a regex-matching string in a file
|
||||
# Arguments:
|
||||
# $1 - filename
|
||||
# $2 - match regex
|
||||
# $3 - substitute regex
|
||||
# $4 - use POSIX regex. Default: true
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
replace_in_file() {
|
||||
local filename="${1:?filename is required}"
|
||||
local match_regex="${2:?match regex is required}"
|
||||
local substitute_regex="${3:?substitute regex is required}"
|
||||
local posix_regex=${4:-true}
|
||||
|
||||
local result
|
||||
|
||||
# We should avoid using 'sed in-place' substitutions
|
||||
# 1) They are not compatible with files mounted from ConfigMap(s)
|
||||
# 2) We found incompatibility issues with Debian10 and "in-place" substitutions
|
||||
local -r del=$'\001' # Use a non-printable character as a 'sed' delimiter to avoid issues
|
||||
if [[ $posix_regex = true ]]; then
|
||||
result="$(sed -E "s${del}${match_regex}${del}${substitute_regex}${del}g" "$filename")"
|
||||
else
|
||||
result="$(sed "s${del}${match_regex}${del}${substitute_regex}${del}g" "$filename")"
|
||||
fi
|
||||
echo "$result" > "$filename"
|
||||
}
|
||||
|
||||
########################
|
||||
# Replace a regex-matching multiline string in a file
|
||||
# Arguments:
|
||||
# $1 - filename
|
||||
# $2 - match regex
|
||||
# $3 - substitute regex
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
replace_in_file_multiline() {
|
||||
local filename="${1:?filename is required}"
|
||||
local match_regex="${2:?match regex is required}"
|
||||
local substitute_regex="${3:?substitute regex is required}"
|
||||
|
||||
local result
|
||||
local -r del=$'\001' # Use a non-printable character as a 'sed' delimiter to avoid issues
|
||||
result="$(perl -pe "BEGIN{undef $/;} s${del}${match_regex}${del}${substitute_regex}${del}sg" "$filename")"
|
||||
echo "$result" > "$filename"
|
||||
}
|
||||
|
||||
########################
|
||||
# Remove a line in a file based on a regex
|
||||
# Arguments:
|
||||
# $1 - filename
|
||||
# $2 - match regex
|
||||
# $3 - use POSIX regex. Default: true
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
remove_in_file() {
|
||||
local filename="${1:?filename is required}"
|
||||
local match_regex="${2:?match regex is required}"
|
||||
local posix_regex=${3:-true}
|
||||
local result
|
||||
|
||||
# We should avoid using 'sed in-place' substitutions
|
||||
# 1) They are not compatible with files mounted from ConfigMap(s)
|
||||
# 2) We found incompatibility issues with Debian10 and "in-place" substitutions
|
||||
if [[ $posix_regex = true ]]; then
|
||||
result="$(sed -E "/$match_regex/d" "$filename")"
|
||||
else
|
||||
result="$(sed "/$match_regex/d" "$filename")"
|
||||
fi
|
||||
echo "$result" > "$filename"
|
||||
}
|
||||
|
||||
########################
|
||||
# Appends text after the last line matching a pattern
|
||||
# Arguments:
|
||||
# $1 - file
|
||||
# $2 - match regex
|
||||
# $3 - contents to add
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
append_file_after_last_match() {
|
||||
local file="${1:?missing file}"
|
||||
local match_regex="${2:?missing pattern}"
|
||||
local value="${3:?missing value}"
|
||||
|
||||
# We read the file in reverse, replace the first match (0,/pattern/s) and then reverse the results again
|
||||
result="$(tac "$file" | sed -E "0,/($match_regex)/s||${value}\n\1|" | tac)"
|
||||
echo "$result" > "$file"
|
||||
}
|
||||
|
||||
########################
|
||||
# Wait until certain entry is present in a log file
|
||||
# Arguments:
|
||||
# $1 - entry to look for
|
||||
# $2 - log file
|
||||
# $3 - max retries. Default: 12
|
||||
# $4 - sleep between retries (in seconds). Default: 5
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
wait_for_log_entry() {
|
||||
local -r entry="${1:-missing entry}"
|
||||
local -r log_file="${2:-missing log file}"
|
||||
local -r retries="${3:-12}"
|
||||
local -r interval_time="${4:-5}"
|
||||
local attempt=0
|
||||
|
||||
check_log_file_for_entry() {
|
||||
if ! grep -qE "$entry" "$log_file"; then
|
||||
debug "Entry \"${entry}\" still not present in ${log_file} (attempt $((++attempt))/${retries})"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
debug "Checking that ${log_file} log file contains entry \"${entry}\""
|
||||
if retry_while check_log_file_for_entry "$retries" "$interval_time"; then
|
||||
debug "Found entry \"${entry}\" in ${log_file}"
|
||||
true
|
||||
else
|
||||
error "Could not find entry \"${entry}\" in ${log_file} after ${retries} retries"
|
||||
debug_execute cat "$log_file"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for file system actions
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Ensure a file/directory is owned (user and group) but the given user
|
||||
# Arguments:
|
||||
# $1 - filepath
|
||||
# $2 - owner
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
owned_by() {
|
||||
local path="${1:?path is missing}"
|
||||
local owner="${2:?owner is missing}"
|
||||
local group="${3:-}"
|
||||
|
||||
if [[ -n $group ]]; then
|
||||
chown "$owner":"$group" "$path"
|
||||
else
|
||||
chown "$owner":"$owner" "$path"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure a directory exists and, optionally, is owned by the given user
|
||||
# Arguments:
|
||||
# $1 - directory
|
||||
# $2 - owner
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
ensure_dir_exists() {
|
||||
local dir="${1:?directory is missing}"
|
||||
local owner_user="${2:-}"
|
||||
local owner_group="${3:-}"
|
||||
|
||||
mkdir -p "${dir}"
|
||||
if [[ -n $owner_user ]]; then
|
||||
owned_by "$dir" "$owner_user" "$owner_group"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Checks whether a directory is empty or not
|
||||
# arguments:
|
||||
# $1 - directory
|
||||
# returns:
|
||||
# boolean
|
||||
#########################
|
||||
is_dir_empty() {
|
||||
local -r path="${1:?missing directory}"
|
||||
# Calculate real path in order to avoid issues with symlinks
|
||||
local -r dir="$(realpath "$path")"
|
||||
if [[ ! -e "$dir" ]] || [[ -z "$(ls -A "$dir")" ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Checks whether a mounted directory is empty or not
|
||||
# arguments:
|
||||
# $1 - directory
|
||||
# returns:
|
||||
# boolean
|
||||
#########################
|
||||
is_mounted_dir_empty() {
|
||||
local dir="${1:?missing directory}"
|
||||
|
||||
if is_dir_empty "$dir" || find "$dir" -mindepth 1 -maxdepth 1 -not -name ".snapshot" -not -name "lost+found" -exec false {} +; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Checks whether a file can be written to or not
|
||||
# arguments:
|
||||
# $1 - file
|
||||
# returns:
|
||||
# boolean
|
||||
#########################
|
||||
is_file_writable() {
|
||||
local file="${1:?missing file}"
|
||||
local dir
|
||||
dir="$(dirname "$file")"
|
||||
|
||||
if [[ (-f "$file" && -w "$file") || (! -f "$file" && -d "$dir" && -w "$dir") ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Relativize a path
|
||||
# arguments:
|
||||
# $1 - path
|
||||
# $2 - base
|
||||
# returns:
|
||||
# None
|
||||
#########################
|
||||
relativize() {
|
||||
local -r path="${1:?missing path}"
|
||||
local -r base="${2:?missing base}"
|
||||
pushd "$base" >/dev/null || exit
|
||||
realpath -q --no-symlinks --relative-base="$base" "$path" | sed -e 's|^/$|.|' -e 's|^/||'
|
||||
popd >/dev/null || exit
|
||||
}
|
||||
|
||||
########################
|
||||
# Configure permisions and ownership recursively
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# $1 - paths (as a string).
|
||||
# Flags:
|
||||
# -f|--file-mode - mode for directories.
|
||||
# -d|--dir-mode - mode for files.
|
||||
# -u|--user - user
|
||||
# -g|--group - group
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
configure_permissions_ownership() {
|
||||
local -r paths="${1:?paths is missing}"
|
||||
local dir_mode=""
|
||||
local file_mode=""
|
||||
local user=""
|
||||
local group=""
|
||||
|
||||
# Validate arguments
|
||||
shift 1
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-f | --file-mode)
|
||||
shift
|
||||
file_mode="${1:?missing mode for files}"
|
||||
;;
|
||||
-d | --dir-mode)
|
||||
shift
|
||||
dir_mode="${1:?missing mode for directories}"
|
||||
;;
|
||||
-u | --user)
|
||||
shift
|
||||
user="${1:?missing user}"
|
||||
;;
|
||||
-g | --group)
|
||||
shift
|
||||
group="${1:?missing group}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
read -r -a filepaths <<<"$paths"
|
||||
for p in "${filepaths[@]}"; do
|
||||
if [[ -e "$p" ]]; then
|
||||
if [[ -n $dir_mode ]]; then
|
||||
find -L "$p" -type d -exec chmod "$dir_mode" {} \;
|
||||
fi
|
||||
if [[ -n $file_mode ]]; then
|
||||
find -L "$p" -type f -exec chmod "$file_mode" {} \;
|
||||
fi
|
||||
if [[ -n $user ]] && [[ -n $group ]]; then
|
||||
chown -LR "$user":"$group" "$p"
|
||||
elif [[ -n $user ]] && [[ -z $group ]]; then
|
||||
chown -LR "$user" "$p"
|
||||
elif [[ -z $user ]] && [[ -n $group ]]; then
|
||||
chgrp -LR "$group" "$p"
|
||||
fi
|
||||
else
|
||||
stderr_print "$p does not exist"
|
||||
fi
|
||||
done
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library to use for scripts expected to be used as Kubernetes lifecycle hooks
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load generic libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
|
||||
# Override functions that log to stdout/stderr of the current process, so they print to process 1
|
||||
for function_to_override in stderr_print debug_execute; do
|
||||
# Output is sent to output of process 1 and thus end up in the container log
|
||||
# The hook output in general isn't saved
|
||||
eval "$(declare -f "$function_to_override") >/proc/1/fd/1 2>/proc/1/fd/2"
|
||||
done
|
||||
@@ -0,0 +1,112 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for logging functions
|
||||
|
||||
# Constants
|
||||
RESET='\033[0m'
|
||||
RED='\033[38;5;1m'
|
||||
GREEN='\033[38;5;2m'
|
||||
YELLOW='\033[38;5;3m'
|
||||
MAGENTA='\033[38;5;5m'
|
||||
CYAN='\033[38;5;6m'
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Print to STDERR
|
||||
# Arguments:
|
||||
# Message to print
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
stderr_print() {
|
||||
# 'is_boolean_yes' is defined in libvalidations.sh, but depends on this file so we cannot source it
|
||||
local bool="${BITNAMI_QUIET:-false}"
|
||||
# comparison is performed without regard to the case of alphabetic characters
|
||||
shopt -s nocasematch
|
||||
if ! [[ "$bool" = 1 || "$bool" =~ ^(yes|true)$ ]]; then
|
||||
printf "%b\\n" "${*}" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Log message
|
||||
# Arguments:
|
||||
# Message to log
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
log() {
|
||||
stderr_print "${CYAN}${MODULE:-} ${MAGENTA}$(date "+%T.%2N ")${RESET}${*}"
|
||||
}
|
||||
########################
|
||||
# Log an 'info' message
|
||||
# Arguments:
|
||||
# Message to log
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
info() {
|
||||
log "${GREEN}INFO ${RESET} ==> ${*}"
|
||||
}
|
||||
########################
|
||||
# Log message
|
||||
# Arguments:
|
||||
# Message to log
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
warn() {
|
||||
log "${YELLOW}WARN ${RESET} ==> ${*}"
|
||||
}
|
||||
########################
|
||||
# Log an 'error' message
|
||||
# Arguments:
|
||||
# Message to log
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
error() {
|
||||
log "${RED}ERROR${RESET} ==> ${*}"
|
||||
}
|
||||
########################
|
||||
# Log a 'debug' message
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
debug() {
|
||||
# 'is_boolean_yes' is defined in libvalidations.sh, but depends on this file so we cannot source it
|
||||
local bool="${BITNAMI_DEBUG:-false}"
|
||||
# comparison is performed without regard to the case of alphabetic characters
|
||||
shopt -s nocasematch
|
||||
if [[ "$bool" = 1 || "$bool" =~ ^(yes|true)$ ]]; then
|
||||
log "${MAGENTA}DEBUG${RESET} ==> ${*}"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Indent a string
|
||||
# Arguments:
|
||||
# $1 - string
|
||||
# $2 - number of indentation characters (default: 4)
|
||||
# $3 - indentation character (default: " ")
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
indent() {
|
||||
local string="${1:-}"
|
||||
local num="${2:?missing num}"
|
||||
local char="${3:-" "}"
|
||||
# Build the indentation unit string
|
||||
local indent_unit=""
|
||||
for ((i = 0; i < num; i++)); do
|
||||
indent_unit="${indent_unit}${char}"
|
||||
done
|
||||
# shellcheck disable=SC2001
|
||||
# Complex regex, see https://github.com/koalaman/shellcheck/wiki/SC2001#exceptions
|
||||
echo "$string" | sed "s/^/${indent_unit}/"
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for network functions
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Resolve IP address for a host/domain (i.e. DNS lookup)
|
||||
# Arguments:
|
||||
# $1 - Hostname to resolve
|
||||
# $2 - IP address version (v4, v6), leave empty for resolving to any version
|
||||
# Returns:
|
||||
# IP
|
||||
#########################
|
||||
dns_lookup() {
|
||||
local host="${1:?host is missing}"
|
||||
local ip_version="${2:-}"
|
||||
getent "ahosts${ip_version}" "$host" | awk '/STREAM/ {print $1 }' | head -n 1
|
||||
}
|
||||
|
||||
#########################
|
||||
# Wait for a hostname and return the IP
|
||||
# Arguments:
|
||||
# $1 - hostname
|
||||
# $2 - number of retries
|
||||
# $3 - seconds to wait between retries
|
||||
# Returns:
|
||||
# - IP address that corresponds to the hostname
|
||||
#########################
|
||||
wait_for_dns_lookup() {
|
||||
local hostname="${1:?hostname is missing}"
|
||||
local retries="${2:-5}"
|
||||
local seconds="${3:-1}"
|
||||
check_host() {
|
||||
if [[ $(dns_lookup "$hostname") == "" ]]; then
|
||||
false
|
||||
else
|
||||
true
|
||||
fi
|
||||
}
|
||||
# Wait for the host to be ready
|
||||
retry_while "check_host ${hostname}" "$retries" "$seconds"
|
||||
dns_lookup "$hostname"
|
||||
}
|
||||
|
||||
########################
|
||||
# Get machine's IP
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# Machine IP
|
||||
#########################
|
||||
get_machine_ip() {
|
||||
local -a ip_addresses
|
||||
local hostname
|
||||
hostname="$(hostname)"
|
||||
read -r -a ip_addresses <<< "$(dns_lookup "$hostname" | xargs echo)"
|
||||
if [[ "${#ip_addresses[@]}" -gt 1 ]]; then
|
||||
warn "Found more than one IP address associated to hostname ${hostname}: ${ip_addresses[*]}, will use ${ip_addresses[0]}"
|
||||
elif [[ "${#ip_addresses[@]}" -lt 1 ]]; then
|
||||
error "Could not find any IP address associated to hostname ${hostname}"
|
||||
exit 1
|
||||
fi
|
||||
echo "${ip_addresses[0]}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a resolved hostname
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_hostname_resolved() {
|
||||
local -r host="${1:?missing value}"
|
||||
if [[ -n "$(dns_lookup "$host")" ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Parse URL
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# $1 - uri - String
|
||||
# $2 - component to obtain. Valid options (scheme, authority, userinfo, host, port, path, query or fragment) - String
|
||||
# Returns:
|
||||
# String
|
||||
parse_uri() {
|
||||
local uri="${1:?uri is missing}"
|
||||
local component="${2:?component is missing}"
|
||||
|
||||
# Solution based on https://tools.ietf.org/html/rfc3986#appendix-B with
|
||||
# additional sub-expressions to split authority into userinfo, host and port
|
||||
# Credits to Patryk Obara (see https://stackoverflow.com/a/45977232/6694969)
|
||||
local -r URI_REGEX='^(([^:/?#]+):)?(//((([^@/?#]+)@)?([^:/?#]+)(:([0-9]+))?))?(/([^?#]*))?(\?([^#]*))?(#(.*))?'
|
||||
# || | ||| | | | | | | | | |
|
||||
# |2 scheme | ||6 userinfo 7 host | 9 port | 11 rpath | 13 query | 15 fragment
|
||||
# 1 scheme: | |5 userinfo@ 8 :... 10 path 12 ?... 14 #...
|
||||
# | 4 authority
|
||||
# 3 //...
|
||||
local index=0
|
||||
case "$component" in
|
||||
scheme)
|
||||
index=2
|
||||
;;
|
||||
authority)
|
||||
index=4
|
||||
;;
|
||||
userinfo)
|
||||
index=6
|
||||
;;
|
||||
host)
|
||||
index=7
|
||||
;;
|
||||
port)
|
||||
index=9
|
||||
;;
|
||||
path)
|
||||
index=10
|
||||
;;
|
||||
query)
|
||||
index=13
|
||||
;;
|
||||
fragment)
|
||||
index=14
|
||||
;;
|
||||
*)
|
||||
stderr_print "unrecognized component $component"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
[[ "$uri" =~ $URI_REGEX ]] && echo "${BASH_REMATCH[${index}]}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Wait for a HTTP connection to succeed
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# $1 - URL to wait for
|
||||
# $2 - Maximum amount of retries (optional)
|
||||
# $3 - Time between retries (optional)
|
||||
# Returns:
|
||||
# true if the HTTP connection succeeded, false otherwise
|
||||
#########################
|
||||
wait_for_http_connection() {
|
||||
local url="${1:?missing url}"
|
||||
local retries="${2:-}"
|
||||
local sleep_time="${3:-}"
|
||||
if ! retry_while "debug_execute curl --silent ${url}" "$retries" "$sleep_time"; then
|
||||
error "Could not connect to ${url}"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,466 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for operating system actions
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libfs.sh
|
||||
. /opt/bitnami/scripts/libvalidations.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Check if an user exists in the system
|
||||
# Arguments:
|
||||
# $1 - user
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
user_exists() {
|
||||
local user="${1:?user is missing}"
|
||||
id "$user" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if a group exists in the system
|
||||
# Arguments:
|
||||
# $1 - group
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
group_exists() {
|
||||
local group="${1:?group is missing}"
|
||||
getent group "$group" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
########################
|
||||
# Create a group in the system if it does not exist already
|
||||
# Arguments:
|
||||
# $1 - group
|
||||
# Flags:
|
||||
# -i|--gid - the ID for the new group
|
||||
# -s|--system - Whether to create new user as system user (uid <= 999)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
ensure_group_exists() {
|
||||
local group="${1:?group is missing}"
|
||||
local gid=""
|
||||
local is_system_user=false
|
||||
|
||||
# Validate arguments
|
||||
shift 1
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-i | --gid)
|
||||
shift
|
||||
gid="${1:?missing gid}"
|
||||
;;
|
||||
-s | --system)
|
||||
is_system_user=true
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
if ! group_exists "$group"; then
|
||||
local -a args=("$group")
|
||||
if [[ -n "$gid" ]]; then
|
||||
if group_exists "$gid"; then
|
||||
error "The GID $gid is already in use." >&2
|
||||
return 1
|
||||
fi
|
||||
args+=("--gid" "$gid")
|
||||
fi
|
||||
$is_system_user && args+=("--system")
|
||||
groupadd "${args[@]}" >/dev/null 2>&1
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Create an user in the system if it does not exist already
|
||||
# Arguments:
|
||||
# $1 - user
|
||||
# Flags:
|
||||
# -i|--uid - the ID for the new user
|
||||
# -g|--group - the group the new user should belong to
|
||||
# -a|--append-groups - comma-separated list of supplemental groups to append to the new user
|
||||
# -h|--home - the home directory for the new user
|
||||
# -s|--system - whether to create new user as system user (uid <= 999)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
ensure_user_exists() {
|
||||
local user="${1:?user is missing}"
|
||||
local uid=""
|
||||
local group=""
|
||||
local append_groups=""
|
||||
local home=""
|
||||
local is_system_user=false
|
||||
|
||||
# Validate arguments
|
||||
shift 1
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-i | --uid)
|
||||
shift
|
||||
uid="${1:?missing uid}"
|
||||
;;
|
||||
-g | --group)
|
||||
shift
|
||||
group="${1:?missing group}"
|
||||
;;
|
||||
-a | --append-groups)
|
||||
shift
|
||||
append_groups="${1:?missing append_groups}"
|
||||
;;
|
||||
-h | --home)
|
||||
shift
|
||||
home="${1:?missing home directory}"
|
||||
;;
|
||||
-s | --system)
|
||||
is_system_user=true
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
if ! user_exists "$user"; then
|
||||
local -a user_args=("-N" "$user")
|
||||
if [[ -n "$uid" ]]; then
|
||||
if user_exists "$uid"; then
|
||||
error "The UID $uid is already in use."
|
||||
return 1
|
||||
fi
|
||||
user_args+=("--uid" "$uid")
|
||||
else
|
||||
$is_system_user && user_args+=("--system")
|
||||
fi
|
||||
useradd "${user_args[@]}" >/dev/null 2>&1
|
||||
fi
|
||||
|
||||
if [[ -n "$group" ]]; then
|
||||
local -a group_args=("$group")
|
||||
$is_system_user && group_args+=("--system")
|
||||
ensure_group_exists "${group_args[@]}"
|
||||
usermod -g "$group" "$user" >/dev/null 2>&1
|
||||
fi
|
||||
|
||||
if [[ -n "$append_groups" ]]; then
|
||||
local -a groups
|
||||
read -ra groups <<<"$(tr ',;' ' ' <<<"$append_groups")"
|
||||
for group in "${groups[@]}"; do
|
||||
ensure_group_exists "$group"
|
||||
usermod -aG "$group" "$user" >/dev/null 2>&1
|
||||
done
|
||||
fi
|
||||
|
||||
if [[ -n "$home" ]]; then
|
||||
mkdir -p "$home"
|
||||
usermod -d "$home" "$user" >/dev/null 2>&1
|
||||
configure_permissions_ownership "$home" -d "775" -f "664" -u "$user" -g "$group"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the script is currently running as root
|
||||
# Arguments:
|
||||
# $1 - user
|
||||
# $2 - group
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
am_i_root() {
|
||||
if [[ "$(id -u)" = "0" ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Print OS metadata
|
||||
# Arguments:
|
||||
# $1 - Flag name
|
||||
# Flags:
|
||||
# --id - Distro ID
|
||||
# --version - Distro version
|
||||
# --branch - Distro branch
|
||||
# --codename - Distro codename
|
||||
# Returns:
|
||||
# String
|
||||
#########################
|
||||
get_os_metadata() {
|
||||
local -r flag_name="${1:?missing flag}"
|
||||
# Helper function
|
||||
get_os_release_metadata() {
|
||||
local -r env_name="${1:?missing environment variable name}"
|
||||
(
|
||||
. /etc/os-release
|
||||
echo "${!env_name}"
|
||||
)
|
||||
}
|
||||
case "$flag_name" in
|
||||
--id)
|
||||
get_os_release_metadata ID
|
||||
;;
|
||||
--version)
|
||||
get_os_release_metadata VERSION_ID
|
||||
;;
|
||||
--branch)
|
||||
get_os_release_metadata VERSION_ID | sed 's/\..*//'
|
||||
;;
|
||||
--codename)
|
||||
get_os_release_metadata VERSION_CODENAME
|
||||
;;
|
||||
*)
|
||||
error "Unknown flag ${flag_name}"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
########################
|
||||
# Get total memory available
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# Memory in bytes
|
||||
#########################
|
||||
get_total_memory() {
|
||||
echo $(($(grep MemTotal /proc/meminfo | awk '{print $2}') / 1024))
|
||||
}
|
||||
|
||||
########################
|
||||
# Get machine size depending on specified memory
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# None
|
||||
# Flags:
|
||||
# --memory - memory size (optional)
|
||||
# Returns:
|
||||
# Detected instance size
|
||||
#########################
|
||||
get_machine_size() {
|
||||
local memory=""
|
||||
# Validate arguments
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--memory)
|
||||
shift
|
||||
memory="${1:?missing memory}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
if [[ -z "$memory" ]]; then
|
||||
debug "Memory was not specified, detecting available memory automatically"
|
||||
memory="$(get_total_memory)"
|
||||
fi
|
||||
sanitized_memory=$(convert_to_mb "$memory")
|
||||
if [[ "$sanitized_memory" -gt 26000 ]]; then
|
||||
echo 2xlarge
|
||||
elif [[ "$sanitized_memory" -gt 13000 ]]; then
|
||||
echo xlarge
|
||||
elif [[ "$sanitized_memory" -gt 6000 ]]; then
|
||||
echo large
|
||||
elif [[ "$sanitized_memory" -gt 3000 ]]; then
|
||||
echo medium
|
||||
elif [[ "$sanitized_memory" -gt 1500 ]]; then
|
||||
echo small
|
||||
else
|
||||
echo micro
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Get machine size depending on specified memory
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# $1 - memory size (optional)
|
||||
# Returns:
|
||||
# Detected instance size
|
||||
#########################
|
||||
get_supported_machine_sizes() {
|
||||
echo micro small medium large xlarge 2xlarge
|
||||
}
|
||||
|
||||
########################
|
||||
# Convert memory size from string to amount of megabytes (i.e. 2G -> 2048)
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# $1 - memory size
|
||||
# Returns:
|
||||
# Result of the conversion
|
||||
#########################
|
||||
convert_to_mb() {
|
||||
local amount="${1:-}"
|
||||
if [[ $amount =~ ^([0-9]+)(m|M|g|G) ]]; then
|
||||
size="${BASH_REMATCH[1]}"
|
||||
unit="${BASH_REMATCH[2]}"
|
||||
if [[ "$unit" = "g" || "$unit" = "G" ]]; then
|
||||
amount="$((size * 1024))"
|
||||
else
|
||||
amount="$size"
|
||||
fi
|
||||
fi
|
||||
echo "$amount"
|
||||
}
|
||||
|
||||
#########################
|
||||
# Redirects output to /dev/null if debug mode is disabled
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# Arguments:
|
||||
# $@ - Command to execute
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
debug_execute() {
|
||||
if is_boolean_yes "${BITNAMI_DEBUG:-false}"; then
|
||||
"$@"
|
||||
else
|
||||
"$@" >/dev/null 2>&1
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Retries a command a given number of times
|
||||
# Arguments:
|
||||
# $1 - cmd (as a string)
|
||||
# $2 - max retries. Default: 12
|
||||
# $3 - sleep between retries (in seconds). Default: 5
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
retry_while() {
|
||||
local cmd="${1:?cmd is missing}"
|
||||
local retries="${2:-12}"
|
||||
local sleep_time="${3:-5}"
|
||||
local return_value=1
|
||||
|
||||
read -r -a command <<<"$cmd"
|
||||
for ((i = 1; i <= retries; i += 1)); do
|
||||
"${command[@]}" && return_value=0 && break
|
||||
sleep "$sleep_time"
|
||||
done
|
||||
return $return_value
|
||||
}
|
||||
|
||||
########################
|
||||
# Generate a random string
|
||||
# Arguments:
|
||||
# -t|--type - String type (ascii, alphanumeric, numeric), defaults to ascii
|
||||
# -c|--count - Number of characters, defaults to 32
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
# Returns:
|
||||
# String
|
||||
#########################
|
||||
generate_random_string() {
|
||||
local type="ascii"
|
||||
local count="32"
|
||||
local filter
|
||||
local result
|
||||
# Validate arguments
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-t | --type)
|
||||
shift
|
||||
type="$1"
|
||||
;;
|
||||
-c | --count)
|
||||
shift
|
||||
count="$1"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
# Validate type
|
||||
case "$type" in
|
||||
ascii)
|
||||
filter="[:print:]"
|
||||
;;
|
||||
alphanumeric)
|
||||
filter="a-zA-Z0-9"
|
||||
;;
|
||||
numeric)
|
||||
filter="0-9"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid type ${type}" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
# Obtain count + 10 lines from /dev/urandom to ensure that the resulting string has the expected size
|
||||
# Note there is a very small chance of strings starting with EOL character
|
||||
# Therefore, the higher amount of lines read, this will happen less frequently
|
||||
result="$(head -n "$((count + 10))" /dev/urandom | tr -dc "$filter" | head -c "$count")"
|
||||
echo "$result"
|
||||
}
|
||||
|
||||
########################
|
||||
# Create md5 hash from a string
|
||||
# Arguments:
|
||||
# $1 - string
|
||||
# Returns:
|
||||
# md5 hash - string
|
||||
#########################
|
||||
generate_md5_hash() {
|
||||
local -r str="${1:?missing input string}"
|
||||
echo -n "$str" | md5sum | awk '{print $1}'
|
||||
}
|
||||
|
||||
########################
|
||||
# Create sha1 hash from a string
|
||||
# Arguments:
|
||||
# $1 - string
|
||||
# $2 - algorithm - 1 (default), 224, 256, 384, 512
|
||||
# Returns:
|
||||
# sha1 hash - string
|
||||
#########################
|
||||
generate_sha_hash() {
|
||||
local -r str="${1:?missing input string}"
|
||||
local -r algorithm="${2:-1}"
|
||||
echo -n "$str" | "sha${algorithm}sum" | awk '{print $1}'
|
||||
}
|
||||
|
||||
########################
|
||||
# Converts a string to its hexadecimal representation
|
||||
# Arguments:
|
||||
# $1 - string
|
||||
# Returns:
|
||||
# hexadecimal representation of the string
|
||||
#########################
|
||||
convert_to_hex() {
|
||||
local -r str=${1:?missing input string}
|
||||
local -i iterator
|
||||
local char
|
||||
for ((iterator = 0; iterator < ${#str}; iterator++)); do
|
||||
char=${str:iterator:1}
|
||||
printf '%x' "'${char}"
|
||||
done
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Bitnami persistence library
|
||||
# Used for bringing persistence capabilities to applications that don't have clear separation of data and logic
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/libfs.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libversion.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Persist an application directory
|
||||
# Globals:
|
||||
# BITNAMI_ROOT_DIR
|
||||
# BITNAMI_VOLUME_DIR
|
||||
# Arguments:
|
||||
# $1 - App folder name
|
||||
# $2 - List of app files to persist
|
||||
# Returns:
|
||||
# true if all steps succeeded, false otherwise
|
||||
#########################
|
||||
persist_app() {
|
||||
local -r app="${1:?missing app}"
|
||||
local -a files_to_restore
|
||||
read -r -a files_to_persist <<< "$(tr ',;:' ' ' <<< "$2")"
|
||||
local -r install_dir="${BITNAMI_ROOT_DIR}/${app}"
|
||||
local -r persist_dir="${BITNAMI_VOLUME_DIR}/${app}"
|
||||
# Persist the individual files
|
||||
if [[ "${#files_to_persist[@]}" -le 0 ]]; then
|
||||
warn "No files are configured to be persisted"
|
||||
return
|
||||
fi
|
||||
pushd "$install_dir" >/dev/null || exit
|
||||
local file_to_persist_relative file_to_persist_destination file_to_persist_destination_folder
|
||||
local -r tmp_file="/tmp/perms.acl"
|
||||
for file_to_persist in "${files_to_persist[@]}"; do
|
||||
if [[ ! -f "$file_to_persist" && ! -d "$file_to_persist" ]]; then
|
||||
error "Cannot persist '${file_to_persist}' because it does not exist"
|
||||
return 1
|
||||
fi
|
||||
file_to_persist_relative="$(relativize "$file_to_persist" "$install_dir")"
|
||||
file_to_persist_destination="${persist_dir}/${file_to_persist_relative}"
|
||||
file_to_persist_destination_folder="$(dirname "$file_to_persist_destination")"
|
||||
# Get original permissions for existing files, which will be applied later
|
||||
# Exclude the root directory with 'sed', to avoid issues when copying the entirety of it to a volume
|
||||
getfacl -R "$file_to_persist_relative" | sed -E '/# file: (\..+|[^.])/,$!d' > "$tmp_file"
|
||||
# Copy directories to the volume
|
||||
ensure_dir_exists "$file_to_persist_destination_folder"
|
||||
cp -Lr --preserve=links "$file_to_persist_relative" "$file_to_persist_destination_folder"
|
||||
# Restore permissions
|
||||
pushd "$persist_dir" >/dev/null || exit
|
||||
if am_i_root; then
|
||||
setfacl --restore="$tmp_file"
|
||||
else
|
||||
# When running as non-root, don't change ownership
|
||||
setfacl --restore=<(grep -E -v '^# (owner|group):' "$tmp_file")
|
||||
fi
|
||||
popd >/dev/null || exit
|
||||
done
|
||||
popd >/dev/null || exit
|
||||
rm -f "$tmp_file"
|
||||
# Install the persisted files into the installation directory, via symlinks
|
||||
restore_persisted_app "$@"
|
||||
}
|
||||
|
||||
########################
|
||||
# Restore a persisted application directory
|
||||
# Globals:
|
||||
# BITNAMI_ROOT_DIR
|
||||
# BITNAMI_VOLUME_DIR
|
||||
# FORCE_MAJOR_UPGRADE
|
||||
# Arguments:
|
||||
# $1 - App folder name
|
||||
# $2 - List of app files to restore
|
||||
# Returns:
|
||||
# true if all steps succeeded, false otherwise
|
||||
#########################
|
||||
restore_persisted_app() {
|
||||
local -r app="${1:?missing app}"
|
||||
local -a files_to_restore
|
||||
read -r -a files_to_restore <<< "$(tr ',;:' ' ' <<< "$2")"
|
||||
local -r install_dir="${BITNAMI_ROOT_DIR}/${app}"
|
||||
local -r persist_dir="${BITNAMI_VOLUME_DIR}/${app}"
|
||||
# Restore the individual persisted files
|
||||
if [[ "${#files_to_restore[@]}" -le 0 ]]; then
|
||||
warn "No persisted files are configured to be restored"
|
||||
return
|
||||
fi
|
||||
local file_to_restore_relative file_to_restore_origin file_to_restore_destination
|
||||
for file_to_restore in "${files_to_restore[@]}"; do
|
||||
file_to_restore_relative="$(relativize "$file_to_restore" "$install_dir")"
|
||||
# We use 'realpath --no-symlinks' to ensure that the case of '.' is covered and the directory is removed
|
||||
file_to_restore_origin="$(realpath --no-symlinks "${install_dir}/${file_to_restore_relative}")"
|
||||
file_to_restore_destination="$(realpath --no-symlinks "${persist_dir}/${file_to_restore_relative}")"
|
||||
rm -rf "$file_to_restore_origin"
|
||||
ln -sfn "$file_to_restore_destination" "$file_to_restore_origin"
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if an application directory was already persisted
|
||||
# Globals:
|
||||
# BITNAMI_VOLUME_DIR
|
||||
# Arguments:
|
||||
# $1 - App folder name
|
||||
# Returns:
|
||||
# true if all steps succeeded, false otherwise
|
||||
#########################
|
||||
is_app_initialized() {
|
||||
local -r app="${1:?missing app}"
|
||||
local -r persist_dir="${BITNAMI_VOLUME_DIR}/${app}"
|
||||
if ! is_mounted_dir_empty "$persist_dir"; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for managing services
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/libvalidations.sh
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Read the provided pid file and returns a PID
|
||||
# Arguments:
|
||||
# $1 - Pid file
|
||||
# Returns:
|
||||
# PID
|
||||
#########################
|
||||
get_pid_from_file() {
|
||||
local pid_file="${1:?pid file is missing}"
|
||||
|
||||
if [[ -f "$pid_file" ]]; then
|
||||
if [[ -n "$(< "$pid_file")" ]] && [[ "$(< "$pid_file")" -gt 0 ]]; then
|
||||
echo "$(< "$pid_file")"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if a provided PID corresponds to a running service
|
||||
# Arguments:
|
||||
# $1 - PID
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_service_running() {
|
||||
local pid="${1:?pid is missing}"
|
||||
|
||||
kill -0 "$pid" 2>/dev/null
|
||||
}
|
||||
|
||||
########################
|
||||
# Stop a service by sending a termination signal to its pid
|
||||
# Arguments:
|
||||
# $1 - Pid file
|
||||
# $2 - Signal number (optional)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
stop_service_using_pid() {
|
||||
local pid_file="${1:?pid file is missing}"
|
||||
local signal="${2:-}"
|
||||
local pid
|
||||
|
||||
pid="$(get_pid_from_file "$pid_file")"
|
||||
[[ -z "$pid" ]] || ! is_service_running "$pid" && return
|
||||
|
||||
if [[ -n "$signal" ]]; then
|
||||
kill "-${signal}" "$pid"
|
||||
else
|
||||
kill "$pid"
|
||||
fi
|
||||
|
||||
local counter=10
|
||||
while [[ "$counter" -ne 0 ]] && is_service_running "$pid"; do
|
||||
sleep 1
|
||||
counter=$((counter - 1))
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Start cron daemon
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# true if started correctly, false otherwise
|
||||
#########################
|
||||
cron_start() {
|
||||
if [[ -x "/usr/sbin/cron" ]]; then
|
||||
/usr/sbin/cron
|
||||
elif [[ -x "/usr/sbin/crond" ]]; then
|
||||
/usr/sbin/crond
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Generate a cron configuration file for a given service
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# $2 - Command
|
||||
# Flags:
|
||||
# --run-as - User to run as (default: root)
|
||||
# --schedule - Cron schedule configuration (default: * * * * *)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
generate_cron_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local cmd="${2:?command is missing}"
|
||||
local run_as="root"
|
||||
local schedule="* * * * *"
|
||||
local clean="true"
|
||||
|
||||
local clean="true"
|
||||
|
||||
# Parse optional CLI flags
|
||||
shift 2
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--run-as)
|
||||
shift
|
||||
run_as="$1"
|
||||
;;
|
||||
--schedule)
|
||||
shift
|
||||
schedule="$1"
|
||||
;;
|
||||
--no-clean)
|
||||
clean="false"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag ${1}" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
mkdir -p /etc/cron.d
|
||||
if "$clean"; then
|
||||
echo "${schedule} ${run_as} ${cmd}" > /etc/cron.d/"$service_name"
|
||||
else
|
||||
echo "${schedule} ${run_as} ${cmd}" >> /etc/cron.d/"$service_name"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Remove a cron configuration file for a given service
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
remove_cron_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local cron_conf_dir="/etc/monit/conf.d"
|
||||
rm -f "${cron_conf_dir}/${service_name}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Generate a monit configuration file for a given service
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# $2 - Pid file
|
||||
# $3 - Start command
|
||||
# $4 - Stop command
|
||||
# Flags:
|
||||
# --disable - Whether to disable the monit configuration
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
generate_monit_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local pid_file="${2:?pid file is missing}"
|
||||
local start_command="${3:?start command is missing}"
|
||||
local stop_command="${4:?stop command is missing}"
|
||||
local monit_conf_dir="/etc/monit/conf.d"
|
||||
local disabled="no"
|
||||
|
||||
# Parse optional CLI flags
|
||||
shift 4
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--disable)
|
||||
disabled="yes"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag ${1}" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
is_boolean_yes "$disabled" && conf_suffix=".disabled"
|
||||
mkdir -p "$monit_conf_dir"
|
||||
cat >"${monit_conf_dir}/${service_name}.conf${conf_suffix:-}" <<EOF
|
||||
check process ${service_name}
|
||||
with pidfile "${pid_file}"
|
||||
start program = "${start_command}" with timeout 90 seconds
|
||||
stop program = "${stop_command}" with timeout 90 seconds
|
||||
EOF
|
||||
}
|
||||
|
||||
########################
|
||||
# Remove a monit configuration file for a given service
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
remove_monit_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local monit_conf_dir="/etc/monit/conf.d"
|
||||
rm -f "${monit_conf_dir}/${service_name}.conf"
|
||||
}
|
||||
|
||||
########################
|
||||
# Generate a logrotate configuration file
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# $2 - Log files pattern
|
||||
# Flags:
|
||||
# --period - Period
|
||||
# --rotations - Number of rotations to store
|
||||
# --extra - Extra options (Optional)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
generate_logrotate_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local log_path="${2:?log path is missing}"
|
||||
local period="weekly"
|
||||
local rotations="150"
|
||||
local extra=""
|
||||
local logrotate_conf_dir="/etc/logrotate.d"
|
||||
local var_name
|
||||
# Parse optional CLI flags
|
||||
shift 2
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--period|--rotations|--extra)
|
||||
var_name="$(echo "$1" | sed -e "s/^--//" -e "s/-/_/g")"
|
||||
shift
|
||||
declare "$var_name"="${1:?"$var_name" is missing}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag ${1}" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
mkdir -p "$logrotate_conf_dir"
|
||||
cat <<EOF | sed '/^\s*$/d' >"${logrotate_conf_dir}/${service_name}"
|
||||
${log_path} {
|
||||
${period}
|
||||
rotate ${rotations}
|
||||
dateext
|
||||
compress
|
||||
copytruncate
|
||||
missingok
|
||||
$(indent "$extra" 2)
|
||||
}
|
||||
EOF
|
||||
}
|
||||
|
||||
########################
|
||||
# Remove a logrotate configuration file
|
||||
# Arguments:
|
||||
# $1 - Service name
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
remove_logrotate_conf() {
|
||||
local service_name="${1:?service name is missing}"
|
||||
local logrotate_conf_dir="/etc/logrotate.d"
|
||||
rm -f "${logrotate_conf_dir}/${service_name}"
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Validation functions library
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Check if the provided argument is an integer
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_int() {
|
||||
local -r int="${1:?missing value}"
|
||||
if [[ "$int" =~ ^-?[0-9]+ ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a positive integer
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_positive_int() {
|
||||
local -r int="${1:?missing value}"
|
||||
if is_int "$int" && (( "${int}" >= 0 )); then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a boolean or is the string 'yes/true'
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_boolean_yes() {
|
||||
local -r bool="${1:-}"
|
||||
# comparison is performed without regard to the case of alphabetic characters
|
||||
shopt -s nocasematch
|
||||
if [[ "$bool" = 1 || "$bool" =~ ^(yes|true)$ ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a boolean yes/no value
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_yes_no_value() {
|
||||
local -r bool="${1:-}"
|
||||
if [[ "$bool" =~ ^(yes|no)$ ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a boolean true/false value
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_true_false_value() {
|
||||
local -r bool="${1:-}"
|
||||
if [[ "$bool" =~ ^(true|false)$ ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is a boolean 1/0 value
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_1_0_value() {
|
||||
local -r bool="${1:-}"
|
||||
if [[ "$bool" =~ ^[10]$ ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if the provided argument is an empty string or not defined
|
||||
# Arguments:
|
||||
# $1 - Value to check
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
is_empty_value() {
|
||||
local -r val="${1:-}"
|
||||
if [[ -z "$val" ]]; then
|
||||
true
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Validate if the provided argument is a valid port
|
||||
# Arguments:
|
||||
# $1 - Port to validate
|
||||
# Returns:
|
||||
# Boolean and error message
|
||||
#########################
|
||||
validate_port() {
|
||||
local value
|
||||
local unprivileged=0
|
||||
|
||||
# Parse flags
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-unprivileged)
|
||||
unprivileged=1
|
||||
;;
|
||||
--)
|
||||
shift
|
||||
break
|
||||
;;
|
||||
-*)
|
||||
stderr_print "unrecognized flag $1"
|
||||
return 1
|
||||
;;
|
||||
*)
|
||||
break
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
if [[ "$#" -gt 1 ]]; then
|
||||
echo "too many arguments provided"
|
||||
return 2
|
||||
elif [[ "$#" -eq 0 ]]; then
|
||||
stderr_print "missing port argument"
|
||||
return 1
|
||||
else
|
||||
value=$1
|
||||
fi
|
||||
|
||||
if [[ -z "$value" ]]; then
|
||||
echo "the value is empty"
|
||||
return 1
|
||||
else
|
||||
if ! is_int "$value"; then
|
||||
echo "value is not an integer"
|
||||
return 2
|
||||
elif [[ "$value" -lt 0 ]]; then
|
||||
echo "negative value provided"
|
||||
return 2
|
||||
elif [[ "$value" -gt 65535 ]]; then
|
||||
echo "requested port is greater than 65535"
|
||||
return 2
|
||||
elif [[ "$unprivileged" = 1 && "$value" -lt 1024 ]]; then
|
||||
echo "privileged port requested"
|
||||
return 3
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Validate if the provided argument is a valid IPv4 address
|
||||
# Arguments:
|
||||
# $1 - IP to validate
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
validate_ipv4() {
|
||||
local ip="${1:?ip is missing}"
|
||||
local stat=1
|
||||
|
||||
if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
|
||||
read -r -a ip_array <<< "$(tr '.' ' ' <<< "$ip")"
|
||||
[[ ${ip_array[0]} -le 255 && ${ip_array[1]} -le 255 \
|
||||
&& ${ip_array[2]} -le 255 && ${ip_array[3]} -le 255 ]]
|
||||
stat=$?
|
||||
fi
|
||||
return $stat
|
||||
}
|
||||
|
||||
########################
|
||||
# Validate a string format
|
||||
# Arguments:
|
||||
# $1 - String to validate
|
||||
# Returns:
|
||||
# Boolean
|
||||
#########################
|
||||
validate_string() {
|
||||
local string
|
||||
local min_length=-1
|
||||
local max_length=-1
|
||||
|
||||
# Parse flags
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-min-length)
|
||||
shift
|
||||
min_length=${1:-}
|
||||
;;
|
||||
-max-length)
|
||||
shift
|
||||
max_length=${1:-}
|
||||
;;
|
||||
--)
|
||||
shift
|
||||
break
|
||||
;;
|
||||
-*)
|
||||
stderr_print "unrecognized flag $1"
|
||||
return 1
|
||||
;;
|
||||
*)
|
||||
break
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
if [ "$#" -gt 1 ]; then
|
||||
stderr_print "too many arguments provided"
|
||||
return 2
|
||||
elif [ "$#" -eq 0 ]; then
|
||||
stderr_print "missing string"
|
||||
return 1
|
||||
else
|
||||
string=$1
|
||||
fi
|
||||
|
||||
if [[ "$min_length" -ge 0 ]] && [[ "${#string}" -lt "$min_length" ]]; then
|
||||
echo "string length is less than $min_length"
|
||||
return 1
|
||||
fi
|
||||
if [[ "$max_length" -ge 0 ]] && [[ "${#string}" -gt "$max_length" ]]; then
|
||||
echo "string length is great than $max_length"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Library for managing versions strings
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
# Functions
|
||||
########################
|
||||
# Gets semantic version
|
||||
# Arguments:
|
||||
# $1 - version: string to extract major.minor.patch
|
||||
# $2 - section: 1 to extract major, 2 to extract minor, 3 to extract patch
|
||||
# Returns:
|
||||
# array with the major, minor and release
|
||||
#########################
|
||||
get_sematic_version () {
|
||||
local version="${1:?version is required}"
|
||||
local section="${2:?section is required}"
|
||||
local -a version_sections
|
||||
|
||||
#Regex to parse versions: x.y.z
|
||||
local -r regex='([0-9]+)(\.([0-9]+)(\.([0-9]+))?)?'
|
||||
|
||||
if [[ "$version" =~ $regex ]]; then
|
||||
local i=1
|
||||
local j=1
|
||||
local n=${#BASH_REMATCH[*]}
|
||||
|
||||
while [[ $i -lt $n ]]; do
|
||||
if [[ -n "${BASH_REMATCH[$i]}" ]] && [[ "${BASH_REMATCH[$i]:0:1}" != '.' ]]; then
|
||||
version_sections[$j]=${BASH_REMATCH[$i]}
|
||||
((j++))
|
||||
fi
|
||||
((i++))
|
||||
done
|
||||
|
||||
local number_regex='^[0-9]+$'
|
||||
if [[ "$section" =~ $number_regex ]] && (( section > 0 )) && (( section <= 3 )); then
|
||||
echo "${version_sections[$section]}"
|
||||
return
|
||||
else
|
||||
stderr_print "Section allowed values are: 1, 2, and 3"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,458 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Bitnami web server handler library
|
||||
|
||||
# shellcheck disable=SC1090,SC1091
|
||||
|
||||
# Load generic libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
########################
|
||||
# Execute a command (or list of commands) with the web server environment and library loaded
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_execute() {
|
||||
local -r web_server="${1:?missing web server}"
|
||||
shift
|
||||
# Run program in sub-shell to avoid web server environment getting loaded when not necessary
|
||||
(
|
||||
. "/opt/bitnami/scripts/lib${web_server}.sh"
|
||||
. "/opt/bitnami/scripts/${web_server}-env.sh"
|
||||
"$@"
|
||||
)
|
||||
}
|
||||
|
||||
########################
|
||||
# Prints the list of enabled web servers
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_list() {
|
||||
local -r -a supported_web_servers=(apache nginx)
|
||||
local -a existing_web_servers=()
|
||||
for web_server in "${supported_web_servers[@]}"; do
|
||||
[[ -f "/opt/bitnami/scripts/${web_server}-env.sh" ]] && existing_web_servers+=("$web_server")
|
||||
done
|
||||
echo "${existing_web_servers[@]:-}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Prints the currently-enabled web server type (only one, in order of preference)
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_type() {
|
||||
local -a web_servers
|
||||
read -r -a web_servers <<< "$(web_server_list)"
|
||||
echo "${web_servers[0]:-}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Validate that a supported web server is configured
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_validate() {
|
||||
local error_code=0
|
||||
local supported_web_servers=("apache" "nginx")
|
||||
|
||||
# Auxiliary functions
|
||||
print_validation_error() {
|
||||
error "$1"
|
||||
error_code=1
|
||||
}
|
||||
|
||||
if [[ -z "$(web_server_type)" || ! " ${supported_web_servers[*]} " == *" $(web_server_type) "* ]]; then
|
||||
print_validation_error "Could not detect any supported web servers. It must be one of: ${supported_web_servers[*]}"
|
||||
elif ! web_server_execute "$(web_server_type)" type -t "is_$(web_server_type)_running" >/dev/null; then
|
||||
print_validation_error "Could not load the $(web_server_type) web server library from /opt/bitnami/scripts. Check that it exists and is readable."
|
||||
fi
|
||||
|
||||
return "$error_code"
|
||||
}
|
||||
|
||||
########################
|
||||
# Check whether the web server is running
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# true if the web server is running, false otherwise
|
||||
#########################
|
||||
is_web_server_running() {
|
||||
"is_$(web_server_type)_running"
|
||||
}
|
||||
|
||||
########################
|
||||
# Start web server
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_start() {
|
||||
info "Starting $(web_server_type) in background"
|
||||
"${BITNAMI_ROOT_DIR}/scripts/$(web_server_type)/start.sh"
|
||||
}
|
||||
|
||||
########################
|
||||
# Stop web server
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_stop() {
|
||||
info "Stopping $(web_server_type)"
|
||||
"${BITNAMI_ROOT_DIR}/scripts/$(web_server_type)/stop.sh"
|
||||
}
|
||||
|
||||
########################
|
||||
# Restart web server
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_restart() {
|
||||
info "Restarting $(web_server_type)"
|
||||
"${BITNAMI_ROOT_DIR}/scripts/$(web_server_type)/restart.sh"
|
||||
}
|
||||
|
||||
########################
|
||||
# Reload web server
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_reload() {
|
||||
"${BITNAMI_ROOT_DIR}/scripts/$(web_server_type)/reload.sh"
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure a web server application configuration exists (i.e. Apache virtual host format or NGINX server block)
|
||||
# It serves as a wrapper for the specific web server function
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# $1 - App name
|
||||
# Flags:
|
||||
# --type - Application type, which has an effect on which configuration template to use
|
||||
# --hosts - Host listen addresses
|
||||
# --server-name - Server name
|
||||
# --server-aliases - Server aliases
|
||||
# --allow-remote-connections - Whether to allow remote connections or to require local connections
|
||||
# --disable - Whether to render server configurations with a .disabled prefix
|
||||
# --disable-http - Whether to render the app's HTTP server configuration with a .disabled prefix
|
||||
# --disable-https - Whether to render the app's HTTPS server configuration with a .disabled prefix
|
||||
# --http-port - HTTP port number
|
||||
# --https-port - HTTPS port number
|
||||
# --document-root - Path to document root directory
|
||||
# Apache-specific flags:
|
||||
# --apache-additional-configuration - Additional vhost configuration (no default)
|
||||
# --apache-additional-http-configuration - Additional HTTP vhost configuration (no default)
|
||||
# --apache-additional-https-configuration - Additional HTTPS vhost configuration (no default)
|
||||
# --apache-before-vhost-configuration - Configuration to add before the <VirtualHost> directive (no default)
|
||||
# --apache-allow-override - Whether to allow .htaccess files (only allowed when --move-htaccess is set to 'no' and type is not defined)
|
||||
# --apache-extra-directory-configuration - Extra configuration for the document root directory
|
||||
# --apache-proxy-address - Address where to proxy requests
|
||||
# --apache-proxy-configuration - Extra configuration for the proxy
|
||||
# --apache-proxy-http-configuration - Extra configuration for the proxy HTTP vhost
|
||||
# --apache-proxy-https-configuration - Extra configuration for the proxy HTTPS vhost
|
||||
# --apache-move-htaccess - Move .htaccess files to a common place so they can be loaded during Apache startup (only allowed when type is not defined)
|
||||
# NGINX-specific flags:
|
||||
# --nginx-additional-configuration - Additional server block configuration (no default)
|
||||
# --nginx-external-configuration - Configuration external to server block (no default)
|
||||
# Returns:
|
||||
# true if the configuration was enabled, false otherwise
|
||||
########################
|
||||
ensure_web_server_app_configuration_exists() {
|
||||
local app="${1:?missing app}"
|
||||
shift
|
||||
local -a apache_args nginx_args web_servers args_var
|
||||
apache_args=("$app")
|
||||
nginx_args=("$app")
|
||||
# Validate arguments
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
# Common flags
|
||||
--disable \
|
||||
| --disable-http \
|
||||
| --disable-https \
|
||||
)
|
||||
apache_args+=("$1")
|
||||
nginx_args+=("$1")
|
||||
;;
|
||||
--hosts \
|
||||
| --server-name \
|
||||
| --server-aliases \
|
||||
| --type \
|
||||
| --allow-remote-connections \
|
||||
| --http-port \
|
||||
| --https-port \
|
||||
| --document-root \
|
||||
)
|
||||
apache_args+=("$1" "${2:?missing value}")
|
||||
nginx_args+=("$1" "${2:?missing value}")
|
||||
shift
|
||||
;;
|
||||
|
||||
# Specific Apache flags
|
||||
--apache-additional-configuration \
|
||||
| --apache-additional-http-configuration \
|
||||
| --apache-additional-https-configuration \
|
||||
| --apache-before-vhost-configuration \
|
||||
| --apache-allow-override \
|
||||
| --apache-extra-directory-configuration \
|
||||
| --apache-proxy-address \
|
||||
| --apache-proxy-configuration \
|
||||
| --apache-proxy-http-configuration \
|
||||
| --apache-proxy-https-configuration \
|
||||
| --apache-move-htaccess \
|
||||
)
|
||||
apache_args+=("${1//apache-/}" "${2:?missing value}")
|
||||
shift
|
||||
;;
|
||||
|
||||
# Specific NGINX flags
|
||||
--nginx-additional-configuration \
|
||||
| --nginx-external-configuration)
|
||||
nginx_args+=("${1//nginx-/}" "${2:?missing value}")
|
||||
shift
|
||||
;;
|
||||
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
read -r -a web_servers <<< "$(web_server_list)"
|
||||
for web_server in "${web_servers[@]}"; do
|
||||
args_var="${web_server}_args[@]"
|
||||
web_server_execute "$web_server" "ensure_${web_server}_app_configuration_exists" "${!args_var}"
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure a web server application configuration does not exist anymore (i.e. Apache virtual host format or NGINX server block)
|
||||
# It serves as a wrapper for the specific web server function
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# $1 - App name
|
||||
# Returns:
|
||||
# true if the configuration was disabled, false otherwise
|
||||
########################
|
||||
ensure_web_server_app_configuration_not_exists() {
|
||||
local app="${1:?missing app}"
|
||||
local -a web_servers
|
||||
read -r -a web_servers <<< "$(web_server_list)"
|
||||
for web_server in "${web_servers[@]}"; do
|
||||
web_server_execute "$web_server" "ensure_${web_server}_app_configuration_not_exists" "$app"
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure the web server loads the configuration for an application in a URL prefix
|
||||
# It serves as a wrapper for the specific web server function
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# $1 - App name
|
||||
# Flags:
|
||||
# --allow-remote-connections - Whether to allow remote connections or to require local connections
|
||||
# --document-root - Path to document root directory
|
||||
# --prefix - URL prefix from where it will be accessible (i.e. /myapp)
|
||||
# --type - Application type, which has an effect on what configuration template will be used
|
||||
# Apache-specific flags:
|
||||
# --apache-additional-configuration - Additional vhost configuration (no default)
|
||||
# --apache-allow-override - Whether to allow .htaccess files (only allowed when --move-htaccess is set to 'no')
|
||||
# --apache-extra-directory-configuration - Extra configuration for the document root directory
|
||||
# --apache-move-htaccess - Move .htaccess files to a common place so they can be loaded during Apache startup
|
||||
# NGINX-specific flags:
|
||||
# --nginx-additional-configuration - Additional server block configuration (no default)
|
||||
# Returns:
|
||||
# true if the configuration was enabled, false otherwise
|
||||
########################
|
||||
ensure_web_server_prefix_configuration_exists() {
|
||||
local app="${1:?missing app}"
|
||||
shift
|
||||
local -a apache_args nginx_args web_servers args_var
|
||||
apache_args=("$app")
|
||||
nginx_args=("$app")
|
||||
# Validate arguments
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
# Common flags
|
||||
--allow-remote-connections \
|
||||
| --document-root \
|
||||
| --prefix \
|
||||
| --type \
|
||||
)
|
||||
apache_args+=("$1" "${2:?missing value}")
|
||||
nginx_args+=("$1" "${2:?missing value}")
|
||||
shift
|
||||
;;
|
||||
|
||||
# Specific Apache flags
|
||||
--apache-additional-configuration \
|
||||
| --apache-allow-override \
|
||||
| --apache-extra-directory-configuration \
|
||||
| --apache-move-htaccess \
|
||||
)
|
||||
apache_args+=("${1//apache-/}" "$2")
|
||||
shift
|
||||
;;
|
||||
|
||||
# Specific NGINX flags
|
||||
--nginx-additional-configuration)
|
||||
nginx_args+=("${1//nginx-/}" "$2")
|
||||
shift
|
||||
;;
|
||||
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
read -r -a web_servers <<< "$(web_server_list)"
|
||||
for web_server in "${web_servers[@]}"; do
|
||||
args_var="${web_server}_args[@]"
|
||||
web_server_execute "$web_server" "ensure_${web_server}_prefix_configuration_exists" "${!args_var}"
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure a web server application configuration is updated with the runtime configuration (i.e. ports)
|
||||
# It serves as a wrapper for the specific web server function
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# $1 - App name
|
||||
# Flags:
|
||||
# --hosts - Host listen addresses
|
||||
# --server-name - Server name
|
||||
# --server-aliases - Server aliases
|
||||
# --enable-http - Enable HTTP app configuration (if not enabled already)
|
||||
# --enable-https - Enable HTTPS app configuration (if not enabled already)
|
||||
# --disable-http - Disable HTTP app configuration (if not disabled already)
|
||||
# --disable-https - Disable HTTPS app configuration (if not disabled already)
|
||||
# --http-port - HTTP port number
|
||||
# --https-port - HTTPS port number
|
||||
# Returns:
|
||||
# true if the configuration was updated, false otherwise
|
||||
########################
|
||||
web_server_update_app_configuration() {
|
||||
local app="${1:?missing app}"
|
||||
shift
|
||||
local -a args web_servers
|
||||
args=("$app")
|
||||
# Validate arguments
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
# Common flags
|
||||
--enable-http \
|
||||
| --enable-https \
|
||||
| --disable-http \
|
||||
| --disable-https \
|
||||
)
|
||||
args+=("$1")
|
||||
;;
|
||||
--hosts \
|
||||
| --server-name \
|
||||
| --server-aliases \
|
||||
| --http-port \
|
||||
| --https-port \
|
||||
)
|
||||
args+=("$1" "${2:?missing value}")
|
||||
shift
|
||||
;;
|
||||
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
read -r -a web_servers <<< "$(web_server_list)"
|
||||
for web_server in "${web_servers[@]}"; do
|
||||
web_server_execute "$web_server" "${web_server}_update_app_configuration" "${args[@]}"
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Enable loading page, which shows users that the initialization process is not yet completed
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_enable_loading_page() {
|
||||
ensure_web_server_app_configuration_exists "__loading" --hosts "_default_" \
|
||||
--apache-additional-configuration "
|
||||
# Show a HTTP 503 Service Unavailable page by default
|
||||
RedirectMatch 503 ^/$
|
||||
# Show index.html if server is answering with 404 Not Found or 503 Service Unavailable status codes
|
||||
ErrorDocument 404 /index.html
|
||||
ErrorDocument 503 /index.html" \
|
||||
--nginx-additional-configuration "
|
||||
# Show a HTTP 503 Service Unavailable page by default
|
||||
location / {
|
||||
return 503;
|
||||
}
|
||||
# Show index.html if server is answering with 404 Not Found or 503 Service Unavailable status codes
|
||||
error_page 404 @installing;
|
||||
error_page 503 @installing;
|
||||
location @installing {
|
||||
rewrite ^(.*)$ /index.html break;
|
||||
}"
|
||||
web_server_reload
|
||||
}
|
||||
|
||||
########################
|
||||
# Enable loading page, which shows users that the initialization process is not yet completed
|
||||
# Globals:
|
||||
# *
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
web_server_disable_install_page() {
|
||||
ensure_web_server_app_configuration_not_exists "__loading"
|
||||
web_server_reload
|
||||
}
|
||||
24
bitnami/airflow-scheduler/2/debian-10/prebuildfs/usr/sbin/install_packages
Executable file
24
bitnami/airflow-scheduler/2/debian-10/prebuildfs/usr/sbin/install_packages
Executable file
@@ -0,0 +1,24 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
set -u
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
n=0
|
||||
max=2
|
||||
until [ $n -gt $max ]; do
|
||||
set +e
|
||||
(
|
||||
apt-get update -qq &&
|
||||
apt-get install -y --no-install-recommends "$@"
|
||||
)
|
||||
CODE=$?
|
||||
set -e
|
||||
if [ $CODE -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
if [ $n -eq $max ]; then
|
||||
exit $CODE
|
||||
fi
|
||||
echo "apt failed, retrying"
|
||||
n=$(($n + 1))
|
||||
done
|
||||
rm -r /var/lib/apt/lists /var/cache/apt/archives
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Environment configuration for airflow-scheduler
|
||||
|
||||
# The values for all environment variables will be set in the below order of precedence
|
||||
# 1. Custom environment variables defined below after Bitnami defaults
|
||||
# 2. Constants defined in this file (environment variables with no default), i.e. BITNAMI_ROOT_DIR
|
||||
# 3. Environment variables overridden via external files using *_FILE variables (see below)
|
||||
# 4. Environment variables set externally (i.e. current Bash context/Dockerfile/userdata)
|
||||
|
||||
# Load logging library
|
||||
# shellcheck disable=SC1090,SC1091
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
export BITNAMI_ROOT_DIR="/opt/bitnami"
|
||||
export BITNAMI_VOLUME_DIR="/bitnami"
|
||||
|
||||
# Logging configuration
|
||||
export MODULE="${MODULE:-airflow-scheduler}"
|
||||
export BITNAMI_DEBUG="${BITNAMI_DEBUG:-false}"
|
||||
|
||||
# By setting an environment variable matching *_FILE to a file path, the prefixed environment
|
||||
# variable will be overridden with the value specified in that file
|
||||
airflow_scheduler_env_vars=(
|
||||
AIRFLOW_EXECUTOR
|
||||
AIRFLOW_FERNET_KEY
|
||||
AIRFLOW_SECRET_KEY
|
||||
AIRFLOW_WEBSERVER_HOST
|
||||
AIRFLOW_WEBSERVER_PORT_NUMBER
|
||||
AIRFLOW_LOAD_EXAMPLES
|
||||
AIRFLOW_HOSTNAME_CALLABLE
|
||||
AIRFLOW_DATABASE_HOST
|
||||
AIRFLOW_DATABASE_PORT_NUMBER
|
||||
AIRFLOW_DATABASE_NAME
|
||||
AIRFLOW_DATABASE_USERNAME
|
||||
AIRFLOW_DATABASE_PASSWORD
|
||||
AIRFLOW_DATABASE_USE_SSL
|
||||
AIRFLOW_REDIS_USE_SSL
|
||||
REDIS_HOST
|
||||
REDIS_PORT_NUMBER
|
||||
REDIS_USER
|
||||
REDIS_PASSWORD
|
||||
)
|
||||
for env_var in "${airflow_scheduler_env_vars[@]}"; do
|
||||
file_env_var="${env_var}_FILE"
|
||||
if [[ -n "${!file_env_var:-}" ]]; then
|
||||
if [[ -r "${!file_env_var:-}" ]]; then
|
||||
export "${env_var}=$(< "${!file_env_var}")"
|
||||
unset "${file_env_var}"
|
||||
else
|
||||
warn "Skipping export of '${env_var}'. '${!file_env_var:-}' is not readable."
|
||||
fi
|
||||
fi
|
||||
done
|
||||
unset airflow_scheduler_env_vars
|
||||
|
||||
# Airflow paths
|
||||
export AIRFLOW_BASE_DIR="${BITNAMI_ROOT_DIR}/airflow"
|
||||
export AIRFLOW_HOME="${AIRFLOW_BASE_DIR}"
|
||||
export AIRFLOW_BIN_DIR="${AIRFLOW_BASE_DIR}/venv/bin"
|
||||
export AIRFLOW_LOGS_DIR="${AIRFLOW_BASE_DIR}/logs"
|
||||
export AIRFLOW_SCHEDULER_LOGS_DIR="${AIRFLOW_LOGS_DIR}/scheduler"
|
||||
export AIRFLOW_LOG_FILE="${AIRFLOW_LOGS_DIR}/airflow-scheduler.log"
|
||||
export AIRFLOW_CONF_FILE="${AIRFLOW_BASE_DIR}/airflow.cfg"
|
||||
export AIRFLOW_TMP_DIR="${AIRFLOW_BASE_DIR}/tmp"
|
||||
export AIRFLOW_PID_FILE="${AIRFLOW_TMP_DIR}/airflow-scheduler.pid"
|
||||
export AIRFLOW_DAGS_DIR="${AIRFLOW_BASE_DIR}/dags"
|
||||
|
||||
# System users (when running with a privileged user)
|
||||
export AIRFLOW_DAEMON_USER="airflow"
|
||||
export AIRFLOW_DAEMON_GROUP="airflow"
|
||||
|
||||
# Airflow configuration
|
||||
export AIRFLOW_EXECUTOR="${AIRFLOW_EXECUTOR:-SequentialExecutor}"
|
||||
export AIRFLOW_FERNET_KEY="${AIRFLOW_FERNET_KEY:-}"
|
||||
export AIRFLOW_SECRET_KEY="${AIRFLOW_SECRET_KEY:-}"
|
||||
export AIRFLOW_WEBSERVER_HOST="${AIRFLOW_WEBSERVER_HOST:-127.0.0.1}"
|
||||
export AIRFLOW_WEBSERVER_PORT_NUMBER="${AIRFLOW_WEBSERVER_PORT_NUMBER:-8080}"
|
||||
export AIRFLOW_LOAD_EXAMPLES="${AIRFLOW_LOAD_EXAMPLES:-yes}"
|
||||
export AIRFLOW_HOSTNAME_CALLABLE="${AIRFLOW_HOSTNAME_CALLABLE:-}"
|
||||
|
||||
# Airflow database configuration
|
||||
export AIRFLOW_DATABASE_HOST="${AIRFLOW_DATABASE_HOST:-postgresql}"
|
||||
export AIRFLOW_DATABASE_PORT_NUMBER="${AIRFLOW_DATABASE_PORT_NUMBER:-5432}"
|
||||
export AIRFLOW_DATABASE_NAME="${AIRFLOW_DATABASE_NAME:-bitnami_airflow}"
|
||||
export AIRFLOW_DATABASE_USERNAME="${AIRFLOW_DATABASE_USERNAME:-bn_airflow}"
|
||||
export AIRFLOW_DATABASE_PASSWORD="${AIRFLOW_DATABASE_PASSWORD:-}"
|
||||
export AIRFLOW_DATABASE_USE_SSL="${AIRFLOW_DATABASE_USE_SSL:-no}"
|
||||
export AIRFLOW_REDIS_USE_SSL="${AIRFLOW_REDIS_USE_SSL:-no}"
|
||||
export REDIS_HOST="${REDIS_HOST:-redis}"
|
||||
export REDIS_PORT_NUMBER="${REDIS_PORT_NUMBER:-6379}"
|
||||
export REDIS_USER="${REDIS_USER:-}"
|
||||
export REDIS_PASSWORD="${REDIS_PASSWORD:-}"
|
||||
|
||||
# Custom environment variables may be defined below
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/bin/bash
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
# set -o xtrace # Uncomment this line for debugging purpose
|
||||
|
||||
# Load Airflow environment variables
|
||||
. /opt/bitnami/scripts/airflow-scheduler-env.sh
|
||||
|
||||
# Load libraries
|
||||
. /opt/bitnami/scripts/libbitnami.sh
|
||||
. /opt/bitnami/scripts/libairflowscheduler.sh
|
||||
|
||||
print_welcome_page
|
||||
|
||||
if ! am_i_root && [[ -e "$LIBNSS_WRAPPER_PATH" ]]; then
|
||||
info "Enabling non-root system user with nss_wrapper"
|
||||
echo "airflow:x:$(id -u):$(id -g):Airflow:$AIRFLOW_HOME:/bin/false" > "$NSS_WRAPPER_PASSWD"
|
||||
echo "airflow:x:$(id -g):" > "$NSS_WRAPPER_GROUP"
|
||||
|
||||
export LD_PRELOAD="$LIBNSS_WRAPPER_PATH"
|
||||
fi
|
||||
|
||||
# Install custom python package if requirements.txt is present
|
||||
if [[ -f "/bitnami/python/requirements.txt" ]]; then
|
||||
. /opt/bitnami/airflow/venv/bin/activate
|
||||
pip install -r /bitnami/python/requirements.txt
|
||||
deactivate
|
||||
fi
|
||||
|
||||
if [[ "$*" = *"/opt/bitnami/scripts/airflow-scheduler/run.sh"* || "$*" = *"/run.sh"* ]]; then
|
||||
info "** Starting Airflow setup **"
|
||||
/opt/bitnami/scripts/airflow-scheduler/setup.sh
|
||||
info "** Airflow setup finished! **"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
exec "$@"
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/bin/bash
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
# set -o xtrace # Uncomment this line for debugging purpose
|
||||
|
||||
# Load Airflow environment variables
|
||||
. /opt/bitnami/scripts/airflow-scheduler-env.sh
|
||||
|
||||
# Load libraries
|
||||
. /opt/bitnami/scripts/libairflowscheduler.sh
|
||||
. /opt/bitnami/scripts/libfs.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
|
||||
ensure_dir_exists "$AIRFLOW_BASE_DIR"
|
||||
# Ensure the needed directories exist with write permissions
|
||||
for dir in "$AIRFLOW_TMP_DIR" "$AIRFLOW_LOGS_DIR" "$AIRFLOW_DAGS_DIR"; do
|
||||
ensure_dir_exists "$dir"
|
||||
configure_permissions_ownership "$dir" -d "775" -f "664" -g "root"
|
||||
done
|
||||
|
||||
chmod -R g+rwX "$AIRFLOW_BASE_DIR"
|
||||
@@ -0,0 +1,24 @@
|
||||
#!/bin/bash
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
# set -o xtrace # Uncomment this line for debugging purposes
|
||||
|
||||
# Load Airflow environment variables
|
||||
. /opt/bitnami/scripts/airflow-scheduler-env.sh
|
||||
|
||||
# Load libraries
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/libairflowscheduler.sh
|
||||
|
||||
args=("--pid" "$AIRFLOW_PID_FILE" "$@")
|
||||
|
||||
info "** Starting Airflow **"
|
||||
if am_i_root; then
|
||||
exec gosu "$AIRFLOW_DAEMON_USER" "${AIRFLOW_BIN_DIR}/airflow" "scheduler" "${args[@]}"
|
||||
else
|
||||
exec "${AIRFLOW_BIN_DIR}/airflow" "scheduler" "${args[@]}"
|
||||
fi
|
||||
@@ -0,0 +1,23 @@
|
||||
#!/bin/bash
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
# set -o xtrace # Uncomment this line for debugging purposes
|
||||
|
||||
# Load Airflow environment variables
|
||||
. /opt/bitnami/scripts/airflow-scheduler-env.sh
|
||||
|
||||
# Load libraries
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/libfs.sh
|
||||
. /opt/bitnami/scripts/libairflowscheduler.sh
|
||||
|
||||
# Ensure Airflow environment variables settings are valid
|
||||
airflow_scheduler_validate
|
||||
# Ensure Airflow daemon user exists when running as root
|
||||
am_i_root && ensure_user_exists "$AIRFLOW_DAEMON_USER" --group "$AIRFLOW_DAEMON_GROUP"
|
||||
# Ensure Airflow is initialized
|
||||
airflow_scheduler_initialize
|
||||
@@ -0,0 +1,456 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Bitnami Airflow library
|
||||
|
||||
# shellcheck disable=SC1091,SC2153
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/libfile.sh
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libnet.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/libservice.sh
|
||||
. /opt/bitnami/scripts/libvalidations.sh
|
||||
. /opt/bitnami/scripts/libpersistence.sh
|
||||
|
||||
# Load database library
|
||||
if [[ -f /opt/bitnami/scripts/libpostgresqlclient.sh ]]; then
|
||||
. /opt/bitnami/scripts/libpostgresqlclient.sh
|
||||
elif [[ -f /opt/bitnami/scripts/libpostgresql.sh ]]; then
|
||||
. /opt/bitnami/scripts/libpostgresql.sh
|
||||
fi
|
||||
|
||||
# Functions
|
||||
|
||||
########################
|
||||
# Validate Airflow inputs
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# 0 if the validation succeeded, 1 otherwise
|
||||
#########################
|
||||
airflow_validate() {
|
||||
local error_code=0
|
||||
|
||||
# Auxiliary functions
|
||||
print_validation_error() {
|
||||
error "$1"
|
||||
error_code=1
|
||||
}
|
||||
|
||||
check_multi_value() {
|
||||
if [[ " ${2} " != *" ${!1} "* ]]; then
|
||||
print_validation_error "The allowed values for ${1} are: ${2}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Check postgresql host
|
||||
[[ -z "$AIRFLOW_DATABASE_HOST" ]] && print_validation_error "Missing AIRFLOW_DATABASE_HOST"
|
||||
|
||||
# Check LDAP parameters
|
||||
if is_boolean_yes "$AIRFLOW_LDAP_ENABLE"; then
|
||||
[[ -z "$AIRFLOW_LDAP_URI" ]] && print_validation_error "Missing AIRFLOW_LDAP_URI"
|
||||
[[ -z "$AIRFLOW_LDAP_SEARCH" ]] && print_validation_error "Missing AIRFLOW_LDAP_SEARCH"
|
||||
[[ -z "$AIRFLOW_LDAP_UID_FIELD" ]] && print_validation_error "Missing AIRFLOW_LDAP_UID_FIELD"
|
||||
[[ -z "$AIRFLOW_LDAP_BIND_USER" ]] && print_validation_error "Missing AIRFLOW_LDAP_BIND_USER"
|
||||
[[ -z "$AIRFLOW_LDAP_BIND_PASSWORD" ]] && print_validation_error "Missing AIRFLOW_LDAP_BIND_PASSWORD"
|
||||
[[ -z "$AIRFLOW_LDAP_ROLES_MAPPING" ]] && print_validation_error "Missing AIRFLOW_LDAP_ROLES_MAPPING"
|
||||
[[ -z "$AIRFLOW_LDAP_ROLES_SYNC_AT_LOGIN" ]] && print_validation_error "Missing AIRFLOW_LDAP_ROLES_SYNC_AT_LOGIN"
|
||||
[[ -z "$AIRFLOW_LDAP_USER_REGISTRATION" ]] && print_validation_error "Missing AIRFLOW_LDAP_USER_REGISTRATION"
|
||||
[[ -z "$AIRFLOW_LDAP_USER_REGISTRATION_ROLE" ]] && print_validation_error "Missing AIRFLOW_LDAP_USER_REGISTRATION_ROLE"
|
||||
|
||||
# Chack boolean env vars contain valid values
|
||||
for var in "AIRFLOW_LDAP_USER_REGISTRATION" "AIRFLOW_LDAP_ROLES_SYNC_AT_LOGIN" "AIRFLOW_LDAP_USE_TLS"; do
|
||||
check_multi_value "$var" "True False"
|
||||
done
|
||||
|
||||
if [[ "$AIRFLOW_LDAP_USE_TLS" == "True" ]]; then
|
||||
[[ -z "$AIRFLOW_LDAP_ALLOW_SELF_SIGNED" ]] && print_validation_error "Missing AIRFLOW_LDAP_ALLOW_SELF_SIGNED"
|
||||
[[ -z "$AIRFLOW_LDAP_TLS_CA_CERTIFICATE" ]] && print_validation_error "Missing AIRFLOW_LDAP_TLS_CA_CERTIFICATE"
|
||||
fi
|
||||
|
||||
fi
|
||||
|
||||
# Check pool parameters
|
||||
if [[ -n "$AIRFLOW_POOL_NAME" ]]; then
|
||||
[[ -z "$AIRFLOW_POOL_DESC" ]] && print_validation_error "Provided AIRFLOW_POOL_NAME but missing AIRFLOW_POOL_DESC"
|
||||
[[ -z "$AIRFLOW_POOL_SIZE" ]] && print_validation_error "Provided AIRFLOW_POOL_NAME but missing AIRFLOW_POOL_SIZE"
|
||||
fi
|
||||
|
||||
return "$error_code"
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure Airflow is initialized
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_initialize() {
|
||||
info "Initializing Airflow ..."
|
||||
|
||||
# Change permissions if running as root
|
||||
for dir in "$AIRFLOW_TMP_DIR" "$AIRFLOW_LOGS_DIR" "$AIRFLOW_DAGS_DIR"; do
|
||||
ensure_dir_exists "$dir"
|
||||
am_i_root && chown "$AIRFLOW_DAEMON_USER:$AIRFLOW_DAEMON_GROUP" "$dir"
|
||||
done
|
||||
|
||||
# The configuration file is not persisted. If it is not provided, generate it based on env vars
|
||||
if [[ ! -f "$AIRFLOW_CONF_FILE" ]]; then
|
||||
info "No injected configuration file found. Creating default config file"
|
||||
airflow_generate_config
|
||||
else
|
||||
info "Configuration file found, loading configuration"
|
||||
fi
|
||||
|
||||
info "Trying to connect to the database server"
|
||||
airflow_wait_for_postgresql_connection
|
||||
# Check if the Airflow database has been already initialized
|
||||
if ! debug_execute airflow db check-migrations; then
|
||||
# Delete pid file
|
||||
rm -f "$AIRFLOW_PID_FILE"
|
||||
|
||||
# Initialize database
|
||||
info "Populating database"
|
||||
debug_execute airflow db init
|
||||
|
||||
airflow_create_admin_user
|
||||
airflow_create_pool
|
||||
else
|
||||
# Upgrade database
|
||||
info "Upgrading database schema"
|
||||
debug_execute airflow db upgrade
|
||||
true # Avoid return false when I am not root
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
########################
|
||||
# Generate Airflow conf file
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_generate_config() {
|
||||
# Generate Airflow default files
|
||||
debug_execute airflow version
|
||||
|
||||
# Setup Airflow base URL
|
||||
airflow_configure_base_url
|
||||
# Configure Airflow Hostname
|
||||
[[ -n "$AIRFLOW_HOSTNAME_CALLABLE" ]] && airflow_conf_set "core" "hostname_callable" "$AIRFLOW_HOSTNAME_CALLABLE"
|
||||
# Configure Airflow webserver authentication
|
||||
airflow_configure_webserver_authentication
|
||||
# Configure Airflow to load examples
|
||||
if is_boolean_yes "$AIRFLOW_LOAD_EXAMPLES"; then
|
||||
airflow_conf_set "core" "load_examples" "True"
|
||||
else
|
||||
airflow_conf_set "core" "load_examples" "False"
|
||||
fi
|
||||
# Configure Airflow database
|
||||
airflow_configure_database
|
||||
|
||||
# Configure the Webserver port
|
||||
airflow_conf_set "webserver" "web_server_port" "$AIRFLOW_WEBSERVER_PORT_NUMBER"
|
||||
|
||||
# Setup the secret keys for database connection and flask application (fernet key and secret key)
|
||||
# ref: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#fernet-key
|
||||
# ref: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key
|
||||
[[ -n "$AIRFLOW_FERNET_KEY" ]] && airflow_conf_set "core" "fernet_key" "$AIRFLOW_FERNET_KEY"
|
||||
[[ -n "$AIRFLOW_SECRET_KEY" ]] && airflow_conf_set "webserver" "secret_key" "$AIRFLOW_SECRET_KEY"
|
||||
|
||||
# Configure Airflow executor
|
||||
airflow_conf_set "core" "executor" "$AIRFLOW_EXECUTOR"
|
||||
[[ "$AIRFLOW_EXECUTOR" == "CeleryExecutor" || "$AIRFLOW_EXECUTOR" == "CeleryKubernetesExecutor" ]] && airflow_configure_celery_executor
|
||||
true # Avoid the function to fail due to the check above
|
||||
}
|
||||
|
||||
########################
|
||||
# Set property on the Airflow configuration file
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_conf_set() {
|
||||
local -r section="${1:?section is required}"
|
||||
local -r key="${2:?key is required}"
|
||||
local -r value="${3:?value is required}"
|
||||
local -r file="${4:-${AIRFLOW_CONF_FILE}}"
|
||||
|
||||
ini-file set --section "$section" --key "$key" --value "$value" "$file"
|
||||
}
|
||||
|
||||
########################
|
||||
# Configure Airflow base url
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_configure_base_url() {
|
||||
if [[ -z "$AIRFLOW_BASE_URL" ]]; then
|
||||
airflow_conf_set "webserver" "base_url" "http://${AIRFLOW_WEBSERVER_HOST}:${AIRFLOW_WEBSERVER_PORT_NUMBER}"
|
||||
else
|
||||
airflow_conf_set "webserver" "base_url" "$AIRFLOW_BASE_URL"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Configure Airflow webserver authentication
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_configure_webserver_authentication() {
|
||||
info "Configuring Airflow webserver authentication"
|
||||
airflow_conf_set "webserver" "rbac" "true"
|
||||
|
||||
if is_boolean_yes "$AIRFLOW_LDAP_ENABLE"; then
|
||||
info "Enabling LDAP authentication"
|
||||
# Based on PR https://github.com/apache/airflow/pull/16647
|
||||
replace_in_file "$AIRFLOW_WEBSERVER_CONF_FILE" "# from airflow.www.fab_security.manager import AUTH_LDAP" "from airflow.www.fab_security.manager import AUTH_LDAP"
|
||||
replace_in_file "$AIRFLOW_WEBSERVER_CONF_FILE" "from airflow.www.fab_security.manager import AUTH_DB" "# from airflow.www.fab_security.manager import AUTH_DB"
|
||||
|
||||
# webserver config
|
||||
airflow_webserver_conf_set "AUTH_TYPE" "AUTH_LDAP"
|
||||
airflow_webserver_conf_set "AUTH_LDAP_SERVER" "$AIRFLOW_LDAP_URI" "yes"
|
||||
|
||||
# searches
|
||||
airflow_webserver_conf_set "AUTH_LDAP_SEARCH" "$AIRFLOW_LDAP_SEARCH" "yes"
|
||||
airflow_webserver_conf_set "AUTH_LDAP_UID_FIELD" "$AIRFLOW_LDAP_UID_FIELD" "yes"
|
||||
|
||||
# Special account for searches
|
||||
airflow_webserver_conf_set "AUTH_LDAP_BIND_USER" "$AIRFLOW_LDAP_BIND_USER" "yes"
|
||||
airflow_webserver_conf_set "AUTH_LDAP_BIND_PASSWORD" "$AIRFLOW_LDAP_BIND_PASSWORD" "yes"
|
||||
|
||||
# User self registration
|
||||
airflow_webserver_conf_set "AUTH_USER_REGISTRATION" "$AIRFLOW_LDAP_USER_REGISTRATION"
|
||||
airflow_webserver_conf_set "AUTH_USER_REGISTRATION_ROLE" "$AIRFLOW_LDAP_USER_REGISTRATION_ROLE" "yes"
|
||||
|
||||
# Mapping from LDAP DN to list of FAB roles
|
||||
airflow_webserver_conf_set "AUTH_ROLES_MAPPING" "$AIRFLOW_LDAP_ROLES_MAPPING"
|
||||
|
||||
# Replace user's roles at login
|
||||
airflow_webserver_conf_set "AUTH_ROLES_SYNC_AT_LOGIN" "$AIRFLOW_LDAP_ROLES_SYNC_AT_LOGIN"
|
||||
|
||||
# Allowing/Denying of self signed certs for StartTLS OR SSL ldaps:// connections
|
||||
airflow_webserver_conf_set "AUTH_LDAP_ALLOW_SELF_SIGNED" "$AIRFLOW_LDAP_ALLOW_SELF_SIGNED"
|
||||
|
||||
# If StartTLS supply cert
|
||||
if [[ "$AIRFLOW_LDAP_USE_TLS" == "True" ]]; then
|
||||
airflow_webserver_conf_set "AUTH_LDAP_TLS_CACERTFILE" "$AIRFLOW_LDAP_TLS_CA_CERTIFICATE" "yes"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Set properties in Airflow's webserver_config.py
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_webserver_conf_set() {
|
||||
local -r key="${1:?missing key}"
|
||||
local -r value="${2:?missing key}"
|
||||
local -r is_literal="${3:-no}"
|
||||
shift 2
|
||||
|
||||
local -r file="$AIRFLOW_WEBSERVER_CONF_FILE"
|
||||
# Check if the value was set before
|
||||
if grep -q "^#*\\s*${key} =.*$" "$file"; then
|
||||
local entry
|
||||
is_boolean_yes "$is_literal" && entry="${key} = '${value}'" || entry="${key} = ${value}"
|
||||
# Update the existing key
|
||||
replace_in_file "$file" "^#*\\s*${key} =.*$" "$entry" false
|
||||
else
|
||||
# Add a new key
|
||||
local new_value="$value"
|
||||
is_boolean_yes "$is_literal" && new_value="'${value}'"
|
||||
printf '\n%s = %s' "$key" "$new_value" >>"$file"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Configure Airflow database
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_configure_database() {
|
||||
local -r user=$(airflow_encode_url "$AIRFLOW_DATABASE_USERNAME")
|
||||
local -r password=$(airflow_encode_url "$AIRFLOW_DATABASE_PASSWORD")
|
||||
local extra_options
|
||||
is_boolean_yes "$AIRFLOW_DATABASE_USE_SSL" && extra_options="?sslmode=require"
|
||||
|
||||
info "Configuring Airflow database"
|
||||
airflow_conf_set "core" "sql_alchemy_conn" "postgresql+psycopg2://${user}:${password}@${AIRFLOW_DATABASE_HOST}:${AIRFLOW_DATABASE_PORT_NUMBER}/${AIRFLOW_DATABASE_NAME}${extra_options:-}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Return URL encoded string in the airflow conf format
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_encode_url() {
|
||||
local -r url="${1?Missing url}"
|
||||
|
||||
urlencode() {
|
||||
old_lc_collate="${LC_COLLATE:-}"
|
||||
LC_COLLATE=C
|
||||
|
||||
local length="${#1}"
|
||||
for ((i = 0; i < length; i++)); do
|
||||
local c="${1:$i:1}"
|
||||
case $c in
|
||||
[a-zA-Z0-9.~_-]) printf '%s' "$c" ;;
|
||||
*) printf '%%%02X' "'$c" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
LC_COLLATE="$old_lc_collate"
|
||||
}
|
||||
|
||||
local -r url_encoded=$(urlencode "$url")
|
||||
# Replace % by %%
|
||||
echo "${url_encoded//\%/\%\%}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Configure Airflow celery executor
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_configure_celery_executor() {
|
||||
info "Configuring Celery Executor"
|
||||
|
||||
# Configure celery Redis url
|
||||
local -r redis_user=$(airflow_encode_url "$REDIS_USER")
|
||||
local -r redis_password=$(airflow_encode_url "$REDIS_PASSWORD")
|
||||
airflow_conf_set "celery" "broker_url" "redis://${redis_user}:${redis_password}@${REDIS_HOST}:${REDIS_PORT_NUMBER}/1"
|
||||
is_boolean_yes "$AIRFLOW_REDIS_USE_SSL" && airflow_conf_set "celery" "redis_backend_use_ssl" "true"
|
||||
|
||||
# Configure celery backend
|
||||
local -r database_user=$(airflow_encode_url "$AIRFLOW_DATABASE_USERNAME")
|
||||
local -r database_password=$(airflow_encode_url "$AIRFLOW_DATABASE_PASSWORD")
|
||||
local database_extra_options
|
||||
is_boolean_yes "$AIRFLOW_DATABASE_USE_SSL" && database_extra_options="?sslmode=require"
|
||||
airflow_conf_set "celery" "result_backend" "db+postgresql://${database_user}:${database_password}@${AIRFLOW_DATABASE_HOST}:${AIRFLOW_DATABASE_PORT_NUMBER}/${AIRFLOW_DATABASE_NAME}${database_extra_options:-}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Wait until the database is accessible
|
||||
# Globals:
|
||||
# None
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# true if the database connection succeeded, false otherwise
|
||||
#########################
|
||||
airflow_wait_for_postgresql_connection() {
|
||||
if ! retry_while "debug_execute airflow db check"; then
|
||||
error "Could not connect to the database"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Airflow create admin user
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_create_admin_user() {
|
||||
info "Creating Airflow admin user"
|
||||
debug_execute airflow users create -r "Admin" -u "$AIRFLOW_USERNAME" -e "$AIRFLOW_EMAIL" -p "$AIRFLOW_PASSWORD" -f "$AIRFLOW_FIRSTNAME" -l "$AIRFLOW_LASTNAME"
|
||||
}
|
||||
|
||||
########################
|
||||
# Airflow create pool
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_create_pool() {
|
||||
if [[ -n "$AIRFLOW_POOL_NAME" ]] && [[ -n "$AIRFLOW_POOL_SIZE" ]] && [[ -n "$AIRFLOW_POOL_DESC" ]]; then
|
||||
info "Creating Airflow pool"
|
||||
debug_execute airflow pool -s "$AIRFLOW_POOL_NAME" "$AIRFLOW_POOL_SIZE" "$AIRFLOW_POOL_DESC"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if Airflow is running
|
||||
# Globals:
|
||||
# AIRFLOW_PID_FILE
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# Whether Airflow is running
|
||||
########################
|
||||
is_airflow_running() {
|
||||
local pid
|
||||
pid="$(get_pid_from_file "$AIRFLOW_PID_FILE")"
|
||||
if [[ -n "$pid" ]]; then
|
||||
is_service_running "$pid"
|
||||
else
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Check if Airflow is running
|
||||
# Globals:
|
||||
# AIRFLOW_PID_FILE
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# Whether Airflow is not running
|
||||
########################
|
||||
is_airflow_not_running() {
|
||||
! is_airflow_running
|
||||
}
|
||||
|
||||
########################
|
||||
# Stop Airflow
|
||||
# Globals:
|
||||
# AIRFLOW*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_stop() {
|
||||
info "Stopping Airflow..."
|
||||
stop_service_using_pid "$AIRFLOW_PID_FILE"
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Bitnami Airflow library
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/libfile.sh
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libnet.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/libservice.sh
|
||||
. /opt/bitnami/scripts/libvalidations.sh
|
||||
. /opt/bitnami/scripts/libpersistence.sh
|
||||
|
||||
# Load airflow library
|
||||
. /opt/bitnami/scripts/libairflow.sh
|
||||
|
||||
########################
|
||||
# Validate Airflow Scheduler inputs
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_scheduler_validate() {
|
||||
# Check webserver host
|
||||
[[ -z "$AIRFLOW_WEBSERVER_HOST" ]] && print_validation_error "Missing AIRFLOW_WEBSERVER_HOST"
|
||||
[[ -z "$AIRFLOW_WEBSERVER_PORT_NUMBER" ]] && print_validation_error "Missing AIRFLOW_WEBSERVER_PORT_NUMBER"
|
||||
# Check postgresql host
|
||||
[[ -z "$AIRFLOW_DATABASE_HOST" ]] && print_validation_error "Missing AIRFLOW_DATABASE_HOST"
|
||||
|
||||
# Avoid to fail because of the above check
|
||||
true
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure Airflow Scheduler is initialized
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_scheduler_initialize() {
|
||||
# Change permissions if running as root
|
||||
for dir in "$AIRFLOW_TMP_DIR" "$AIRFLOW_LOGS_DIR" "$AIRFLOW_SCHEDULER_LOGS_DIR"; do
|
||||
ensure_dir_exists "$dir"
|
||||
am_i_root && chown "$AIRFLOW_DAEMON_USER:$AIRFLOW_DAEMON_GROUP" "$dir"
|
||||
done
|
||||
|
||||
# The configuration file is not persisted. If it is not provided, generate it based on env vars
|
||||
if [[ ! -f "$AIRFLOW_CONF_FILE" ]]; then
|
||||
info "No injected configuration file found. Creating default config file"
|
||||
airflow_scheduler_generate_config
|
||||
else
|
||||
info "Configuration file found, loading configuration"
|
||||
fi
|
||||
|
||||
info "Trying to connect to the database server"
|
||||
airflow_wait_for_postgresql_connection
|
||||
# Change the permissions after restoring the persisted data in case we are root
|
||||
for dir in "$AIRFLOW_TMP_DIR" "$AIRFLOW_LOGS_DIR"; do
|
||||
ensure_dir_exists "$dir"
|
||||
am_i_root && chown "$AIRFLOW_DAEMON_USER:$AIRFLOW_DAEMON_GROUP" "$dir"
|
||||
done
|
||||
|
||||
# Wait for airflow webserver to be available
|
||||
airflow_scheduler_wait_for_webserver "$AIRFLOW_WEBSERVER_HOST" "$AIRFLOW_WEBSERVER_PORT_NUMBER"
|
||||
[[ "$AIRFLOW_EXECUTOR" == "CeleryExecutor" || "$AIRFLOW_EXECUTOR" == "CeleryKubernetesExecutor" ]] && wait-for-port --host "$REDIS_HOST" "$REDIS_PORT_NUMBER"
|
||||
|
||||
# Avoid to fail when the executor is not celery
|
||||
true
|
||||
}
|
||||
|
||||
########################
|
||||
# Generate Airflow Scheduler conf file
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_scheduler_generate_config() {
|
||||
# Generate Airflow default files
|
||||
debug_execute airflow version
|
||||
|
||||
# Configure Airflow Hostname
|
||||
[[ -n "$AIRFLOW_HOSTNAME_CALLABLE" ]] && airflow_conf_set "core" "hostname_callable" "$AIRFLOW_HOSTNAME_CALLABLE"
|
||||
# Configure Airflow to load examples
|
||||
if is_boolean_yes "$AIRFLOW_LOAD_EXAMPLES"; then
|
||||
airflow_conf_set "core" "load_examples" "True"
|
||||
else
|
||||
airflow_conf_set "core" "load_examples" "False"
|
||||
fi
|
||||
|
||||
# Configure Airflow database
|
||||
airflow_configure_database
|
||||
|
||||
# Configure the Webserver port
|
||||
airflow_conf_set "webserver" "web_server_port" "$AIRFLOW_WEBSERVER_PORT_NUMBER"
|
||||
|
||||
# Setup the secret keys for database connection and flask application (fernet key and secret key)
|
||||
# ref: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#fernet-key
|
||||
# ref: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key
|
||||
[[ -n "$AIRFLOW_FERNET_KEY" ]] && airflow_conf_set "core" "fernet_key" "$AIRFLOW_FERNET_KEY"
|
||||
[[ -n "$AIRFLOW_SECRET_KEY" ]] && airflow_conf_set "webserver" "secret_key" "$AIRFLOW_SECRET_KEY"
|
||||
|
||||
# Configure Airflow executor
|
||||
airflow_conf_set "core" "executor" "$AIRFLOW_EXECUTOR"
|
||||
[[ "$AIRFLOW_EXECUTOR" == "CeleryExecutor" || "$AIRFLOW_EXECUTOR" == "CeleryKubernetesExecutor" ]] && airflow_configure_celery_executor
|
||||
true # Avoid the function to fail due to the check above
|
||||
}
|
||||
|
||||
########################
|
||||
# Wait Ariflow webserver
|
||||
# Globals:
|
||||
# AIRFLOW_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
airflow_scheduler_wait_for_webserver() {
|
||||
local -r webserver_host="${1:?missing database host}"
|
||||
local -r webserver_port="${2:?missing database port}"
|
||||
check_webserver_connection() {
|
||||
wait-for-port --host "$webserver_host" "$webserver_port"
|
||||
}
|
||||
if ! retry_while "check_webserver_connection"; then
|
||||
error "Could not connect to the Airflow webserver"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,417 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Bitnami PostgreSQL Client library
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
# Load Generic Libraries
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
. /opt/bitnami/scripts/libos.sh
|
||||
. /opt/bitnami/scripts/libvalidations.sh
|
||||
|
||||
########################
|
||||
# Validate settings in POSTGRESQL_CLIENT_* environment variables
|
||||
# Globals:
|
||||
# POSTGRESQL_CLIENT_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_client_validate() {
|
||||
info "Validating settings in POSTGRESQL_CLIENT_* env vars"
|
||||
local error_code=0
|
||||
|
||||
# Auxiliary functions
|
||||
print_validation_error() {
|
||||
error "$1"
|
||||
error_code=1
|
||||
}
|
||||
|
||||
empty_password_enabled_warn() {
|
||||
warn "You set the environment variable ALLOW_EMPTY_PASSWORD=${ALLOW_EMPTY_PASSWORD}. For safety reasons, do not use this flag in a production environment."
|
||||
}
|
||||
empty_password_error() {
|
||||
print_validation_error "The $1 environment variable is empty or not set. Set the environment variable ALLOW_EMPTY_PASSWORD=yes to allow the container to be started with blank passwords. This is recommended only for development."
|
||||
}
|
||||
|
||||
# Only validate environment variables if any action needs to be performed
|
||||
local -a database_names
|
||||
read -r -a database_names <<< "$(tr ',;' ' ' <<< "$POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES")"
|
||||
if [[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" || "${#database_names[@]}" -gt 0 ]]; then
|
||||
if is_boolean_yes "$ALLOW_EMPTY_PASSWORD"; then
|
||||
empty_password_enabled_warn
|
||||
else
|
||||
if [[ -z "$POSTGRESQL_CLIENT_POSTGRES_PASSWORD" ]]; then
|
||||
empty_password_error "POSTGRESQL_CLIENT_POSTGRES_PASSWORD"
|
||||
fi
|
||||
if [[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" ]] && [[ -z "$POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD" ]]; then
|
||||
empty_password_error "POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
# When enabling extensions, the DB name must be provided
|
||||
local -a extensions
|
||||
read -r -a extensions <<< "$(tr ',;' ' ' <<< "$POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS")"
|
||||
if [[ "${#database_names[@]}" -le 0 && "${#extensions[@]}" -gt 0 ]]; then
|
||||
print_validation_error "POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS requires POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES to be set."
|
||||
fi
|
||||
return "$error_code"
|
||||
}
|
||||
|
||||
########################
|
||||
# Perform actions to a database
|
||||
# Globals:
|
||||
# POSTGRESQL_CLIENT_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_client_initialize() {
|
||||
local -a database_names
|
||||
read -r -a database_names <<< "$(tr ',;' ' ' <<< "$POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES")"
|
||||
# Wait for the database to be accessible if any action needs to be performed
|
||||
if [[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" || "${#database_names[@]}" -gt 0 ]]; then
|
||||
info "Trying to connect to the database server"
|
||||
check_postgresql_connection() {
|
||||
echo "SELECT 1" | postgresql_remote_execute "$POSTGRESQL_CLIENT_DATABASE_HOST" "$POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER" "postgres" "$POSTGRESQL_CLIENT_POSTGRES_USER" "$POSTGRESQL_CLIENT_POSTGRES_PASSWORD"
|
||||
}
|
||||
if ! retry_while "check_postgresql_connection"; then
|
||||
error "Could not connect to the database server"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
# Ensure a database user exists in the server
|
||||
if [[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" ]]; then
|
||||
info "Creating database user ${POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME}"
|
||||
local -a args=("$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" "--host" "$POSTGRESQL_CLIENT_DATABASE_HOST" "--port" "$POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER")
|
||||
[[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD" ]] && args+=("-p" "$POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD")
|
||||
postgresql_ensure_user_exists "${args[@]}"
|
||||
fi
|
||||
# Ensure a database exists in the server (and that the user has write privileges, if specified)
|
||||
if [[ "${#database_names[@]}" -gt 0 ]]; then
|
||||
local -a createdb_args extensions
|
||||
read -r -a extensions <<< "$(tr ',;' ' ' <<< "$POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS")"
|
||||
for database_name in "${database_names[@]}"; do
|
||||
info "Creating database ${database_name}"
|
||||
createdb_args=("$database_name" "--host" "$POSTGRESQL_CLIENT_DATABASE_HOST" "--port" "$POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER")
|
||||
[[ -n "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME" ]] && createdb_args+=("-u" "$POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME")
|
||||
postgresql_ensure_database_exists "${createdb_args[@]}"
|
||||
# Ensure the list of extensions are enabled in the specified database
|
||||
if [[ "${#extensions[@]}" -gt 0 ]]; then
|
||||
for extension_to_create in "${extensions[@]}"; do
|
||||
echo "CREATE EXTENSION IF NOT EXISTS ${extension_to_create}" | postgresql_remote_execute "$POSTGRESQL_CLIENT_DATABASE_HOST" "$POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER" "$database_name" "$POSTGRESQL_CLIENT_POSTGRES_USER" "$POSTGRESQL_CLIENT_POSTGRES_PASSWORD"
|
||||
done
|
||||
fi
|
||||
done
|
||||
fi
|
||||
# Execute a custom SQL script
|
||||
if [[ -n "$POSTGRESQL_CLIENT_EXECUTE_SQL" ]]; then
|
||||
info "Executing custom SQL script"
|
||||
echo "$POSTGRESQL_CLIENT_EXECUTE_SQL" | postgresql_remote_execute "$POSTGRESQL_CLIENT_DATABASE_HOST" "$POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER" "postgres" "$POSTGRESQL_CLIENT_POSTGRES_USER" "$POSTGRESQL_CLIENT_POSTGRES_PASSWORD"
|
||||
fi
|
||||
# Avoid exit code of previous commands to affect the result of this function
|
||||
true
|
||||
}
|
||||
|
||||
########################
|
||||
# Return PostgreSQL major version
|
||||
# Globals:
|
||||
# POSTGRESQL_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# String
|
||||
#########################
|
||||
postgresql_get_major_version() {
|
||||
psql --version | grep -oE "[0-9]+\.[0-9]+" | grep -oE "^[0-9]+"
|
||||
}
|
||||
|
||||
########################
|
||||
# Gets an environment variable name based on the suffix
|
||||
# Arguments:
|
||||
# $1 - environment variable suffix
|
||||
# Returns:
|
||||
# environment variable name
|
||||
#########################
|
||||
get_env_var_value() {
|
||||
local env_var_suffix="${1:?missing suffix}"
|
||||
local env_var_name
|
||||
for env_var_prefix in POSTGRESQL POSTGRESQL_CLIENT; do
|
||||
env_var_name="${env_var_prefix}_${env_var_suffix}"
|
||||
if [[ -n "${!env_var_name:-}" ]]; then
|
||||
echo "${!env_var_name}"
|
||||
break
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
########################
|
||||
# Execute an arbitrary query/queries against the running PostgreSQL service and print the output
|
||||
# Stdin:
|
||||
# Query/queries to execute
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# POSTGRESQL_*
|
||||
# Arguments:
|
||||
# $1 - Database where to run the queries
|
||||
# $2 - User to run queries
|
||||
# $3 - Password
|
||||
# $4 - Extra options (eg. -tA)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_execute_print_output() {
|
||||
local -r db="${1:-}"
|
||||
local -r user="${2:-postgres}"
|
||||
local -r pass="${3:-}"
|
||||
local opts
|
||||
read -r -a opts <<<"${@:4}"
|
||||
|
||||
local args=("-U" "$user" "-p" "${POSTGRESQL_PORT_NUMBER:-5432}")
|
||||
[[ -n "$db" ]] && args+=("-d" "$db")
|
||||
[[ "${#opts[@]}" -gt 0 ]] && args+=("${opts[@]}")
|
||||
|
||||
# Execute the Query/queries from stdin
|
||||
PGPASSWORD=$pass psql "${args[@]}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Execute an arbitrary query/queries against the running PostgreSQL service
|
||||
# Stdin:
|
||||
# Query/queries to execute
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# POSTGRESQL_*
|
||||
# Arguments:
|
||||
# $1 - Database where to run the queries
|
||||
# $2 - User to run queries
|
||||
# $3 - Password
|
||||
# $4 - Extra options (eg. -tA)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_execute() {
|
||||
if [[ "${BITNAMI_DEBUG:-false}" = true ]]; then
|
||||
"postgresql_execute_print_output" "$@"
|
||||
elif [[ "${NO_ERRORS:-false}" = true ]]; then
|
||||
"postgresql_execute_print_output" "$@" 2>/dev/null
|
||||
else
|
||||
"postgresql_execute_print_output" "$@" >/dev/null 2>&1
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Execute an arbitrary query/queries against a remote PostgreSQL service and print to stdout
|
||||
# Stdin:
|
||||
# Query/queries to execute
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# DB_*
|
||||
# Arguments:
|
||||
# $1 - Remote PostgreSQL service hostname
|
||||
# $2 - Remote PostgreSQL service port
|
||||
# $3 - Database where to run the queries
|
||||
# $4 - User to run queries
|
||||
# $5 - Password
|
||||
# $6 - Extra options (eg. -tA)
|
||||
# Returns:
|
||||
# None
|
||||
postgresql_remote_execute_print_output() {
|
||||
local -r hostname="${1:?hostname is required}"
|
||||
local -r port="${2:?port is required}"
|
||||
local -a args=("-h" "$hostname" "-p" "$port")
|
||||
shift 2
|
||||
"postgresql_execute_print_output" "$@" "${args[@]}"
|
||||
}
|
||||
|
||||
########################
|
||||
# Execute an arbitrary query/queries against a remote PostgreSQL service
|
||||
# Stdin:
|
||||
# Query/queries to execute
|
||||
# Globals:
|
||||
# BITNAMI_DEBUG
|
||||
# DB_*
|
||||
# Arguments:
|
||||
# $1 - Remote PostgreSQL service hostname
|
||||
# $2 - Remote PostgreSQL service port
|
||||
# $3 - Database where to run the queries
|
||||
# $4 - User to run queries
|
||||
# $5 - Password
|
||||
# $6 - Extra options (eg. -tA)
|
||||
# Returns:
|
||||
# None
|
||||
postgresql_remote_execute() {
|
||||
if [[ "${BITNAMI_DEBUG:-false}" = true ]]; then
|
||||
"postgresql_remote_execute_print_output" "$@"
|
||||
elif [[ "${NO_ERRORS:-false}" = true ]]; then
|
||||
"postgresql_remote_execute_print_output" "$@" 2>/dev/null
|
||||
else
|
||||
"postgresql_remote_execute_print_output" "$@" >/dev/null 2>&1
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Optionally create the given database user
|
||||
# Flags:
|
||||
# -p|--password - database password
|
||||
# --host - database host
|
||||
# --port - database port
|
||||
# Arguments:
|
||||
# $1 - user
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_ensure_user_exists() {
|
||||
local -r user="${1:?user is missing}"
|
||||
local password=""
|
||||
# For accessing an external database
|
||||
local db_host=""
|
||||
local db_port=""
|
||||
|
||||
# Validate arguments
|
||||
shift 1
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-p | --password)
|
||||
shift
|
||||
password="${1:?missing password}"
|
||||
;;
|
||||
--host)
|
||||
shift
|
||||
db_host="${1:?missing database host}"
|
||||
;;
|
||||
--port)
|
||||
shift
|
||||
db_port="${1:?missing database port}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
local -a postgresql_execute_cmd=("postgresql_execute")
|
||||
[[ -n "$db_host" && -n "$db_port" ]] && postgresql_execute_cmd=("postgresql_remote_execute" "$db_host" "$db_port")
|
||||
local -a postgresql_execute_flags=("postgres" "$(get_env_var_value POSTGRES_USER)" "$(get_env_var_value POSTGRES_PASSWORD)")
|
||||
|
||||
"${postgresql_execute_cmd[@]}" "${postgresql_execute_flags[@]}" <<EOF
|
||||
DO
|
||||
\$do\$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT FROM pg_catalog.pg_roles WHERE rolname = '${user}'
|
||||
) THEN
|
||||
CREATE ROLE "${user}" LOGIN PASSWORD '${password}';
|
||||
END IF;
|
||||
END
|
||||
\$do\$;
|
||||
EOF
|
||||
}
|
||||
|
||||
########################
|
||||
# Ensure a user has all privileges to access a database
|
||||
# Arguments:
|
||||
# $1 - database name
|
||||
# $2 - database user
|
||||
# $3 - database host (optional)
|
||||
# $4 - database port (optional)
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_ensure_user_has_database_privileges() {
|
||||
local -r user="${1:?user is required}"
|
||||
local -r database="${2:?db is required}"
|
||||
local -r db_host="${3:-}"
|
||||
local -r db_port="${4:-}"
|
||||
|
||||
local -a postgresql_execute_cmd=("postgresql_execute")
|
||||
[[ -n "$db_host" && -n "$db_port" ]] && postgresql_execute_cmd=("postgresql_remote_execute" "$db_host" "$db_port")
|
||||
local -a postgresql_execute_flags=("postgres" "$(get_env_var_value POSTGRES_USER)" "$(get_env_var_value POSTGRES_PASSWORD)")
|
||||
|
||||
debug "Providing privileges to username ${user} on database ${database}"
|
||||
"${postgresql_execute_cmd[@]}" "${postgresql_execute_flags[@]}" <<EOF
|
||||
GRANT ALL PRIVILEGES ON DATABASE "${database}" TO "${user}";
|
||||
ALTER DATABASE "${database}" OWNER TO "${user}";
|
||||
EOF
|
||||
}
|
||||
|
||||
########################
|
||||
# Optionally create the given database, and then optionally give a user
|
||||
# full privileges on the database.
|
||||
# Flags:
|
||||
# -u|--user - database user
|
||||
# --host - database host
|
||||
# --port - database port
|
||||
# Arguments:
|
||||
# $1 - database name
|
||||
# Returns:
|
||||
# None
|
||||
#########################
|
||||
postgresql_ensure_database_exists() {
|
||||
local -r database="${1:?database is missing}"
|
||||
local user=""
|
||||
# For accessing an external database
|
||||
local db_host=""
|
||||
local db_port=""
|
||||
|
||||
# Validate arguments
|
||||
shift 1
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
-u | --user)
|
||||
shift
|
||||
user="${1:?missing database user}"
|
||||
;;
|
||||
--host)
|
||||
shift
|
||||
db_host="${1:?missing database host}"
|
||||
;;
|
||||
--port)
|
||||
shift
|
||||
db_port="${1:?missing database port}"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid command line flag $1" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
local -a postgresql_execute_cmd=("postgresql_execute")
|
||||
[[ -n "$db_host" && -n "$db_port" ]] && postgresql_execute_cmd=("postgresql_remote_execute" "$db_host" "$db_port")
|
||||
local -a postgresql_execute_flags=("postgres" "$(get_env_var_value POSTGRES_USER)" "$(get_env_var_value POSTGRES_PASSWORD)")
|
||||
|
||||
"${postgresql_execute_cmd[@]}" "${postgresql_execute_flags[@]}" <<EOF
|
||||
SELECT 'CREATE DATABASE "${database}"'
|
||||
WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = '${database}')\gexec
|
||||
EOF
|
||||
if [[ -n "$user" ]]; then
|
||||
local -a grant_flags=("$user" "$database")
|
||||
[[ -n "$db_host" ]] && grant_flags+=("$db_host")
|
||||
[[ -n "$db_port" ]] && grant_flags+=("$db_port")
|
||||
postgresql_ensure_user_has_database_privileges "${grant_flags[@]}"
|
||||
fi
|
||||
}
|
||||
|
||||
########################
|
||||
# Retrieves the WAL directory in use by PostgreSQL / to use if not initialized yet
|
||||
# Globals:
|
||||
# REPMGR_*
|
||||
# Arguments:
|
||||
# None
|
||||
# Returns:
|
||||
# the path to the WAL directory, or empty if not set
|
||||
#########################
|
||||
postgresql_get_waldir() {
|
||||
if [[ -L "${POSTGRESQL_DATA_DIR}/pg_wal" && -d "${POSTGRESQL_DATA_DIR}/pg_wal" ]]; then
|
||||
readlink -f "${POSTGRESQL_DATA_DIR}/pg_wal"
|
||||
else
|
||||
# Uninitialized - using value from $POSTGRESQL_INITDB_WAL_DIR if set
|
||||
echo "$POSTGRESQL_INITDB_WAL_DIR"
|
||||
fi
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Environment configuration for postgresql-client
|
||||
|
||||
# The values for all environment variables will be set in the below order of precedence
|
||||
# 1. Custom environment variables defined below after Bitnami defaults
|
||||
# 2. Constants defined in this file (environment variables with no default), i.e. BITNAMI_ROOT_DIR
|
||||
# 3. Environment variables overridden via external files using *_FILE variables (see below)
|
||||
# 4. Environment variables set externally (i.e. current Bash context/Dockerfile/userdata)
|
||||
|
||||
# Load logging library
|
||||
# shellcheck disable=SC1090,SC1091
|
||||
. /opt/bitnami/scripts/liblog.sh
|
||||
|
||||
export BITNAMI_ROOT_DIR="/opt/bitnami"
|
||||
export BITNAMI_VOLUME_DIR="/bitnami"
|
||||
|
||||
# Logging configuration
|
||||
export MODULE="${MODULE:-postgresql-client}"
|
||||
export BITNAMI_DEBUG="${BITNAMI_DEBUG:-false}"
|
||||
|
||||
# By setting an environment variable matching *_FILE to a file path, the prefixed environment
|
||||
# variable will be overridden with the value specified in that file
|
||||
postgresql_client_env_vars=(
|
||||
ALLOW_EMPTY_PASSWORD
|
||||
POSTGRESQL_CLIENT_DATABASE_HOST
|
||||
POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER
|
||||
POSTGRESQL_CLIENT_POSTGRES_USER
|
||||
POSTGRESQL_CLIENT_POSTGRES_PASSWORD
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS
|
||||
POSTGRESQL_CLIENT_EXECUTE_SQL
|
||||
POSTGRESQL_HOST
|
||||
POSTGRESQL_PORT_NUMBER
|
||||
POSTGRESQL_CLIENT_ROOT_USER
|
||||
POSTGRESQL_POSTGRES_USER
|
||||
POSTGRESQL_ROOT_USER
|
||||
POSTGRESQL_CLIENT_ROOT_PASSWORD
|
||||
POSTGRESQL_POSTGRES_PASSWORD
|
||||
POSTGRESQL_ROOT_PASSWORD
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_NAME
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_USER
|
||||
)
|
||||
for env_var in "${postgresql_client_env_vars[@]}"; do
|
||||
file_env_var="${env_var}_FILE"
|
||||
if [[ -n "${!file_env_var:-}" ]]; then
|
||||
if [[ -r "${!file_env_var:-}" ]]; then
|
||||
export "${env_var}=$(< "${!file_env_var}")"
|
||||
unset "${file_env_var}"
|
||||
else
|
||||
warn "Skipping export of '${env_var}'. '${!file_env_var:-}' is not readable."
|
||||
fi
|
||||
fi
|
||||
done
|
||||
unset postgresql_client_env_vars
|
||||
|
||||
# Paths
|
||||
export POSTGRESQL_BASE_DIR="/opt/bitnami/postgresql"
|
||||
export POSTGRESQL_BIN_DIR="$POSTGRESQL_BASE_DIR/bin"
|
||||
export PATH="${POSTGRESQL_BIN_DIR}:${PATH}"
|
||||
|
||||
# PostgreSQL settings
|
||||
export ALLOW_EMPTY_PASSWORD="${ALLOW_EMPTY_PASSWORD:-no}"
|
||||
POSTGRESQL_CLIENT_DATABASE_HOST="${POSTGRESQL_CLIENT_DATABASE_HOST:-"${POSTGRESQL_HOST:-}"}"
|
||||
export POSTGRESQL_CLIENT_DATABASE_HOST="${POSTGRESQL_CLIENT_DATABASE_HOST:-postgresql}"
|
||||
POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER="${POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER:-"${POSTGRESQL_PORT_NUMBER:-}"}"
|
||||
export POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER="${POSTGRESQL_CLIENT_DATABASE_PORT_NUMBER:-5432}"
|
||||
POSTGRESQL_CLIENT_POSTGRES_USER="${POSTGRESQL_CLIENT_POSTGRES_USER:-"${POSTGRESQL_CLIENT_ROOT_USER:-}"}"
|
||||
POSTGRESQL_CLIENT_POSTGRES_USER="${POSTGRESQL_CLIENT_POSTGRES_USER:-"${POSTGRESQL_POSTGRES_USER:-}"}"
|
||||
POSTGRESQL_CLIENT_POSTGRES_USER="${POSTGRESQL_CLIENT_POSTGRES_USER:-"${POSTGRESQL_ROOT_USER:-}"}"
|
||||
export POSTGRESQL_CLIENT_POSTGRES_USER="${POSTGRESQL_CLIENT_POSTGRES_USER:-postgres}" # only used during the first initialization
|
||||
POSTGRESQL_CLIENT_POSTGRES_PASSWORD="${POSTGRESQL_CLIENT_POSTGRES_PASSWORD:-"${POSTGRESQL_CLIENT_ROOT_PASSWORD:-}"}"
|
||||
POSTGRESQL_CLIENT_POSTGRES_PASSWORD="${POSTGRESQL_CLIENT_POSTGRES_PASSWORD:-"${POSTGRESQL_POSTGRES_PASSWORD:-}"}"
|
||||
POSTGRESQL_CLIENT_POSTGRES_PASSWORD="${POSTGRESQL_CLIENT_POSTGRES_PASSWORD:-"${POSTGRESQL_ROOT_PASSWORD:-}"}"
|
||||
export POSTGRESQL_CLIENT_POSTGRES_PASSWORD="${POSTGRESQL_CLIENT_POSTGRES_PASSWORD:-}" # only used during the first initialization
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES="${POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES:-"${POSTGRESQL_CLIENT_CREATE_DATABASE_NAME:-}"}"
|
||||
export POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES="${POSTGRESQL_CLIENT_CREATE_DATABASE_NAMES:-}" # only used during the first initialization
|
||||
POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME="${POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME:-"${POSTGRESQL_CLIENT_CREATE_DATABASE_USER:-}"}"
|
||||
export POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME="${POSTGRESQL_CLIENT_CREATE_DATABASE_USERNAME:-}" # only used during the first initialization
|
||||
export POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD="${POSTGRESQL_CLIENT_CREATE_DATABASE_PASSWORD:-}" # only used during the first initialization
|
||||
export POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS="${POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS:-}" # only used during the first initialization
|
||||
export POSTGRESQL_CLIENT_EXECUTE_SQL="${POSTGRESQL_CLIENT_EXECUTE_SQL:-}" # only used during the first initialization
|
||||
|
||||
# Custom environment variables may be defined below
|
||||
@@ -0,0 +1,19 @@
|
||||
#!/bin/bash
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
# set -o xtrace # Uncomment this line for debugging purposes
|
||||
|
||||
# Load libraries
|
||||
. /opt/bitnami/scripts/libpostgresqlclient.sh
|
||||
|
||||
# Load PostgreSQL Client environment variables
|
||||
. /opt/bitnami/scripts/postgresql-client-env.sh
|
||||
|
||||
# Ensure PostgreSQL Client environment variables settings are valid
|
||||
postgresql_client_validate
|
||||
# Ensure PostgreSQL Client is initialized
|
||||
postgresql_client_initialize
|
||||
17
bitnami/airflow-scheduler/CODE_OF_CONDUCT.md
Normal file
17
bitnami/airflow-scheduler/CODE_OF_CONDUCT.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Contributor Code of Conduct
|
||||
|
||||
As contributors and maintainers of this project, we pledge to respect everyone who contributes by posting issues, updating documentation, submitting pull requests, providing feedback in comments, and any other activities.
|
||||
|
||||
Communication through any of Bitnami's channels (GitHub, mailing lists, Twitter, and so on) must be constructive and never resort to personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
|
||||
|
||||
We promise to extend courtesy and respect to everyone involved in this project, regardless of gender, gender identity, sexual orientation, disability, age, race, ethnicity, religion, or level of experience. We expect anyone contributing to this project to do the same.
|
||||
|
||||
If any member of the community violates this code of conduct, the maintainers of this project may take action, including removing issues, comments, and PRs or blocking accounts, as deemed appropriate.
|
||||
|
||||
If you are subjected to or witness unacceptable behavior, or have any other concerns, please communicate with us.
|
||||
|
||||
If you have suggestions to improve this Code of Conduct, please submit an issue or PR.
|
||||
|
||||
**Attribution**
|
||||
|
||||
This Code of Conduct is adapted from the Angular project available at this page: https://github.com/angular/code-of-conduct/blob/master/CODE_OF_CONDUCT.md
|
||||
52
bitnami/airflow-scheduler/CONTRIBUTING.md
Normal file
52
bitnami/airflow-scheduler/CONTRIBUTING.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Contributing Guidelines
|
||||
|
||||
Contributions are welcome via GitHub Pull Requests. This document outlines the process to help get your contribution accepted.
|
||||
|
||||
Any type of contribution is welcome: new features, bug fixes, documentation improvements, etc.
|
||||
|
||||
## How to Contribute
|
||||
|
||||
1. Fork this repository, develop, and test your changes.
|
||||
2. Submit a pull request.
|
||||
|
||||
### Requirements
|
||||
|
||||
When submitting a PR make sure that:
|
||||
- It must pass CI jobs for linting and test the changes (if any).
|
||||
- It must follow [container best practices](https://engineering.bitnami.com/articles/best-practices-writing-a-dockerfile.html).
|
||||
- The title of the PR is clear enough.
|
||||
- If necessary, add information to the repository's `README.md`.
|
||||
|
||||
#### Sign Your Work
|
||||
|
||||
The sign-off is a simple line at the end of the explanation for a commit. All commits needs to be signed. Your signature certifies that you wrote the patch or otherwise have the right to contribute the material. The rules are pretty simple, you only need to certify the guidelines from [developercertificate.org](https://developercertificate.org/).
|
||||
|
||||
Then you just add a line to every git commit message:
|
||||
|
||||
Signed-off-by: Joe Smith <joe.smith@example.com>
|
||||
|
||||
Use your real name (sorry, no pseudonyms or anonymous contributions.)
|
||||
|
||||
If you set your `user.name` and `user.email` git configs, you can sign your commit automatically with `git commit -s`.
|
||||
|
||||
Note: If your git config information is set properly then viewing the `git log` information for your commit will look something like this:
|
||||
|
||||
```
|
||||
Author: Joe Smith <joe.smith@example.com>
|
||||
Date: Thu Feb 2 11:41:15 2018 -0800
|
||||
|
||||
Update README
|
||||
|
||||
Signed-off-by: Joe Smith <joe.smith@example.com>
|
||||
```
|
||||
|
||||
Notice the `Author` and `Signed-off-by` lines match. If they don't your PR will be rejected by the automated DCO check.
|
||||
|
||||
### PR Approval and Release Process
|
||||
|
||||
1. Changes are manually reviewed by Bitnami team members usually within a business day.
|
||||
2. Once the changes are accepted, the PR is tested (if needed) into the Bitnami CI pipeline, the container is deployed and tested (verification and functional tests) using docker-compose and Helm (if there is an associated Helm Chart).
|
||||
3. The PR is merged by the reviewer(s) in the GitHub `master` branch.
|
||||
4. Then our CI/CD system is going to push the container image to the different registries including the recently merged changes.
|
||||
|
||||
***NOTE***: Please note that, in terms of time, may be a slight difference between the appearance of the code in GitHub and the image with the changes in the different registries.
|
||||
13
bitnami/airflow-scheduler/LICENSE.md
Normal file
13
bitnami/airflow-scheduler/LICENSE.md
Normal file
@@ -0,0 +1,13 @@
|
||||
Copyright © 2022 Bitnami
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
395
bitnami/airflow-scheduler/README.md
Normal file
395
bitnami/airflow-scheduler/README.md
Normal file
@@ -0,0 +1,395 @@
|
||||
# Apache Airflow Scheduler packaged by Bitnami
|
||||
|
||||
## What is Apache Airflow Scheduler?
|
||||
|
||||
> Apache Airflow is a tool to express and execute workflows as directed acyclic graphs (DAGs). The Airflow scheduler triggers tasks and provides tools to monitor task progress.
|
||||
|
||||
[Overview of Apache Airflow Scheduler](https://airflow.apache.org/)
|
||||
|
||||
Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.
|
||||
|
||||
## TL;DR
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```console
|
||||
$ curl -LO https://raw.githubusercontent.com/bitnami/bitnami-docker-airflow-scheduler/master/docker-compose.yml
|
||||
$ docker-compose up
|
||||
```
|
||||
|
||||
You can find the default credentials and available configuration options in the [Environment Variables](#environment-variables) section.
|
||||
|
||||
## Why use Bitnami Images?
|
||||
|
||||
* Bitnami closely tracks upstream source changes and promptly publishes new versions of this image using our automated systems.
|
||||
* With Bitnami images the latest bug fixes and features are available as soon as possible.
|
||||
* Bitnami containers, virtual machines and cloud images use the same components and configuration approach - making it easy to switch between formats based on your project needs.
|
||||
* All our images are based on [minideb](https://github.com/bitnami/minideb) a minimalist Debian based container image which gives you a small base container image and the familiarity of a leading Linux distribution.
|
||||
* All Bitnami images available in Docker Hub are signed with [Docker Content Trust (DCT)](https://docs.docker.com/engine/security/trust/content_trust/). You can use `DOCKER_CONTENT_TRUST=1` to verify the integrity of the images.
|
||||
* Bitnami container images are released daily with the latest distribution packages available.
|
||||
|
||||
|
||||
> This [CVE scan report](https://quay.io/repository/bitnami/airflow-scheduler?tab=tags) contains a security report with all open CVEs. To get the list of actionable security issues, find the "latest" tag, click the vulnerability report link under the corresponding "Security scan" field and then select the "Only show fixable" filter on the next page.
|
||||
|
||||
|
||||
## Supported tags and respective `Dockerfile` links
|
||||
|
||||
Learn more about the Bitnami tagging policy and the difference between rolling tags and immutable tags [in our documentation page](https://docs.bitnami.com/tutorials/understand-rolling-tags-containers/).
|
||||
|
||||
|
||||
* [`2`, `2-debian-10`, `2.2.5`, `2.2.5-debian-10-r2`, `latest` (2/debian-10/Dockerfile)](https://github.com/bitnami/bitnami-docker-airflow-scheduler/blob/2.2.5-debian-10-r2/2/debian-10/Dockerfile)
|
||||
|
||||
Subscribe to project updates by watching the [bitnami/airflow GitHub repo](https://github.com/bitnami/bitnami-docker-airflow-scheduler).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
To run this application you need [Docker Engine](https://www.docker.com/products/docker-engine) >= `1.10.0`. [Docker Compose](https://docs.docker.com/compose/) is recommended with a version `1.6.0` or later.
|
||||
|
||||
## How to use this image
|
||||
|
||||
Apache Airflow Scheduler is a component of an Airflow solution configuring with the `CeleryExecutor`. Hence, you will need to rest of Airflow components for this image to work.
|
||||
You will need an [Airflow Webserver](https://www.github.com/bitnami/bitnami-docker-airflow), one or more [Airflow Workers](https://www.github.com/bitnami/bitnami-docker-airflow-worker), a [PostgreSQL database](https://www.github.com/bitnami/bitnami-docker-postgresql) and a [Redis(TM) server](https://www.github.com/bitnami/bitnami-docker-redis).
|
||||
|
||||
### Using Docker Compose
|
||||
|
||||
The main folder of this repository contains a functional [`docker-compose.yml`](https://github.com/bitnami/bitnami-docker-airflow-scheduler/blob/master/docker-compose.yml) file. Run the application using it as shown below:
|
||||
|
||||
```console
|
||||
$ curl -sSL https://raw.githubusercontent.com/bitnami/bitnami-docker-airflow-scheduler/master/docker-compose.yml > docker-compose.yml
|
||||
$ docker-compose up -d
|
||||
```
|
||||
|
||||
### Using the Docker Command Line
|
||||
|
||||
If you want to run the application manually instead of using `docker-compose`, these are the basic steps you need to run:
|
||||
|
||||
1. Create a network
|
||||
|
||||
```console
|
||||
$ docker network create airflow-tier
|
||||
```
|
||||
|
||||
2. Create a volume for PostgreSQL persistence and create a PostgreSQL container
|
||||
|
||||
```console
|
||||
$ docker volume create --name postgresql_data
|
||||
$ docker run -d --name postgresql \
|
||||
-e POSTGRESQL_USERNAME=bn_airflow \
|
||||
-e POSTGRESQL_PASSWORD=bitnami1 \
|
||||
-e POSTGRESQL_DATABASE=bitnami_airflow \
|
||||
--net airflow-tier \
|
||||
--volume postgresql_data:/bitnami/postgresql \
|
||||
bitnami/postgresql:latest
|
||||
```
|
||||
|
||||
3. Create a volume for Redis(TM) persistence and create a Redis(TM) container
|
||||
|
||||
```console
|
||||
$ docker volume create --name redis_data
|
||||
$ docker run -d --name redis \
|
||||
-e ALLOW_EMPTY_PASSWORD=yes \
|
||||
--net airflow-tier \
|
||||
--volume redis_data:/bitnami \
|
||||
bitnami/redis:latest
|
||||
```
|
||||
|
||||
4. Launch the Apache Airflow Scheduler web container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow -p 8080:8080 \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
-e AIRFLOW_LOAD_EXAMPLES=yes \
|
||||
-e AIRFLOW_PASSWORD=bitnami123 \
|
||||
-e AIRFLOW_USERNAME=user \
|
||||
-e AIRFLOW_EMAIL=user@example.com \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow:latest
|
||||
```
|
||||
|
||||
5. Launch the Apache Airflow Scheduler scheduler container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow-scheduler \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
-e AIRFLOW_LOAD_EXAMPLES=yes \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow-scheduler:latest
|
||||
```
|
||||
|
||||
6. Launch the Apache Airflow Scheduler worker container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow-worker \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow-worker:latest
|
||||
```
|
||||
|
||||
Access your application at `http://your-ip:8080`
|
||||
|
||||
### Persisting your application
|
||||
|
||||
The Bitnami Airflow container relies on the PostgreSQL database & Redis to persist the data. This means that Airflow does not persist anything. To avoid loss of data, you should mount volumes for persistence of [PostgreSQL data](https://github.com/bitnami/bitnami-docker-mariadb#persisting-your-database) and [Redis(TM) data](https://github.com/bitnami/bitnami-docker-redis#persisting-your-database)
|
||||
|
||||
The above examples define docker volumes namely `postgresql_data`, and `redis_data`. The Airflow application state will persist as long as these volumes are not removed.
|
||||
|
||||
To avoid inadvertent removal of these volumes you can [mount host directories as data volumes](https://docs.docker.com/engine/tutorials/dockervolumes/). Alternatively you can make use of volume plugins to host the volume data.
|
||||
|
||||
#### Mount host directories as data volumes with Docker Compose
|
||||
|
||||
The following `docker-compose.yml` template demonstrates the use of host directories as data volumes.
|
||||
|
||||
```yaml
|
||||
version: '2'
|
||||
services:
|
||||
postgresql:
|
||||
image: 'bitnami/postgresql:latest'
|
||||
environment:
|
||||
- POSTGRESQL_DATABASE=bitnami_airflow
|
||||
- POSTGRESQL_USERNAME=bn_airflow
|
||||
- POSTGRESQL_PASSWORD=bitnami1
|
||||
volumes:
|
||||
- /path/to/postgresql-persistence:/bitnami
|
||||
redis:
|
||||
image: 'bitnami/redis:latest'
|
||||
environment:
|
||||
- ALLOW_EMPTY_PASSWORD=yes
|
||||
volumes:
|
||||
- /path/to/redis-persistence:/bitnami
|
||||
airflow-worker:
|
||||
image: bitnami/airflow-worker:latest
|
||||
environment:
|
||||
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
|
||||
- AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08=
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_LOAD_EXAMPLES=yes
|
||||
airflow-scheduler:
|
||||
image: bitnami/airflow-scheduler:latest
|
||||
environment:
|
||||
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
|
||||
- AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08=
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_LOAD_EXAMPLES=yes
|
||||
airflow:
|
||||
image: bitnami/airflow:latest
|
||||
environment:
|
||||
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
|
||||
- AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08=
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_PASSWORD=bitnami123
|
||||
- AIRFLOW_USERNAME=user
|
||||
- AIRFLOW_EMAIL=user@example.com
|
||||
ports:
|
||||
- '8080:8080'
|
||||
```
|
||||
|
||||
#### Mount host directories as data volumes using the Docker command line
|
||||
|
||||
1. Create a network (if it does not exist)
|
||||
|
||||
```console
|
||||
$ docker network create airflow-tier
|
||||
```
|
||||
|
||||
2. Create the PostgreSQL container with host volumes
|
||||
|
||||
```console
|
||||
$ docker run -d --name postgresql \
|
||||
-e POSTGRESQL_USERNAME=bn_airflow \
|
||||
-e POSTGRESQL_PASSWORD=bitnami1 \
|
||||
-e POSTGRESQL_DATABASE=bitnami_airflow \
|
||||
--net airflow-tier \
|
||||
--volume /path/to/postgresql-persistence:/bitnami \
|
||||
bitnami/postgresql:latest
|
||||
```
|
||||
|
||||
3. Create the Redis(TM) container with host volumes
|
||||
|
||||
```console
|
||||
$ docker run -d --name redis \
|
||||
-e ALLOW_EMPTY_PASSWORD=yes \
|
||||
--net airflow-tier \
|
||||
--volume /path/to/redis-persistence:/bitnami \
|
||||
bitnami/redis:latest
|
||||
```
|
||||
|
||||
4. Create the Airflow container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow -p 8080:8080 \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
-e AIRFLOW_LOAD_EXAMPLES=yes \
|
||||
-e AIRFLOW_PASSWORD=bitnami123 \
|
||||
-e AIRFLOW_USERNAME=user \
|
||||
-e AIRFLOW_EMAIL=user@example.com \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow:latest
|
||||
```
|
||||
|
||||
5. Create the Apache Airflow Scheduler container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow-scheduler \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
-e AIRFLOW_LOAD_EXAMPLES=yes \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow-scheduler:latest
|
||||
```
|
||||
|
||||
6. Create the Airflow Worker container
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow-worker \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
--net airflow-tier \
|
||||
bitnami/airflow-worker:latest
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Installing additional python modules
|
||||
|
||||
This container supports the installation of additional python modules at start-up time. In order to do that, you can mount a `requirements.txt` file with your specific needs under the path `/bitnami/python/requirements.txt`.
|
||||
|
||||
### Environment variables
|
||||
|
||||
The Apache Airflow Scheduler instance can be customized by specifying environment variables on the first run. The following environment values are provided to customize Apache Airflow Scheduler:
|
||||
|
||||
###### Apache Airflow Scheduler configuration
|
||||
|
||||
- `AIRFLOW_EXECUTOR`: Apache Airflow Scheduler executor. Default: **SequentialExecutor**
|
||||
- `AIRFLOW_FERNET_KEY`: Apache Airflow Scheduler Fernet key. No defaults.
|
||||
- `AIRFLOW_SECRET_KEY`: Apache Airflow Scheduler Secret key. No defaults.
|
||||
- `AIRFLOW_WEBSERVER_HOST`: Apache Airflow Scheduler webserver host. Default: **airflow**
|
||||
- `AIRFLOW_WEBSERVER_PORT_NUMBER`: Apache Airflow Scheduler webserver port. Default: **8080**
|
||||
- `AIRFLOW_LOAD_EXAMPLES`: To load example tasks into the application. Default: **yes**
|
||||
- `AIRFLOW_HOSTNAME_CALLABLE`: Method to obtain the hostname. No defaults.
|
||||
|
||||
###### Use an existing database
|
||||
|
||||
- `AIRFLOW_DATABASE_HOST`: Hostname for PostgreSQL server. Default: **postgresql**
|
||||
- `AIRFLOW_DATABASE_PORT_NUMBER`: Port used by PostgreSQL server. Default: **5432**
|
||||
- `AIRFLOW_DATABASE_NAME`: Database name that Apache Airflow Scheduler will use to connect with the database. Default: **bitnami_airflow**
|
||||
- `AIRFLOW_DATABASE_USERNAME`: Database user that Apache Airflow Scheduler will use to connect with the database. Default: **bn_airflow**
|
||||
- `AIRFLOW_DATABASE_PASSWORD`: Database password that Apache Airflow Scheduler will use to connect with the database. No defaults.
|
||||
- `AIRFLOW_DATABASE_USE_SSL`: Set to yes if the database uses SSL. Default: **no**
|
||||
- `AIRFLOW_REDIS_USE_SSL`: Set to yes if Redis(TM) uses SSL. Default: **no**
|
||||
- `REDIS_HOST`: Hostname for Redis(TM) server. Default: **redis**
|
||||
- `REDIS_PORT_NUMBER`: Port used by Redis(TM) server. Default: **6379**
|
||||
- `REDIS_USER`: USER that Apache Airflow Scheduler will use to connect with Redis(TM). No defaults.
|
||||
- `REDIS_PASSWORD`: Password that Apache Airflow Scheduler will use to connect with Redis(TM). No defaults.
|
||||
|
||||
> In addition to the previous environment variables, all the parameters from the configuration file can be overwritten by using environment variables with this format: `AIRFLOW__{SECTION}__{KEY}`. Note the double underscores.
|
||||
|
||||
#### Specifying Environment variables using Docker Compose
|
||||
|
||||
```yaml
|
||||
version: '2'
|
||||
|
||||
services:
|
||||
airflow:
|
||||
image: bitnami/airflow:1
|
||||
environment:
|
||||
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
|
||||
- AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08=
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_PASSWORD=bitnami123
|
||||
- AIRFLOW_USERNAME=user
|
||||
- AIRFLOW_EMAIL=user@example.com
|
||||
```
|
||||
|
||||
#### Specifying Environment variables on the Docker command line
|
||||
|
||||
```console
|
||||
$ docker run -d --name airflow -p 8080:8080 \
|
||||
-e AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho= \
|
||||
-e AIRFLOW_SECRET_KEY=a25mQ1FHTUh3MnFRSk5KMEIyVVU2YmN0VGRyYTVXY08= \
|
||||
-e AIRFLOW_EXECUTOR=CeleryExecutor \
|
||||
-e AIRFLOW_DATABASE_NAME=bitnami_airflow \
|
||||
-e AIRFLOW_DATABASE_USERNAME=bn_airflow \
|
||||
-e AIRFLOW_DATABASE_PASSWORD=bitnami1 \
|
||||
-e AIRFLOW_PASSWORD=bitnami123 \
|
||||
-e AIRFLOW_USERNAME=user \
|
||||
-e AIRFLOW_EMAIL=user@example.com \
|
||||
--volume airflow_data:/bitnami \
|
||||
bitnami/airflow:latest
|
||||
```
|
||||
|
||||
## Notable Changes
|
||||
|
||||
### 1.10.15-debian-10-r18 and 2.0.1-debian-10-r52
|
||||
|
||||
- The size of the container image has been decreased.
|
||||
- The configuration logic is now based on Bash scripts in the *rootfs/* folder.
|
||||
|
||||
## Contributing
|
||||
|
||||
We'd love for you to contribute to this container. You can request new features by creating an [issue](https://github.com/bitnami/bitnami-docker-airflow-scheduler/issues), or submit a [pull request](https://github.com/bitnami/bitnami-docker-airflow-scheduler/pulls) with your contribution.
|
||||
|
||||
## Issues
|
||||
|
||||
If you encountered a problem running this container, you can file an [issue](https://github.com/bitnami/bitnami-docker-airflow-scheduler/issues/new). For us to provide better support, be sure to include the following information in your issue:
|
||||
|
||||
- Host OS and version
|
||||
- Docker version (`$ docker version`)
|
||||
- Output of `$ docker info`
|
||||
- Version of this container
|
||||
- The command you used to run the container, and any relevant output you saw (masking any sensitive information)
|
||||
|
||||
## License
|
||||
|
||||
Copyright © 2022 Bitnami
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
<http://www.apache.org/licenses/LICENSE-2.0>
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
15
bitnami/airflow-scheduler/docker-compose.yml
Normal file
15
bitnami/airflow-scheduler/docker-compose.yml
Normal file
@@ -0,0 +1,15 @@
|
||||
version: '2'
|
||||
services:
|
||||
airflow-scheduler:
|
||||
image: docker.io/bitnami/airflow-scheduler:2
|
||||
environment:
|
||||
- AIRFLOW_DATABASE_NAME=bitnami_airflow
|
||||
- AIRFLOW_DATABASE_USERNAME=bn_airflow
|
||||
- AIRFLOW_DATABASE_PASSWORD=bitnami1
|
||||
- AIRFLOW_EXECUTOR=CeleryExecutor
|
||||
volumes:
|
||||
- airflow_scheduler_data:/bitnami
|
||||
command: ["tail", "-f", "/dev/null"]
|
||||
volumes:
|
||||
airflow_scheduler_data:
|
||||
driver: local
|
||||
Reference in New Issue
Block a user