Building Non Root Docker Images OpenShift
- Bummer: You can rarely use an existing Dockerhub image and run it on OpenShift out the box.
- Although with good intentions, this is a massive blow to developer experience coming from standard Kubernetes which is probably hindering adoption of OpenShift in the wider community
- Openshift ignores the USER directive of the Dockerfile and launches the container with a random UUID. (src)
For example, you cannot simply oc new-app jetty:jre8-alpine from dockerhub. Even though this is the official Docker jetty image with > 10 million happy pulls, it won't run out the box on OpenShift. Instead, you will see the error:
/docker-entrypoint.sh: line 91: can't create /var/lib/jetty/jetty.start: Permission denied
You can't blindly change the USER to a number like 1001 for example, as Openshift randomnly assigns a user is (this creates more problems than it fixes). You'll see things like:
whoami: unknown uid 1000140000
Which makes sense, given the user dosn't exist, and you must use a number rather than a username for this else OpenShift has no way of determining if it is a non-root user or not (according to Openshift Docs).
Thanks to the Patroni project (see excellent talks on Postgres on Kubernetes on Youtube by Oleksii Kliukin a major contributor for background) we can see how this project addresses this challange.
We (usually) must re-create every Dockerhub image
This is why bitnami makes a big point about it with their hard work and effort to create openshift specific images. Kudos to that effort. But what about images not rebuilt by bitnami?
- Usually it means rewriting or creating an entrypoint.sh file which (most) official docker hub images provide
- The key point to remember here is that entrypoint refers to the runtime of your docker image, you need to handle the random user id which openshift assigns you here, it can't be done within your Dockerfile context because that's only for when the image is being built, not ran
Working example
Here's an example of jetting vanilla Jetty to run as non-root in a Docker container. Unfortunetly, we can't simply use the official docker hub jetty image as it begins as root by default (even though it eventually drops to non-root, openshift will block this too early). So instead, we must write our own conainter which doesn't start as root.
For the open bank project, we acheived this as follows for our Dockerfile to run an non-root and run vanilla jetty:
Full source: https://github.com/chrisjsimpson/obp-kubernetes/blob/83ea61ddfcab1198fe16fe194b7923a1c02afdb7/Dockerfile
# This creates a jetty jre8 image containing obp-api-1.0.war.
# It is a multi stage build, meaning a small-ish image is the end result.
FROM alpine:latest as repo
# Get repo fron github, store as stage 'repo'
RUN apk add --no-cache git
WORKDIR OBP-API
RUN git clone https://github.com/OpenBankProject/OBP-API.git
FROM maven:3-jdk-8 as maven
# Build the source using maven, source is copied from the 'repo' build.
COPY --from=repo /OBP-API /usr/src
RUN cp /usr/src/OBP-API/obp-api/pom.xml /tmp/pom.xml # For Packaging a local repository within the image
WORKDIR /usr/src/OBP-API
RUN cp obp-api/src/main/resources/props/sample.props.template obp-api/src/main/resources/props/default.props
RUN cp obp-api/src/main/resources/props/test.default.props.template obp-api/src/main/resources/props/test.default.props
RUN mvn install -pl .,obp-commons
RUN mvn install -DskipTests -pl obp-api
FROM openjdk:8-jre-alpine
# Add user
RUN adduser -D obp
# Download jetty
RUN wget https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.15.v20190215/jetty-distribution-9.4.15.v20190215.tar.gz
RUN tar xvf jetty-distribution-9.4.15.v20190215.tar.gz
# Copy OBP source code
# Copy build artifact (.war file) into jetty from 'maven' stage.
COPY --from=maven /usr/src/OBP-API/obp-api/target/obp-api-1.0.war jetty-distribution-9.4.15.v20190215/webapps/ROOT.war
WORKDIR jetty-distribution-9.4.15.v20190215/
# Switch to the obp user (non root)
USER obp
# Starts jetty
ENTRYPOINT ["java", "-jar", "start.jar"]