Last month our team decided to lock down the cluster. Security compliance required readOnlyRootFilesystem: true on every pod. Standard stuff — scan passes green, everyone’s happy.
Then our Spring Boot 3.0 app crashed. Hard. Pods went into CrashLoopBackOff immediately.
The logs? One line: java.io.IOException: No space left on device.
Except it wasn’t a disk space issue. The filesystem literally refused writes.
Symptom: Pod Fails to Start, Logs Show Temp Directory Errors
Run kubectl logs and you’ll see this:
org.apache.catalina.startup.Catalina.start: Error starting Tomcat context
java.io.IOException: The temporary directory is not writable
at org.apache.catalina.startup.Tomcat.init(Tomcat.java:467)
Spring Boot’s embedded Tomcat needs to write temp files at startup — JSP compilation, session serialization, file upload parsing. All of it goes to /tmp or whatever java.io.tmpdir points to.
With readOnlyRootFilesystem: true, the entire root filesystem is locked. /tmp is part of that rootfs, so it’s also read-only. Boom.
Root Cause: Spring Boot’s Temp Directory Dependency
Is this Spring Boot’s fault? Not entirely.
Spring Boot doesn’t have built-in adaptation for read-only filesystems. Tomcat, Undertow, Jetty — they all default to creating temp directories under java.io.tmpdir. Lock the rootfs, and they all break the same way.
The kicker? Some apps also write logs, caches, or Hibernate second-level caches at runtime. No write permission = instant crash.
Fix: Three Steps, No Code Changes
Step 1: Mount an emptyDir Volume to /tmp
This is the most straightforward fix. Add an emptyDir volume to your Deployment and mount it at /tmp.
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-boot-app
spec:
template:
spec:
containers:
- name: app
image: myapp:latest
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
emptyDir is ephemeral storage tied to the Pod’s lifecycle. K8s mounts it as read-write by default, and it’s a separate mount point — not part of the rootfs — so readOnlyRootFilesystem doesn’t touch it.
Step 2: Explicitly Set java.io.tmpdir
Sometimes Spring Boot doesn’t pick up /tmp, or your app defines a different temp path. Force it with an environment variable:
env:
- name: JAVA_TOOL_OPTIONS
value: "-Djava.io.tmpdir=/tmp"
Or add it to your Dockerfile:
ENV JAVA_TOOL_OPTIONS="-Djava.io.tmpdir=/tmp"
Step 3 (Optional): Mount emptyDirs for Other Writable Paths
Your app might need to write elsewhere — logs, uploads, caches.
volumeMounts:
- name: tmp
mountPath: /tmp
- name: logs
mountPath: /var/log/app
- name: uploads
mountPath: /var/uploads
volumes:
- name: tmp
emptyDir: {}
- name: logs
emptyDir: {}
- name: uploads
emptyDir: {}
Heads up: emptyDir data disappears when the Pod dies. If you need persistent logs, use hostPath or a PVC.
Comparison: Temp Storage Options
| Solution | Persistent | Performance | Best For |
|---|---|---|---|
| emptyDir | No (lost on Pod deletion) | High (local disk) | Temp files, caches |
| hostPath | Yes (node-level) | High (local disk) | Log collection, monitoring |
| PVC | Yes (cluster-level) | Medium (depends on backend) | Persistent data |
| tmpfs | No (RAM) | Highest | Sensitive temp data |
Community Frustration: This Should’ve Been Fixed Years Ago
Over on Reddit, someone posted: “We stumbled upon a problem, that we can’t run applications in Docker in a read-only filesystem.” That thread on the Docker repo has been sitting unresolved for two years.
Another commenter summed it up: “The docs are garbage on this part.” They’re right. Spring Boot’s official docs mention “ensure the temp directory is writable” but give zero practical guidance on how to do it in a locked-down container.
FAQ
Q: Why does readOnlyRootFilesystem cause Spring Boot to fail?
A: Spring Boot’s embedded Tomcat creates temp files under java.io.tmpdir (default /tmp) during startup. If the rootfs is read-only, writes throw an IOException, crashing the Pod into CrashLoopBackOff.
Q: What’s the difference between emptyDir and tmpfs?
A: emptyDir uses the node’s local disk and consumes disk space. tmpfs uses RAM — faster but limited by memory, and data is never persisted to disk. For temp files, emptyDir is sufficient; use tmpfs for performance-critical or sensitive data.
Q: Can I fix this without changing application code?
A: Yes. Mounting an emptyDir to /tmp solves it without touching a single line of Java code. Mount additional emptyDir volumes for any other writable directories your app needs.
Q: Is this approach secure?
A: Yes. The write permissions only apply to the mounted directories, not the rootfs. Attackers can’t modify system files — they can only write to the temp directories, which is a controlled, acceptable risk.
The Bottom Line
readOnlyRootFilesystem is a security best practice in Kubernetes. But Spring Boot’s support for it is frankly terrible. The fix isn’t complicated: mount an emptyDir to /tmp, set java.io.tmpdir, and you’re done.
Don’t hold your breath for Spring Boot to fix this natively — the community has been complaining for two years with no movement. Do it yourself. It takes five minutes and saves you hours of debugging.
References & Community Insights
The following authoritative resources were referenced for architectural best practices and specifications:
