Drop unrecognized --workers arg causing crashloop
The libretranslate CLI does not accept --workers, only the gunicorn process inside it does. Passing it caused the pod to fail argparse on boot and CrashLoopBack...
Probe timeout 10s and 2 gunicorn workers
LibreTranslate's /languages probe blocks on the gunicorn worker event loop. With a single worker translating a long chat message, the probe times out at the def...
Rewrite /etc/odbc.ini to point at $MARIADB_HOST on every boot
The escomputers/freepbx:17 image bakes /etc/odbc.ini with Server=db (the docker-compose link target). On this Kubernetes deployment MariaDB lives at \$MARIADB_H...
Configure STUN so browser ICE can complete bidirectionally
Two-party browser-to-browser calls connected at the SIP layer but had no audio either direction. Voicemail (one-way Asterisk → browser) worked because Asterisk'...
Host browser SIP WSS on internal.kamocrm.com (real cert)
Browsers refused the WebSocket connection to **************** with close code 1006 because Traefik was serving its self-signed default cert for that hostname (n...
Make WebRTC config survive pod restart
The previous approach (http_custom.conf with [general] override) didn't actually take effect because Asterisk's INI parser doesn't honor multiple [general] sect...
Enable Asterisk built-in HTTP server on :8089 for WebRTC /ws
A chan_pjsip `protocol=ws` transport rides on top of res_http_websocket, which in turn requires Asterisk's built-in HTTP server to be running. The default FreeP...
Expose port 8088 directly on ClusterIP service
Service only had port 80→8088 mapping. Internal services using **************** (e.g. VOIPService API token requests) were timing out because port 8088 had no l...
Add missing fail2ban-config ConfigMap for k1m1
Deployment references fail2ban-config ConfigMap (jail.local + asterisk-security.conf) but it did not exist, blocking the pod at MountVolume.SetUp and preventing...
Allow /domain/ path prefix on theme IngressRoute for CORS
The theme config at theme.kamocrm.com/domain/... was not matched by the existing route rules (only /public/ and /internal/ were allowed), so Traefik returned a ...
Replace Coqui TTS with Piper TTS (en_US-norman-medium)
Coqui VCTK model has no named speakers — replace with Piper TTS which has the exact norman[medium] voice. Piper binary (~3 MB) and model (~60 MB) are downloaded...
Add local_net for split-DNS / LAN client RTP fix
Phones on the same LAN as k1m1 connect via split DNS (domain resolves to 192.168.4.22 internally). Without local_net, Asterisk still advertises the external dom...
Persist sorcery contact=memory fix and block SIP scanner in startup.sh
- Inject [res_pjsip] contact=memory into sorcery.conf before Asterisk starts; the default sorcery-astdb SQLite backend silently drops all REGISTER contacts - ...
Move hostPath from MinIO (rclone FUSE) to local /var/lib/pbx
MinIO via rclone FUSE does not support POSIX chmod or symlink() — both required by FreePBX's module system. Changed all FreePBX and MariaDB hostPath volumes fro...
Wrap fwconsole/amportal via php for MinIO FUSE hostPath
/var/lib/asterisk is backed by MinIO via rclone FUSE which doesn't support POSIX chmod, so fwconsole scripts can't be made executable on that volume. startup.sh...
Add preStop hook and terminationGracePeriodSeconds
Containers were getting stuck in Terminating because background Asterisk processes kept running after apache2ctl (PID 1) exited. preStop hook sends SIGTERM to A...
Add bootstrap.php require to regenerated /etc/freepbx.conf
The installer writes freepbx.conf with datasource='' and a require_once for bootstrap.php, which config.php depends on to load the FreePBX class. Our regenerate...
Regenerate /etc/freepbx.conf on restart
/etc is ephemeral (not a hostPath), so /etc/freepbx.conf written by the installer is lost on every pod restart. Added else branch to regenerate it from env vars...
Bypass start_asterisk, wait for control socket instead
start_asterisk uses `pidof asterisk` which returns a false positive on pod restarts, causing the wait-for-Asterisk loop to hang. Now startup.sh starts Asterisk ...
Start Asterisk in background, wait for it, then run installer
The entrypoint.sh does not run the FreePBX installer — it only starts services. startup.sh now replicates the entrypoint setup steps, starts Asterisk in backgro...
Let entrypoint handle install, not startup.sh
FreePBX installer (php install -n) requires Asterisk to already be running. Our startup.sh was calling it before Asterisk started, causing "Error communicating ...
Mount /etc/asterisk only, not /etc, to fix PHP simplexml error
Mounting the full /etc as a hostPath volume was corrupting PHP extension loading even after seeding from the image — simplexml_load_file undefined at install ti...
Reduce CPU limits to 2:1 ratio to eliminate overcommitment
Docs 4→1 core, jibri 4→2 cores, libretranslate 2→1 core, bergamot 2→1 core, coturn 1→0.5 core. Saves ~7.5 cores off node limits.
Switch to escomputers/freepbx:17 with correct architecture
tiredofit/freepbx has no FreePBX 17 image (abandoned since 2022). escomputers/freepbx:17 (updated Apr 2026, Asterisk 21 + FreePBX 17) requires a separate MariaD...
Remove nonexistent postfix-lmdb package, install lmdb library instead
postfix-lmdb is not a valid Alpine package; lmdb support is built into the base postfix package. The pod was CrashLoopBackOff due to apk failure.
Fix Docs image registry, add Recreate strategy to coturn/postfix, increase qdrant memory
- Docs: fix wrong image registry (redis.kamo.svc... → **************** add Recreate strategy - coturn: add Recreate strategy to prevent rolling update port conf...
Switch Postfix sasl_passwd from hash to lmdb format
Alpine postfix build does not include hash support — postmap and smtp_sasl_password_maps were silently failing, deferring every outbound relay attempt through s...
Set Traefik externalTrafficPolicy=Local to preserve client IP
With the default "Cluster" policy, kube-proxy SNATs external traffic to the node IP before forwarding to Traefik. That caused Traefik to see the node IP as the ...
Clean up legacy analytics deployment on each CI run
Delete old analytics deployment, service, and ingressroute that conflict with the kamo-analytics deployment managed by its own CI/CD pipeline.
Strip X-Frame-Options header in iframe-compatible middleware
Set X-Frame-Options to empty string to remove the SAMEORIGIN value that was being inherited, allowing CSP frame-ancestors to control iframe embedding instead.
Reset document.title in rebrand() when React overwrites it
React re-applies RSC payload after hydration, resetting the title to "analytics". Add a title check inside the MutationObserver callback so it gets corrected ev...
Replace "analytics" text nodes with "Kamo Analytics" instead of clearing
Use TreeWalker to remove all "analytics" text nodes from rendered DOM
The workspace nav title "analytics" comes from JS bundles loaded after auth, rendered as a text node next to the logo SVG. Uses TreeWalker to find and clear all...
Add guarded MutationObserver for client-side rendered logo/text
Login page and dashboard content is rendered by Next.js JS bundles, not in the HTML body, so sub_filter can't reach it. Adds a script with a busy-flag-guarded M...
Redirect favicons at nginx level, fix RSC payload escaping
- Serve favicon via nginx 302 redirect instead of trying to rewrite HTML/RSC payload href strings (cleaner, always works) - Fix RSC title replacement with pro...
Use pure sub_filter replacements instead of MutationObserver
MutationObserver caused infinite loop (DOM change -> observer fires -> DOM change). Switch to direct nginx sub_filter string replacements targeting both static ...
Comprehensive analytics rebranding for login page and dashboard
- Replace title dynamically (handles "Login | analytics", "Dashboard | analytics" etc.) - Replace "analytics" h2 text on login page with Kamo logo image - Repla...
Use MutationObserver to rebrand analytics client-side rendered UI
analytics is a Next.js app that sets title/logo via JavaScript after initial HTML load. Replaces static sub_filter approach with injected script that uses Mutat...
Purge incomplete model directories in init container
The service crashes on startup if it finds a directory in /models that has missing or partial model files. Add a sweep at init start that removes any directory ...
Remove GitHub LFS downloads, use only data.statmt.org models
GitHub raw URLs return LFS pointer files, not actual binaries. Drop enru/enar downloads entirely — LibreTranslate handles those locales. Only download the 4 pai...
Scale bergamot to 0, update image to mozilla/translation-service
The Google Cloud Storage model download bucket (bergamot-models-sandbox) returns 404 for all files and the firefox-translations-models repo was archived Dec 202...
Scale bergamot back to 1 replica — translation failures caused by missing pod
Bergamot was at 0 replicas with its service still registered. The ProviderRouter in TranslateService had no circuit breaker, so every request for es/fr/de/pt/it...
Remove --load-only restriction from LibreTranslate
All needed language packages are pre-installed on the hostPath volume. --load-only was preventing some packages from loading. Without the flag, LibreTranslate l...
Use startupProbe for LibreTranslate to handle long model downloads
startupProbe with 60 attempts x 30s = 30min startup window for initial model downloads. After startup completes, normal liveness and readiness probes take over ...
Restrict LibreTranslate to only load supported language models
Use --load-only with the 22 supported language codes instead of downloading all available models (~hundreds of GB). Removes --update-models flag which would dow...
Remove invalid --load-awesome-ai-models flag from LibreTranslate
Flag does not exist in current libretranslate image. Replace with --update-models to ensure language packs are downloaded on startup.
Like what you see shipping?
Every one of these updates lands in your workspace automatically. Start free and watch it grow week after week.