Why We Took Ownership of the DigitalOcean External-DNS Webhook

At Amoniac, we manage dozens of complex Kubernetes environments. In our world, automation isn't just a "nice-to-have" — it’s the backbone of our operations. So, when a core component like External-DNS starts failing silently, leaving you to manually clean up DNS records with doctl, you know it’s time to stop patching and start engineering.

As Partners of DigitalOcean, we decided to step up and take over the maintenance of the DigitalOcean provider, moving it from the monolithic external-dns core to a dedicated, high-performance Webhook.

The Pain: Automation That Required a Human Touch

If you’ve used the built-in DigitalOcean provider in External-DNS, you might recognize this scenario: you delete a DNSEndpoint or an Ingress, but the DNS record remains "stuck" in the DigitalOcean dashboard. Or worse, you deploy a complex set of records, and the main domain records simply fail to appear.

We saw this repeatedly when handling multi-record DNSEndpoint resources. Here is a typical example of what we were trying to automate:

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: preview1243.dev.com
spec:
  endpoints:
  - dnsName: preview1243.dev.com
    recordType: TXT
    targets: ["v=spf1 include:mailgun.org ~all"]
  - dnsName: preview1243.dev.com
    recordType: A
    targets: ["1.2.3.4"]
  - dnsName: '*.preview1243.dev.com'
    recordType: A
    targets: ["1.2.3.4"]
  # ... and several more for MX and CNAME

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: preview1243.dev.com
spec:
  endpoints:
  - dnsName: preview1243.dev.com
    recordType: TXT
    targets: ["v=spf1 include:mailgun.org ~all"]
  - dnsName: preview1243.dev.com
    recordType: A
    targets: ["1.2.3.4"]
  - dnsName: '*.preview1243.dev.com'
    recordType: A
    targets: ["1.2.3.4"]
  # ... and several more for MX and CNAME

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: preview1243.dev.com
spec:
  endpoints:
  - dnsName: preview1243.dev.com
    recordType: TXT
    targets: ["v=spf1 include:mailgun.org ~all"]
  - dnsName: preview1243.dev.com
    recordType: A
    targets: ["1.2.3.4"]
  - dnsName: '*.preview1243.dev.com'
    recordType: A
    targets: ["1.2.3.4"]
  # ... and several more for MX and CNAME

The issues were consistent:

1. The Deletion Failure: Records wouldn't be removed after CRD deletion, forcing us to manually purge them via doctl.

2. The Partial Update Problem: Periodically, records for the root domain (A or TXT) simply wouldn't be created, while subdomains worked fine.

3. The Sync Trap: Our temporary fix was a manual purge followed by waiting for the next External-DNS sync cycle, hoping the provider would "catch up."

The Journey: From Patches to "Out-of-Tree"

Initially, we tried the traditional route. We submitted Pull Requests to the main external-dns repository (like adding TXT record support for DO). However, maintaining a custom branch of a massive upstream project is a maintenance nightmare.

While researching, we found the community initiative to move providers from "in-tree" (inside the main binary) to "out-of-tree" (external webhooks). Since the DigitalOcean provider was still in the Alpha stage, we made a strategic decision: we would take the code out of the tree and maintain it ourselves.

Why the Webhook Approach Wins

Moving to a dedicated Webhook wasn't just about fixing bugs; it was about gaining the agility to build a production-grade tool. By maintaining the DigitalOcean Webhook independently, we’ve introduced several critical features:

• Full API Support: We track the DigitalOcean DNS API closely, ensuring all record types and edge cases are covered.

• Resilience by Design: We implemented automatic retries with exponential backoff. If the DigitalOcean API returns a 429 (Rate Limit) or a 5xx error, the webhook handles it gracefully instead of failing the entire sync.

• SoftError Support: We’ve integrated graceful error handling to prevent minor API hiccups from breaking your cluster's DNS state.

• Security & Maintenance: Being a separate project allows us to instantly fix Dependabot issues and update DO client libraries without waiting for the global External-DNS release cycle.

Official Recognition

This isn't just a side project. As part of the broader Kubernetes-sigs effort to modularize External-DNS, our webhook is now officially documented as the recommended provider for DigitalOcean.

We took this on because, as DigitalOcean partners, we feel responsible for the experience of the community. If you are running Kubernetes on DigitalOcean, you shouldn't have to deal with "ghost" records or manual cleanups.

Getting Started

The webhook runs as a simple sidecar container alongside your External-DNS deployment. You can find full installation instructions, Helm charts, and configuration examples on our official documentation site and GitHub:

👉 Amoniac DigitalOcean Webhook on GitHub

Conclusion

At Amoniac, we don't just build software for our clients; we build the infrastructure that makes the internet more reliable. We’re proud to support the DigitalOcean and Kubernetes communities by ensuring that "automated DNS" finally means truly automated.