Skip to content

Router Customization and Debugging (Traefik)

DDEV uses the well-known open-source Traefik for its router container.

DDEV’s router plays an important role in its container architecture, receiving HTTP and HTTPS traffic for requests like *.ddev.site and delivering them to the appropriate project’s web container.

Traefik Configuration

Before continuing, it’s important to note that very few users ever experiment with custom Traefik configuration. This is an advanced topic, and the vast majority of users never need to know anything about it, as DDEV generates all the necessary configuration. In general you don’t need to understand Traefik configuration.

However, you can fully customize the router’s Traefik configuration. (DDEV uses the Traefik v3 rule syntax.)

All Traefik configuration uses the file provider, not the Docker provider. Even though the Traefik daemon itself is running inside the ddev-router container, it uses mounted files for configuration, rather than listening to the Docker socket.

Tip

Like other DDEV configuration, any file with #ddev-generated will be overwritten unless you choose to “take over” it yourself. You can do this by removing the #ddev-generated line. DDEV will stop making changes to that file, and you’ll be responsible for updating it.

Traefik Static Configuration

Static configuration is automatically generated in the $HOME/.ddev/traefik directory (see global configuration directory). “Static” configuration means Traefik configuration which is only read when the router is started.

  • .static_config.yaml (a hidden file) is the configuration that gets used. It is not to be edited; it is generated from DDEV’s base configuration while merging any files named static_config.*.yaml. It is read on router startup, and does not change until the router starts again (normally after ddev poweroff).

  • Additional static configuration may be added by adding static_config.*.yaml files, which will be merged into the generated .static_config.yaml. For example, a static_config.loglevel.yaml might override logging configuration, a static_config.plugin.yaml might contain external Traefik plugins, or a static_config.dnschallenge.yaml might provide configuration for additional certificatesResolvers. Merging is done with an override strategy, meaning that the final file in alphanumeric sort to touch a particular element of the YAML structure wins. Some examples of static_config.*.yaml files are:

    • static_config.loglevel.yaml:

      # Enable extensive error and access logging
log:
  level: DEBUG
accessLog:
  filters:
    statusCodes: {}

    • static_config.cloudflare.yaml:

      certificatesResolvers:
  acme-dnsChallenge:
    acme:
      email: admin@example.com
      dnsChallenge:
        provider: cloudflare

    • static_config.fail2ban.yaml

      experimental:
  plugins:
    fail2ban:
      moduleName: "github.com/tomMoulard/fail2ban"
      version: "v0.8.1"

  • certs/default_cert.* files are the default DDEV-generated certificates, normally created by mkcert.

  • config/default_config.yaml contains global dynamic configuration, including pointers to the default certificates. It’s unusual to change its content, but if you wanted to change the content you would override with a file in custom-global-config.

  • custom-global-config is a directory where you can add custom Traefik dynamic configuration files (such as middleware, routers, or services) that will be automatically included for all projects. Files in this directory are copied to the router’s config volume on each ddev start. For example, a custom-global-config/global_middlewares.yaml file might provide middleware implementations that could be used in all projects:

    http:
  middlewares:
    global-ratelimit:
      rateLimit:
        average: 100
        burst: 50

Project Traefik Configuration

Project-specific configuration is automatically generated in the project’s .ddev/traefik/config directory. For example, a project named example will have a .ddev/traefik/config/example.yaml which describes the routers, middlewares, and services generated by default for that project. These are based on the base hostname, additional_hostnames, and additional_fqdns defined for the project. They also include support for add-ons and services that use HTTP_EXPOSE, HTTPS_EXPOSE, and VIRTUAL_HOST configurations (see Conventions for Defining Additional Services for more details).

  • The certs directory contains the <projectname>.crt and <projectname>.key certificate generated for the project.
  • The config/<projectname>.yaml file contains the base configuration for the project, including information about routers, services, and certificates.

  • Adding Custom Configuration: You can add custom Traefik configuration by creating additional *.yaml files in the .ddev/traefik/config directory. All YAML files in this directory will be automatically merged together, with <projectname>.yaml serving as the base. The merged result is then copied to the global Traefik configuration as <projectname>_merged.yaml. This allows you to add custom routers, middlewares, or services without modifying the main configuration file.

    For example, to add custom middleware to a project named example, you could create .ddev/traefik/config/custom-middleware.yaml:

    http:
  middlewares:
    example-custom-headers:
      headers:
        customRequestHeaders:
          X-Custom-Header: "my-value"

  • Taking Over Configuration: If you want to completely customize the configuration and prevent DDEV from regenerating it, remove the #ddev-generated line from the top of the config/<projectname>.yaml file. DDEV will then stop regenerating it, and you’ll be responsible for maintaining it. If you do this and then change the name of the project, you have to either remove the file and have DDEV regenerate it, or you have to update it yourself.

Router docker-compose Customization

The default Docker Compose configuration for the router container is found in $HOME/.ddev/.router-compose.yaml (see global configuration directory). It is quite unusual to override this configuration, but it can be overridden in the same way project configuration can be overridden (project .ddev/docker-compose.*.yaml). These ultimately get merged into $HOME/.ddev/.router-compose-full.yaml

You can create a $HOME/.ddev/router-compose.*.yaml. For example, as $HOME/.ddev/router-compose.cloudflare.yaml might contain environment variables like this:

services:
  ddev-router:
    environment:
      - CLOUDFLARE_EMAIL=you@example.com
      - CLOUDFLARE_API_KEY=some-key

Troubleshooting Traefik Routing

Traefik provides a dynamic description of its configuration you can visit at http://localhost:10999. When things seem to be going wrong, run ddev poweroff and then start your project again by running ddev start. Examine the router’s logs to see what the Traefik daemon is doing (or failing at) by running docker logs ddev-router or docker logs -f ddev-router. The Traefik logs are set to a minimal set by default, but you can enable much more extensive logging and access logs with a static_config.loglevel.yaml as described above.

Warning: There are router configuration problems

If you see a warning on ddev start about router configuration problems, it is most likely a result of custom configuration problems, which could be an invalid docker-compose.*.yaml or leftover project .ddev/traefik/config/*.yaml files. Only Traefik error-level (ERR) messages are shown there; warning-level (WRN) and debug output from Traefik appear only in docker logs ddev-router.

On Linux systems, file watcher errors related to the *file.Provider are a common cause. See the section below for solutions.

Linux file watcher errors

If you encounter Traefik file watcher errors on Linux systems, these are common solutions:

Error adding file watcher: no space left on device

This error appears during ddev start or in the router logs, docker logs ddev-router, as:

ERR Cannot start the provider *file.Provider error=”error adding file watcher: no space left on device”

This indicates that the system has reached the limit for inotify watches on your Linux computer. First, check the current value:

sysctl fs.inotify.max_user_watches

Then create a configuration file to increase the limit (for example, if the current value is 65536, you can increase it to 524288):

echo 'fs.inotify.max_user_watches=524288' | sudo tee -a /etc/sysctl.d/60-inotify.conf

Apply the changes without restarting the system:

sudo sysctl -p /etc/sysctl.d/60-inotify.conf

Error creating file watcher: too many open files

This error appears during ddev start or in the router logs, docker logs ddev-router, as:

ERR Cannot start the provider *file.Provider error=”error creating file watcher: too many open files”

This indicates that too many inotify instances are open. First, check the current value:

sysctl fs.inotify.max_user_instances

Then create a configuration file to increase the limit (for example, if the current value is 128, you can increase it to 8192):

echo 'fs.inotify.max_user_instances=8192' | sudo tee -a /etc/sysctl.d/60-inotify.conf

Apply the changes without restarting the system:

sudo sysctl -p /etc/sysctl.d/60-inotify.conf