Merge pull request #6704 from dokku/k3s-default-nginx

Switch the default k3s routing layer from traefik to nginx

docs/appendices/0.34.0-migration-guide.md

@@ -3,3 +3,11 @@
 ## Removals
 
 - The `disable-chown` property of the `scheduler-docker-local` plugin has been removed. Mounted paths will no longer have file permissions changed during the pre-deploy phase. Files baked into the container image for herokuish builds will always be owned by the correct user.
+
+## Changes
+
+- The k3s scheduler now defaults to nginx as it's default proxy implementation. The traefik proxy implementation is still available, though users will need to set the global `ingress-class` k3s property to `traefik` via the following command:
+
+    ```shell
+    dokku scheduler-k3s:set --global ingress-class traefik
+    ```

docs/deployment/schedulers/k3s.md

@@ -58,12 +58,12 @@ By default, Dokku will attempt to auto-detect the IP address of the server. In c
 dokku scheduler-k3s:initialize --server-ip 192.168.20.15
 ```
 
-Dokku's k3s integration natively uses `Traefik` as it's ingress load balancer via [Traefik's CRDs](https://doc.traefik.io/traefik/providers/kubernetes-crd/), but a cluster can be set to use `nginx` via the [ingress-nginx](https://github.com/kubernetes/ingress-nginx) project. Switching to nginx will result in any `nginx` plugin settings being respected, either by turning them into annotations or creating a custom server snippet.
+Dokku's k3s integration natively uses `nginx` as it's ingress load balancer via [ingress-nginx](https://github.com/kubernetes/ingress-nginx). Properties set by the `nginx` plugin will be respected, either by turning them into annotations or creating a custom server/location snippet that the `ingress-nginx` project can use.
 
-To change the ingress, set the `--ingress-class` flag:
+Dokku can also use Traefik on cluster initialization via the [Traefik's CRDs](https://doc.traefik.io/traefik/providers/kubernetes-crd/). To change the ingress, set the `--ingress-class` flag:
 
 ```shell
-dokku scheduler-k3s:initialize --ingress-class nginx
+dokku scheduler-k3s:initialize --ingress-class traefik
 ```
 
 ### Adding nodes to the cluster

plugins/scheduler-k3s/scheduler_k3s.go

@@ -42,7 +42,7 @@ var (
 	}
 )
 
-const DefaultIngressClass = "traefik"
+const DefaultIngressClass = "nginx"
 const GlobalProcessType = "--global"
 const KubeConfigPath = "/etc/rancher/k3s/k3s.yaml"
 const DefaultKubeContext = ""