Installing KubeSphere on Tencent TKE: Step-by-Step Guide & Common Pitfalls

Installation Steps

Follow the official KubeSphere documentation "Install KubeSphere on Tencent TKE". The detailed procedure is available at https://kubesphere.io/zh/docs/installing-on-kubernetes/hosted-kubernetes/install-ks-on-tencent-tke/.

Pitfalls and Tips

1. CBS Disk Size Must Be Multiples of 10Gi

Tencent Cloud Container Service uses CBS disks, which only accept capacities that are multiples of 10Gi. If a PersistentVolumeClaim requests a size that is not a multiple of 10Gi, the volume attachment will fail. Adjust the ClusterConfiguration volumeSize fields for all components to satisfy this requirement.

2. Uninstall May Stall or Leave Residual Resources

When reinstalling, run the kubesphere-delete.sh script from the "Uninstall KubeSphere from Kubernetes" guide. The script can hang because some resources retain a finalizer. Remove the finalizer entries from the affected resources to complete deletion. Incomplete cleanup often leaves MutatingWebhookConfiguration, ValidatingWebhookConfiguration, ClusterRole, and ClusterRoleBinding objects, which cause re‑installation failures.

3. Monitoring Incompatibility – Missing Super‑Node Pod Metrics

KubeSphere’s monitoring collects cAdvisor data by accessing port 10250 on each node. Super‑node IPs are non‑routable placeholder addresses, so the metrics cannot be scraped. The solution is to add a custom scrape configuration.

Prepare a Secret named additional-scrape-configs with the following YAML (saved as scrape-config.yaml):

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: additional-scrape-configs
  namespace: kubesphere-monitoring-system
stringData:
  additional-scrape-configs.yaml: |-
    - job_name: kubelet
      honor_timestamps: true
      metrics_path: '/metrics'
      params:
        collect[]:
        - 'ipvs'
      scheme: http
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_tke_cloud_tencent_com_pod_type]
        regex: eklet
        action: keep
      - source_labels: [__meta_kubernetes_pod_phase]
        regex: Running
        action: keep
      - source_labels: [__meta_kubernetes_pod_ip]
        separator: ';'
        regex: (.*)
        target_label: __address__
        replacement: ${1}:9100
        action: replace
      - source_labels: [__meta_kubernetes_pod_name]
        separator: ';'
        regex: (.*)
        target_label: pod
        replacement: ${1}
        action: replace
      - source_labels: [__meta_kubernetes_namespace]
        separator: ';'
        regex: (.*)
        target_label: namespace
        replacement: ${1}
        action: replace
      metric_relabel_configs:
      - source_labels: [__name__]
        separator: ';'
        regex: container_.*
        replacement: $1
        action: keep
      - target_label: metrics_path
        replacement: /metrics/cadvisor
        action: replace
    - job_name: eks
      honor_timestamps: true
      metrics_path: '/metrics'
      params:
        collect[]:
        - 'ipvs'
      scheme: http
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_tke_cloud_tencent_com_pod_type]
        regex: eklet
        action: keep
      - source_labels: [__meta_kubernetes_pod_phase]
        regex: Running
        action: keep
      - source_labels: [__meta_kubernetes_pod_ip]
        separator: ';'
        regex: (.*)
        target_label: __address__
        replacement: ${1}:9100
        action: replace
      - source_labels: [__meta_kubernetes_pod_name]
        separator: ';'
        regex: (.*)
        target_label: pod
        replacement: ${1}
        action: replace
      - source_labels: [__meta_kubernetes_namespace]
        separator: ';'
        regex: (.*)
        target_label: namespace
        replacement: ${1}
        action: replace
      metric_relabel_configs:
      - source_labels: [__name__]
        separator: ';'
        regex: (container_.*|pod_.*|kubelet_.*)
        replacement: $1
        action: keep

Create the secret: kubectl apply -f scrape-config.yaml Edit the Prometheus custom resource in the monitoring namespace to reference the secret:

kubectl -n kubesphere-monitoring-system edit prometheuses.monitoring.coreos.com k8s

Add the following snippet under spec:

spec:
  additionalScrapeConfigs:
    key: additional-scrape-configs.yaml
    name: additional-scrape-configs

4. s‑apiserver Crash

Crashes are often caused by incomplete KubeSphere chart CRDs. Manually install the missing CRDs with:

kubectl apply -f https://raw.githubusercontent.com/kubesphere/notification-manager/master/config/bundle.yaml

Reference: https://kubesphere.com.cn/forum/d/7610-ks-330-ks-apiserver-crash/3