相关链接特性清单: https://gitcode.com/openFuyao/release-management/blob/main/openFuyao-26.03/release-plan.md安装部署前置环境校验工具使用指导: https://gitcode.com/openFuyao/sig-installation/blob/master/docs/zh/user_guide/cluster_installation_deployment/environment_pre_check_tool_guide.md测试环境CPU: Kunpeng-920OS: openEuler 24.03 LTS SP3 aarch64Fuyao Version: v26.03 rc3docker: 2:18.09.0-346.oe2403sp3测试特性[ ] 在线部署；[ ] 离线包制备；[ ] 离线部署；[ ] 安装部署前置检查工具；[ ] NPU Operator；[ ] AI推理套件；建议优化点环境检测工具，检查 iptables 默认策略是否放行，若未放行可能在部署成功后无法访问；默认防火墙策略为 FORWARD DROP ，对集群运行和访问带来的潜在问题；运行 cli 前检查是否存在命令并及时抛出错误；检查 tar / unzip 是否安装，安装过程有很多地方会用到，而且出错时不会得到明显的解压失败报错，难以定位问题。安装命令变化，考虑上下兼容性？场景记录离线部署管理面和业务面集群CPU: Kunpeng-920OS: openEuler 24.03 LTS SP3 aarch64Fuyao Version: v26.03 rc3docker: 2:18.09.0-346.oe2403sp3arm64 环境下构建离线制品包为什么会执行 amd64 的 bin[bke][2026-03-26 07:48:49][INFO] The bke binary file version is . sh: line 1: /root/fuyao-26-03/packages/usr/bin/bkeadm_linux_amd64: cannot execute binary file: Exec format error完整日志[root@master1 fuyao-26-03]# cat build-offline-package.log | grep -v sha256 2026-03-26T06:49:56.467+0800 info infrastructure/infrastructure.go:53 The docker client is ready. [bke][2026-03-26 06:49:56][step.1] Configuration file check [bke][2026-03-26 06:49:56][step.2] Creates a workspace in the current directory [bke][2026-03-26 06:49:56][step.5] Collect the required image files [bke][2026-03-26 06:49:56][INFO] Try pulling away the mirror image cr.openfuyao.cn/openfuyao/registry:2.8.1 [bke][2026-03-26 06:49:56][step.3] Collect host dependency packages and package files [bke][2026-03-26 06:49:56][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/kubernetes/kubernetes/releases/download/1.34.3-of.1/bin/linux/arm64/kubectl to /root/fuyao-26-03/packages/tmp/packages/files/kubectl-v1.34.3-of.1-arm64 [bke][2026-03-26 06:50:07][INFO] Try pulling away the mirror image cr.openfuyao.cn/openfuyao/registry:2.8.1 [bke][2026-03-26 06:50:18][step.6] Collect images from the source repository to the target repository [bke][2026-03-26 06:50:18][INFO] Remove the image repository 2026-03-26T06:50:18.026+0800 info infrastructure/infrastructure.go:53 The docker client is ready. 2026-03-26T06:50:18.028+0800 info infrastructure/infrastructure.go:53 The docker client is ready. [bke][2026-03-26 06:50:20][WARN] Image cr.openfuyao.cn/openfuyao/registry:2.8.1 inspect failed: Error response from daemon: no such image: cr.openfuyao.cn/openfuyao/registry:2.8.1: No such image: cr.openfuyao.cn/openfuyao/registry:2.8.1, retrying (1/3)... [bke][2026-03-26 06:50:21][WARN] Image cr.openfuyao.cn/openfuyao/registry:2.8.1 inspect failed: Error response from daemon: no such image: cr.openfuyao.cn/openfuyao/registry:2.8.1: No such image: cr.openfuyao.cn/openfuyao/registry:2.8.1, retrying (2/3)... [bke][2026-03-26 06:50:22][WARN] Image cr.openfuyao.cn/openfuyao/registry:2.8.1 inspect failed: Error response from daemon: no such image: cr.openfuyao.cn/openfuyao/registry:2.8.1: No such image: cr.openfuyao.cn/openfuyao/registry:2.8.1, retrying (3/3)... [bke][2026-03-26 06:50:23][WARN] Get image cr.openfuyao.cn/openfuyao/registry:2.8.1 inspect failed: failed to inspect image cr.openfuyao.cn/openfuyao/registry:2.8.1 after 3 attempts: Error response from daemon: no such image: cr.openfuyao.cn/openfuyao/registry:2.8.1: No such image: cr.openfuyao.cn/openfuyao/registry:2.8.1 [bke][2026-03-26 06:50:23][INFO] Image cr.openfuyao.cn/openfuyao/registry:2.8.1 is downloading [bke][2026-03-26 06:50:37][INFO] Wait for the container mirroring service to start... [bke][2026-03-26 06:50:42][INFO] The container mirroring service is started. Getting image list signatures Copying 6 images generated from 6 images in list Getting image source signatures [bke][2026-03-26 06:50:56][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/kubernetes/kubernetes/releases/download/1.34.3-of.1/bin/linux/arm64/kubelet to /root/fuyao-26-03/packages/tmp/packages/files/kubelet-v1.34.3-of.1-arm64 Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 7 images generated from 7 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures [bke][2026-03-26 06:52:10][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/kubernetes/kubernetes/releases/download/1.34.3-of.1/bin/linux/amd64/kubectl to /root/fuyao-26-03/packages/tmp/packages/files/kubectl-v1.34.3-of.1-amd64 Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures [bke][2026-03-26 06:53:23][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/kubernetes/kubernetes/releases/download/1.34.3-of.1/bin/linux/amd64/kubelet to /root/fuyao-26-03/packages/tmp/packages/files/kubelet-v1.34.3-of.1-amd64 Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures [bke][2026-03-26 06:54:34][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/containerd/containerd/releases/download/v2.1.1-origin/containerd-v2.1.1-linux-amd64.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/containerd-v2.1.1-linux-amd64.tar.gz Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 3 images generated from 3 images in list Getting image source signatures [bke][2026-03-26 06:55:53][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/containerd/containerd/releases/download/v2.1.1-origin/containerd-v2.1.1-linux-arm64.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/containerd-v2.1.1-linux-arm64.tar.gz Writing manifest to image destination Getting image source signatures [bke][2026-03-26 06:57:07][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/containernetworking/plugins/releases/download/v1.4.1/cni-plugins-linux-amd64-v1.4.1.tgz to /root/fuyao-26-03/packages/tmp/packages/files/cni-plugins-linux-amd64-v1.4.1.tgz [bke][2026-03-26 06:58:00][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/containernetworking/plugins/releases/download/v1.4.1/cni-plugins-linux-arm64-v1.4.1.tgz to /root/fuyao-26-03/packages/tmp/packages/files/cni-plugins-linux-arm64-v1.4.1.tgz Writing manifest to image destination Getting image source signatures [bke][2026-03-26 06:58:49][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/helm/releases/download/v3.14.2/helm-v3.14.2-linux-amd64.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/helm-v3.14.2-linux-amd64.tar.gz [bke][2026-03-26 06:59:08][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/helm/releases/download/v3.14.2/helm-v3.14.2-linux-arm64.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/helm-v3.14.2-linux-arm64.tar.gz [bke][2026-03-26 06:59:24][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/mikefarah/yq/releases/download/v4.43.1/yq_linux_arm64 to /root/fuyao-26-03/packages/tmp/packages/files/yq_linux_arm64 [bke][2026-03-26 06:59:35][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/mikefarah/yq/releases/download/v4.43.1/yq_linux_amd64 to /root/fuyao-26-03/packages/tmp/packages/files/yq_linux_amd64 Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 10 images generated from 10 images in list Getting image source signatures [bke][2026-03-26 06:59:44][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/jqlang/jq/releases/download/v1.7.1/jq-linux-arm64 to /root/fuyao-26-03/packages/tmp/packages/files/jq-linux-arm64 [bke][2026-03-26 06:59:46][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/jqlang/jq/releases/download/v1.7.1/jq-linux-amd64 to /root/fuyao-26-03/packages/tmp/packages/files/jq-linux-amd64 [bke][2026-03-26 06:59:49][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl-certinfo_1.6.4_linux_arm64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssl-certinfo_1.6.4_linux_arm64 [bke][2026-03-26 07:00:00][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl-certinfo_1.6.4_linux_amd64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssl-certinfo_1.6.4_linux_amd64 [bke][2026-03-26 07:00:11][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl_1.6.4_linux_arm64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssl_1.6.4_linux_arm64 [bke][2026-03-26 07:00:24][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl_1.6.4_linux_amd64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssl_1.6.4_linux_amd64 [bke][2026-03-26 07:00:38][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssljson_1.6.4_linux_arm64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssljson_1.6.4_linux_arm64 [bke][2026-03-26 07:00:46][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/cloudflare/cfssl/releases/download/v1.6.4/cfssljson_1.6.4_linux_amd64 to /root/fuyao-26-03/packages/tmp/packages/files/cfssljson_1.6.4_linux_amd64 [bke][2026-03-26 07:00:55][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/opencontainers/runc/releases/download/v1.1.12/runc-arm64 to /root/fuyao-26-03/packages/tmp/packages/files/runc-arm64 [bke][2026-03-26 07:01:07][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/opencontainers/runc/releases/download/v1.1.12/runc-amd64 to /root/fuyao-26-03/packages/tmp/packages/files/runc-amd64 [bke][2026-03-26 07:01:19][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/etcd-io/etcd/releases/download/v3.5.6/etcdctl-v3.5.6-linux-amd64 to /root/fuyao-26-03/packages/tmp/packages/files/etcdctl-v3.5.6-linux-amd64 [bke][2026-03-26 07:01:40][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/etcd-io/etcd/releases/download/v3.5.6/etcdctl-v3.5.6-linux-arm64 to /root/fuyao-26-03/packages/tmp/packages/files/etcdctl-v3.5.6-linux-arm64 Writing manifest to image destination Getting image source signatures [bke][2026-03-26 07:01:58][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/rpm/releases/download/v0.0.1/rpm.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/rpm.tar.gz Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 7 images generated from 7 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 8 images generated from 8 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures [bke][2026-03-26 07:45:30][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/nfs/nfsshare.tar.gz to /root/fuyao-26-03/packages/tmp/packages/files/nfsshare.tar.gz [bke][2026-03-26 07:45:30][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/bkeadm/releases/download/1.2.1/bkeadm_linux_amd64 to /root/fuyao-26-03/packages/tmp/packages/files/bkeadm_linux_amd64 [bke][2026-03-26 07:46:48][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/bkeadm/releases/download/1.2.1/bkeadm_linux_arm64 to /root/fuyao-26-03/packages/tmp/packages/files/bkeadm_linux_arm64 [bke][2026-03-26 07:48:03][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/oauth-webhook-1.0.2.tgz to /root/fuyao-26-03/packages/tmp/charts/oauth-webhook-1.0.2.tgz Writing manifest to image destination [bke][2026-03-26 07:48:03][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/oauth-server-1.0.3.tgz to /root/fuyao-26-03/packages/tmp/charts/oauth-server-1.0.3.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/console-website-1.0.4.tgz to /root/fuyao-26-03/packages/tmp/charts/console-website-1.0.4.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/monitoring-service-1.0.4.tgz to /root/fuyao-26-03/packages/tmp/charts/monitoring-service-1.0.4.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/console-service-1.0.4.tgz to /root/fuyao-26-03/packages/tmp/charts/console-service-1.0.4.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/marketplace-service-1.0.3.tgz to /root/fuyao-26-03/packages/tmp/charts/marketplace-service-1.0.3.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/application-management-service-1.0.2.tgz to /root/fuyao-26-03/packages/tmp/charts/application-management-service-1.0.2.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/plugin-management-service-1.0.2.tgz to /root/fuyao-26-03/packages/tmp/charts/plugin-management-service-1.0.2.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/user-management-operator-1.0.2.tgz to /root/fuyao-26-03/packages/tmp/charts/user-management-operator-1.0.2.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/web-terminal-service-1.0.3.tgz to /root/fuyao-26-03/packages/tmp/charts/web-terminal-service-1.0.3.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/installer-service-1.2.1.tgz to /root/fuyao-26-03/packages/tmp/charts/installer-service-1.2.1.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/installer-website-1.2.1.tgz to /root/fuyao-26-03/packages/tmp/charts/installer-website-1.2.1.tgz [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/harbor-1.11.4.tgz to /root/fuyao-26-03/packages/tmp/charts/harbor-1.11.4.tgz Getting image source signatures [bke][2026-03-26 07:48:04][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/bke-console-website-1.0.3.tgz to /root/fuyao-26-03/packages/tmp/charts/bke-console-website-1.0.3.tgz [bke][2026-03-26 07:48:05][INFO] Collecting file packages https://openfuyao.obs.cn-north-4.myhuaweicloud.com/charts/releases/download/bke-console-service-1.0.2.tgz to /root/fuyao-26-03/packages/tmp/charts/bke-console-service-1.0.2.tgz [bke][2026-03-26 07:48:49][step.4] Collect the bke binary file [bke][2026-03-26 07:48:49][INFO] The bke binary file version is . sh: line 1: /root/fuyao-26-03/packages/usr/bin/bkeadm_linux_amd64: cannot execute binary file: Exec format error Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 17 images generated from 17 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 16 images generated from 16 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 2 images generated from 2 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 10 images generated from 10 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 4 images generated from 4 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures Getting image list signatures Copying 5 images generated from 5 images in list Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Getting image source signatures Writing manifest to image destination Writing manifest list to image destination Storing list signatures [bke][2026-03-26 08:41:56][INFO] The system starts to pack the image file. [bke][2026-03-26 08:43:49][INFO] Remove the image repository 2026-03-26T08:43:49.940+0800 info infrastructure/infrastructure.go:53 The docker client is ready. [bke][2026-03-26 08:43:51][step.7] Build the bke package, please wait for the larger package... [bke][2026-03-26 08:46:22][step.8] Packaging complete bke.tar.gz openEuler 24.03 lts 部署 npu operator 有 pod 无法启动：Every 1.0s: kubectl get pod -A master1: Fri Mar 27 10:33:04 2026 NAMESPACE NAME READY STATUS RESTARTS AGE cluster-system bkeagent-deployer-fcnfd 1/1 Running 0 29m default mindio-acp-g67n9 1/1 Running 0 7m56s default mindio-tft-bk8mj 1/1 Running 0 7m57s ingress-nginx ingress-nginx-admission-create-fh4rv 0/1 Completed 0 28m ingress-nginx ingress-nginx-admission-patch-nn79l 0/1 Completed 2 28m ingress-nginx ingress-nginx-controller-jdb5s 1/1 Running 0 28m kube-system ascend-device-plugin-5hmhf 0/1 CrashLoopBackOff 6 (33s ago) 7m59s kube-system calico-kube-controllers-9c987b475-db86h 1/1 Running 0 29m kube-system calico-node-sfvjm 1/1 Running 0 29m kube-system coredns-6bd9b76578-2mqzh 1/1 Running 2 (29m ago) 29m kube-system coredns-6bd9b76578-94pmp 1/1 Running 2 (21m ago) 29m kube-system etcd-master1 1/1 Running 0 28m kube-system kube-apiserver-master1 1/1 Running 0 27m kube-system kube-controller-manager-master1 1/1 Running 1 (27m ago) 29m kube-system kube-proxy-rwpzc 1/1 Running 0 29m kube-system kube-scheduler-master1 1/1 Running 1 (27m ago) 29m kube-system metrics-server-db68b78d-b8kkn 1/1 Running 0 26m mindx-dl ascend-operator-manager-5d4d89f675-8dzn4 0/1 Pending 0 7m59s mindx-dl clusterd-5588c5dc88-qmblm 0/1 Pending 0 7m57s mindx-dl noded-rc5lj 1/1 Running 0 7m59s mindx-dl resilience-controller-8686bbd76f-qzc2z 0/1 Pending 0 7m57s monitoring alertmanager-main-0 2/2 Running 0 28m monitoring alertmanager-main-1 2/2 Running 0 28m monitoring alertmanager-main-2 2/2 Running 0 28m monitoring blackbox-exporter-6877f5c5f7-66qn4 3/3 Running 0 28m monitoring kube-state-metrics-6f6c47f5f4-4pt9s 3/3 Running 0 28m monitoring node-exporter-ng468 2/2 Running 0 28m monitoring prometheus-k8s-0 2/2 Running 0 28m monitoring prometheus-k8s-1 2/2 Running 0 28m monitoring prometheus-operator-6698d7bc85-72xwx 2/2 Running 0 28m npu-exporter npu-exporter-vnzk6 0/1 Pending 0 7m57s npu ascend-runtime-containerd-7s6jv 1/1 Running 0 8m npu npu-driver-l7rjp 1/1 Running 0 8m npu npu-feature-discovery-xr7nt 1/1 Running 0 19m npu npu-node-feature-discovery-gc-5d97746dbc-wvnx6 1/1 Running 0 9m52s npu npu-node-feature-discovery-master-664666b7bb-7qjfh 1/1 Running 0 9m43s npu npu-node-feature-discovery-worker-8mj5w 1/1 Running 0 9m38s npu npu-operator-5678cd59d4-ljvc8 1/1 Running 0 19m openfuyao-system-controller modify-manifests-master1-rhfqf 0/1 Completed 0 27m openfuyao-system-controller openfuyao-system-controller-798c4f6598-npmjs 1/1 Running 0 29m openfuyao-system application-management-service-77457c5c85-g7b6z 2/2 Running 0 28m openfuyao-system console-service-7fdb88c9c6-g2kp5 1/1 Running 0 28m openfuyao-system console-website-c78945fcc-dm6s4 1/1 Running 0 28m openfuyao-system local-harbor-chartmuseum-57fdd9949d-5xwh6 1/1 Running 0 28m openfuyao-system local-harbor-core-6f7d4cc767-9rp6l 1/1 Running 0 28m openfuyao-system local-harbor-database-0 1/1 Running 0 28m openfuyao-system local-harbor-jobservice-7b8c9bf798-ms8qh 1/1 Running 4 (27m ago) 28m openfuyao-system local-harbor-nginx-78b94f7b74-nflct 1/1 Running 0 28m openfuyao-system local-harbor-portal-6b8cbf6747-gq6wd 1/1 Running 0 28m openfuyao-system local-harbor-redis-0 1/1 Running 0 28m openfuyao-system local-harbor-registry-7879c9d46d-d8wq6 2/2 Running 0 28m openfuyao-system marketplace-service-5cf7cd6f5b-bmknq 2/2 Running 0 28m openfuyao-system monitoring-service-6fd8dbd59f-bwq96 2/2 Running 0 28m openfuyao-system oauth-server-64f6545c48-2hm7z 1/1 Running 0 26m openfuyao-system oauth-webhook-78cb864fc5-gsmw9 1/1 Running 0 26m openfuyao-system plugin-management-service-6fd8f64cc4-zvql8 2/2 Running 0 26m openfuyao-system user-management-operator-9bb7bf64-6q9xg 1/1 Running 0 26m openfuyao-system web-terminal-service-6f64b888f9-gdxpb 1/1 Running 0 26m volcano-system volcano-controllers-6ffb787f8d-chk69 0/1 Pending 0 7m58s volcano-system volcano-scheduler-867f9784bb-tvhzh 0/1 Pending 0 7m57s [root@master1 fuyao-26-03]# kubectl -n kube-system logs ascend-device-plugin-5hmhf Defaulted container "device-plugin-01" out of: device-plugin-01, init-permission (init) [INFO] 2026/03/27 02:26:56.030212 1 hwlog/api.go:108 devicePlugin.log's logger init success [INFO] 2026/03/27 02:26:56.030715 1 main.go:187 ascend device plugin starting and the version is v6.0.0_linux-aarch64 [INFO] 2026/03/27 02:26:56.030759 1 main.go:188 ascend device plugin starting scene is center 2026/03/27 02:26:56 command exec failed, exit status 1 [ERROR] 2026/03/27 02:26:56.032950 1 devmanager/devmanager.go:95 deviceManager init failed, prepare dcmi failed, err: cannot found valid driver lib, fromEnv: lib path is invalid, [], fromLdCmd: can't find valid lib: EOF [ERROR] 2026/03/27 02:26:56.033013 1 main.go:203 init devmanager failed, err: auto init failed, err: get chip info failed, err: device Manager is nil, may encounter an exception during initialization. You can check the system log to confirm 环境检查参考 https://gitcode.com/openFuyao/sig-installation/blob/master/docs/zh/user_guide/cluster_installation_deployment/environment_pre_check_tool_guide.mdwget https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/env-check/releases/download/latest/bin/linux/arm64/envCheck wget https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/env-check/releases/download/latest/bin/linux/arm64/config.json文件检查：[root@localhost env]# ./envCheck query [INFO][2026-03-27 02:53:25] envCheck tool start [INFO][2026-03-27 02:53:25] mode: fileQuery [INFO][2026-03-27 02:53:25] file query start ...... [INFO][2026-03-27 02:53:25] query path: [/root/.kube /etc/kubernetes /usr/local/bin/crictl /etc/sysctl.d/k8s.conf /etc/systemd/system/kubelet.service /etc/systemd/system/kubelet.service.d /var/lib/etcd /var/lib/kubelet /run/containerd/containerd.sock /usr/lib/systemd/system/kubelet.service.d /var/run/containerd/containerd.sock /var/run/docker.sock] [INFO][2026-03-27 02:53:25] path not exist: /root/.kube [INFO][2026-03-27 02:53:25] path not exist: /etc/kubernetes [INFO][2026-03-27 02:53:25] path not exist: /usr/local/bin/crictl [INFO][2026-03-27 02:53:25] path not exist: /etc/sysctl.d/k8s.conf [INFO][2026-03-27 02:53:25] path not exist: /etc/systemd/system/kubelet.service [INFO][2026-03-27 02:53:25] path not exist: /etc/systemd/system/kubelet.service.d [INFO][2026-03-27 02:53:25] path not exist: /var/lib/etcd [INFO][2026-03-27 02:53:25] path not exist: /var/lib/kubelet [INFO][2026-03-27 02:53:25] path not exist: /run/containerd/containerd.sock [INFO][2026-03-27 02:53:25] path not exist: /usr/lib/systemd/system/kubelet.service.d [INFO][2026-03-27 02:53:25] path not exist: /var/run/containerd/containerd.sock [WARNING][2026-03-27 02:53:25] file exist: /var/run/docker.sock [INFO][2026-03-27 02:53:25] file query completed Query Time: 2026-03-27 02:53:25 +-------------------------------------------+---------+---------+-------+--------+-------------+ | Path | Exists | Type | Owner | Group | Permissions | +-------------------------------------------+---------+---------+-------+--------+-------------+ | /root/.kube | Missing | Missing |-------+--------+-------------+ | /etc/kubernetes | Missing | Missing |-------+--------+-------------+ | /usr/local/bin/crictl | Missing | Missing |-------+--------+-------------+ | /etc/sysctl.d/k8s.conf | Missing | Missing |-------+--------+-------------+ | /etc/systemd/system/kubelet.service | Missing | Missing |-------+--------+-------------+ | /etc/systemd/system/kubelet.service.d | Missing | Missing |-------+--------+-------------+ | /var/lib/etcd | Missing | Missing |-------+--------+-------------+ | /var/lib/kubelet | Missing | Missing |-------+--------+-------------+ | /run/containerd/containerd.sock | Missing | Missing |-------+--------+-------------+ | /usr/lib/systemd/system/kubelet.service.d | Missing | Missing |-------+--------+-------------+ | /var/run/containerd/containerd.sock | Missing | Missing |-------+--------+-------------+ | /var/run/docker.sock | Exists | File | root | docker | Srw-rw---- | +-------------------------------------------+---------+---------+-------+--------+-------------+ +-------------------+-------+ | Summary | Count | +-------------------+-------+ | Total Checked | 12 | | Total Exists | 1 | | Total Missing | 11 | | Total Directories | 0 | | Total Files | 1 | +-------------------+-------+ [INFO][2026-03-27 02:53:25] completed 程序存在性检测：[root@localhost env]# ./envCheck check [INFO][2026-03-27 02:54:04] envCheck tool start [INFO][2026-03-27 02:54:04] mode: programCheck [INFO][2026-03-27 02:54:04] program check start...... [INFO][2026-03-27 02:54:04] os: linux, arch: arm64 [INFO][2026-03-27 02:54:04] program list to check: [docker kubectl containerd] [INFO][2026-03-27 02:54:04] check program: docker [WARNING][2026-03-27 02:54:04] docker installed - version: Docker version 18.09.0, build d51e3ad [INFO][2026-03-27 02:54:04] check program: kubectl [INFO][2026-03-27 02:54:04] not install: kubectl [INFO][2026-03-27 02:54:04] check program: containerd [WARNING][2026-03-27 02:54:04] containerd installed - version: time="2026-03-27T02:54:04Z" level=warning msg="init error, wrong runtimeTimeout format: time: invalid duration """ containerd version:1.2.0.320.oe2203sp4 871075eb7cc979944ba2d987719cb534bbb87e5c [INFO][2026-03-27 02:54:04] program check completed [WARNING][2026-03-27 02:54:04] detected installed application(s): docker, containerd. Please uninstall it(them) yourself Check Time: 2026-03-27 02:54:04 +------------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ | Program | Status | Version | Path | +------------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ | docker | Installed | Docker version 18.09.0, build d51e3ad | /usr/bin/docker | | kubectl | Not Installed | Unknown | Not found | | containerd | Installed | time="2026-03-27T02:54:04Z" level=warning msg="init error, wrong runtimeTimeout format: time: invalid duration """ containerd version:1.2.0.320.oe2203sp4 871075eb7cc979944ba2d987719cb534bbb87e5c | /usr/bin/containerd | +------------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ +-----------------+-------+ | Summary | Count | +-----------------+-------+ | Total Checked | 3 | | Total Installed | 2 | | Total Missing | 1 | +-----------------+-------+ [INFO][2026-03-27 02:54:04] completed 在线部署部署完毕浏览 NPU Operator 文档才发现需满足 openEuler 22.03 lts 条件，故重装https://docs.openfuyao.cn/zh/docs/v25.12/user_guide/npu_operator.html#%E5%AE%89%E8%A3%85CPU: Kunpeng-920OS: openEuler 22.03 (LTS-SP4) aarch64Fuyao Version: v26.03 rc3docker: 2:18.09.0-346.oe2403sp3使用 openEuler 22.03 (LTS-SP4) aarch64 cloud 镜像全新安装、扩容硬盘后部署。[root@localhost fuyao-26.3-rc3]# ./bkeadm_linux_arm64 init --otherRepo cr.openfuyao.cn/openfuyao/bke-online-installed:latest --hostIP: 10.17.30.131 --domain: deploy.bocloud.k8s --kubernetesPort: 36443 --imageRepoPort: 40443 --yumRepoPort: 40080 --chartRepoPort: 38080 --ntpServer: cn.pool.ntp.org:123 --runtime: containerd --runtimeStorage: /var/lib/containerd --clusterAPI: 1.2.1 --oFVersion: v26.03-rc.3 --versionUrl: https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/version-config/ --enableNTP: true --agentHealthPort: 58080 --otherRepo: cr.openfuyao.cn/openfuyao/bke-online-installed:latest Confirm the parameters, press Y to continue N will exit. [Y/N]? y [bke][2026-03-27 02:57:16][INFO] BKE initialize ... [bke][2026-03-27 02:57:16][INFO] HOSTNAME: localhost [bke][2026-03-27 02:57:16][INFO] PLATFORM: openeuler [bke][2026-03-27 02:57:16][INFO] Version: 22.03 [bke][2026-03-27 02:57:16][INFO] KERNEL: 5.10.0-216.0.0.115.oe2203sp4.aarch64 [bke][2026-03-27 02:57:16][INFO] GOOS: linux [bke][2026-03-27 02:57:16][INFO] ARCH: arm64 [bke][2026-03-27 02:57:16][INFO] CPU: 8 [bke][2026-03-27 02:57:16][INFO] MEMORY: 16G [bke][2026-03-27 02:57:16][INFO] BKE Console: ENABLED [bke][2026-03-27 02:57:16][INFO] BKE initialize environment check... [bke][2026-03-27 02:57:16][WARN] Client authentication enabled but no credentials provided 2026-03-27T02:57:16.485Z info infrastructure/infrastructure.go:53 The docker client is ready. [bke][2026-03-27 02:57:16][INFO] set up the host machine zone [bke][2026-03-27 02:57:16][INFO] set ntp server Fri Mar 27 02:57:15 AM CST 2026 [bke][2026-03-26 18:57:15][INFO] config local source [bke][2026-03-26 18:57:15][INFO] 在线模式：domain：deploy.bocloud.k8s 绑定到默认IP：10.17.30.131 [bke][2026-03-26 18:57:49][INFO] iptables -V output: iptables v1.8.7 (legacy) [bke][2026-03-26 18:57:49][INFO] workDir /bke mod permission success [bke][2026-03-26 18:57:49][ERROR] Failed to start the container service, open /bke/mount/source_registry/files: no such file or directory最新版似乎修改了命令参数，使用最新参数正常。[root@stl-fuyao-2603 fuyao-26.3-rc3]# ./bkeadm_linux_arm64 init --onlineImage cr.openfuyao.cn/openfuyao/bke-online-installed:latest --hostIP: 10.17.30.131 --domain: deploy.bocloud.k8s --kubernetesPort: 36443 --imageRepoPort: 40443 --yumRepoPort: 40080 --chartRepoPort: 38080 --ntpServer: cn.pool.ntp.org:123 --runtime: containerd --runtimeStorage: /var/lib/containerd --clusterAPI: 1.2.1 --oFVersion: v26.03-rc.3 --versionUrl: https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/version-config/ --enableNTP: true --agentHealthPort: 58080 --onlineImage: cr.openfuyao.cn/openfuyao/bke-online-installed:latest Confirm the parameters, press Y to continue N will exit. [Y/N]? y [bke][2026-03-27 11:36:51][INFO] BKE initialize ... [bke][2026-03-27 11:36:51][INFO] HOSTNAME: stl-fuyao-2603 [bke][2026-03-27 11:36:51][INFO] PLATFORM: openeuler [bke][2026-03-27 11:36:51][INFO] Version: 22.03 [bke][2026-03-27 11:36:51][INFO] KERNEL: 5.10.0-216.0.0.115.oe2203sp4.aarch64 [bke][2026-03-27 11:36:51][INFO] GOOS: linux [bke][2026-03-27 11:36:51][INFO] ARCH: arm64 [bke][2026-03-27 11:36:51][INFO] CPU: 8 [bke][2026-03-27 11:36:51][INFO] MEMORY: 16G [bke][2026-03-27 11:36:51][INFO] BKE Console: ENABLED [bke][2026-03-27 11:36:51][INFO] BKE initialize environment check... [bke][2026-03-27 11:36:51][WARN] Client authentication enabled but no credentials provided [bke][2026-03-27 11:36:51][INFO] set up the host machine zone [bke][2026-03-27 11:36:51][INFO] set ntp server Fri Mar 27 11:36:51 AM CST 2026 [bke][2026-03-27 11:36:51][INFO] config local source [bke][2026-03-27 11:36:51][INFO] 在线模式：domain：deploy.bocloud.k8s 绑定到默认IP：10.17.30.131 [bke][2026-03-27 11:36:51][WARN] 无法解析私有仓库地址，跳过CA证书配置 [bke][2026-03-27 11:36:51][INFO] Using client certificate authentication(CA only) [bke][2026-03-27 11:36:51][INFO] Download source file... [bke][2026-03-27 11:36:53][INFO] Writing data... ... [bke][2026-03-27 13:48:19][INFO] containerd sandbox image: hub.oepkgs.net/openfuyao/pause:3.9 [bke][2026-03-27 13:48:19][INFO] Created offline special hosts.toml: /etc/containerd/certs.d/127.0.0.1:40443/hosts.toml [bke][2026-03-27 13:48:19][INFO] wait for containerd to start [bke][2026-03-27 13:48:19][INFO] Waiting for containerd to be ready ... 成功解压： /opt/cni/bin/vrf , 共处理了 4007138 个字符 [bke][2026-03-27 13:48:20][INFO] Start the base dependency service [bke][2026-03-27 13:48:21][INFO] Image hub.oepkgs.net/openfuyao/registry:2.8.1 is downloading [bke][2026-03-27 13:48:29][INFO] Wait for the container mirroring service to start... [bke][2026-03-27 13:48:34][INFO] The container mirroring service is started by containerd. [bke][2026-03-27 13:48:34][INFO] Image hub.oepkgs.net/openfuyao/nginx:1.23.0-alpine is downloading [bke][2026-03-27 13:48:43][INFO] Wait for the container yum service to start... [bke][2026-03-27 13:48:48][INFO] The container yum service is started. [bke][2026-03-27 13:48:48][INFO] Image hub.oepkgs.net/openfuyao/helm/chartmuseum:v0.16.2 is downloading [bke][2026-03-27 13:49:05][INFO] Wait for the chart mirroring service to start... [bke][2026-03-27 13:49:10][INFO] The chart mirroring service is started. [bke][2026-03-27 13:49:10][INFO] Image hub.oepkgs.net/openfuyao/openebs/nfs-server-alpine:0.9.0 is downloading [bke][2026-03-27 13:49:17][INFO] Wait for the nfs mirroring service to start... [bke][2026-03-27 13:49:22][INFO] The nfs mirroring service is started. [bke][2026-03-27 13:49:22][INFO] Image hub.oepkgs.net/openfuyao/rancher/k3s:v1.25.16-k3s4 is downloading 2026-03-27T13:50:07.914+0800 info k3s/k3s.go:314 params: onlineImage=cr.openfuyao.cn/openfuyao/bke-online-installed:latest otherRepo=, otherRepoIp=, hostIP=10.17.30.131, imageRepo=deploy.bocloud.k8s, imageRepoPort=40443, kubernetesPort=36443 [bke][2026-03-27 13:50:07][INFO] Start the local Kubernetes cluster... [bke][2026-03-27 13:50:10][ERROR] Failed to copy kubectl from the container [bke][2026-03-27 13:50:10][ERROR] Failed to start kubernetes exit status 1 [bke][2026-03-27 13:50:10][ERROR] Failed to start cluster API, exit status 1似乎启动失败，居然是因为 tar 没有安装？[root@stl-fuyao-2603 fuyao-26.3-rc3]# nerdctl cp kubernetes:/bin/k3s /tmp/test-k3s FATA[0000] unable to copy: failed to find `tar` binary 安装 tar 后解决，顺利部署。Npu OperatorNAMESPACE NAME READY STATUS RESTARTS AGE cluster-system bkeagent-deployer-h2flc 1/1 Running 0 65m ingress-nginx ingress-nginx-admission-create-xqwmf 0/1 Completed 0 62m ingress-nginx ingress-nginx-admission-patch-4dcjr 0/1 Completed 1 62m ingress-nginx ingress-nginx-controller-p5nrm 1/1 Running 0 62m kube-system ascend-device-plugin-xtfkb 0/1 CrashLoopBackOff 7 (4m34s ago) 16m kube-system calico-kube-controllers-6d75d78f5d-tfjvf 1/1 Running 0 65m kube-system calico-node-xtd99 1/1 Running 0 65m kube-system coredns-6c6fdbdb66-24cn4 1/1 Running 0 65m kube-system coredns-6c6fdbdb66-w29mt 1/1 Running 1 (63m ago) 65m kube-system etcd-master1 1/1 Running 0 64m kube-system kube-apiserver-master1 1/1 Running 0 55m kube-system kube-controller-manager-master1 1/1 Running 1 (55m ago) 65m kube-system kube-proxy-xxgz4 1/1 Running 0 65m kube-system kube-scheduler-master1 1/1 Running 1 (55m ago) 65m kube-system metrics-server-586f979f47-4b6fz 1/1 Running 0 54m mindx-dl ascend-operator-manager-5d4d89f675-tbjpv 0/1 Pending 0 15m mindx-dl clusterd-5588c5dc88-2fd5g 1/1 Running 0 16m mindx-dl resilience-controller-8686bbd76f-jv5lp 1/1 Running 0 16m monitoring alertmanager-main-0 2/2 Running 0 51m monitoring alertmanager-main-1 2/2 Running 0 51m monitoring alertmanager-main-2 2/2 Running 0 51m monitoring blackbox-exporter-6d6fbbfc96-cl6sg 3/3 Running 0 56m monitoring kube-state-metrics-677558db89-9rtbl 3/3 Running 0 56m monitoring node-exporter-4mzzl 2/2 Running 0 56m monitoring prometheus-k8s-0 2/2 Running 0 51m monitoring prometheus-k8s-1 2/2 Running 0 51m monitoring prometheus-operator-5cb64c846d-8m55t 2/2 Running 0 56m npu ascend-runtime-containerd-pg5q4 1/1 Running 0 16m npu npu-driver-66dqf 0/1 Init:0/1 0 103s npu npu-feature-discovery-rr68r 1/1 Running 0 17m npu npu-operator-5858d99c89-lgd24 1/1 Running 0 17m npu npu-operator-node-feature-discovery-gc-5cf8bc768d-6w4b4 1/1 Running 0 17m npu npu-operator-node-feature-discovery-master-5985b5cfcd-swdqs 1/1 Running 0 17m npu npu-operator-node-feature-discovery-worker-q5ct6 1/1 Running 0 17m openfuyao-system-controller modify-manifests-master1-sjxrv 0/1 Completed 0 55m openfuyao-system-controller openfuyao-system-controller-8444679b95-jv8jg 1/1 Running 0 65m openfuyao-system application-management-service-75799d4dd6-8vm9w 2/2 Running 0 55m openfuyao-system console-service-84bbd85575-grtwx 1/1 Running 0 55m openfuyao-system console-website-855c9d8f65-8btkl 1/1 Running 0 61m openfuyao-system local-harbor-chartmuseum-7f96745849-d7vzk 1/1 Running 0 56m openfuyao-system local-harbor-core-5f847798b8-khkf4 1/1 Running 1 (51m ago) 56m openfuyao-system local-harbor-database-0 1/1 Running 0 56m openfuyao-system local-harbor-jobservice-7d67f4f887-svklq 1/1 Running 3 (47m ago) 56m openfuyao-system local-harbor-nginx-6449749746-pbzkc 1/1 Running 0 56m openfuyao-system local-harbor-portal-78bf65c9-rqmk6 1/1 Running 0 56m openfuyao-system local-harbor-redis-0 1/1 Running 0 56m openfuyao-system local-harbor-registry-65884895bf-ff2wd 2/2 Running 0 56m openfuyao-system marketplace-service-5c79cbcbfc-swg9r 2/2 Running 0 55m openfuyao-system monitoring-service-79fc57c6b4-47vbm 2/2 Running 0 56m openfuyao-system oauth-server-68b6655d95-c8pzx 1/1 Running 0 54m openfuyao-system oauth-webhook-6995d46758-g47xn 1/1 Running 0 54m openfuyao-system plugin-management-service-84bfcd6565-2bmz2 2/2 Running 0 54m openfuyao-system user-management-operator-8d79bd8b8-jr4sk 1/1 Running 0 54m openfuyao-system web-terminal-service-6d858d974-hpjw6 1/1 Running 0 54m volcano-system volcano-controllers-6ffb787f8d-mxxph 1/1 Running 0 16m volcano-system volcano-scheduler-867f9784bb-vsswv 0/1 Pending 0 16m kube-system ascend-device-plugin-sbxr7 0/1 CrashLoopBackOff 6 (100s ago) 9m57s[root@master1 ~]# kubectl -n kube-system logs ascend-device-plugin-sbxr7 Defaulted container "device-plugin-01" out of: device-plugin-01, init-permission (init) [INFO] 2026/03/27 07:27:47.189809 1 hwlog/api.go:108 devicePlugin.log's logger init success [INFO] 2026/03/27 07:27:47.190348 1 main.go:187 ascend device plugin starting and the version is v6.0.0_linux-aarch64 [INFO] 2026/03/27 07:27:47.190416 1 main.go:188 ascend device plugin starting scene is center 2026/03/27 07:27:47 command exec failed, exit status 1 [ERROR] 2026/03/27 07:27:47.192892 1 devmanager/devmanager.go:95 deviceManager init failed, prepare dcmi failed, err: cannot found valid driver lib, fromEnv: lib path is invalid, [], fromLdCmd: can't find valid lib: EOF [ERROR] 2026/03/27 07:27:47.192970 1 main.go:203 init devmanager failed, err: auto init failed, err: get chip info failed, err: device Manager is nil, may encounter an exception during initialization. You can check the system log to confirm驱动安装失败：[root@master1 ~]# kubectl -n npu logs -f npu-driver-lxv5c -c npu-driver-installer Checking if /mnt/usr/local/sbin/npu-smi exists... master1 is not an option, please use -h to view help [2026-03-27 15:55:00, [INFO] No operation specified, default install operation on node: -- [2026-03-27 15:55:00, [INFO] install npu-driver [2026-03-27 15:55:00, [INFO] copy npu-install to host [2026-03-27 15:55:00, [INFO] copy npu-install to host success [2026-03-27 15:55:00,303677070] [INFO] Install dependency packages [2026-03-27 15:55:00,305026980] [INFO] Using yum for package installation 5 files removed repo 4.3 MB/s | 33 kB 00:00 Metadata cache created. [2026-03-27 15:55:01,143913450] [INFO] jq is already installed, skipping... [2026-03-27 15:55:01,150749780] [INFO] wget is already installed, skipping... [2026-03-27 15:55:01,156894190] [INFO] Installing unzip... No match for argument: unzip Error: Unable to find a match: unzip [2026-03-27 15:55:01,599984660] [FATAL] Failed to install unzip using yum [2026-03-27 15:55:01, [FATAL] install failed: -- 因为主机缺少 unizp 导致失败，但是该容器没有任何异常，无法感知错误。从清单可以看到 ascend-device-plugin-sbxr7 容器一直由于找不到驱动异常退出，而实际原因是 npu-driver-66dqf pod 没有正常安装驱动，但没有退出，反而正常运行。模糊了实际错误点。安装 unzip 后正常：[root@master1 ~]# kubectl -n npu logs -f npu-driver-66dqf -c npu-driver-installer Checking if /mnt/usr/local/sbin/npu-smi exists... master1 is not an option, please use -h to view help [2026-03-27 15:57:58, [INFO] No operation specified, default install operation on node: -- [2026-03-27 15:57:58, [INFO] install npu-driver [2026-03-27 15:57:58, [INFO] copy npu-install to host [2026-03-27 15:57:58, [INFO] copy npu-install to host success [2026-03-27 15:57:58,069791170] [INFO] Install dependency packages [2026-03-27 15:57:58,070929430] [INFO] Using yum for package installation 48 files removed repo 5.1 MB/s | 33 kB 00:00 OS 67 MB/s | 3.3 MB 00:00 everything 76 MB/s | 17 MB 00:00 EPOL 65 MB/s | 4.7 MB 00:00 debuginfo 69 MB/s | 3.9 MB 00:00 source 59 MB/s | 1.8 MB 00:00 update 79 MB/s | 71 MB 00:00 可以看到花费了大量时间在 init 阶段，这样才是正常的：npu npu-driver-66dqf 0/1 Init:0/1 0 3m51s虚拟机运行似乎会有这个错误kubectl -n kube-system delete pod ascend-device-plugin-xtfkb [Driver] [2026-03-27 16:03:40] [INFO]upgradePercentage:100% [Driver] [2026-03-27 16:03:42] [INFO]Driver package installed successfully! The new version takes effect immediately. [Driver] [2026-03-27 16:03:42] [INFO]End time: 2026-03-27 16:03:42 [Firmware] [2026-03-27 16:03:43] [INFO]Start time: 2026-03-27 16:03:43 [Firmware] [2026-03-27 16:03:43] [INFO]LogFile: /var/log/ascend_seclog/ascend_install.log [Firmware] [2026-03-27 16:03:43] [INFO]OperationLogFile: /var/log/ascend_seclog/operation.log [Firmware] [2026-03-27 16:03:43] [WARNING]Do not power off or restart the system during the installation/upgrade [Firmware] [2026-03-27 16:03:43] [ERROR]Not a physical-machine, firmware upgrade does not support. [Firmware] [2026-03-27 16:03:43] [INFO]End time: 2026-03-27 16:03:43 但驱动等已经部署完毕：[root@master1 ~]# npu-smi info +--------------------------------------------------------------------------------------------------------+ | npu-smi 24.1.rc3 Version: 24.1.rc3 | +-------------------------------+-----------------+------------------------------------------------------+ | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) | | Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) | +===============================+=================+======================================================+ | 176 310P3 | OK | NA 56 0 / 0 | | 0 0 | 0000:00:16.0 | 0 1838 / 21527 | +===============================+=================+======================================================+ +-------------------------------+-----------------+------------------------------------------------------+ | NPU Chip | Process id | Process name | Process memory(MB) | +===============================+=================+======================================================+ | No running processes found in NPU 176 | +===============================+=================+======================================================+但是这个容器依然无法正常运行：[root@master1 ~]# kubectl -n kube-system logs -f -l name=ascend-device-plugin-ds -c device-plugin-01 [INFO] 2026/03/27 08:19:41.677826 1 hwlog/api.go:108 devicePlugin.log's logger init success [INFO] 2026/03/27 08:19:41.678051 1 main.go:187 ascend device plugin starting and the version is v6.0.0_linux-aarch64 [INFO] 2026/03/27 08:19:41.678116 1 main.go:188 ascend device plugin starting scene is center [INFO] 2026/03/27 08:19:41.900653 1 devmanager/devmanager.go:104 the dcmi version is 24.1.rc3 [ERROR] 2026/03/27 08:19:41.900744 1 devmanager/devmanager.go:211 get error card quantity: 0 [ERROR] 2026/03/27 08:19:41.900780 1 devmanager/devmanager.go:195 get card list failed for init [ERROR] 2026/03/27 08:19:41.900828 1 main.go:203 init devmanager failed, err: auto init failed, err: get card list failed for init部署后防火墙问题另外发现 openEuler iptables 默认，还是 openFuyao 默认配置，iptables 默认 FORWARD 为 Drop，会导致部署后无法访问。[root@stl-fuyao-2603 ~]# iptables -L -n Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy DROP) target prot opt source destination CNI-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0 /* CNI firewall plugin rules (ingressPolicy: same-bridge) */ CNI-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* CNI firewall plugin rules */ Chain OUTPUT (policy ACCEPT) target prot opt source destination 需 配置解决iptables -P FORWARD ACCEPT部署时软件源问题[root@master1 ~]# cp /etc/yum.repos.d/b bak/ bke.repo默认移除系统自带软件源，部署后无法再安装软件，需自行配置。能否通过更加灵活的方式，如配置优先级的方式来规避直接移除软件源。压缩工具检查建议参考这种：root@hosthatch-us1:~# sudo -v ; curl https://rclone.org/install.sh | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4734 100 4734 0 0 8458 0 --:--:-- --:--:-- --:--:-- 8468 None of the supported tools for extracting zip archives (unzip 7z busybox) were found. Please install one of them and try again.带有明确错误说明。