本文主要介绍使用云效在安装、主机部署接入中,常见的问题及解决方案。
主机代理问题 — 流水线 Runner
同一台 ECS 实例被添加到多个主机组导致部署失败或频繁断连怎么办?
同一台 ECS 实例被添加到多个主机组时,可能出现 Runner 离线、版本过旧或配置冲突,导致部署失败或 Runner 频繁断连。
快速恢复
-
在服务器上查看 Runner 服务名称:
ls -al /etc/systemd/system | grep runner -
重启对应的 Runner 服务:
systemctl restart runner-{版本}-{租户名称}.service
彻底解决
如果重启后问题持续,建议卸载旧版 Runner 并安装最新版本(如 0.3.4)。操作步骤:
-
停止 Runner 服务:
systemctl stop runner-{版本}-{租户名称}.service -
删除 systemd 配置文件:
rm -rf /etc/systemd/system/runner-{版本}-{租户名称}.service -
删除 Runner 配置目录:
rm -rf /root/yunxiao -
在云效控制台重新安装最新版 Runner 插件。
流水线 Runner 问题排查方法
在开始处理主机部署、私有构建机申请环境失败问题之前请先按以下步骤确认机器上云效runner状态。
使用工具排查
只支持Linux系统
-
下载工具:
wget "https://rdc-public-software.oss-cn-hangzhou.aliyuncs.com/runner/runnerStatusCheck" -O runnerStatusCheck -
设置执行权限:
chmod u+x runnerStatusCheck -
执行工具:
./runnerStatusCheck -
按照工具提示信息进行处理,示例如下。
[INFO] prepare to check disk check disk output Filesystem Size Used Avail Use% Mounted on /dev/root 99G 59G 36G 62% / devtmpfs 3.7G 0 3.7G 0% /dev tmpfs 3.7G 0 3.7G 0% /dev/shm tmpfs 3.7G 600K 3.7G 1% /run tmpfs 3.7G 0 3.7G 0% /sys/fs/cgroup tmpfs 748M 0 748M 0% /run/user/0 [INFO] prepare to check runner status RunnerService: runner-v0.0.7-be-1fy1ifhg1ylcvqv7rwinqpgh.service status is active running RunnerService: runner-v0.0.7-be-3n5goas4jkp3clmcaopzeebj.service status is active running RunnerService: runner-v0.0.7-be-6fcm3z72s2fsnrkcjl7pbpsa.service status is not running , please exeute 'systemctl restart runner-v0.0.7-be-6fcm3z72s2fsnrkcjl7pbpsa.service ' to restart runner service RunnerService: runner-v0.0.7-be-7srpetjhkcy40btoontpsy8a.service status is not running , please exeute 'systemctl restart runner-v0.0.7-be-7srpetjhkcy40btoontpsy8a.service ' to restart runner service RunnerService: runner-v0.0.7-be-7ypo76lza21lljapgp5xtopi.service status is active running RunnerService: runner-v0.0.7-be-9zouh99ta6ycx2dq3iggekk9.service status is not running , please exeute 'systemctl restart runner-v0.0.7-be-9zouh99ta6ycx2dq3iggekk9.service ' to restart runner service RunnerService: runner-v0.0.7-be-ao81clhgw56bo0jopj5jgwn3.service status is active running RunnerService: runner-v0.0.7-be-bn9wuh8n4kiba0aa0a8dcirb.service status is active running RunnerService: runner-v0.0.7-be-fsr2gvdefe5yxl3nyijp36d3.service status is active running RunnerService: runner-v0.0.7-be-haphfbrhuudbxkzwpcuhn0hk.service status is not running , please exeute 'systemctl restart runner-v0.0.7-be-haphfbrhuudbxkzwpcuhn0hk.service ' to restart runner service RunnerService: runner-v0.0.7-be-mwkcqcv2hbwozrwmmczopzgm.service status is active running RunnerService: runner-v0.0.7-be-qdiyjifmn0bbkafobtfpkrst.service status is not running , please exeute 'systemctl restart runner-v0.0.7-be-qdiyjifmn0bbkafobtfpkrst.service ' to restart runner service RunnerService: runner-v0.0.7-be-qx6gaxojjjpzeblor692hrwk.service status is active running RunnerService: runner-v0.0.7-be-s5f7tixziynazefazqhls9ob.service status is active running RunnerService: runner-v0.0.7-be-uwdlmn516wd8emqebhgqwq6x.service status is active running RunnerService: runner-v0.0.7-be-xjmxrlbzrtksuafv4oyg8qtf.service status is active running Check runner service status finished.
手动排查
-
检查系统 Linux 版本是否匹配。
通过命令
lsb_release -a查看 Linux 发行版本信息是否满足条件。目前云效统一 Runner 支持以下 Linux 发行版本:-
centos 6+ 及以上
-
ubuntu 16.04+ 及以上
-
Alibaba Cloud Linux 2/3
-
-
检查 Runner service 状态和日志。
通过命令
ls -al /etc/systemd/system | grep runner查看 Runner 服务名称,通常 Runner 服务的名称为 runner-{版本}-{租户名称}.service,如下所示。[root@ecs-for-batch-deploy-1 ~]# lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 8.2.2004 (Core) Release: 8.2.2004 Codename: Core [root@ecs-for-batch-deploy-1 ~]# ls -al /etc/systemd/system | grep runner -rw-r--r-- 1 root root 550 10月 11 10:50 runner-v0.xxx t.service [root@ecs-for-batch-deploy-1 ~]#通过命令
systemctl status runner-{版本}-{租户名称}.service查看 Runner 服务状态,如果服务状态为 active(running),则服务正常,如下所示。Release: 8.2.2004 Codename: Core [root@ecs-for-batch-deploy-1 ~]# ls -al /etc/systemd/system | gr -rw-r--r-- 1 root root 550 10月 11 10:50 runner-v0.0.5-be-bnyxxx xxx yt.service [root@ecs-for-batch-deploy-1 ~]# systemctl status runner-v0.0.5-be-bnyxxxpci0q6|06yt.service ● runner-v0.0.5-be-bnxxx yt.service - Aliyun yunxiao runner polling jo Loaded: loaded (/etc/systemd/system/runner-v0.xxx yt.service) Active: active (running) since Tue 2022-10-11 10.xxx ago Main PID: 1405243 (runner) Tasks: 10 (limit: 48875) Memory: 13.6M CGroup: /system.slice/runner-v0.0.5-be-brxxx06yt.service └─1405243 /usr/local/share/yunxiao-runner/v0.0.5/runner run --configPath=/r 10月 11 11:01:03 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:01:03+08:00] 10月 11 11:01:03 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:01:03+08:00] 10月 11 11:01:18 ecs-for-batch-deploy-1 runner[1405243]: WARN[2022-10-11T11:01:18+08:00]另外,你也可以通过命令
journalctl -u runner-{版本}-{租户名称}.service -a --no-pager --since '5 minutes ago' -f查看 Runner 执行日志。
构建/部署运行中,无日志或者主机状态显示为离线
-
检查主机磁盘是否满了
df -hl,如果磁盘已满请尝试清理后重试。 -
检查 Runner service 状态,如果状态非
active(running),通过以下步骤重启 Runner service:-
获取服务名称,Runner 的服务名称格式为:
runner-{版本}-{租户名称}.service。systemctl | grep "runner-v" | awk '{print $1}' -
使用获取到的服务名称替换以下命令中
$SERVICE_NAME并执行,重启 Runner service。systemctl restart $SERVICE_NAME
-
-
检查网络连通性是否正常。
-
通过
systemctl status runner-{版本}-{租户名称}.service查询 Runner service 状态,并截取 service 进程参数 --configPath=***,如下所示:[root@ecs-for-batch-deploy-1 ~]# systemctl status runner-v0.0.5-be-bnyxxx1llzpci0q606yt.service ● runner-v0.0.5-be-xxx_llzpci0q606yt.service - Aliyun yunxiao runner polling jobs to execute Loaded: loaded (/etc/systemd/system/runner-v0.0.5-be-bxxx pci0q606yt.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2022-10-11 10:50:56 CST; 41min ago Main PID: 1405243 (runner) Tasks: 10 (limit: 48875) Memory: 13.7M CGroup: /system.slice/runner-v0.0.5-be-bnyxxx q606yt.service └─1405243 /usr/local/share/yunxiao-runner/v0.0.5/runner run --configPath=/root/yunxiao/be-bnykv60t831llzpci0q606yt/runner/config 10月 11 11:31:46 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:31:46+08:00] [runner] no new job, skip. runner=9e7073c92b2fa8b35bc0efd3eec051f8 10月 11 11:31:46 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:31:46+08:00] counter current: 0, limit: 50 10月 11 11:32:01 ecs-for-batch-deploy-1 runner[1405243]: WARN[2022-10-11T11:32:01+08:00] Response status code: 204 body data: 10月 11 11:32:01 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:32:01+08:00] POST /api/v2/builds/request, time spent 15.07 s 10月 11 11:32:01 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:32:01+08:00] [runner] no new job, skip. runner=9e7073c92b2fa8b35bc0efd3eec051f8 10月 11 11:32:01 ecs-for-batch-deploy-1 runner[1405243]: INFO[2022-10-11T11:32:01+08:00] counter current: 0, limit: 50 10月 11 11:32:16 ecs-for-batch-deploy-1 runner[1405243]: WARN[2022-10-11T11:32:16+08:00] ... -
查看 configPath 中URL
cat {***}/config.yml | grep url。[root@ecs-for-batch-deploy-1 config]# cat /root/yunxiao/be-bnykv60t83lllzpci0q606yt/runner/config/config.yml | grep url url: https://xxx.com [root@ecs-for-batch-deploy-1 config]# -
通过以下命令判断是否可以正常访问。
# 注意替换下面命令中的url curl '{url}/api/v2/runner/storage/latest?os=linux&arch=amd64' -
跨地域部署失败排查
如果部署涉及跨地域场景(如从中国香港部署到马来西亚等),除了检查 Runner 状态和磁盘空间外,还需重点排查跨地域网络的连通性和稳定性。
在目标主机上测试到云效服务端的网络延迟与丢包率:
# 测试网络延迟 ping {云效服务端地址} # 测试网络连通性和丢包 mtr {云效服务端地址}如网络质量不佳,建议优化网络链路(如使用专线或加速服务),或联系网络管理员排查跨地域公网链路问题。
-
多节点部署时如何避免服务启动期间请求进入导致报错?
云效无法直接控制外部流量,建议采取以下措施实现部署期间的流量控制:
-
将主机分成两个主机组,在流水线中配置为顺序部署(先部署第一组,完成后再部署第二组),确保始终有可用节点对外提供服务。
-
结合网关层面的流量拦截或使用应用型负载均衡(ALB)配置转发规则,在部署期间将流量从待部署节点上摘除,待服务启动完成后再恢复转发,避免请求进入未完全启动的服务。