全部產品
Search
文件中心

MaxCompute:並發過高導致DNS解析失敗

更新時間:Feb 07, 2025

本文為您介紹大量並發請求訪問對端服務的網域名稱導致DNS解析失敗的解決方案。

報錯資訊

Logview中的報錯資訊包含name or service not known

問題描述

UDF或Spark任務在運行過程中,產生了大量的並發請求以訪問對端服務的網域名稱,導致DNS解析失敗。

解決方案

建議您在任務的初始化階段將網域名稱解析為IP地址,並在執行階段使用解析得到的IP地址進行訪問。樣本如下:

# -*- coding:UTF-8 -*-
from odps.udf import annotate

@annotate("string->string")
class test_udf(object):
    __ip_address = ''

    def evaluate(self, inputPath):
        import requests
        output = 'false'
        retries = 3
        print(self.__ip_address)
        url = f"http://{self.__ip_address}/{inputPath}"
        print(url)
        for i in range(retries):
            try:
                response = requests.get(url)
                if response.status_code == 200:
                    output = 'true'
                else:
                    raise
            except Exception as e:
                if i < retries:
                    print('connect retry: ' + str(i + 1))
                    print('error: ' + e.message)
                    continue
                else:
                    raise
            break

        return output

    def __init__(self):
        import socket
        retries = 3
        for i in range(retries):
            try:
                self.__ip_address = socket.gethostbyname("xxx-vpc.cn-shanghai.aliyuncs.com")
                print(self.__ip_address)
            except socket.gaierror as e:
                print('Failed to resolv domain' + e.strerror)
                if i < retries:
                    print('resolv domain retry: ' + str(i + 1))
                    continue
                else:
                    raise
            break