本文為您介紹大量並發請求訪問對端服務的網域名稱導致DNS解析失敗的解決方案。
報錯資訊
Logview中的報錯資訊包含name or service not known。
問題描述
UDF或Spark任務在運行過程中,產生了大量的並發請求以訪問對端服務的網域名稱,導致DNS解析失敗。
解決方案
建議您在任務的初始化階段將網域名稱解析為IP地址,並在執行階段使用解析得到的IP地址進行訪問。樣本如下:
# -*- coding:UTF-8 -*-
from odps.udf import annotate
@annotate("string->string")
class test_udf(object):
__ip_address = ''
def evaluate(self, inputPath):
import requests
output = 'false'
retries = 3
print(self.__ip_address)
url = f"http://{self.__ip_address}/{inputPath}"
print(url)
for i in range(retries):
try:
response = requests.get(url)
if response.status_code == 200:
output = 'true'
else:
raise
except Exception as e:
if i < retries:
print('connect retry: ' + str(i + 1))
print('error: ' + e.message)
continue
else:
raise
break
return output
def __init__(self):
import socket
retries = 3
for i in range(retries):
try:
self.__ip_address = socket.gethostbyname("xxx-vpc.cn-shanghai.aliyuncs.com")
print(self.__ip_address)
except socket.gaierror as e:
print('Failed to resolv domain' + e.strerror)
if i < retries:
print('resolv domain retry: ' + str(i + 1))
continue
else:
raise
break