GCP — облачные задачи — Grpc.Core.RpcException ⇐ C#

[Anonymous]

Периодически при создании облачной задачи на gcp я получаю это исключение.

Код: Выделить всё

Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. IOException: An HTTP/2 connection could not be established because the server did not complete the HTTP/2 handshake.  ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Security.SslStream'.", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.")
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: An HTTP/2 connection could not be established because the server did not complete the HTTP/2 handshake.
--->  System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Security.SslStream'.
at System.Net.Security.SslStream.g__ThrowExceptional|126_0(ExceptionDispatchInfo e)
at System.Net.Http.Http2Connection.SetupAsync(CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.Http2Connection.SetupAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConstructHttp2ConnectionAsync(Stream stream, HttpRequestMessage request, IPEndPoint remoteEndPoint, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConstructHttp2ConnectionAsync(Stream stream, HttpRequestMessage request, IPEndPoint remoteEndPoint, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(QueueItem queueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
--- End of inner exception stack trace ---
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.c__DisplayClass0_0`2.d.MoveNext()
--- End of stack trace from previous location ---

В этом методе

Код: Выделить всё

  public async System.Threading.Tasks.Task Publish(CloudTaskIntegrationEvent taskIntegrationEvent)
{
var parent = new QueueName(_projectId, _location, taskIntegrationEvent.Queue);
var baseUrl = taskIntegrationEvent.BaseUrl ?? _baseUrl;

var fullEndpointUrl = $"{baseUrl}/{taskIntegrationEvent.Endpoint}";

_logger.LogInformation("Full Endpoint Url for Cloud Task: {FullEndPointUrl}", fullEndpointUrl);

var httpRequest = new HttpRequest
{
HttpMethod = HttpMethod.Post,
Url = fullEndpointUrl,
Body = ByteString.CopyFromUtf8(taskIntegrationEvent.Payload),
Headers =
{
{"Content-Type", "application/json"}
}
};

if(_gcpServiceAccountId != null)
{
var oidcToken = new OidcToken
{
ServiceAccountEmail = _gcpServiceAccountId
};
httpRequest.OidcToken = oidcToken;
}

var task = new Google.Cloud.Tasks.V2.Task
{
HttpRequest = httpRequest
};

if (!string.IsNullOrWhiteSpace(taskIntegrationEvent.TaskName))
task.Name = $"{parent}/tasks/{taskIntegrationEvent.TaskName}";

try
{
_logger.LogInformation("Creating cloud task in {Queue} with payload {Payload}", taskIntegrationEvent.Queue, taskIntegrationEvent.Payload);

var request = new CreateTaskRequest
{
Parent = parent.ToString(),
Task = task
};

request.Task.ScheduleTime = Timestamp.FromDateTime(
DateTime.UtcNow.AddSeconds(taskIntegrationEvent.Delay + _defaultDelay));
var response = await _cloudTasksClient.CreateTaskAsync(request);
_logger.LogInformation("Task {Name} created", response.Name);
}
catch (Exception e)
{
_logger.LogError(e, "Error while creating could task for task |  {Queue}, {EventType}, {Payload}", taskIntegrationEvent.Queue, taskIntegrationEvent.EventType, taskIntegrationEvent.Payload);
}
}

Для отправки запроса я использую пакет nuget GCP — пакет «Google.Cloud.Tasks.V2» -v 3.5.0
Я в настоящее время использую .Net8.
Я отправляю сотни тысяч запросов, и это происходит примерно 100 раз в день.
У меня не было ни одного удачи в отладке.
Единственное, что я могу подумать о том, чтобы добавить повторную попытку к вызову. Однако мне интересно, есть ли способ полностью предотвратить проблему.
Метод с ошибкой выглядит так:

Подробнее здесь: https://stackoverflow.com/questions/791 ... cexception