[DmtpRpc]观察到一个高并发的接口出现超时异常，设定超时5秒，实际过了35秒才报错，再过16秒还调用成功了

### 组件

Dmtp-Http

### 版本号

3.1.18

### .NET SDK

.NET9

### 项目类型

WebApi

### 操作系统

Ubuntu

### 运行环境

生产环境 (Production)

### 这个问题是否已经存在？

- [x] 我已经搜索过现有的问题 (https://gitee.com/RRQM_Home/TouchSocket/issues)

### 如何复现

因为是偶发，概率极低（服务器高负载时，4000次调用期间触发8次），正常时每小时20w次调用无异常，就是普通的WaitSend调用，Demo无法重现。
这种问题打扰作者很抱歉，求助提供一些方向，有没有什么参数调用让我看到当时待处理的，我再添加日志等待观察结果。

### 预期结果

求指点，求检查方向，我应该观察哪些日志参数，上传到生产环境，我在进一步观察日志排查问题。

### 实际结果

流程如下：
```
00:00:00 Client1 以FeedbackType.WaitSend调用Service1，设定超时5秒
00:00:35 Client1提示调用超时：
	System.TimeoutException: The operation has timed out.
	   at TouchSocket.Dmtp.Rpc.DmtpRpcActor.ThrowExceptionIfNotSetRunning(WaitDataStatus status) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Dmtp\Features\Rpc\Actor\DmtpRpcActor.cs:line 229
	   at TouchSocket.Dmtp.Rpc.DmtpRpcActor.InvokeAsync(String invokeKey, Type returnType, IInvokeOption invokeOption, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Dmtp\Features\Rpc\Actor\DmtpRpcActor.cs:line 433
	   at TouchSocket.Rpc.RpcClientExtension.InvokeTAsync[T](IRpcClient client, String invokeKey, IInvokeOption invokeOption, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Rpc\Extensions\RpcClientExtension.cs:line 43
	   at Client.ActiveWindowAsync(Byte[] data) in /src/Server/Client.cs:line 154
	   at CCServer.TcpDmtp.Connection.ClientBridgeRpcServer.ClientBridgeLimit(Byte[] rpcData) in /src/Server/CCServer.TcpDmtp/Connection/RpcServices/ClientBridgeRpcServer.cs:line 25
	   at TouchSocket.Core.DynamicMethodInfoBase.GetResultAsync(Object result) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Core\Reflection\DynamicMethodInfoBase.cs:line 64
	   at TouchSocket.Core.Method.InvokeAsync(Object instance, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Core\Reflection\Method.cs:line 202
	   at TouchSocket.Rpc.InternalRpcServerProvider.ExecuteAsync(ICallContext callContext, InvokeResult invokeResult) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Rpc\RpcServerProvider\InternalRpcServerProvider.cs:line 60
00:00:51 Service1收到请求并且执行完成
```
1分钟内，接口调用了4000次，出现超时8次，概率较低，但问题比较奇怪。
于此同时，也有Servie1调用Client1的超时情况，方法调用陈功，Client1的方法也执行完成，在返回结果的地方也明确打印日志执行完成了，但是Service1迟迟没有收到结果，一直到超时。
流程如下：
00:00:00 Service1调用Client1，设定超时10秒
00:00:01 Client1收到请求，开始执行业务
00:00:02 Client2执行完成，返回结果，日志打印在return前一行，确保所有业务都执行完成
00:00:10 Service1迟迟没有收到结果，提示超时
……
感觉这个问题和上面的类似，Client1自己执行也无没问题，接受外部访问也没问题，但是将数据传给Service1就异常了。
想知道什么情况会出现这种，我应该从哪方面排查问题。能否提点一下，出现这个问题，我可能需要打印哪些日志出来看看。
当时服务器出现高负荷，但服务器资源消耗没有到达上限（内存、CPU、带宽都低于60%），出现了各种调用超时。
所有接口的调用也没有明显异常（还没找到。。），只看到了这2个错误，就想从这里入手看看，求指点，求检查方向。
通过内存转储，发现ProtectedDefaultSendAsync堆积数据，是否于此有关？有什么参数，检测到Socket堵了就直接丢弃数据，不要再拥堵进来了？
```
		AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Dmtp.DmtpActor+<SendAsync>d__105> <0x00007FC72FEA63C8> (检测到循环)
AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Dmtp.DmtpActor+<SendAsync>d__105> <0x00007FC72FEA63C8>
	AsyncTaskMethodBuilder+AsyncStateMachineBox<Object, TouchSocket.Dmtp.Rpc.DmtpRpcActor+<InvokeAsync>d__29> <0x00007FC72FEA6468>
	AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Sockets.TcpClientBase+<ProtectedDefaultSendAsync>d__61> <0x00007FC72FEA6338>
		AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Sockets.TcpCore+<SendAsync>d__56> <0x00007FC72FEA62B8>
			AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Sockets.TcpClientBase+<ProtectedDefaultSendAsync>d__61> <0x00007FC72FEA6338> (检测到循环)
			SemaphoreSlim+TaskNode <0x00007FC72FEA6260>
				SemaphoreSlim+TaskNode <0x00007FC72FEBA5B0>
					SemaphoreSlim+TaskNode <0x00007FC72FEBC748>
						SemaphoreSlim+TaskNode <0x00007FC72FECFFE0>
							SemaphoreSlim+TaskNode <0x00007FC72FED1B68>
								SemaphoreSlim+TaskNode <0x00007FC72FEE5AB0>
									SemaphoreSlim+TaskNode <0x00007FC72FEF9968>
										SemaphoreSlim+TaskNode <0x00007FC72FF0D8A0>
											SemaphoreSlim+TaskNode <0x00007FC72FF218E8>
												SemaphoreSlim+TaskNode <0x00007FC72FF36090>
													SemaphoreSlim+TaskNode <0x00007FC72FF49788>
														SemaphoreSlim+TaskNode <0x00007FC72FF5D530>
															SemaphoreSlim+TaskNode <0x00007FC72FF719C0>
																SemaphoreSlim+TaskNode <0x00007FC72FF73218>
																	SemaphoreSlim+TaskNode <0x00007FC72FF74978>
																		SemaphoreSlim+TaskNode <0x00007FC72FF88788>
																			SemaphoreSlim+TaskNode <0x00007FC72FF9E4F0>
																				SemaphoreSlim+TaskNode <0x00007FC72FFB2BF8>
																					SemaphoreSlim+TaskNode <0x00007FC72FFC6718>
																						SemaphoreSlim+TaskNode <0x00007FC72FFC84E8>
																							SemaphoreSlim+TaskNode <0x00007FC72FFCB278>
																								SemaphoreSlim+TaskNode <0x00007FC72FFDEC48>
																									SemaphoreSlim+TaskNode <0x00007FC72FFE3240>
																										SemaphoreSlim+TaskNode <0x00007FC72FFF8D58>
																											SemaphoreSlim+TaskNode <0x00007FC72FFF9BC8>
																												SemaphoreSlim+TaskNode <0x00007FC72FFFB500>
																													SemaphoreSlim+TaskNode <0x00007FC72FFFCF28>
																														SemaphoreSlim+TaskNode <0x00007FC72FFFEA50>
																															SemaphoreSlim+TaskNode <0x00007FC7300001D0>
																																SemaphoreSlim+TaskNode <0x00007FC730002BD8>
																																	SemaphoreSlim+TaskNode <0x00007FC730016D68>
																																		SemaphoreSlim+TaskNode <0x00007FC73002C810>
																																			SemaphoreSlim+TaskNode <0x00007FC73002D548>
																																				SemaphoreSlim+TaskNode <0x00007FC73002F2D0>
																																					SemaphoreSlim+TaskNode <0x00007FC730030688>
																																						SemaphoreSlim+TaskNode <0x00007FC730043D58>
																																							SemaphoreSlim+TaskNode <0x00007FC730045870>
																																								SemaphoreSlim+TaskNode <0x00007FC730059808>
																																									SemaphoreSlim+TaskNode <0x00007FC73006A8D8>
																																										SemaphoreSlim+TaskNode <0x00007FC73007A340>
																																											SemaphoreSlim+TaskNode <0x00007FC73007BF38>
																																												SemaphoreSlim+TaskNode <0x00007FC73007DAB0>
																																													SemaphoreSlim+TaskNode <0x00007FC73008D7C8>
																																														SemaphoreSlim+TaskNode <0x00007FC73009D280>
																																															SemaphoreSlim+TaskNode <0x00007FC73009EC40>
																																																SemaphoreSlim+TaskNode <0x00007FC7300A07B8>
																																																	SemaphoreSlim+TaskNode <0x00007FC7300B0438>
																																																		SemaphoreSlim+TaskNode <0x00007FC7300B0E68>
																																																			SemaphoreSlim+TaskNode <0x00007FC7300BFE30>
																																																				SemaphoreSlim+TaskNode <0x00007FC7300CE9A8>
																																																					SemaphoreSlim+TaskNode <0x00007FC7300DD520>
																																																						SemaphoreSlim+TaskNode <0x00007FC7300ED188>
																																																							SemaphoreSlim+TaskNode <0x00007FC7300FD2C0>
																																																								SemaphoreSlim+TaskNode <0x00007FC73010D0F8>
																																																									SemaphoreSlim+TaskNode <0x00007FC73010E848>
																																																										SemaphoreSlim+TaskNode <0x00007FC73011E950>
																																																											SemaphoreSlim+TaskNode <0x00007FC730120650>
																																																												SemaphoreSlim+TaskNode <0x00007FC73012FA98>
																																																													SemaphoreSlim+TaskNode <0x00007FC73013F920>
																																																														SemaphoreSlim+TaskNode <0x00007FC73014F5E8>
																																																															SemaphoreSlim+TaskNode <0x00007FC730150F18>
																																																																SemaphoreSlim+TaskNode <0x00007FC730160358>
																																																																	SemaphoreSlim+TaskNode <0x00007FC73016FE98>
																																																																		SemaphoreSlim+TaskNode <0x00007FC730185D20>
																																																																			SemaphoreSlim+TaskNode <0x00007FC730187110>
																																																																				SemaphoreSlim+TaskNode <0x00007FC73019D130>
																																																																					SemaphoreSlim+TaskNode <0x00007FC7301B2F88>
																																																																						SemaphoreSlim+TaskNode <0x00007FC7301C1E48>
																																																																							SemaphoreSlim+TaskNode <0x00007FC7301D1F18>
																																																																								SemaphoreSlim+TaskNode <0x00007FC7301E0ED0>
																																																																									SemaphoreSlim+TaskNode <0x00007FC7301E3010>
																																																																										SemaphoreSlim+TaskNode <0x00007FC7301F2978>
																																																																											SemaphoreSlim+TaskNode <0x00007FC7302021C0>
																																																																												SemaphoreSlim+TaskNode <0x00007FC730211A10>
																																																																													SemaphoreSlim+TaskNode <0x00007FC730221738>
																																																																														SemaphoreSlim+TaskNode <0x00007FC730230F68>
																																																																															SemaphoreSlim+TaskNode <0x00007FC730232D70>
																																																																																SemaphoreSlim+TaskNode <0x00007FC730241D70>
																																																																																	AsyncTaskMethodBuilder+AsyncStateMachineBox<VoidTaskResult, TouchSocket.Sockets.TcpCore+<SendAsync>d__56> <0x00007FC730241DC8>
																																																																																	SemaphoreSlim+TaskNode <0x00007FC730232D70> (检测到循环)
																																																																																	SemaphoreSlim+TaskNode <0x00007FC712060850>
																																																																																		SemaphoreSlim+TaskNode <0x00007FC730241D70> (检测到循环)
																																																																																		SemaphoreSlim+TaskNode <0x00007FC72EF92AB8>
																																																																																			SemaphoreSlim+TaskNode <0x00007FC72EFA2398>
																																																																																				SemaphoreSlim+TaskNode <0x00007FC72EFB2500>
																																																																																					SemaphoreSlim+TaskNode <0x00007FC72EFC1ED0>
																																																																																						SemaphoreSlim+TaskNode <0x00007FC72EFD2020>
																																																																																							SemaphoreSlim+TaskNode <0x00007FC72EFD2B30>
																																																																																								SemaphoreSlim+TaskNode <0x00007FC7302703F0>
																																																																																									SemaphoreSlim+TaskNode <0x00007FC73029FF70>
																																																																																										SemaphoreSlim+TaskNode <0x00007FC7302BFA40>
																																																																																											SemaphoreSlim+TaskNode <0x00007FC7302D1AA0>
																																																																																												SemaphoreSlim+TaskNode <0x00007FC7303017B0>
																																																																																													...
																																																																																													...
																																																																																													SemaphoreSlim+TaskNode <0x00007FC712072568>
																																																																																														...
																																																																																														SemaphoreSlim+TaskNode <0x00007FC7120C67D8>
																																																																																															SemaphoreSlim+TaskNode <0x00007FC712132950>
																																																																																																SemaphoreSlim+TaskNode <0x00007FC712146428>
																																																																																																	SemaphoreSlim+TaskNode <0x00007FC7121C5FC8>
																																																																																																		SemaphoreSlim+TaskNode <0x00007FC7121C7668>
																																																																																																			SemaphoreSlim+TaskNode <0x00007FC712291D68>
																																																																																																				SemaphoreSlim+TaskNode <0x00007FC712367758>
																																																																																																					...
																																																																																																					...
																																																																																																					...
																																																																																																				...
																																																																																																				...
																																																																																																			...
																																																																																																			...
																																																																																																		...
																																																																																																		...
																																																																																																	...
																																																																																																	...
																																																																																																...
																																																																																																...
																																																																																															...
																																																																																															...
																																																																																														...
																																																																																												...
																																																																																												...
																																																																																											...
																																																																																											...
																																																																																										...
																																																																																										...
																																																																																									...
																																																																																									...
																																																																																								...
																																																																																								...
																																																																																							...
																																																																																							...
																																																																																						...
																																																																																						...
																																																																																					...
																																																																																					...
																																																																																				...
																																																																																				...
																																																																																			...
																																																																																			...
																																																																																		...
																																																																																...
																																																																																...
																																																																															...
																																																																															...
																																																																														...
																																																																														...
																																																																													...
																																																																													...
																																																																												...
																																																																												...
																																																																											...
																																																																											...
																																																																										...
																																																																										...
																																																																									...
																																																																									...
																																																																								...
																																																																								...
																																																																							...
																																																																							...
																																																																						...
																																																																						...
																																																																					...
																																																																					...
																																																																				...
																																																																				...
																																																																			...
																																																																			...
																																																																		...
																																																																		...
																																																																	...
																																																																	...
																																																																...
																																																																...
																																																															...
																																																															...
																																																														...
																																																														...
																																																													...
																																																													...
																																																												...
																																																												...
																																																											...
																																																											...
																																																										...
																																																										...
																																																									...
																																																									...
																																																								...
																																																								...
																																																							...
																																																							...
																																																						...
																																																						...
																																																					...
																																																					...
																																																				...
																																																				...
																																																			...
																																																			...
																																																		...
																																																		...
																																																	...
																																																	...
																																																...
																																																...
																																															...
																																															...
																																														...
																																														...
																																													...
																																													...
																																												...
																																												...
																																											...
																																											...
																																										...
																																										...
																																									...
																																									...
																																								...
																																								...
																																							...
																																							...
																																						...
																																						...
																																					...
																																					...
																																				...
																																				...
																																			...
																																			...
																																		...
																																		...
																																	...
																																	...
																																...
																																...
																															...
																															...
																														...
																														...
																													...
																													...
																												...
																												...
																											...
																											...
																										...
																										...
																									...
																									...
																								...
																								...
																							...
																							...
																						...
																						...
																					...
																					...
																				...
																				...
																			...
																			...
																		...
																		...
																	...
																	...
																...
																...
															...
															...
														...
														...
													...
													...
												...
												...
											...
											...
										...
										...
									...
									...
								...
								...
							...
							...
						...
						...
					...
					...
				...
				...

```

### 异常信息

System.TimeoutException: The operation has timed out.
	   at TouchSocket.Dmtp.Rpc.DmtpRpcActor.ThrowExceptionIfNotSetRunning(WaitDataStatus status) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Dmtp\Features\Rpc\Actor\DmtpRpcActor.cs:line 229
	   at TouchSocket.Dmtp.Rpc.DmtpRpcActor.InvokeAsync(String invokeKey, Type returnType, IInvokeOption invokeOption, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Dmtp\Features\Rpc\Actor\DmtpRpcActor.cs:line 433
	   at TouchSocket.Rpc.RpcClientExtension.InvokeTAsync[T](IRpcClient client, String invokeKey, IInvokeOption invokeOption, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Rpc\Extensions\RpcClientExtension.cs:line 43
	   at Client.ActiveWindowAsync(Byte[] data) in /src/Server/Client.cs:line 154
	   at CCServer.TcpDmtp.Connection.ClientBridgeRpcServer.ClientBridgeLimit(Byte[] rpcData) in /src/Server/CCServer.TcpDmtp/Connection/RpcServices/ClientBridgeRpcServer.cs:line 25
	   at TouchSocket.Core.DynamicMethodInfoBase.GetResultAsync(Object result) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Core\Reflection\DynamicMethodInfoBase.cs:line 64
	   at TouchSocket.Core.Method.InvokeAsync(Object instance, Object[] parameters) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Core\Reflection\Method.cs:line 202
	   at TouchSocket.Rpc.InternalRpcServerProvider.ExecuteAsync(ICallContext callContext, InvokeResult invokeResult) in D:\CodeOpen\TouchSocketPro\Src\TouchSocket.Rpc\RpcServerProvider\InternalRpcServerProvider.cs:line

### Demo 地址（仅限Git地址）

Demo无法重现

### 承诺支持

- [x] 我确定我已经对TouchSocket项目进行了“Star”操作。

### 承诺规范

- [x] 我确定已完整阅读[Issue提问规范](https://touchsocket.net/docs/current/troubleshootissue)，并按照要求填写。

### 承诺友好

- [x] 我承诺将本着相互尊重、理解和友善的态度进行交流，共同维护好 TouchSocket 来之不易的良好的社区氛围。

GVP 若汝棋茗/TouchSocket

内容风险标识

评论 (2)

GVP若汝棋茗/TouchSocket

内容风险标识