并发超过临界点，Error req数量陡然增加

[youngstu]

您好，并发超过临界点，Error req数量陡然增加，什么情况下会导致Error req呢？帮忙解答一下，谢谢～

如下是Tracer日志：

DAGExecutor:
2021-11-04 16:02:40,743 	Query count[1000]
2021-11-04 16:02:40,743 	QPS[100.0 q/s]
2021-11-04 16:02:40,743 	Succ[0.33699999999999997]
2021-11-04 16:02:40,743 	Error req[4603, 4605, 4607, 4608, 4609, 4610, 4611, 4613, 4614, 4616, 4615, 4617, 4619, 4618, 4620, 4621, 4623, 4624, 4625, 4626, 4628, 4629, 4627, 4630, 4632, 4631, 4633, 4634, 4635, 4636, 4637, 4638, 4639, 4642, 4643, 4644, 4648, 4649, 4651, 4652, 4653, 4655, 4656, 4657, 4658, 4660, 4661, 4662, 4663, 4664, 4665, 4666, 4667, 4668, 4669, 4670, 4671, 4672, 4673, 4674, 4675, 4676, 4677, 4679, 4681, 4682, 4683, 4684, 4685, 4686, 4687, 4688, 4689, 4690, 4691, 4692, 4693, 4694, 4695, 4697, 4698, 4699, 4701, 4702, 4703, 4704, 4705, 4706, 4708, 4709, 4710, 4711, 4712, 4713, 4714, 4715, 4716, 4718, 4719, 4720, 4721, 4723, 4725, 4726, 4727, 4730, 4731, 4732, 4733, 4734, 4736, 4737, 4738, 4739, 4740, 4741, 4742, 4743, 4744, 4745, 4746, 4747, 4749, 4750, 4751, 4752, 4753, 4755, 4757, 4758, 4759, 4760, 4761, 4762, 4763, 4764, 4765, 4766, 4767, 4768, 4769, 4770, 4771, 4772, 4776, 4777, 4778, 4780, 4781, 4782, 4783, 4784, 4785, 4787, 4788, 4789, 4791, 4792, 4793, 4794, 4795, 4796, 4797, 4798, 4799, 4800, 4801, 4802, 4803, 4804, 4805, 4806, 4807, 4808, 4810, 4811, 4812, 4813, 4814, 4815, 4819, 4820, 4821, 4822, 4823, 4824, 4825, 4826, 4828, 4829, 4831, 4832, 4833, 4834, 4835, 4837, 4838, 4839, 4840, 4841, 4842, 4844, 4845, 4846, 4848, 4849, 4850, 4851, 4852, 4853, 4854, 4855, 4856, 4857, 4858, 4859, 4860, 4861, 4862, 4863, 4864, 4865, 4866, 4868, 4869, 4870, 4871, 4872, 4873, 4874, 4875, 4876, 4877, 4879, 4881, 4882, 4883, 4885, 4884, 4886, 4887, 4889, 4890, 4891, 4892, 4893, 4895, 4896, 4899, 4900, 4902, 4903, 4904, 4905, 4906, 4907, 4909, 4910, 4911, 4912, 4913, 4914, 4915, 4916, 4917, 4919, 4921, 4922, 4923, 4924, 4926, 4927, 4928, 4930, 4931, 4932, 4933, 4934, 4935, 4937, 4938, 4939, 4940, 4941, 4942, 4943, 4944, 4945, 4946, 4948, 4949, 4950, 4951, 4952, 4955, 4956, 4958, 4959, 4960, 4961, 4962, 4963, 4964, 4965, 4966, 4967, 4968, 4969, 4970, 4972, 4973, 4974, 4975, 4976, 4977, 4978, 4979, 4981, 4983, 4984, 4985, 4986, 4987, 4988, 4989, 4990, 4991, 4992, 4994, 4995, 4997, 4998, 4999, 5000, 5001, 5002, 5003, 5004, 5005, 5006, 5007, 5008, 5011, 5012, 5013, 5014, 5015, 5016, 5018, 5019, 5020, 5021, 5022, 5023, 5024, 5025, 5026, 5027, 5029, 5030, 5031, 5032, 5033, 5034, 5035, 5037, 5038, 5039, 5040, 5041, 5042, 5046, 5047, 5049, 5050, 5051, 5052, 5053, 5054, 5055, 5056, 5057, 5059, 5060, 5061, 5062, 5064, 5065, 5066, 5067, 5069, 5071, 5072, 5073, 5074, 5075, 5076, 5077, 5079, 5080, 5081, 5082, 5083, 5084, 5085, 5086, 5087, 5088, 5089, 5090, 5091, 5092, 5093, 5096, 5097, 5099, 5101, 5102, 5103, 5104, 5105, 5106, 5107, 5108, 5109, 5110, 5112, 5113, 5114, 5116, 5117, 5119, 5120, 5121, 5123, 5124, 5125, 5126, 5127, 5128, 5129, 5130, 5131, 5132, 5133, 5135, 5136, 5137, 5139, 5141, 5142, 5143, 5144, 5145, 5146, 5147, 5148, 5150, 5151, 5153, 5154, 5155, 5156, 5157, 5159, 5160, 5163, 5164, 5165, 5166, 5167, 5168, 5169, 5170, 5171, 5172, 5173, 5174, 5175, 5177, 5178, 5179, 5180, 5181, 5183, 5184, 5185, 5186, 5188, 5189, 5190, 5191, 5192, 5194, 5196, 5197, 5198, 5199, 5200, 5201, 5202, 5203, 5204, 5206, 5207, 5208, 5210, 5211, 5214, 5215, 5216, 5217, 5218, 5220, 5221, 5222, 5223, 5225, 5226, 5228, 5230, 5231, 5232, 5233, 5234, 5235, 5236, 5237, 5238, 5239, 5240, 5241, 5242, 5243, 5245, 5246, 5248, 5249, 5250, 5251, 5252, 5253, 5254, 5255, 5256, 5258, 5259, 5260, 5261, 5262, 5263, 5265, 5266, 5268, 5269, 5271, 5272, 5273, 5275, 5276, 5277, 5278, 5280, 5281, 5282, 5283, 5284, 5285, 5286, 5287, 5288, 5289, 5290, 5292, 5293, 5294, 5295, 5296, 5297, 5298, 5299, 5300, 5301, 5302, 5303, 5304, 5305, 5306, 5307, 5308, 5309, 5310, 5312, 5313, 5314, 5315, 5322, 5323, 5325, 5326, 5327, 5329, 5330, 5331, 5332, 5333, 5334, 5335, 5336, 5337, 5338, 5339, 5340, 5341, 5342, 5343, 5344, 5345, 5346, 5347, 5348, 5349, 5350, 5351, 5352, 5353, 5354, 5356, 5357, 5358, 5360, 5362, 5363, 5367, 5368, 5369, 5370, 5371, 5372, 5373, 5374, 5377, 5378, 5380, 5381, 5382, 5383, 5384, 5385, 5386, 5387, 5388, 5389, 5390, 5391, 5392, 5393, 5394, 5399, 5400, 5401, 5403, 5404]

[TeslaZhao]

并发超过100了？服务端请求超时时间是多少？ 再看下PipelineServingLogs/pipeline.log.wf 的错误信息

[youngstu]

并发超过100了？服务端请求超时时间是多少？ 再看下PipelineServingLogs/pipeline.log.wf 的错误信息

请求线程数是100情况下卡住了。服务端日志响应时间在300-400ms。pipeline.log.wf日志报错显示cannot start new thread.

[TeslaZhao]

client开太多了吧，client和server是在同一个机器上吗？

[youngstu]

client开太多了吧，client和server是在同一个机器上吗？

是的，但client也就100线程，应该问题不大。我再尝试把fe和be调用逻辑切换为threadpool 方案看看。