Azure continuous integration testing fails with System.OperationCanceledException error - multithreading

Question

I have this netcore 3.1 C# Web application that we setup a batch of integration tests. The tests pass when run locally, but a few keep failing when run on the CI server with the System.OperationCanceledException error.
Here is log for the CI testing environment:
==============================================================================
Task : Visual Studio Test
Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version : 2.170.1
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : Z:\a\1\s
Action when minimum tests threshold not met : donothing
Minimum tests expected to be run: 0
VisualStudio version selected for test execution : 16.0
Attempting to find vstest.console from a visual studio installation with version [15.0,17.0).
Run in parallel : true
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : true
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Source filter: **\*Test.dll,**\*TestIntegration.dll,!**\*TestAdapter.dll,!**\obj\**,!**\submodules\**
SystemVssConnection exists true
[command]Z:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420cc\2.170.1\Modules\DTAExecutionHost.exe --inputFile D:\a\_temp\input_eedfb280-073a-11eb-bdb8-5118fbbc2831.json
======================================================
##########################################################################
DtaExecutionHost version 18.170.30112.1.
Starting TestExecution Model...
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Updated Run Settings:
<RunSettings>
<RunConfiguration>
<MaxCpuCount>0</MaxCpuCount>
<BatchSize>1000</BatchSize>
<ResultsDirectory>Z:\c\_temp\TestResults</ResultsDirectory>
</RunConfiguration>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="blame" enabled="True">
<Configuration>
<ResultsDirectory>Z:\c\_temp\TestResults</ResultsDirectory>
<CollectDump />
<CollectDump />
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<LoggerRunSettings>
<Loggers>
<Logger friendlyName="blame" enabled="True" />
</Loggers>
</LoggerRunSettings>
</RunSettings>
The raw log that shows the exception is as below:
Error Message:
System.OperationCanceledException : The operation was canceled.
Stack Trace:
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at System.IO.Pipelines.Pipe.ReadAsync(CancellationToken token)
at System.IO.Pipelines.Pipe.DefaultPipeReader.ReadAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.TestHost.ResponseBodyReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)