Issues with SignalR service that was dropping connections causing our server to hit critical hot code paths, which then caused thread contention and thread pool exhaustion.

Steps taken to resolve:

1. When a call is completed we previously use synchronous api calls to update that call record to communicate that it was hung up.  These API calls in cases would wait for a response which created performance issues during high volume hours.  Moving these API calls to async operations allows them to run in parallel and keep the application stable.
2. We optimized code paths to improve cpu utilization.
3. We added capacity to our real-time communication service, MS Azure SignalR, essentially scaling up the service to handle a higher volume of real-time connections and messages.
4. We made improvements to our database by adding or modifying four indexes which help speed up data retrieval for call specific tables.
5. We updated maintenance task on our DB which are crucial activities performed to optimize the performance of a database by organizing and refreshing index structures and statistics.
6. Deeper Monitoring, Tracing and Profiling tools implemented to help us identify root cause more efficiently, provide better visibility into our products and enable incident managers to respond before an issue has impact on important services.