TheCompWiz
Well-known member
- Joined
- Apr 20, 2016
- Messages
- 142
- Points
- 18
Load average fine, memory fine, disk fine, uptime graph a flat green line, and a customer on the phone telling you the site is unusable. Had this often enough that I stopped treating the graphs as an answer.
Averages are the first thing that hides it. A mean response time looks healthy while the slowest five percent of requests are terrible, and if the customer's admin pages happen to live in that five percent then their experience of the site is the slow one. Logging per request duration and looking at the 95th percentile changed what I could see. Apache has a format token for microseconds, nginx has the request time variable, neither is on by default.
The other half is that a lot of it is genuinely not the server. Splitting the load into DNS, connect, TLS, time to first byte and download tells you which segment is eating the time, and often it is third party scripts on the page or a resolver on their end.
The part I have never done well is presenting this to a customer. Correct evidence delivered badly still sounds like deflection. How do you handle that conversation?
Averages are the first thing that hides it. A mean response time looks healthy while the slowest five percent of requests are terrible, and if the customer's admin pages happen to live in that five percent then their experience of the site is the slow one. Logging per request duration and looking at the 95th percentile changed what I could see. Apache has a format token for microseconds, nginx has the request time variable, neither is on by default.
The other half is that a lot of it is genuinely not the server. Splitting the load into DNS, connect, TLS, time to first byte and download tells you which segment is eating the time, and often it is third party scripts on the page or a resolver on their end.
The part I have never done well is presenting this to a customer. Correct evidence delivered badly still sounds like deflection. How do you handle that conversation?
