Incident Alerted
Incident Report for Olark Live Chat
Postmortem

We know you rely on Olark to help support your customers and we take that responsibility very seriously and we apologize for the outage on November 14th. Whenever we don’t live up to our end of the agreement, it erodes trust and that takes time to rebuild. Over the past year, we have been working hard to improve stability and make our infrastructure more resilient to unforeseen issues.

On November 14th, 2019 we shipped a minor code change to production which causes our software to reconnect to our database provider. Unknown to us, there was a change made on our provider’s side which made our database connections fail, rendering our login and website as unusable. Our engineers immediately began investigating the issue. As soon as we identified the root cause, we immediately began making changes to work around the database connection issue. Login and our website came back up an hour after they went down.

Even though this issue was ultimately caused by our service provider, it’s our responsibility to provide reliable service for you and your customers. We are working with our service provider to understand what happened in more detail and come up with ways to improve communication around changes impacting our services.

Posted Nov 25, 2019 - 11:54 EST

Resolved
This incident has been resolved.
Posted Nov 14, 2019 - 19:32 EST
Monitoring
We're back online. Looks like encountered a network issue but have now recovered. You should be able to log in to chat.olark.com and log in as normal. We'll continue to monitor.
Posted Nov 14, 2019 - 16:41 EST
Identified
Olark.com is offline currently and some users may have issues logging in to chat. We've identified the cause of this issue and are working to get things back online. We'll update as we have updates.
Posted Nov 14, 2019 - 15:53 EST
Investigating
We've detected an issue and are working to resolve this quickly. We'll have an update within the hour.
Posted Nov 14, 2019 - 15:41 EST
This incident affected: chat.olark.com.