Incident History

During a recent configuration of additional DDoS mitigation measures, our engineering team inadvertently caused a brief interruption in access to our Agent Portal. The change has since been reversed, and full access should now be restored. We apologize for any inconvenience this may have caused.

Our engineers identified the issue and have restored the service

Our engineers our working to restore the Zingtree service. More updates to follow

Our engineers have isolated the issue and service has been restored

Our engineers our working to restore the Zingtree service. More updates to follow

Zingtree engineers have isolated the issues to DNS resolution and have implement a fix. The issue appears to be resolved but we will continue to monitor.

Observing intermittent latency issues. We are currently investigating the cause.

We are currently investigating reports of latency when accessing the Zingtree application.

Investigation complete. Site back up. Down for about 5 minutes.

Issue is now fully resolved and emails are flowing.

Emails are starting to flow and we are monitoring the recovery. We are still waiting for confirmation from Mailchimp/Mandrill that this issue is fully resolved

Per Mailchimp/Mandrill:

Our Engineers are still working on a solution for the connection issues affecting Mailchimp Transactional. Thanks for sticking with us.

https://twitter.com/MailchimpStatus

Per Mailchimp/Mandrill:

Connection issues for Mailchimp Transactional are continuing to be investigated by our Engineers. We greatly appreciate your continued patience.

@MailchimpStatus

The Zingtree technical team has identified an issue sending transactional emails from the system. This issue is caused by an outage reported by our vendor Mandrill. We will continue to monitor the situation and work with the vendor to resolve the issue. Thanks for your patience.

The issue is now resolved.

We have identified the root cause and are working to remediate it.

We are currently investigating reports of latency when accessing the Zingtree application.

As per the AWS status page (https://health.aws.amazon.com/health/status), their systems are coming up slowly. Our internal validations show all our systems are operating normally. At this point, we consider this issue has been resolved but we will continue monitoring.

AWS is having issues in the US-EAST-2 Region. https://health.aws.amazon.com/health/status.

This is causing a degradation of our services at ph.zingtree.com. All other services at www.zingtree.com should be available without interruption.

We will be monitoring AWS statuses closely and update the status.

DB Upgrade complete

DB Upgrade in progress

Database Upgrade In progress

On Saturday, 5/14/2022 at 11:59 PM EST we will perform some database upgrades to keep Zingtree reliable and fast. During this upgrade window zingtree.com and related services will be unavailable.