Double Counting in Analytics

Dear Valued Customer,

We wanted to inform you about an issue we recently identified and resolved to ensure transparency and maintain your trust in our platform.

Incident Summary

During the week of November 12th, we discovered an issue in our analytics where some metrics were double-counted. This occurred after transitioning our database from PostgreSQL to ClickHouse to support faster data processing.

What Happened

The switch to ClickHouse was made to enhance data processing speed for our large datasets, some of which contain close to half a billion rows. This transition significantly improved data ingestion and reduced processing time. However, during this update, an error was introduced into the way we calculated analytics metrics.

The issue stemmed from our transformation processes, where both pre-processed and post-processed data were included in the calculations, resulting in double-counted metrics in the analytics dashboards.

Importantly, this issue did not impact the accuracy or integrity of your raw data. Only the pre-calculated tables used for analytics were affected.

What We've Done

Once we identified the issue, we worked closely with ClickHouse support to resolve it. The problem has now been fixed, and the pre-calculated tables have been corrected. Your analytics dashboards are accurate once again.

Moving Forward

To prevent issues like this in the future, we’ve:

• Conducted a thorough review of our data transformation processes.

• Implemented additional safeguards to ensure data integrity during future updates.

• Strengthened our collaboration with ClickHouse to promptly address any unforeseen challenges.

We take the reliability of your analytics very seriously and are committed to continuous improvement to provide you with the best possible experience.

Thank you for your understanding and trust. If you have any questions or need further clarification, please don’t hesitate to reach out to our support team.

Sincerely,

The Because Team