The Post Erasure Event, as it has now been named, was a catastrophic data loss event on the Cracked Egg Studios web site that took place on or about May 4, 2009, during testing of the upgrade process for VaultWiki 2.3.0 that was in development.
According to public development notes, the new upgrade process was developed from April 27-May 4, 2009, with a test upgrade being completed by 2 AM on May 4, 2009. Within that upgrade process, the newest script for the upgrade to 2.3.0 included a database query designed to synchronize the content of wiki pages, which VaultWiki 2.x stored as forum posts, against the current revision of those pages. However, the query had a flaw that resulted in all wiki pages that had no revision history, as well as all posts that were not even wiki pages, having their content replaced with empty string text, making those posts blank.
The issue was not discovered until several days later, when developers began to notice posts that had no content. This was most likely May 7, based on an early AM development note on May 8 by Ted Phillips:
The event prompted administrators to attempt to restore the web site against the last database backup, but they discovered that automated database backups had been failing for over a year, so there was no usable backup available.
Instead, administrators attempted to reconstruct the affected posts using various cache entries, which allowed them to save the content of many posts. As a result, only the original content of the following content was irreparably lost, many of which happened to be Card of the Day-related wiki pages:
According to public development notes, the new upgrade process was developed from April 27-May 4, 2009, with a test upgrade being completed by 2 AM on May 4, 2009. Within that upgrade process, the newest script for the upgrade to 2.3.0 included a database query designed to synchronize the content of wiki pages, which VaultWiki 2.x stored as forum posts, against the current revision of those pages. However, the query had a flaw that resulted in all wiki pages that had no revision history, as well as all posts that were not even wiki pages, having their content replaced with empty string text, making those posts blank.
The issue was not discovered until several days later, when developers began to notice posts that had no content. This was most likely May 7, based on an early AM development note on May 8 by Ted Phillips:
The event prompted administrators to attempt to restore the web site against the last database backup, but they discovered that automated database backups had been failing for over a year, so there was no usable backup available.
Instead, administrators attempted to reconstruct the affected posts using various cache entries, which allowed them to save the content of many posts. As a result, only the original content of the following content was irreparably lost, many of which happened to be Card of the Day-related wiki pages: