My phone rang.

It was around 10 o'clock, and no one calls this late.

It was my brother.

He told me the PosterBurner website wouldn't load.

No big deal, I thought.

Sometimes a service goes down and you click a few buttons to fix it.

I told him we'd be back online in 10 minutes.

But I was wrong.

And we were about to have a really bad day.

(If you missed the start of the PosterBurner story, it is here)

No matter what I tried, the server would not come back online.

And after a lot of back and forth with the hosting company, we learned that our hard drives were toast.

Back at the office we had nearly 1000 orders in progress.  

One of those orders was a big canvas order for the Emmy Awards.

And we only had four days to ship it in time.

And without the data on the hard drives, we wouldn't know what print gets shipped to what customer.

The support guy on the phone told me that we needed to use our backup drives.

I went to pull the backups.

But the backups weren't there.  

The service that does the backups had been failing for two straight months.  We were so busy I didn't notice.

And when I realized this, I looked like this:

The tech on the phone gave me two options.

We could wipe the data and be back online in an hour, or we could try to recover the data on the drives.  

But if we tried to recover the data, PosterBurner would be offline for 24 hours.

We didn't have a choice.

We needed the data so we could ship all the orders we had in progress and get the canvas prints to the Emmy Award party.

So they started emergency data recovery.

And the data recovery did not go smoothly.  I'll explain in the next post.

Rick @ PosterBurner