I watched the geo-location app for several minutes and it just didn’t look all that busy. If “it’s 5 o’clock somewhere”, it’s 8:30 am somewhere else … so it doesn’t look like anyone could have lost a serious amount of data.

My guess is that you lost next to nothing. At the very most, it would have taken you 32 minutes to re-transmit it.

By the time you read these comments, you could have been done.

You guys seriously need to get a grip on reality. If you’re instinct is to move away from FB for 30 minutes of downtown, I can’t imagine how you’re even still in business. Seriously… get a grip! I have had clients in the past for my own service I provide who wine and complain always looking for freebies and looking for any opportunity to complain. I can understand if it’s a regular occurrence or there was no support, but seriously… for this???? GET A GRIP!

This service is stable, reliable, affordable and obviously the owners are accountable.

I guess it’s a fair assumption that your own businesses are perfect????

And no, we’re not retyping the data, but we’re only copying over data for users that request it: finding out what the user needs, and exporting and importing just that data for that account.

surely wheather you are using Oracle or PostGres or even microsoft sql, you can build a script to identify the affected records then export those records to a compressed data file optionally automating an scp or other secureftp to send the file to the primary data centre.
then from the same terminal, log into the primary database server and reverse the script to import all the selected data.

thats what I call a manual data transfer or recovery.

are you refering to using the hand to move the fingers to retype all of the data?

unfortunately I know a lot about the challenges of remote collaborative databases and how putting it online brings extra challenges to crafting something that is not just functional but viably worth while for a small business to actually use it.

So far I liked everything I see and used toping off the list with comupter morons and accounting morons can actually use it easily.
before I make any further recommendations to anyone about your service offering I need to know two things;
- What Database are you actually using on the back end.
- That you have On Site Staff database administrator who are capable of doing what I said above.

to me this naturally extrapolats to you do not relly on any external technical support contracts except for fast hardware replacement. That includes Oracle and Microsoft and who ever manages the data centres you essentially have to lease instead of operate by hand.

Id really appreciate if you could clear this up for me.

Thanks

Feb 1/10
5:20 pm
Rich Lafferty says:

Geoffrey: We don’t have a systematic, automated way of bringing data back that doesn’t mean making the entire application unavailable for too long, since we couldn’t automatedly restore things and have people using FreshBooks at the same time.

(The automated way is replication, which is the thing that wasn’t working in the first place.)

The number of affected customers is pretty small, though, so we’re doing it by hand on request: looking to see what changed during the window, and then figuring out what’s necessary to accomplish that given the current state of their account.

2. Suggest you offer the affected customers one free month of usage. This will show that you are not only sorry, but that you are also fair and reasonable.

Unfortunately some people will never understand that and those are the ones that will move on. Strange that they don’t abandon Microsoft every time that product crashes their computer and loses the changes in a document.

This post was mentioned on Twitter by MikeMcDerment: @brightwhite In a nutshell, we have the data, but we recommend recreating it. Details here: http://bit.ly/b5pp6u...

Perhaps a percentage uptime figure might help shed a little more light on what was more of a hiccup in the long term of things. Perhaps the “haters” in the crowd could compare that with their own uptime statistics before making hasty comments.

I, for one, am more comfortable in the fact that this wasn’t a cover up, and that this in effect has tested the DRP and allowed corrections that could have been much more serious if not found.