[rspec-users] scenarios on production data

[Matt Wynne]

On 9 Sep 2008, at 14:54, Ashley Moran wrote:

>
> On 8 Sep 2008, at 17:21, Jonathan Linowes wrote:
>
>> Here's one example: lets say my app is a specialized CMS, where  
>> account owners can setup their own projects, pages and forms. I'd  
>> like to run scenarios against setups that users have created.  
>> Thinking further, wouldn't it be neat to make this a user feature,  
>> eg via "Validate This Project" button
>
> The more I think about it, the more I come to the conclusion you  
> should just take (very) regular backups, and when a problem occurs,  
> isolate the situation.
>
> Dan's export suggestion would be really useful here.  In case of  
> data loss, you could fire up a backup version of the app, export  
> missing/corrupt pages, and re-import into the production environment.
>
> This would encourage you to define a robust data format too.  It  
> wouldn't directly lead you to catching edge cases, but it will give  
> you more confidence about recovering from them.  And anyway, if  
> your tests-on-real-data throw up edge cases, they will still miss  
> the ones that real data doesn't exist for.  So maybe being able to  
> say "well, I'm only 98% sure your data is safe, but I'm 99.9% sure  
> I can get it back if there's a problem", is more reassuring than  
> hoping you covered everything.
>
> I've found that testing can sometimes drive out new features from  
> an app.  The one I'm starting to work on now, for example, is a  
> daemon process that sits and polls some RSS and XML services.  But  
> to test it I've been driven to start writing a (crude, right now)  
> socket-based remote control and corresponding client interface.  I  
> can't say if it will ever be used, but it's reassuring to know I  
> can always go in and prod it if I'm not sure it's working right live.
>
> Sometimes it feels like waste, but then insurance products have an  
> expected net loss, and people still consider them valuable.
>
> Ashley
>
> -- 
> http://www.patchspace.co.uk/
> http://aviewfromafar.net/
>

The great thing about using tools like RSpec is that you write a lot  
less bugs in your code, freeing testers up to do exploratory testing 
[1] which is where you find the sort of issues I think Jonathan is  
worried about.

[1]http://en.wikipedia.org/wiki/Exploratory_testing

cheers,
Matt
----
http://blog.mattwynne.net
http://songkick.com

In case you wondered: The opinions expressed in this email are my own  
and do not necessarily reflect the views of any former, current or  
future employers of mine.