github / scientist

# Scientist!
A Ruby library for carefully refactoring critical paths. 
## How do I science?
Let's pretend you're changing the way you handle permissions in a large web app. Tests can help guide your refactoring, but you really want to compare the current and refactored behaviors under load.
 [code] 
Wrap a `use` block around the code's original behavior, and wrap `try` around the new behavior. `experiment.run` will always return whatever the `use` block returns, but it does a bunch of stuff behind the scenes:
* It decides whether or not to run the `try` block,
* Randomizes the order in which `use` and `try` blocks are run,
* Measures the wall time and cpu time of all behaviors in seconds,
* Compares the result of `try` to the result of `use`,
* Swallow and record exceptions raised in the `try` block when overriding `raised`, and
* Publishes all this information.
The `use` block is called the **control**. The `try` block is called the **candidate**.
Creating an experiment is wordy, but when you include the `Scientist` module, the `science` helper will instantiate an experiment and call `run` for you:
 [code] 
If you don't declare any `try` blocks, none of the Scientist machinery is invoked and the control value is always returned.
## Making science useful
The examples above will run, but they're not really *doing* anything. The `try` blocks don't run yet and none of the results get published. Replace the default experiment implementation to control execution and reporting:
 [code] 
When `Scientist::Experiment` is included in a class, it automatically sets it as the default implementation via `Scientist::Experiment.set_default`. This `set_default` call is skipped if you include `Scientist::Experiment` in a module.
Now calls to the `science` helper will load instances of `MyExperiment`.
### Controlling comparison
Scientist compares control and candidate values using `==`. To override this behavior, use `compare` to define how to compare observed values instead:
 [code] 
If either the control block or candidate block raises an error, Scientist compares the two observations' classes and messages using `==`. To override this behavior, use `compare_errors` to define how to compare observed errors instead:
 [code] 
### Adding context
Results aren't very useful without some way to identify them. Use the `context` method to add to or retrieve the context for an experiment:
 [code] 
`context` takes a Symbol-keyed Hash of extra data. The data is available in `Experiment#publish` via the `context` method. If you're using the `science` helper a lot in a class, you can provide a default context:
 [code] 
The `widget-permissions` and `widget-destruction` experiments will both have a `:widget` key in their contexts.
### Expensive setup
If an experiment requires expensive setup that should only occur when the experiment is going to be run, define it with the `before_run` method:
 [code] 
### Keeping it clean
Sometimes you don't want to store the full value for lat