jQuery: Press button to add class AND remove all other classes

so I have the following HTML Code

<div class="btn" id="b">Blue</div>
<div class="btn" id="r">Red</div>
<div class="btn" id="g">Green</div>
<div class="colorfield wide"></div>

This is what I try to achieve:

If I press the blue button, I need the .colorfield to become:

<div class="colorfield wide BLUE"></div>

If I press any other button, I need .BLUE removed and the other color-class added instead.

I know that this is a rather simple toggle, but I just can't get it to work.