The magic wand in TIFFany is very flexible and powerful, much more so than Photoshop's "tolerance"-based magic wand tool. But it can be maddening to people who aren't used to it. Hopefully, I can save people some aggravation and explain what the tool is really trying to do and some tricks to get what you're expecting a little easier.
Technically speaking, the magic wand is a tool to create a bitmap marquee. It doesn't choose a region defined by a piece of geometry like a rectangle or spline marquee. It actually picks the pixels and how much they're picked (more on this later) according to the four criteria in the Magic Wand inspector: Hue, Saturation, Brightness and Transparency. Notice that these are the values that we use to define a color.
When you click on a spot in your image, that sets a baseline color. The criteria in the inspector sets the amount deviation from that color and all the pixels in that range are ten selected. You're basically filtering for color.
Now I think I was first confused about why there were two arrows/brackets in each of the sliders in the Magic Wand's inspector. The slider brackets set percentage values for tolerance and intolerance. The left bracket sets how much a value can vary from the one you picked and still be included in the selection. The right bracket sets how much will not be allowed into the selection based on how much it deviates from the picked color. For example, if you set the Brightness slider to 10 for the left bracket and 15 for the right, any color that is less than 10% brighter or darker than the color you clicked on will be part of the selection. Likewise, all colors that are more than 15% brighter or darker will be left out. If both were set to the same point, say 10%, the closest 10% are in, everything else is out (talking about brightness only). You can't set the intolerance to a lower value than the tolerance because then they would be fighting over the same pixels.
Like Photoshop But So Much More
This last example is how Photoshop's magic wand tool works. It assumes a value for tolerance relative to the picked color and everything outside of that tolerance range is excluded. This means that it gets you crisp areas of color without any smooth transition, and you can't control how it blends out. By making the tolerance 10% and the intolerance 15% in the first example, the pixels in between these values are partially picked. This means that, in that first example, if you use the simple paint tool to color the selected area bright green, the closest 10% to your original picked point would all be bright green. Makes sense, they're selected and they were painted. Obviously the pixels that are more than 15% brighter or darker than your original pick won't change color; they were excluded and are not part of the selected area. That leaves all the pixels that were between 10% and 15% different from the picked color.
If they're not really included and not really excluded, what happens to them? Those pixels would get some green. The pixels that are 11% different will turn more green than the ones that were 14% different. The ones that are off by 11% are almost in, and the ones off by 14% were almost completely out. So instead of a crisp, possibly jagged line where the selection ends, you have a smooth gradient where the green is blended out. You can use this to your advantage to make much more subtle and convincing changes to an image without something popping out as looking "tinkered with."
Putting It All Together
So that was one slider, and there are four of them. It can be hard to juggle all those tolerances and exclusions. So don't!
Caffeine uses the example of a bright red car in their user guide. Let's say you know you want to pick all the red there, no matter if it's in shadow, in the sun, with glare off its surface, etc. You want red. So you set the Hue slider to something like 10% tolerance of the point you pick and exclude any colors that are more than 15% off the red hue you select. If you do the same with brightness, 10% inclusion, 15% exclusion, it's going to leave out the highlights and shadows you want on the car body. However, instead of including nearest 10% in terms of brightness, you can include all of the reds no matter their brightness. When I say all, I mean 100% tolerance for red brightness: darkest to lightest. By sliding the left-hand tolerance bracket for brightness up to 100, there is no exclusion. So if you want all that red, no matter how bright or how saturated it is, set the tolerances for everything else to 100, and they're out of the way. Now you can get the hue just right and get everything you want and nothing you didn't. If you need to, you can slowly start to exclude some areas of brightness, saturation or transparency by sliding them to the left a little.
In real world examples, this is usually how it goes. There's one main color criteria you're aiming for, and the others you want out of the way. However, you usually need to exclude a little bit (meaning, push the sliders to the left a little bit) from the other color criteria to get the best selection.
The most extreme settings should help demonstrate how this works while you're trying it.
If you push all the sliders to the right, you pick all the pixels. If you push all the sliders to the left, you usually only pick that one pixel you clicked on. In the first case, you've set the relative tolerance to 100%, in the second, you pushed the tolerance to 0%. If you push the left-hand bracket of each slider all the way to the left and the right-hand brackets all the way to the right, you partially pick all the pixels: 0% tolerance and 0% exclusion means that a gradient is placed on everything but nothing changes completely.
Try testing these examples and use the plain color action on the selection to see the gradients and the extreme examples I just mentioned. the last one with tolerance and intolerance both set to 0 is interesting, kind of a bug really.