Add spin loop recv option

[rzezeski]

I want this feature for two reasons:

1. The Solarflare pingpong test has it and this is our replacement for that benchmark tool.
2. The upper-half of the recv() path is different on illumos depending on if you make a blocking call or not. I want to be able to test that path.

This option can reduce latency by 10-14% on SmartOS so I think it's a good feature to have to demonstrate the amount of overhead the blocking call can add.

[gdamore]

Ok, looking at this, the code looks reasonable. I'm not sure if "spin" is the right way to describe it -- the main difference is that you're doing a non-blocking operation (and yes, spinning), instead of blocking. I guess its ok though.

That said, of course spinning is a tragically bad idea to use in production. :-)

[rzezeski]

I call this a spin loop since it is spinning on recv. It could also be called a busy-wait or busy-looping according to wikipedia (https://en.wikipedia.org/wiki/Busy_waiting). If you want me to change the name then I'm fine with that, but I think spin best describes what it does.