Hearing that a Zumo can tumble down the road and be OK is good news.
Personally I think any phone is built to be fairly tough, particularly basic Nokias, but that must be compromised by both the need to be pocketable, and the fact they don't really want one lasting for 5 years plus. If your phone breaks, well, thats why they sell you insurance, and you can pop to the high street to buy the latest whizz bang smartphone in your next break between meetings.
I would hope something "designed" to go on a bike's handlebars will be tougher than that! A 5" screen phone thats only a few mm thick won't stand much flexing, and I've had one touchscreen phone that got a small crack despite being in its supplied case in my pocket that rendered it useless

In fact, maybe the very fact that a stand alone gps isn't carried round as a phone makes it less open to being damaged?
I'd totally agree with the need for a speedy "high sensitivity receiver" unit too. Stationary, and able to stop at a junction of doubt then maybe not. Otherwise, throw in a few trees or tall buildings with heavy traffic and your blood pressure will rise if it won't keep up with what you're doing! How good a smart phone is in that respect though I don't know though?
Can you for instance download all of the UK, France, Spain and Morocco road maps to a smart phone, as well as having Olaf ready to go on arrival? An old Garmin I had you needed to select only map tiles that covered your route which was really annoying, and I don't want to do that again!
As for not needing a paper map? Good gracious me, surely not?!?