Your post title says this is not about ISO, but how do you determine what you can hand-hold without taking ISO into consideration? For the pictures I normally take (scenics with deep depth of field), my older 1.5X crop factor DSLR is good down to about EV10--I usually need f/8 to ensure getting the depth of field I need at my favored 35mm focal length (50mm equivalent), I can avoid excessive camera shake fairly reliably at 1/50 second or higher, and I like my camera's output at ISO 400. With a Nikon P7100 compact, though restricted to ISO 100 for clean results, I could get to EV 7 thanks to its image stabilization and great depth of field even wide open at f/3.5 (at the same 50mm equivalent focal length). Had I liked its output at ISO 400, I could have shot handheld down to EV5.It is about available light. For handheld shots 2-3 EV100 is about the limit for me.
One of the interesting things to me is the very low light levels that people in these forums regularly shoot in. The exposure guides I remember from my film days typically covered EV16 (bright sun on sand or snow) to EV12 (heavy overcast or open shade), and some guides would go to EV11 (deep shade). One of my favorite books is Brian Bower's Lens, Light and Landscape. Like most of my picture taking, the book consists of outdoor scenes mostly under daylight conditions, Of the 121 example photos in the book, 115 were taken at EV11 or higher (in fact, 112 of them were within the three-stop range of EV15 to EV12). EV11 is about as low as I need to go outdoors, and when I'm indoors (such as for a birthday party), I'm happy to use flash, just like I did with a film camera loaded up with ISO 100 Fuji Astia.