The zone map (http://planthardiness.ars.usda.gov/PHZMWeb/Downloads.aspx) is a good starting point to think about winter hardiness of plants, but it is only a starting point. The zone map looks only at average winter low temperatures, and many other things influence how well a plant survived winter.
Summer heat: Long, warm summers allow plants to mature fully and finish their normal growth cycle before heading into preparing for winter, and that can have a huge impact on how well plants survive the winter. Daphnes are a good example, where species like D. jasminea is totally hardy for us here in zone 5, but require protection to survive the much, much milder winters in England where summers are much too cool for them. I suspect this might also be the reason we can't grow Magnolia grandiflora here, even though I saw huge gorgeous ones near Springfield Illinois, where winters are just as cold, if not colder, than here.
Winter wet: Many plants, especially those from deserts and high altitudes, can take incredibly extreme amounts of cold – provided they are kept perfectly dry. This results in what we call the “hardy in Denver” phenomenon. All sorts of agaves and delosperma and other succulents are totally hardy in Denver, and never survive here despite us both being in zone 5. Building raised beds can help this a lot, as will siting these plants under the overhanging eve of the roof of your house to keep the rain off them.
Snow cover: Snow is a terrific insulator, and perennials under a thick blanket of snow will be protected from the most extreme temperature fluctuations. This of course provides no help for taller trees and shrubs. We usually have reliable snow cover here in Michigan, so take that into account when I say something is hardy for us.
Late spring freezes: Dormant plants are very cold tolerant, but once they start growth in the spring, a late freeze can be devastating. We often have early warm spells followed by hard freezes here, and many plants suffer from it. Most early flowering trees, like magnolias or cherries, will have their flowers frozen off, but the over all health of the plant won't be harmed. Others, like ginkgos in our experience, are frequently totally killed if it freezes hard once they've started active growth. Urban areas, with all the extra heat produced in them, often are spared these late freezes that are so devastating in rural areas.
Plant maturity: Trees in particular often show significantly better winter hardiness once they've gotten big and mature than they do when a small sapling, so if you take care to protect them when small, you may not need to once they grow up.
Zonal creep: Strings of years with mild winters can give gardeners a false sense of the true intensity of their winters... not a problem for us anymore after the intense winters of 2012/13 and 2013/14. We try to keep our plant descriptions updated to reflect what we are actually seeing survive after these very cold winters