ordinarily I would agree with you but not too sure about access to shrines , temple and other places of interest. Streets are okay, as are railway stations; you know how far back you can get and how big the subjects are likely to be: trains, cars, people and bridges are fairly standard but its non standard subjects which I am unsure of. Is a 28mm (effective Focal Length) enough for low, wide buildings, is there normally a resonable distance to move back or are they cramped. Are they cordoned off? Stonehenge used to be a wideangle/standard subject 'cos you just wandered through it - now you need a telephoto.
Normally wander around with one lens and then move around to frame shot; if the lens of the day is wrong and the shot needs a longer or shorter one then I just return and reshoot but Japan is a bit far!