What i think is so very important and a must have feature is the ability for alternate two factor methods. As far as im aware no bank in NZ allows two factor other than insecure SMS. I am recommending the ability to at a minimum use a authentication app or as a plus allow the use of two factor physical keys such as yubikey!
Rabo offers a physical key, BNZ also has a physical key (but very low tech key )
ASB will let you get a physical key for 2FA if you want. Personally I would find that annoying but I can understand the need for business banking or high-value transactions