You're not missing anything, it's just a matter of trust. There are a lot of middlemen between you and your money, no matter the scenario. For online banking, these middlemen include the browser, your OS, and the underlying communication network. You have developed a trust relationship with those components. Mint wants to be another middleman, and that's why they're developing a brand known for security and trustworthiness.
And if they get hacked and my password is abused to steal money from my account, will my bank cover the liability? I doubt it - I gave my password away. I assume it's like a stolen ATM PIN, I eat the loss. No thanks.
Actually, your bank is required by law to limit your liability to $50 if reported within 2 days, or to $500 if reported within 60 days. Many states are even more friendly towards consumers, and some banks have voluntarily made their policies more consumer-friendly than that, limiting liability to $50 in all cases, for example.