Story points were made up to keep project managers happy so that they leave the team alone.

This is a good indicator of how much time you should spend estimating.