Okay, sure, for a document-based website with a few bells and whistles. If you want to build one of these new-fangled "web apps" that I hear so much about, you will absolutely need to manipulate hundreds or thousands of DOM elements and do so fast and consistently across god-knows-how-many versions of - to make the list short - IE, FF, Safari, Chrome, and Opera. There is a major push for JavaScript optimization going on industry-wide, and the libraries are a crucial component of this.
First, I own one of those new-fangled web apps (http://agilezen.com/). It's built on jQuery, has a fairly complex Javascript-driven UI, and nowhere do I manipulate hundreds or thousands of DOM elements.
Second, I wasn't arguing against libraries -- I was saying that the difference in performance is probably not what you want to use to decide which library to use.