An international team including NASA has plans in development for a follow up to JWST called The Large Ultraviolet Optical Infrared Surveyor (LUVOIR) [1]. There's two versions, LUVOIR-A with a 15m primary mirror and LUVOIR-B with a 8m primary mirror (compared to JWST 6.5m primary mirror) [2]
From the Wiki "LUVOIR would be able to analyze the structure and composition of exoplanet atmospheres and surfaces. It could also detect biosignatures arising from life in the atmosphere of a distant exoplanet."
Launch cost is actually a _significant_ fraction. Even more so, because a lot of the complexity arises from design and systems to get things right on the first try, since launch costs are so prohibitive.
Put another way, if launch costs were cheaper, it would also have been cheaper to design and build the telescope.
JWST is expensive for two reasons. One is the giant heat shield, as it is an infrared telescope. There is no real way around that.
Secondly it is the space in the rocket, the mirror (and everything else) had to be folded to fit into the rocket. Getting this folding mechanism to reliably unfold was the big challenge - if anything had gone wrong, there would have been no recovery.
With its 8m diameter, Starship could house larger telescopes without folding. Considering how cheap a Starship by itself is, the Starship itself could be the permanent housing of a large telescope. It would just need to jettison the tip to expose the telescope inside.
I believe much of the complexity comes from having to fit it in the nose cone of the Ariane 5 rocket. With a much larger launch vehicle, you can make some different design trade offs to maybe reduce that complexity.
I very much doubt you can build a cheap 6.5m gold plated mirror aligned to a standard that it can get clear images of the farthest corners of the universe, but that can also survive a rocket launch.
