For telescopes in general, ever since they were invented, it's the size of the objective lens (in the front) or mirror (in the back).
For the most hugest humongous telescopes, built in the past few decades, it's limited by the turbulence of the atmosphere. That's the main reason we started building the big ones on mountain tops and in deserts, and lately we've started putting them in Earth orbit, where they don't have to look through any air at all.
