You would be surprised how large you can get without having an office. I've had big offices, but now I have a 100% virtual team and we do fine. I think the question is WHY do you want an office? If you want separation from your home life, an office can be great. If you want to entertain clients, an office can be useful. If you are trying to impress, an office is critical.
What is your plan? You don't need an office to get bigger, unless you want to get bigger in a specific way, which is to have a big team all working together. Offices are VERY expensive, it goes way beyond the rent as you will soon find. Supplies and utilities, desks and chairs, etc. really add up so be sure that you are investing in a growth strategy that you really want.