Well if other people agree with a move I have another forum setup for it already. I'll give moderator status to a few people on the new forum as well and try and I'm working on a webpage for it.
It's less a matter of quantity, and more a matter of quality. Having a bunch of users who are following a fad(and thus, don't really care) is far less rewarding than having a few good members who make the forums glint in the eyes of like-minded people who pass by. I know this particular bit of insight from personal experience. It would be nice if more people were involved, but not at the expense of the quality of the current user base.