There's nothing you can do to make your new forum popular. That's something that comes from the people who visit the site. However, there are things you can do to make your forum more successful (which is what I think you meant).
Others have offered some good advise in this thread. Here are a few other things to consider:
Identify your audience.
Who is the forum for? I know it's a photography forum, but that is a very broad category. Take some time to refine it and determine exactly who it is you want to reach. A good place to start is to look at yourself and where you fit in the area of photography. Then think about being in a room full of people who have similar interests. Who are they and what are they like?
Identify your niche.
Again, photography is a very broad and general category. You need to carve out a small corner that is uniquely yours. sbjsbj brings up a good point. There are many photography forums that have died because they can't compete with instagram or flickr, etc.
If all you want is a forum that lets people post their photos, you probably will struggle, not only with getting members, but also with rising bandwidth costs. On the other hand, a forum which focuses on some specific aspect of photography or a specific type of photography, providing useful, in-depth discussion, could be very successful in attracting those who share that interest.
Make a plan and stick to it.
Once you have your audience and your niche determined, make a plan for the forum to deliver serve that audience. Organize your forum with that specific niche mind and avoid grabbing for the common, low-hanging fruit by adding forum features and topic areas that don't serve your primary audience. Most importantly, stick with it. Forums take time to develop to the point where they are interesting to others. Expect slow growth, and plan to be the only poster for at least 6 months. When you start to get discouraged (which will happen) take a look at your forum's views and remind yourself that many of those viewers can be members. They are just waiting for the right topic or the right amount of content to push them into joining.
The engagement trap is a race to community activity for the sake of activity. Its usually measured by simple aggregate numbers like the total number of posts, topics, likes, or members. You dont want members to chat. You want members to learn, to advocate, to innovate, to educate, to support, to ...
Instead, focus on what makes your website unique, compelling, educational, enlightening, thoughtful, or valuable. Members will come if you provide the content and value.