Hi shishirbd123. Welcome to the fourms.
There are lots of forum add-ons for WP, such as http://bbpress.org/ and http://simple-press.com/.
But first ask if you need this. Is your site just a forum? In that case, consider an actual forum CMS, like vBulletin (on which this site runs) or a number of other options out there.
If you are adding this to a larger WP site, though, you can also Google "forum add-ons for WordPress".