On my profile page it shows a number after "flag weight". I understand it's related to the number of flagged questions/answers, but what exactly does the number signify?
It represents how useful your close/reopen/moderator etc flaggings have been.
Every time you flag something and the action you flagged it for gets taken your flag weight increases. If the flag is 'declined' then your flag weight decreases.
People with larger flag weights get their flags higher up the lists on the moderator pages so they are more likely to be acted on faster.
There is a formula somewhere that defines how much your flag weight increases with each correct flagging. After a flag weight of 500 you start getting less and less weight per flag.
The official FAQ entry is here: https://meta.stackexchange.com/questions/80170/what-is-flag-weight