There in lies the problem, all those Facebook widgets on various 3rd party websites are used to track you. If you block FB's network ranges then it gets much harder for them to do that.
In effect you are "using" Facebook whether you want to or not; this is the issue some people have with shadow FB profiles.