So where do you draw the line when the potential for harm to others is increased?

We’ve had a few rousing conversations/debates today about guns and rights.
Can we safely all agree that an individual should not have access to things like, oh..dynamite. Any type of nuclear weapon. Biological or chemical agents that could cause harm to others. And I do not mean household chemicals.
We agree, I think, that the individual rich man who can afford a scooped out volcano should not be allowed access to a team of scientists who can invent and create a weapon that could harm others. We’ve all seen the James Bond movies, and others in the same category, like XXX. This is why we have government controls. We can’t have that much potential to harm others resting in one person’s hands. Or the hands of a small group of people with an axe to grind, and that answer to no other authority.
So what about these anti government nut cases that hole up somewhere with a huge stash of weapons and ammo. Or the religious whak jobs? Remember Waco? How did these people manage to acquire such a large amount of hardware? Should they have been allowed to do so?
Where should the line be drawn? Or do you believe that there should be no line?
Where do the rights of the individual to own a firearm end, and my right to be safe from nutcases initiating standoffs with the police begin?
People who think that the government should be aware of the activities of these potentially dangerous groups, how do you suggest they do so? Should they respect the rights of these individuals not to be investigated?
Just curious to see which way the wind blows on this aspect of the whole gun control thing.