I've got ants in my pants! garden.... What, if anything, should I do?

Today I was taking a bit of soil out of my raised bed to fill up a pot and came across a massive ants nest. I’ve seen ants in my garden before and never really bothered with them because it was only a few here and there and I figured they would kill some of the bad bugs in my garden. This however is an insane amount. Like “holy crap the ground is moving!” level of ants. I haven’t noticed any negative effects yet but I don’t want to risk anything down the road as their next is rather close to a lot of my plants roots.

Here are some pictures to give everyone an idea of what I’m dealing with.

If I do have to remove the ants what is the best way to go about this without using pesticides?