Figured out the cause, it's an exploit, but luckily only can be used for deleting media from posts and not full posts.
Had to commandeer a bunch of boards to disable the setting, transferred back them to the respective BOs immediately. Sorry about that.

Thanks for all the users who helped out, reporting the issue and giving tips.