Hackers News AI bots everywhere. Does anyone have a good whitelist for robots.txt? adminJanuary 28, 20250 My niche little site, http://golfcourse.wiki seems to be very popular with AI bots. They basically become most of my traffic….