以下是部分nginx日志,第一个URL是REQUEST_URI,第二个是HTTP_REFERER。
可以看到,网络爬虫一直在user.reg/user.login/link.tpl之间循环重定向,甚至嵌套了好几层,这导致爬取永远无法停止,导致高峰期每秒有超过50个请求在访问这些登录注册页面。
我已添加代码阻止这些循环重定向,服务器压力似乎下降了一些。
https://github.com/hu60t/hu60wap6/commit/cd16fcb1cf85cebe477f840446ac673f37cf6d05
183.199.75.86 - - [28/Aug/2026:16:19:56 +0800] "GET /q.php/user.reg.html?u=user.login.html%3Fu%3Duser.reg.html%253Fu%253Duser.reg.html%25253Fu%25253Dbbs.topic.102231.2.html%2525253Ffloor%2525253D25 HTTP/2.0" 400 15 "https://www.hu60.cn/q.php/HTTP;/user.login.html?u=user.reg.html%3Fu%3Duser.reg.html%253Fu%253Dbbs.topic.102231.2.html%25253Ffloor%25253D25" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36"
111.49.253.245 - - [28/Aug/2026:16:19:56 +0800] "GET /q.php/link.tpl.classic.html?url64=dXNlci5yZWcuaHRtbD91PXVzZXIucmVnLmh0bWwlM0Z1JTNEdXNlci5yZWcuaHRtbCUyNTNGdSUyNTNEdXNlci5yZWcuaHRtbCUyNTI1M0Z1JTI1MjUzRHVzZXIucmVnLmh0bWwlMjUyNTI1M0Z1JTI1MjUyNTNEdXNlci5yZWcuaHRtbCUyNTI1MjUyNTNGdSUyNTI1MjUyNTNEdXNlci5yZWcuaHRtbCUyNTI1MjUyNTI1M0Z1JTI1MjUyNTI1MjUzRHVzZXIucmVnLmh0bWwlMjUyNTI1MjUyNTI1M0Z1JTI1MjUyNTI1MjUyNTNEdXNlci5yZWcuaHRtbCUyNTI1MjUyNTI1MjUyNTNGdSUyNTI1MjUyNTI1MjUyNTNEdXNlci5yZWcuaHRtbCUyNTI1MjUyNTI1MjUyNTI1M0Z1JTI1MjUyNTI1MjUyNTI1MjUzRGJicy50b3BpYy4xMDUxODguaHRtbCUyNTI1MjUyNTI1MjUyNTI1MjUzRmZsb29yJTI1MjUyNTI1MjUyNTI1MjUyNTNEOA.. HTTP/2.0" 400 15 "https://www.hu60.cn/q.php/user.reg.html?u=user.reg.html%3Fu%3Duser.reg.html%253Fu%253Duser.reg.html%25253Fu%25253Duser.reg.html%2525253Fu%2525253Duser.reg.html%252525253Fu%252525253Duser.reg.html%25252525253Fu%25252525253Duser.reg.html%2525252525253Fu%2525252525253Duser.reg.html%252525252525253Fu%252525252525253Duser.reg.html%25252525252525253Fu%25252525252525253Dbbs.topic.105188.html%2525252525252525253Ffloor%2525252525252525253D8" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36 Edg/145.0.0.0"
120.219.224.58 - - [28/Aug/2026:16:19:56 +0800] "GET /q.php/user.reg.html?u=user.reg.html%3Fu%3Duser.reg.html%253Fu%253Duser.reg.html%25253Fu%25253Duser.reg.html%2525253Fu%2525253Duser.reg.html%252525253Fu%252525253Dbbs.topic.95066.html%25252525253Ffloor%25252525253D14 HTTP/2.0" 400 15 "https://www.hu60.cn/q.php/user.reg.html?u=user.reg.html%3Fu%3Duser.reg.html%253Fu%253Duser.reg.html%25253Fu%25253Duser.reg.html%2525253Fu%2525253Dbbs.topic.95066.html%252525253Ffloor%252525253D14" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36"