Requests-get（基于‘hao123.com’,实现网页采集） 与Requests-post（基于百度，实现文本翻译）

一、Requests-get

import requests

keyword = input('请输入要查询的关键字：')

#1、指定url

url = f'https://www.baidu.com/s?tn=50000021_hao_pg&ie=utf-8&sc=UWd1pgw-pA7EnHc1FMfqnHTdnWn3njmYP10krauW5y99U1Dznzu9m1Y1nHfkPjD4n1cz&ssl_sample=normal&srcqid=-8651530116834638384&H123Tmp=nunew7&word={keyword}'

#2、网络请求

header = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0'}

response = requests.get(url,headers = header)#用的是get方法

print(response.text)#若是删去'headers = header'，则打印含有'python-requests/2.32.3'的语句，意味着服务器认为你使用python工具爬取数据，此时是无法获得正确的'response.text'，在控制台按'ctrl'+f可以查询是否有你要的关键字

with open(keyword + '.html','w',encoding = 'utf-8') as f:

        f.write(response.text) print(f'已下载...{keyword}') print(response.request.headers)                  print(response.url)

检验：
print(response.request.headers)
print(response.url)

输出：

{'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
https://www.baidu.com/s?tn=50000021_hao_pg&ie=utf-8&sc=UWd1pgw-pA7EnHc1FMfqnHTdnWn3njmYP10krauW5y99U1Dznzu9m1Y1nHfkPjD4n1cz&ssl_sample=normal&srcqid=-8651530116834638384&H123Tmp=nunew7&word=%E7%8E%8B%E7%BB%B4

这表示能获取到正确的response

二、Requests-post

有了上面的经验，此项目也能迎刃而解

import requests
import pprint
keyword = input('请输入要查询的关键字：')
url = 'https://fanyi.baidu.com/sug'
header = {
'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0'
}

data = {
'kw':keyword
}#此处看上面，先看预览内容，确认是否是要找的包，接着把负载里的内容作为data
res = requests.post(url,headers = header,data=data)
result = res.json()['data'][0]['v']
pprint.pprint(result)#打印的字典排列更加整洁