• 周四. 12 月 12th, 2024

5G编程聚合网

5G时代下一个聚合的编程学习网

热门标签

python爬虫笔记(1-2)requests模块实例

admin

11 月 28, 2021
# 需求:获取网页,用requests
import requests
url = "http://www.baidu.com/"
resp = requests.get(url)
resp.encoding = "utf-8"
# 抓取内容写入文件
with open("mybaidu222.html",mode="w",encoding="utf-8") as f:
    f.write(resp.text)
print("over!")
# 需求:搜狗搜索内容爬取,用requests
# 1、安装requests
# 2、导入requests
# 3、准备URL;加入headers处理反爬;获取get请求返回的值;拿到页面源代码
import requests
query = input("输入你的查询内容:")
url = f'https://www.sogou.com/web?query={query}'
# 加header伪装成浏览器发送请求,绕过反爬
header = {
  "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.106 Safari/537.36"
           }

resp = requests.get(url=url,headers=header)
print(resp.text)
resp.close() #关掉resp

 实例来源:https://www.bilibili.com/video/BV1i54y1h75W?p=1

发表回复