Python3 Requests模块详解

参考链接 : https://www.cnblogs.com/ranxf/p/7808537.html


一、安装requests

  • 通过pip安装

    pip install requests

  • 或者下载代码后通过git安装

    $ git clone git://github.com/kennethreitz/requests.git
    $ cd requests
    $ python setup.py install


二、基础入门

1)导入模块

1
import requests

2)发送请求的简洁
  
   示例代码:获取一个网页(个人github)

1
2
3
4
import requests

r1 = requests.get('https://github.com/QzmVc1') # 最基本的不带参数的get请求
r2 = requests.get(url='http://dict.baidu.com/s', params={'wd': 'python'}) # 带参数的get请求

我们就可以使用该方式使用以下各种方法

requests.get(‘https://github.com/timeline.json’)       # GET请求
requests.post(“http://httpbin.org/post”)           # POST请求
requests.put(“http://httpbin.org/put”)            # PUT请求
requests.delete(“http://httpbin.org/delete”)         # DELETE请求
requests.head(“http://httpbin.org/get”)           # HEAD请求
requests.options(“http://httpbin.org/get” )         # OPTIONS请求

3)响应的内容                      

响应内容 意义
r.encoding 获取当前的编码
r.encoding = ‘utf-8’ 设置编码
r.text 以encoding解析返回内容。字符串方式的响应体,会自动根据响应头部的字符编码进行解码。
r.content 以字节形式(二进制)返回。字节方式的响应体,会自动为你解码 gzip 和 deflate 压缩。
r.headers 以字典对象存储服务器响应头,但是这个字典比较特殊,字典键不区分大小写,若键不存在则返回None
r.status_code 响应状态码
r.ok 查看r.ok的布尔值便可以知道是否登陆成功
r.json() Requests中内置的JSON解码器,以json形式返回,前提返回的内容确保是json格式的,不然解析出错会抛异常
r.raise_for_status() 失败请求(非200响应)抛出异常
r.requests.headers 返回发送到服务器的头信息
r.cookies 返回cookie
r.history 返回重定向信息,当然可以在请求是加上allow_redirects = false 阻止重定向

4)为get请求传递参数

1
2
3
4
5
6
import requests

url_params = {'key':'value'} #字典传递参数
r = requests.get('your url',params = url_params)

print(r.url)

输出结果

your url?key=value

5)为post请求传递参数

1
2
3
4
import requests

url_params = {'key':'value'}
r = requests.post('https://api.github.com/some/endpoint',data=url_params)

6)定制header和cookie信息

1
2
3
header = {'user-agent': 'my-app/0.0.1'}
cookie = {'key':'value'}
r = requests.get/post('your url',headers=header,cookies=cookie)
1
2
3
4
5
6
7
8
9
import requests
import json

data = {'some': 'data'}
headers = {'content-type': 'application/json',
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0'}

r = requests.post('https://api.github.com/some/endpoint', data=data, headers=headers)
print(r.text)

7)超时

1
r = requests.get('url',timeout=1)           #设置秒数超时,仅对于连接有效

8)会话对象,能够跨请求保持某些参数

1
2
3
4
5
s = requests.Session()
s.auth = ('auth','passwd')
s.headers = {'key':'value'}
r1 = s.get('url')
r2 = s.get('url1')

9)代理

1
2
proxies = {'http':'ip1','https':'ip2' }
requests.get('url',proxies=proxies)

三、示例代码

3.1 GET请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 1、无参数实例

import requests

ret = requests.get('https://github.com/timeline.json')

print(ret.url)
print(ret.text)


# 2、有参数实例

import requests

payload = {'key1': 'value1', 'key2': 'value2'}
ret = requests.get("http://httpbin.org/get", params=payload)

print(ret.url)
print(ret.text)

3.2 POST请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 1、基本POST实例

import requests

payload = {'key1': 'value1', 'key2': 'value2'}
ret = requests.post("http://httpbin.org/post", data=payload)

print(ret.text)


# 2、发送请求头和数据实例

import requests
import json

url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'content-type': 'application/json'}

ret = requests.post(url, data=json.dumps(payload), headers=headers)

print(ret.text)
print(ret.cookies)

3.3 XML请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#! /usr/bin/python3
import requests

class url_request():
def __init__(self):
"""init"""

if __name__ == '__main__':
heards = {'Content-type': 'text/xml'}
XML = '<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><Request xmlns="http://tempuri.org/"><jme><JobClassFullName>WeChatJSTicket.JobWS.Job.JobRefreshTicket,WeChatJSTicket.JobWS</JobClassFullName><Action>RUN</Action><Param>1</Param><HostIP>127.0.0.1</HostIP><JobInfo>1</JobInfo><NeedParallel>false</NeedParallel></jme></Request></soap:Body></soap:Envelope>'
url = 'http://jobws.push.mobile.xxxxxxxx.com/RefreshWeiXInTokenJob/RefreshService.asmx'
r = requests.post(url=url, heards=heards, data=XML)
data = r.text
print(data)

3.4 状态异常处理

1
2
3
4
5
6
7
8
9
10
11
import requests

URL = 'http://ip.taobao.com/service/getIpInfo.php' # 淘宝IP地址库API
try:
r = requests.get(URL, params={'ip': '8.8.8.8'}, timeout=1)
r.raise_for_status() # 如果响应状态码不是 200,就主动抛出异常
except requests.RequestException as e:
print(e)
else:
result = r.json()
print(type(result), result, sep='\n')

3.5 上传文件

1
2
3
4
5
6
7
8
import requests

url = 'http://127.0.0.1:8080/upload'
files = {'file': open('/home/rxf/test.jpg', 'rb')}
#files = {'file': ('report.jpg', open('/home/lyb/sjzl.mpg', 'r'))} #显式的设置文件名

r = requests.post(url, files=files)
print(r.text)

3.6 身份验证

基本身份认证(HTTP Basic Auth)

1
2
3
4
5
6
import requests
from requests.auth import HTTPBasicAuth

r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd', auth=HTTPBasicAuth('user', 'passwd'))
# r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd', auth=('user', 'passwd')) # 简写
print(r.json())

另一种非常流行的HTTP身份认证形式是摘要式身份认证,Requests对它的支持也是开箱即可用的:

1
requests.get(URL, auth=HTTPDigestAuth('user', 'pass'))

有关Session的问题可以参考Session详解

待补充…