Python网络请求：三款库的对比解析

引言

前些时候，我有个念头，想把一个Python服务的接口慢慢改成异步的，那原本用requests的地方就得换成httpx或者aiohttp啦。心里挺好奇异步请求相比同步请求能有啥提升，就做了些小测试。

有个服务A提供接口，这个接口会停顿1秒，模拟数据库操作。服务B去调用服务A的这个接口，然后把响应返回给客户端C。服务B有4个接口，分别用requests、httpx同步、httpx异步和aiohttp来请求服务A。

客户端用wrk来做请求测试。

搭建服务A

服务A用Go编写，用标准库就能搞定

package main

import (
    "net/http"
    "time"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /a", func(w http.ResponseWriter, r *http.Request) {
        time.Sleep(1 * time.Second)
        w.Write([]byte("ok"))
    })
    if err := http.ListenAndServe("127.0.0.1:8000", mux); err != nil {
        panic(err)
    }
}

先用wrk直接请求服务A，把这当作基准来测测。

$ wrk -t8 -c1000 -d30s http://127.0.0.1:8000/a
Running 30s test @ http://127.0.0.1:8000/a
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.00s     7.62ms   1.17s    97.34%
    Req/Sec   194.17    266.07     1.24k    86.82%
  29491 requests in 30.10s, 3.32MB read
Requests/sec:    979.85
Transfer/sec:    112.91KB

服务B：FastAPI

先用FastAPI来搭建服务B试试

from fastapi import FastAPI
import httpx
import aiohttp
import uvicorn
import requests

app = FastAPI()
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1():
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print(f"sync request failed, {e}")
    else:
        return resp.text

@app.get("/sync2")
def sync2():
    try:
        with httpx.Client() as client:
            resp = client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print(f"sync2 request failed, {e}")
    else:
        return resp.text

@app.get("/async1")
async def async1():
    try:
        async with httpx.AsyncClient(timeout=2) as client:
            resp = await client.get(url)
            resp.raise_for_status()
    except Exception as e:
        print(f"async1 request failed, {e}")
    else:
        return resp.text

@app.get("/async2")
async def async2():
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as resp:
                resp.raise_for_status()
                content = await resp.text()
    except Exception as e:
        print(f"async2 request failed, {e}")
    else:
        return content

if __name__ == "__main__":
    uvicorn.run("server1:app", host="127.0.0.1", port=8001, workers=4, access_log=False)

wrk请求结果出来了。发现httpx不光同步请求性能比不上requests，异步请求性能也不如requests。而aiohttp的性能远超其他，是第二名的五倍多。

异步请求性能还比同步差，这挺让人费解的。后来找大模型问了问，大模型说httpx默认配置参数不高，得额外指定参数，而且还得避免反复创建http client。不过同步性能比不上开箱即用的requests，异步性能比不上开箱即用的aiohttp，那为啥还要折腾httpx呢？

服务B：Flask

Flask 2.0也支持异步接口，不过之前测试性能不太好，把它拉出来一块测试看看实力。

Flask版本是3.1.1。因为用gunicorn运行异步接口会报错，所以用的flask内置webserver。

from flask import Flask
import requests
import httpx
import logging
import aiohttp

app = Flask(__name__)
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1():
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text

@app.get("/sync2")
def sync2():
    try:
        with httpx.Client() as client:
            resp = client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text

@app.get("/async1")
async def async1():
    try:
        async with httpx.AsyncClient(timeout=2) as client:
            resp = await client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text

@app.get("/async2")
async def async2():
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as response:
                response.raise_for_status()
                resp = await response.text()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp

if __name__ == "__main__":
    werkzeug_logger = logging.getLogger("werkzeug")
    werkzeug_logger.disabled = True
    app.run(host="127.0.0.1", port=8001)

测试结果显示，看来flask还是和requests更搭，异步性能还不如同步。

服务B：Sanic

再用Sanic测试一遍

from sanic import Sanic
from sanic.response import text
import requests
import httpx
import aiohttp

app = Sanic(__name__)
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1(request):
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print(f"sync1 request failed, {e}")
    else:
        return text(resp.text)

@app.get("/sync2")
def sync2(request):
    try:
        with httpx.Client() as client:
            response = client.get(url, timeout=2)
            response.raise_for_status()
    except Exception as e:
        print(f"sync2 request failed, {e}")
    else:
        return text(response.text)

@app.get("/async1")
async def async1(request):
    try:
        async with httpx.AsyncClient() as client:
            response = await client.get(url, timeout=2)
            response.raise_for_status()
    except Exception as e:
        print(f"async1 request failed, {e}")
    else:
        return text(response.text)

@app.get("/async2")
async def async2(request):
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as response:
                response.raise_for_status()
                content = await response.text()
    except Exception as e:
        print(f"async2 request failed, {e}")
    else:
        return text(content)

if __name__ == "__main__":
    app.run(host="127.0.0.1", port=8001, debug=False, access_log=False, workers=4)

可能我对Sanic了解不多，单从这个测试结果来看，Sanic挺不适合写同步API的。而且用httpx异步请求的时候有好多报错，wrk结果显示Non-2xx or 3xx responses: 1244

服务B：Go

最后用Go实现请求

func GetA(w http.ResponseWriter, r *http.Request) {
    resp, err := http.Get("http://127.0.0.1:8000/a")
    if err != nil {
        log.Println(err)
    }
    defer resp.Body.Close()
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        log.Println(err)
    }
    w.Write(body)
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /b", GetA)
    if err := http.ListenAndServe("127.0.0.1:8001", mux); err != nil {
        panic(err)
    }
}

测试结果和直接请求服务A差别不大。

$ wrk -t8 -c1000 -d30s http://127.0.0.1:8001/b
Running 30s test @ http://127.0.0.1:8001/b
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.02s    85.69ms   1.66s    96.55%
    Req/Sec   210.49    175.05   740.00     63.65%
  29000 requests in 30.08s, 3.26MB read
Requests/sec:    963.97
Transfer/sec:    111.08KB

总结

用Python写同步请求还是老老实实用requests，异步接口应该用aiohttp，httpx的性能勉强能用。