Python's requests library has been the default HTTP client for more than a decade, and for good reason: the common case is one line. That convenience hides three decisions you should make on purpose, though: where your data goes in the request, what counts as failure, and whether each call opens a new connection. This lesson makes each one explicit and ends with a helper you can drop into any script.
Everything runs on Python 3.9 or newer with requests installed:
python -m pip install requestsAs in the last lesson, the key lives in an environment variable called APIVERVE_API_KEY, never in the file.
01One call, read carefully
import os
import requests
res = requests.get(
"https://api.apiverve.com/v1/dictionary",
params={"word": "retry"},
headers={"x-api-key": os.environ["APIVERVE_API_KEY"]},
timeout=15,
)
body = res.json()
print(body["data"]["definitionCount"], body["data"]["definitions"][0])Compare it with the JavaScript version and the differences are all in your favour:
params=builds and encodes the query string for you. You never write?word=by hand, so a value with&, spaces or+in it arrives intact.headers=is a plain dict. Header names are case-insensitive on the wire, sox-api-keyandX-API-Keyare the same header; lowercase is the convention.os.environ["APIVERVE_API_KEY"]uses square brackets on purpose. If the variable is missing you get aKeyErrornaming it, immediately.os.environ.get(...)would handNoneto requests, which drops a header whose value isNoneentirely, and you would be debugging an edge rejection instead of a missing variable.timeout=15is not optional in spirit.requestshas no default timeout: without one, a stalled connection can hang your script indefinitely. Lesson three goes deeper; for now, always pass it.
res.json() parses the body. Unlike fetch, it is synchronous, because by the time requests.get returns, the whole body has been downloaded.
curl "https://api.apiverve.com/v1/dictionary?word=retry" \
-H "x-api-key: YOUR_API_KEY"data.definitionCount tells you how many entries are in data.definitions.
02params versus json
The question beginners ask most about requests is which keyword to put their data in. It depends on where the API expects it, and that is set by the HTTP method on the API's reference page.
| Keyword | Where the data goes | Typical method | Content-Type set for you |
|---|---|---|---|
params= | Query string in the URL | GET | none |
json= | Request body, serialised as JSON | POST | application/json |
data= | Request body, form-encoded (dict) or raw (str/bytes) | POST | application/x-www-form-urlencoded for a dict |
Every API used in this course is a GET, so params= is the answer. You need json= the moment you send a body. APIVerve's batch requests are a real example: one POST that carries many inputs for the same endpoint in a requests array.
import os
import requests
res = requests.post(
"https://api.apiverve.com/v1/emailvalidator/batch",
json={"requests": [
{"email": "support@myspace.com"},
{"email": "not-an-email"},
]},
headers={"x-api-key": os.environ["APIVERVE_API_KEY"]},
timeout=30,
)
body = res.json()
for item in body["results"]:
print(item["status"], item["error"])Two mistakes show up here constantly:
data=with a dict when the API wants JSON. The body is sent form-encoded (requests=...), the server cannot find the field it expected, and you get a400. If you usedata=json.dumps(payload)instead, you must also setContent-Type: application/jsonyourself.json=does both, which is why it is the right default.params=on a POST, orjson=on a GET. The data is sent somewhere the server is not looking. Because undeclared parameters are ignored rather than rejected, a misplaced parameter can produce a successful response to a different question.
An endpoint is documented as POST with a JSON body. Which call sends the body correctly?
03raise_for_status and why it is not enough
requests, like fetch, does not raise on an error status. A 401 comes back as a normal Response object with res.ok set to False. The library gives you a shortcut:
res = requests.get(url, params=params, headers=headers, timeout=15)
res.raise_for_status() # raises requests.HTTPError for 4xx and 5xx
data = res.json()["data"]raise_for_status() is a good habit in throwaway scripts, and it is better than nothing. In a client you intend to keep, it has two weaknesses.
First, the exception message is generic: 401 Client Error: Unauthorized for url: .... The API's own error sentence, which names the actual problem, is still sitting in the body and you have to dig it out of err.response. Second, it says nothing about the envelope. A 200 whose body says "status": "error" sails straight through.
You also need a plan for the body not being JSON at all. APIVerve answers its own errors (including a missing or wrong key, which is a JSON 401) in JSON, but a proxy or gateway in between can answer with an HTML page instead. res.json() on that raises requests.JSONDecodeError (a subclass of ValueError), which reads like a parsing bug rather than an authentication problem.
And some statuses carry meaning beyond "it failed". Ask the dictionary for a word it does not have and you get a 404 with the error word not found. That is not a broken endpoint; it is an answer about your input. Your code should be able to tell those apart, which means keeping the status and the message together.
04A reusable helper with a Session
Here is the Python counterpart of last lesson's apiverve() function. It adds one thing the JavaScript version did not need to think about: a Session.
import os
import requests
BASE = "https://api.apiverve.com/v1"
class ApiError(Exception):
def __init__(self, message, status):
super().__init__(message)
self.status = status
def _make_session():
key = os.environ.get("APIVERVE_API_KEY")
if not key:
raise RuntimeError("APIVERVE_API_KEY is not set")
s = requests.Session()
s.headers.update({"x-api-key": key, "User-Agent": "my-script/1.0"})
return s
_session = _make_session()
def apiverve(api_id, params=None, timeout=15):
res = _session.get(f"{BASE}/{api_id}", params=params or {}, timeout=timeout)
try:
body = res.json()
except ValueError:
body = None # not JSON: something other than the API answered
if not res.ok or not isinstance(body, dict) or body.get("status") != "ok":
message = (body or {}).get("error") or f"HTTP {res.status_code}"
raise ApiError(message, res.status_code)
return body["data"]And using it:
from apiverve import apiverve, ApiError
for word in ["retry", "timeout"]:
try:
data = apiverve("dictionary", {"word": word})
print(word, "->", data["definitions"][0][:60])
except ApiError as err:
if err.status == 404:
print(word, "-> not in the dictionary")
else:
raiseAt the time of writing, retry is in the dictionary and timeout is not, so the second iteration exercises the 404 branch. Notice the shape of that handler: it deals with the one status it understands and re-raises everything else. Catching ApiError and printing it for every status would hide a revoked key behind a message about missing words.
What the Session buys you
A requests.Session does two jobs.
It holds shared settings. Headers set on the session go out with every request, so the key is attached in one place. That is also the one place to change if you rotate it.
It reuses connections. A bare requests.get opens a TCP connection, negotiates TLS, sends one request and throws the connection away. A session keeps connections in a pool and reuses them for the same host. For a loop of a few hundred lookups, skipping the TLS handshake on every call is a visible speed-up, and it is free.
Two rules keep a session from biting you:
- Do not set the timeout on the session expecting it to apply.
requestsignores atimeoutattribute onSession; it has to be passed on each call. That is why the helper takestimeoutas an argument with a default. - Do not share one session across processes. Sharing across threads is common and generally works for simple GETs, but connections do not survive a
fork. If you usemultiprocessing, create the session inside each worker.
curl "https://api.apiverve.com/v1/numbertowords?number=7012" \
-H "x-api-key: YOUR_API_KEY"Run this, then call apiverve('numbertowords', {'number': 7012}) and compare data.words.
You create a requests.Session, set s.headers with your key, and call s.get(url, params=p) in a loop with no timeout argument. What is the risk?
05Where httpx fits
httpx is a newer client with an API deliberately close to requests: httpx.get(url, params=..., headers=..., timeout=...) reads the same. Three differences matter in practice:
- It applies a default timeout (five seconds) instead of none. That is safer, and it also means a slow endpoint that worked with requests can start raising
httpx.ReadTimeout. Set the value you actually want. - It has an async client.
httpx.AsyncClientworks withasyncio, which is the natural fit if your app is already async (FastAPI, for example). Making requests calls from async code blocks the event loop. - The session equivalent is
httpx.Client, used as a context manager so its connections are closed when you are done.
import os
import httpx
with httpx.Client(
base_url="https://api.apiverve.com/v1",
headers={"x-api-key": os.environ["APIVERVE_API_KEY"]},
timeout=15,
) as client:
res = client.get("/numbertowords", params={"number": 1999})
body = res.json()
if res.is_success and body.get("status") == "ok":
print(body["data"]["words"])The envelope check is identical; only the property names differ (res.is_success rather than res.ok). Pick requests for scripts and synchronous code, httpx when you need async or want a timeout by default. Do not mix both in one codebase without a reason.
06Before you move on
You now have the same contract in two languages: the key comes from the environment and fails loudly when missing, parameters are encoded by the library, both the HTTP status and the envelope are checked, a non-JSON body is survived, and errors carry their status so callers can make decisions. In Python, a session makes all of that cheaper per call.
What neither helper does yet is cope with a slow or flaky network. The next lesson adds timeouts you control, and retries that only fire when retrying can actually help: Timeouts, retries and backoff.