Lesson 2 asked whether a visitor is hiding. This lesson asks a different question: has this address, or this domain, already been caught doing something bad? That is reputation, and it comes from blocklists that security teams and volunteer projects maintain by watching attacks happen.
Reputation is useful in two places. On the way in, it scores the address a request comes from: an IP that spent last week brute-forcing SSH servers deserves more scrutiny on your login form. On the way out, it protects your users from each other: if your product lets people post links in comments, messages, profiles or support tickets, someone will eventually post a phishing page, and your domain will be the one that delivered it.
01Reputation is history, not identity
Every blocklist entry records that an address or domain was seen doing something at some point. Three things weaken that record over time, and your code has to account for all of them.
- Addresses are reassigned. A residential ISP hands the same address to different subscribers over weeks. A cloud provider hands a released server address to the next customer within minutes. The new holder inherits the old one's listing.
- Addresses are shared. Behind carrier-grade NAT, one infected phone can get an address listed that thousands of other phones also use. Corporate networks and universities have the same problem at smaller scale.
- Domains are cleaned up, or not yet listed. A compromised WordPress site may host a phishing kit for two days, get listed, get cleaned and stay listed for a while. A brand-new phishing domain registered this morning is on no list at all.
The practical consequence: a blocklist hit raises the risk of a request; it does not establish guilt. A clean result fails to raise it; it does not establish safety. Weight both accordingly.
02Check an IP against blocklists
IP Blacklist Lookup (ipblacklistlookup) takes an ip on a GET and costs 10 credits. Try it on the example from its definition, a well-known Tor exit:
curl "https://api.apiverve.com/v1/ipblacklistlookup?ip=185.220.101.1" \
-H "x-api-key: YOUR_API_KEY"isIPBlacklisted is on every plan. inbound, outbound, threatLevel and isTor are premium.
Every plan gets ipAddress and the one boolean that matters most, isIPBlacklisted. Higher plans break it down:
| Field | Plan | Meaning |
|---|---|---|
isIPBlacklisted | all | Listed on any of the blocklists checked |
inbound.found | premium | Listed as a source of attacks: spam, scanning, brute force |
outbound.found | premium | Listed as a destination: command-and-control server, malware host, phishing |
threatLevel | premium | Overall severity; none for a clean address. The Tor exit above returned critical |
isTor | premium | Whether the address is a Tor exit |
ipDetails, asn, asnName | premium | Geolocation and network, as in lesson 1 |
The inbound and outbound split is worth understanding even if your plan only has the boolean, because it tells you which direction the risk runs. For a visitor hitting your login form, inbound is what matters: this address attacks things. For a server that a user's link points to, outbound is what matters: this address hosts things that hurt people. The Tor exit above is listed both ways, which is common for exits, since some of the traffic leaving them is abuse.
A clean address, such as Google's resolver 8.8.8.8, returns isIPBlacklisted: false with threatLevel: "none" and null for both inbound and outbound.
Cache blocklist results for a few hours. Lists update continually, and an address that was cleaned up should not stay penalised in your system for a week because of your cache.
Your login endpoint sees a request from an address with isIPBlacklisted: true. On review, the address belongs to a large mobile carrier. What is the sensible response?
03Scan links in user content for phishing
The second half of reputation protects your users. Phishing Domain Checker (phishingcheck) checks a domain against a database of known phishing sites built from two independent blocklists and refreshed every six hours. It is a POST with a domain field and costs 10 credits.
curl -X POST "https://api.apiverve.com/v1/phishingcheck" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"domain":"github.com"}'isPhishing is false for a legitimate domain. riskScore, riskLevel and sources are premium.
domain, isPhishing, inputType and originalInput are on every plan. Premium plans add matchedDomain, the sources that listed it, two structural flags, isPunycode and isIpAddress, and a 0-100 riskScore with a riskLevel band that combines the list match with those structural tells.
Two behaviours shape how you use it.
It checks hosts, not pages. Pass https://login.example-bank.com/verify?id=1 as the domain and the response's domain is login.example-bank.com: the scheme, path and query are dropped. That keeps the check fast and cacheable, but it means a phishing page hosted on a legitimate shared platform (a site builder, a file host, a forms service) is judged by the platform's domain, which is not on the list. Host-level checking catches the dedicated phishing domains that make up most campaigns; it will not catch every malicious path on a big shared host.
It knows what has been reported. The example domain in the API's own definition, a site that was listed when the definition was written, now returns isPhishing: false. Lists churn: sites get cleaned and delisted, new ones appear. A false means "not on a list right now", which is why the scanner below also checks structure and why you re-scan old content.
Structural tells you can check yourself
Two patterns show up in phishing links far more often than in honest ones, and you do not need a premium plan to see them.
- Punycode hosts (labels starting
xn--) can render as look-alikes of real brands.xn--pple-43d.comdisplays with a Cyrillic first letter that looks like theainapple.com. The premium response for it reportsisPunycode: trueand ariskScoreof 40 even though it is on no list. - Raw IP hosts such as
http://192.168.1.10/login: real services almost never send users to a bare address. The premium response reportsisIpAddress: trueand ariskScoreof 58.
Both are a one-line check on the parsed hostname, so the scanner computes them locally and saves the call for the list lookup.
04Where is a link hosted?
Sometimes you want to know what is behind a domain, not just whether the domain is listed. URL Lookup (urllookup) resolves a URL to the IP of its first DNS A record and geolocates that address. It is a POST with a url field, 10 credits.
curl -X POST "https://api.apiverve.com/v1/urllookup" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://www.google.com"}'domain, ip, country and continent are on every plan; city, state and coordinates are premium.
The obvious chain is URL Lookup, then IP Blacklist Lookup on the returned ip, looking for an outbound listing. It is a reasonable extra check for links to unfamiliar small sites. It is close to meaningless for anything behind a large CDN: those addresses front millions of unrelated sites, so the IP's reputation says nothing about the one domain you care about. And since DNS answers change, the ip you get today may not be the one a user's browser gets tomorrow. Treat URL Lookup as an investigation tool for your moderation queue rather than a gate on every post; at 20 credits per link for the pair, it is too expensive to run blind.
05Build a link scanner
The scanner takes a block of user text and returns a verdict: publish, hold for a moderator, or reject. It extracts URLs, normalises their hosts, removes duplicates, caps how many hosts one post can make you pay for, applies the free structural checks, then asks the phishing list about whatever is left. Python 3.9+ with requests:
import ipaddress
import os
import re
import time
from urllib.parse import urlsplit
import requests
URL_RE = re.compile(r"""(?:https?://|www\.)[^\s<>"'()]+""", re.IGNORECASE)
MAX_HOSTS = 5 # one post can never cost more than 5 lookups
TTL = 6 * 60 * 60 # matches the list's six-hour refresh
_cache = {}
def hosts_in(text):
seen = []
for raw in URL_RE.findall(text or ""):
url = raw if "://" in raw else f"http://{raw}"
host = (urlsplit(url.rstrip(".,;:!?")).hostname or "").rstrip(".").lower()
if host and host not in seen:
seen.append(host)
return seen
def is_ip(host):
try:
ipaddress.ip_address(host)
return True
except ValueError:
return False
def is_phishing(host):
hit = _cache.get(host)
if hit and hit[1] > time.time():
return hit[0]
res = requests.post(
"https://api.apiverve.com/v1/phishingcheck",
json={"domain": host},
headers={"x-api-key": os.environ["APIVERVE_API_KEY"]},
timeout=3,
)
res.raise_for_status()
listed = bool(res.json()["data"]["isPhishing"])
_cache[host] = (listed, time.time() + TTL)
return listed
def scan(text, trusted=frozenset({"apiverve.com", "github.com"})):
hosts = hosts_in(text)
findings = []
if len(hosts) > MAX_HOSTS:
return {"verdict": "hold", "findings": [{"reason": "too-many-links", "count": len(hosts)}]}
for host in hosts:
if host in trusted or any(host.endswith("." + t) for t in trusted):
continue
if is_ip(host):
findings.append({"host": host, "reason": "ip-host"})
continue
if any(label.startswith("xn--") for label in host.split(".")):
findings.append({"host": host, "reason": "punycode"})
try:
if is_phishing(host):
findings.append({"host": host, "reason": "listed-phishing"})
except requests.RequestException:
findings.append({"host": host, "reason": "check-failed"})
reasons = {f["reason"] for f in findings}
if "listed-phishing" in reasons:
verdict = "reject"
elif reasons:
verdict = "hold"
else:
verdict = "publish"
return {"verdict": verdict, "findings": findings}
if __name__ == "__main__":
print(scan("Docs at https://github.com/apiverve and a login page at http://192.168.1.10/login"))Run against the sample text, it skips github.com as trusted, flags the raw IP host without spending a credit, and returns hold.
Notice the asymmetry in failure handling. In lesson 2 the network gate failed open, because blocking a login over a timeout hurts a real user. Here a failed check sends the post to the moderation queue instead, because publishing an unchecked link to everyone who reads the thread is the riskier mistake, and a short delay on one post is cheap. Same principle, different answer: decide failure behaviour by what each mistake costs on that surface.
The trusted list matters more than it looks. Links to your own docs, your status page and a handful of big platforms make up most links in most communities. Skipping them keeps the per-post cost near zero for ordinary use.
The scanner sends a post to the hold queue because the phishing check timed out. Why not publish it, given that the network gate in lesson 2 fails open?
06What to do with a hit
Reject quietly, but tell the poster. For a listed domain, refuse the post with a plain message: "One of the links in your message points to a site reported for phishing." Most people who post such a link were themselves fooled; telling them helps. Do not explain your thresholds.
Hold, do not delete. Structural flags and failed checks go to a moderator with the findings attached. A punycode link to a legitimate site in a non-Latin script is perfectly normal for some communities; a human makes that call in seconds.
Render user links defensively regardless. Even links that pass should not borrow your domain's authority:
<a href="https://example.org/article" rel="ugc nofollow noopener noreferrer" target="_blank">example.org/article</a>ugc and nofollow tell search engines the link is user content you do not vouch for; noopener stops the opened page from controlling your tab. Showing the real host as the link text, rather than whatever text the user chose, defeats the oldest trick in phishing.
Re-scan what you already published. A link that was clean when posted can be listed a week later. Re-check hosts from recent content on a schedule, newest first, and unpublish anything that turns up. The six-hour cache keeps that affordable, since the same few hosts recur across many posts.
07Privacy and fairness
Blocklists are run by third parties with their own criteria. If a customer's server address is listed because a previous tenant misbehaved, your policy should give them a way out: a support path, an allow-list entry after review, or a step-up they can pass. Log the specific signal behind every decision so that path is possible. Scanning content that users post publicly is ordinary moderation; scanning private messages deserves a line in your terms and privacy notice saying you check links for security.
The last lesson moves from the network to the client itself: what the user agent claims, how to tell a crawler from a browser, and how to combine everything from this course into one decision. Continue to Bots and user agents.