Datafeed files | Specifications | Website Categorization Database | WhoisXML API
Products APIs Data Feeds Web Tools Domain Research & Monitoring Enterprise Packages
WHOIS / WHOIS History
WHOIS / WHOIS History

Provide current and historical ownership information on domains / IPs. Identify all connections between domains, registrants, registrars, and DNS servers.

DNS / DNS History
DNS / DNS History

Look into all current and historical DNS / IP connections between domains and A, MX, NS, and other records. Monitor suspicious changes to DNS records.

IP Geolocation / IP Netblocks
IP Geolocation / IP Netblocks

Get detailed context on an IP address, including its user’s geolocation, time zone, connected domains, connection type, IP range, ASN, and other network ownership details.

Domain Research Suite (DRS)
Domain Research Suite (DRS)

Access our web-based solution to dig into and monitor all domain events of interest.

Domain Research Suite (DRS)
Domain Research Suite (DRS)

Get access to a web-based enterprise-grade solution to search and monitor domain registrations and ownership details for branded terms, fuzzy matches, registrants of interest, and more.

Research
Monitoring
White-Label
WhoisXML API Predictive Threat Intelligence
Predictive Threat Intelligence

Detect and block access to and from dangerous domain names before malicious actors can weaponize them. Contact us today for more information.

WhoisXML API Internet Infrastructure
Internet Infrastructure

Unlock integrated intelligence on Internet properties and their ownership, infrastructure, and other attributes.

Enterprise API Packages
Enterprise API Packages

Our complete set of domain, IP, and DNS intelligence available via API calls as an annual subscription with predictable pricing.

Security Intelligence (SI) Suite
Security Intelligence (SI) Suite

Offers complete access to WHOIS, IP, DNS, and subdomain data for product enrichment, threat hunting and more.

Multi-Level API User Administration Now Available - Manage individual API keys for team members in your organization.

Learn More
×
Contact Us
Specifications Pricing Blog
Datafeed files Output format Supported categories

Datafeed files

A website categorization database is a database that classifies and categorises websites based on their content, purpose and other characteristics. The purpose of website categorization is to provide users with information about the nature of a website, helping them to make informed decisions about accessing or interacting with it. Our Website Categories Database supports 500+ categories.

The dataset comes in the two following forms

1. Daily Website Categorization Datafeed

The entire website categorization database is updated daily at 06:00 UTC.

Data for Daily Website Categorization Datafeed is obtained from Newly Registered Domains Data Feed. This approach helps to keep the database up to date.

  • Data included: domain, ASN, categories, domain created date (WHOIS), website responding (true/false) and country code.
  • Filename format: categories.%DATE%.[csv|json].gz
  • Average file sizes:
File Gzip size Unpacked size Rows
categories.%DATE%.daily.csv.gz 50MiB 250MiB 1.5M
categories.%DATE%.daily.json.gz 60MiB 500MiB 1.5M

2. Custom Website Categorization Database

You can configure your own dataset with your desired categories and countries. All updates from the Daily Website Categorisation Datafeed are guaranteed to hit the dataset for at least 24 hours after the daily release.

  • Data included: domain, ASN, categories, domain created date (WHOIS), website responding (true/false) and country code.
  • Filename format: categorization_db.%DATE%.%DATASET_ID%.categories.[csv|jsonl].gz



How to download the datafeed files

Downloading via HTTPS

Downloading via FTP

  • Host: datafeeds.whoisxmlapi.com
  • Port: 21210
  • Username: 'user'
  • Password: equal to your personal API Key which you can obtain from the My Products page.
  • Base path: ftp://datafeeds.whoisxmlapi.com:21210
  • Folder: Website_Categorization_Database or Website_Categorization_Custom

Downloading via FTPS

Read more about the FTPS connection: https://en.wikipedia.org/wiki/FTPS.

Our FTP server supports explicit FTP over TLS encryption. You may configure your FTP client to use explicit FTP over TLS encryption for secure communications.

Our FTPS server is accessed using the same paths and API keys as a regular FTP server, the instructions for which are described above. To connect via FTPS, select the "Require explicit FTP over TLS" encryption option in your FTP client, if it supports it. FileZilla configuration example:

explicit FTP over TLS encryption