Datafeed files | Specifications | Website Categorization Database | WhoisXML API

Datafeed files

A website categorization database is a database that classifies and categorises websites based on their content, purpose and other characteristics. The purpose of website categorization is to provide users with information about the nature of a website, helping them to make informed decisions about accessing or interacting with it. Our Website Categories Database supports 500+ categories.

The dataset comes in the two following forms

1. Daily Website Categorization Datafeed

The entire website categorization database is updated daily at 06:00 UTC.

Data for Daily Website Categorization Datafeed is obtained from Newly Registered Domains Data Feed. This approach helps to keep the database up to date.

  • Data included: domain, ASN, categories, domain created date (WHOIS), website responding (true/false) and country code.
  • Filename format: categories.%DATE%.[csv|json].gz
  • Average file sizes:
File Gzip size Unpacked size Rows
categories.%DATE%.daily.csv.gz 50MiB 250MiB 1.5M
categories.%DATE%.daily.json.gz 60MiB 500MiB 1.5M

2. Custom Website Categorization Database

You can configure your own dataset with your desired categories and countries. All updates from the Daily Website Categorisation Datafeed are guaranteed to hit the dataset for at least 24 hours after the daily release.

  • Data included: domain, ASN, categories, domain created date (WHOIS), website responding (true/false) and country code.
  • Filename format: categorization_db.%DATE%.%DATASET_ID%.categories.[csv|jsonl].gz



How to download the datafeed files

Downloading via HTTPS

Downloading via FTP

  • Host: datafeeds.whoisxmlapi.com
  • Port: 21210
  • Username: 'user'
  • Password: equal to your personal API Key which you can obtain from the My Products page.
  • Base path: ftp://datafeeds.whoisxmlapi.com:21210
  • Folder: Website_Categorization_Database or Website_Categorization_Custom

Downloading via FTPS

Read more about the FTPS connection: https://en.wikipedia.org/wiki/FTPS.

Our FTP server supports explicit FTP over TLS encryption. You may configure your FTP client to use explicit FTP over TLS encryption for secure communications.

Our FTPS server is accessed using the same paths and API keys as a regular FTP server, the instructions for which are described above. To connect via FTPS, select the "Require explicit FTP over TLS" encryption option in your FTP client, if it supports it. FileZilla configuration example:

explicit FTP over TLS encryption