Introduction

DNSDB Export is a Farsight subscription service that offers access to the full set (or a partial subset) of the underlying database files used by DNSDB.

DNSDB Export is intended to be installed on real or virtual servers owned or controlled by the customer within a secure hosting facility owned and controlled by the customer or their service provider. These servers must be managed securely to guarantee that DNSDB data is only disclosed to authorized users.

Per the subscription contract with Farsight, the DNSDB data must be segregated from other data sets and cannot be merged into other databases or mingled with non-Farsight data. If the Farsight subscription is terminated then all DNSDB data must be deleted and cannot be retained.

DNSDB Export data is stored in MTBL files and Flex files. The MTBL files are sorted string tables files with Non-DNSSEC and DNSSEC data separated into different sets of files. It is possible to be provisioned for any or all of the data sets.

Farsight provides the software to query these files using the same RESTful APIs as seen in the Farsight DNSDB service.

DNSDB Export will perform better as more RAM and faster disks are allocated. Our current recommended configuration (as of March 2021):

DNSDB Storage Requirements

When determining the disk requirements the following considerations must be made per each data set. Growth is the hardest to predict as the Farsight DNS Sensor network will fluctuate over time. Treat these growth numbers as estimations.

Considerations

  1. Historical file size
  2. File size growth
  3. Bufferspace

Non-DNSSEC (dns.* files)

  1. Historical (2010 - 2020): 11 TB, including 3.5 TB in 2020
  2. Growth: 19 TB
  3. Buffer 4 TB

Total: 34 TB

DNSSEC (dnssec.* files)

  1. Historical (2010 - 2020): 19 TB, including 6.4 TB in 2020
  2. Growth: 30 TB
  3. Buffer 5 TB

Total: 54 TB

Flex (*.tgdb files)

  1. Historical (2010 - 2020): 5.2 TB, including 2.7 TB in 2020
  2. Growth: 10 TB
  3. Buffer 3 TB

Total: 15.7 TB

Notes

A customer can limit what data they download and synchronize if necessary (DNS vs DNSSEC vs Flex, current year vs prior years, monthly-only vs monthly+daily vs monthly+daily+hourly). Most security analysts looking for pivot points in DNS are more interested in DNS data and might do without DNSSEC data. Some analysts might care only what is available in the last few weeks or months and not want to include historical data (or they’re not willing to pay for access to historical data).

Data in MTBL files can be exported into a JSON format. This JSON data will typically be 3.8x the size of the MTBL file from which it is exported.

The Flex (.tgdb) files are in a proprietary format from which we do not support extracting the raw data.

Network Requirements

A high speed network connection with sustained excess capacity of 100 Mbps is recommended. The database files are transferred over multiple HTTPS connections and is limited to a maximum of 100 Mbps to mitigate unexpected resource exhaustion. Depending on which file sets are chosen it is possible that 100 Mbps sustained network usage will be experienced for several days during the initial synchronization as well as future updates.

DNSDB Operating System Requirements

DNSDB Export Recommended Software:

We support and have binary packages for:

Consultation

The Farsight Customer Solutions Engineering team is always available for consultation to assist with design and implementation.

About Farsight Security

Farsight Security® Inc. is the world’s largest provider of historical and real-time DNS intelligence solutions. We enable security teams to qualify, enrich and correlate all sources of threat data and ultimately save time when it is most critical - during an attack or investigation. Our solutions provide enterprise, government and security industry personnel and platforms with unmatched global visibility, context and response. Farsight Security is headquartered in San Mateo, California, USA. Learn more about how we can empower your threat platform and security team with Farsight Security passive DNS solutions at www.farsightsecurity.com or follow us on Twitter: @FarsightSecInc.