Sending mail might sound easy, but to avoid getting your mail caught in spam filters, and to prevent others from sending spoofed email in your name, you need to employ different preventive methods. One such method is DMARC, which allows domain administrators to apply policies with regard to email authentication. You also have the possibility to get reports sent to you with the results of the applied policy.

This article shows how you can use existing open-source tools to visualize these reports in a graphical way, self-hosted on your own servers, without having to submit your reports to a third party.

Introduction

The email protocols used today were constructed in a time where computers and users on the network were considered trusted. Today, this causes problems with unsolicited and spoofed email, which pose a threat to users all over the world.

Several methods have been proposed to provide email authentication. DMARC (Domain-based Message Authentication, Reporting and Conformance) is one such protocol. DMARC allows a domain owner to publish a policy of requirements that email sent from this domain should fullfil. The receiving mail server will then evaluate the policy, and if the policy does not match, this could indicate that the email was spoofed. The receiving mail server may then take other actions, such as marking it as spam or discarding it.

DMARC

DMARC is based on the use of two other methods, namely SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail). DMARC ties them together by allowing a domain administrator to state if any or both are expected to be used for sending email with a particular domain as sender.

Briefly, SPF is used to set a predefined set of mail servers as authorized to send email for a particular domain. The SPF record is set on the domain, and the receiving mail server can then check if the email originated from the expected mail server by matching the IP address or the domain name of the server.

DKIM instead adds a signature to the email at the sender mail server. This signature can then be verified by the receiving server, to see if it matches. This also gives an assurance that the email originated from the expected domain, since only mail servers on that domain should be able to create signatures.

DMARC Reports

The DMARC policy supports the sending of DMARC reports, where the receiving mail server sends a (usually) daily report of mail received from a particular domain. These reports can be sent by mail, and if configured as in the example record below, [email protected] would receive these reports.

_dmarc.example.com. 7200    IN    TXT    "v=DMARC1; p=none; 
rua=mailto:[email protected]; pct=100"

The DMARC reports are sent as XML files, but are not very practical for humans to read, especially not since you may receive several of them every day. Instead, what we want to do is to visualize this in a more human-friendly way.

Existing Open-Source Projects

After searching the internet for projects that parse DMARC reports, I started looking at parsedmarc, an open-source project hosted on Github. It has a lot of desirable features, for example:

It can handle the reports both when they are plain xml files, as well as when they are compressed with zip or gzip.
It can read directly from an inbox, if desired, or just read saved reports from disk.
The aggregated output can be outputted either as a JSON file, or sent to Elasticsearch for further processing.
There are several templates available for visualizing the output using e.g., Grafana, Splunk, or Kibana.

While templates are provided inside the Git repository, connecting the different components are non-trivial. You would have to run parsedmarc, set up an Elasticsearch instance, and then connect this to a Grafana instance to visualize it. This requires a lot of manual configuration if you just want to try things out.

However, I will now show how you can connect these components together using Docker.

Connect Everything Together

To simplify things a bit, I decided to use docker-compose to connect three different components together:

parsedmarc to parse the reports
Elasticsearch to store the aggregated data
Grafana to visualize the results

In this way, you can simply spin up a series of Docker containers, without a manual installation process, which provides a Grafana dashboard available in the browser.

I have prepared a repository with the required files available on Github at the following address https://github.com/debricked/dmarc-visualizer. The docker-compose.yml file looks like this:

version: '3.7'
services:
  parsedmarc:
    build: ./parsedmarc/
    volumes:
      - ./files:/input:ro
      - ./output_files:/output
    command: parsedmarc -c /parsedmarc.ini /input/*
    depends_on:
      - elasticsearch
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    environment:
      - discovery.type=single-node
  grafana:
    build: ./grafana/
    ports:
      - 3000:3000
    environment:
        GF_INSTALL_PLUGINS: grafana-piechart-panel,grafana-worldmap-panel
        GF_AUTH_ANONYMOUS_ENABLED: 'true'

In this example, I have chosen to read all DMARC reports from a directory called

files/

, which will be parsed by

parsedmarc

. If desired, you could also configure parsedmarc to read directly from your inbox.

The aggregated results are stored in Elasticsearch. After this, Grafana is configured with the dashboard from parsedmarc, and also configured to connect to the Elasticsearch instance.

This configuration is done in

grafana/Dockerfile

and the

grafana/grafana-provisioning

directory, if you want to see the details.

(An additional note, if you want to be able to see geographical information from the DMARC reports, i.e., the country corresponding to certain IP-addresses, you need to register and download the GeoIP2 database yourself, since we cannot redistribute it. You can find instructions here https://dev.maxmind.com/geoip/geoip2/geolite2/.

You can then copy the correct files to

parsedmarc/

and modify

parsedmarc/Dockerfile

Final Result

You can now bring up the containers with

docker-compose up

. It will take a while for all containers to start, and you might temporarily see some errors until Elasticsearch is up, but after this, parsedmarc will start to parse all DMARC reports. When the parsedmarc container has exited successfully, you can visit http://localhost:3000 to see the Grafana dashboard.

Click Home in the top-left corner, and then click on the “DMARC Reports” dashboard. You will see the following view:

As you can see, you can view a lot of information, for example the individual SPF and DKIM passage, as well as the final DMARC passage. Recall that DMARC Passage can be true if either SPF or DKIM alignment is true. You can also see graphs over time, so that you can monitor how alignment changes over time, for example after a policy or configuration change.

Final Words

In this article I have showed how you can combine some great open-source resources together using Docker, to get a solution that is easy to deploy on your own system. Implementing all parts yourself would have been a huge task, but now when I could use these already existing components, I could quite quickly analyse our ever growing pile of DMARC reports.

Interested in more how to’s? Visit our blog!
Previously published at https://debricked.com/blog/2020/05/14/analyse-and-visualize-dmarc-results-using-open-source-tools/