Django metrics with ELK and Elastic APM

Now that we have our Django project with ELK, adding Elastic APM is a breeze. We will create a Docker container for the server and then configure the client for our Django project to use it.

 

compose/production/apm/Dockerfile

FROM elastic/apm-server:6.2.2

COPY ./compose/production/apm/apm-server.yml /usr/share/apm-server/apm-server.yml

We will use the official Elastic image, the source is located here. Although the official documentation states the apm-server.yml file should be in /etc/apm-server/apm-server.yml, this image uses /usr/share/apm-server/apm-server.yml.

 

compose/production/apm/apm-server.yml

apm-server:
  host: 0.0.0.0:8200

output.elasticsearch:
  hosts: ['elk:9200']
  username: ${ELASTICSEARCH_USER:elastic}
  password: ${ELASTICSEARCH_PASSWORD:changeme}

setup.kibana:
  host: 'elk:5601'

setup.dashboards.enabled: true

First, we want our APM server to listen on all interfaces, so we use 0.0.0.0 as the host. By default, it listens on localhost, so our Django project wouldn't be able to contact the server since it is on a different machine. We'll use the previously configured ElasticSearch as the output, and configure it for Kibana with dashboards.

 

elk.yml

version: '2'

volumes:
  elk_data_local: {}

services:
  elk:
    build:
      context: .
      dockerfile: ./compose/production/elk/Dockerfile
    env_file: .env
    ports:
      - "5601:5601"
      - "9200:9200"
      - "5044:5044"
    volumes:
      - elk_data_local:/var/lib/elasticsearch
  apm:
    build:
      context: .
      dockerfile: ./compose/production/apm/Dockerfile
    env_file: .env
    ports:
      - "8200:8200"
    depends_on:
      - elk

On our elk.yml file, we'll add the APM service. Pretty straightforward, the only issue with this right now is that when starting the services, the APM server will try to contact ElasticSearch and since it's not up yet, it will fail to launch. To avoid this for now we can start ELK first and when everything is up, start APM. Ideally, we should do something like the ELK image does: automatically wait on ES to start before starting Logstash and Kibana.

 

.env

#APM

ELASTIC_APM_SERVICE_NAME=portfolio
ELASTIC_APM_SERVER_URL=http://internal-elk.yourdomain.com:8200

These variables are used for our Django client, you can also set a secret token here. For more information, you can head to the official documentation here.

 

config/settings/production.py

#ELASTIC APM CONFIGURATION
#-------------------------
INSTALLED_APPS += ['elasticapm.contrib.django', ]
ELASTIC_MIDDLEWARE = ['elasticapm.contrib.django.middleware.TracingMiddleware',]
MIDDLEWARE = ELASTIC_MIDDLEWARE + MIDDLEWARE

We'll add the APM app to our installed apps, and the tracing middleware for performance metrics.

 

requirements/production.txt

elastic-apm==2.2.1

And finally, the python library to our requirements.

 

After everything is configured and up and running, head to your Kibana interface and go to APM. You'll see the setup instructions and the possibility to check that the APM Server/Agent status is fine. More work is needed to make this more secure, such as HTTPS on ElasticSearch and using the secret token for the Django project, but for an initial setup, this is all you need!