Deploying RTB Services

Last updated: February 18, 2019

Overview

This section shows how the comonents of the RTB4FREE platform can be deployed as a Docker Swarm.

  • For each physical server in your swarm, you must install Docker.
  • Create your swarm.
    docker swarm init
    
    Note the docker command required for other nodes to join this swarm
  • Add each additional physical server to your swarm.
    docker swarm join --token token master_node_IP:port
    
  • If you want to specify the server for each service, create labels for each node.
  • Create network overlays that are shared among your stacks. We recommend creating separate networks and physical interfaces, if possible, for:
    • Exchange traffic - Inbound from Exchanges to Load Balancer; Load Balancer to Bidders.
    • Log traffic - Load Balancer to Kafka; Kafka to log consumers (Logstash).
    • Control traffic - Crosstalk/Bidders/Zerospike; MySQL to Crosstalk/Campaign Manager; Users to Kibana/Campaign Manger.
    docker network create --driver overlay  rtb_net
    docker network create --driver overlay  kafka_net
    docker network create --driver overlay  control_net
    
  • For services that require persistent data storage across service restarts, create directories on the server running the service, and mapping them using the compose volume directive. For the RTB4FREE platform, these are Elasticsearch, MySQL, and Zerospike. See the compose files below for examples.
  • You can now deploy your stacks by creating compose files for each stack, and deploying:
    docker stack deploy -c compose_file stack_name 
    
  • After starting all stacks, the platform is operational. You can now define campaigns in the Campaign Manager, load the bidders with campaigns, connect the exchange to send requests to the bidders, and see logging activity on Kibana and the Campaign Manger Dashboard.
  • To stop a stack, remove it from the swarm:
    docker stack rm stack_name 
    

RTB Stack

The RTB stack will deploy the following containers:

  • Bidders.
  • Crosstalk.
  • Zerospike.

The docker compose file is:

version: "3"
services:

  zerospike:
    image: "64.94.191.15:5000/zerospike"
    environment:
      BROKERLIST: "kafka1:9093,kafka2:9094,kafka3:9095"
    ports:
      - "2000:2000"
      - "2001:2001"
      - "2002:2002"
#    volumes:
#      - "/tmp/cache.db:/cache.db"
    networks:
      - control_net
      - kafka_net
    depends_on:
      - kafka

  crosstalk:
    image: "64.94.191.15:5000/crosstalk"
    ports:
      - "8200:8200"
    environment:
      REGION: "US"
      GHOST: "elastic1"
      AHOST: "elastic1"
      BROKERLIST: "kafka1:9093,kafka2:9094,kafka3:9095"
      PUBSUB: "zerospike"
      CONTROL: "8100"
      JDBC: "jdbc:mysql://db/rtb4free?user=ben&password=test"
      PASSWORD: "iamspartacus"
#    volumes:
#      - ./crosstalk/config.json:/CROSSTALK/config.json
    networks:
      - kafka_net
      - control_net
    depends_on:
      - kafka
      - zerospike

  bidder1:
    image: "64.94.191.15:5000/bidder3"
    environment:
      BROKERLIST: "kafka1:9093,kafka2:9094,kafka3:9095"
      PUBSUB: "zerospike"
      EXTERNAL: "http://localhost:8080"
    ports:
      - "8080:8080"
      - "8100:8100"
      - "7379:7379"
    networks:
      - rtb_net
      - kafka_net
      - control_net
    depends_on:
      - kafka
      - crosstalk
      - zerospike

networks:
  kafka_net:
    external: true
  rtb_net:
    external: true
  control_net:
    external: true

									

Kafka Stack

The Kafka stack will deploy the following containers:

  • Kafka cluster instances.
  • Zookeper cluster instances.

The docker compose file for a single node kafka and zookeeper is:

version: "3"
services:
  zookeeper:
    image: "jplock/zookeeper"
#    ports:
#      - "2181:2181"
    networks:
      - control_net
  kafka:
    image: "ploh/kafka"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
# This option enables montoring of kafka metrics
#      KAFKA_OPTS: -javaagent:/usr/app/jmx_prometheus_javaagent.jar=7071:/usr/app/prom-jmx-agent-config.yml
    ports:
      - "9092:9092"
      - "7071:7071"
    depends_on:
      - zookeeper
    networks:
      - kafka_net
      - control_net

networks:
  kafka_net:
    external: true

networks:
  kafka_net:
    external: true
  control_net:
    external: true

									

ELK Stack

The ELK stack will deploy the following containers:

  • Elasticsearch cluster instances.
  • Logstash.
  • Kibana.

The docker compose file for a single node Elasticsearch cluster is:

version: "3"
services:
  elastic1:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2
    ports:
      - "9200:9200"
      - "9300:9300"
    environment:
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
    volumes:
      - ./esdata1:/usr/share/elasticsearch/data
    networks:
      - kafka_net
      - control_net

  logstash1:
    image: docker.elastic.co/logstash/logstash:6.1.1
    ports:
      - "5044:5044"
    environment:
      - "XPACK_MONITORING_ELASTICSEARCH_URL=http://elastic1:9200"
      - "XPACK_MONITORING_ENABLED=true"
      - "LS_JAVA_OPTS=-Xmx1g"

    networks:
      - kafka_net
      - control_net
    volumes:
      - "./logstash:/usr/share/logstash/pipeline"

  kibana:
    image: docker.elastic.co/kibana/kibana:6.2.2
    environment:
      - SERVER_NAME=elastic1
      - ELASTICSEARCH_URL=http://elastic1:9200
    ports:
      - "5601:5601"
    networks:
      - control_net

networks:
  kafka_net:
    external: true
  control_net:
    external: true
									

Campaign Manager Stack

The ELK stack will deploy the following containers:

  • Campaign Manager Rails application.
  • MySQL database.

The docker compose file for this stack is:

version: '3'
services:
  db:
    image: mysql/mysql-server:5.7
    ports:
    - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=rtb4free
      - MYSQL_DATABASE=rtb4free
      - MYSQL_USER=ben
      - MYSQL_PASSWORD=test
    volumes:
#  Uncomment to initialize mysql database with default values on startup
      - /home/ubuntu/docker_kafka/db/rtb4free_demo_base_02-26-2018.sql:/docker-entrypoint-initdb.d/rtbdemo.sql
#  Uncomment to use local directory as persistent volume
      - /home/ubuntu/docker_kafka/mysqldata:/var/lib/mysql
    networks:
      - control_net
  web:
    image: ploh/rtbadmin_open
#    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    command: bash -c "./wait_for_it.sh db:3306 --timeout=120; bundle exec rails s -p 3000 -b '0.0.0.0' "
    ports:
      - "3000:3000"
    environment:
      - CUSTOMER_NAME=RTB4FREE
      - RTB4FREE_DATABASE_HOST=db
      - RTB4FREE_DATABASE_PORT=3306
      - RTB4FREE_DATABASE_USERNAME=ben
      - RTB4FREE_DATABASE_PASSWORD=test
      - RTB4FREE_DATABASE_NAME=rtb4free
      - ELASTICSEARCH_ENABLE=false
      - ELASTICSEARCH_HOST=
      - ELASTICSEARCH_KIBANA_URL=
      - RTB_CROSSTALK_PORT=8100
      - RTB_CROSSTALK_USER=ben
      - RTB_CROSSTALK_PASSWORD=iamspartacus
#    volumes:
#      - ./web/1_rtb4free.rb:/rtb4free_admin/config/initializers/1_rtb4free.rb
    depends_on:
      - db
    networks:
      - control_net

networks:
  control_net:
    external: true
									

HAProxy Stack

The HAProxy stack will deploy the following containers:

  • HAProxy.

The docker compose file for this stack is:

version: '3'
services:
  proxy:
    image: haproxy
    ports:
      - 80:80
      - 443:443
      - 4433:4433
      - 9000-9016:9000-9016
    networks:
      - frontend
    volumes:
    # Place the haproxy configuration file here and map to container's config file.
      - ./haproxy:/usr/local/etc/haproxy    
networks:
  rtb_net:
    external: true

All-In-One Stack

For development or demo purposes, you can deploy the entire RTB4FREE serivce as a single stack. This wil deploy the following containers:

  • Bidder.
  • Crosstalk.
  • Zerospike.
  • Kafka.
  • MySQL.
  • Campaign Manger Rails App.
  • Elasticsearch.
  • Logstash.
  • Kibana.
  • Simulator to send sample requests to the bidder.
All clusters are single node, and there is no data persistence when the stack is stopped. A single network is defined.

The docker compose file for this stack is:


version: "3"
services:
  zookeeper:
    image: "zookeeper"
  kafka:
    image: "ches/kafka"
    environment:
      ZOOKEEPER_IP: "zookeeper"
    ports:
      - "9092:9092"
    depends_on:
      - zookeeper

  zerospike:
    image: "jacamars/zerospike:v1"
    environment:
      BROKERLIST: "kafka:9092"
    depends_on:
      - kafka
    command: bash -c "sleep 5 && ./wait-for-it.sh kafka:9092 -t 120 && sleep 1; ./zerospike"

  bidder:
    image: "jacamars/rtb4free:v1"
    environment:
      BROKERLIST: "kafka:9092"
      PUBSUB: "zerospike"
      EXTERNAL: "http://localhost"
      ADMINPORT: "0"
      ACCOUNTING: "accountingsystem"
      FREQGOV: "false"
      INDEXPAGE: "/index.html"
    ports:
      - "80:8080"
      - "8155:8155"
    depends_on:
      - kafka
      - zerospike
    command: bash -c "sleep 5 && ./wait-for-it.sh kafka:9092 -t 120 && ./wait-for-it.sh zerospike:6000 -t 120 && sleep 1; ./rtb4free"
  crosstalk:
    image: "jacamars/crosstalk:v1"
    environment:
      REGION: "US"
      GHOST: "elastic1"
      AHOST: "elastic1"
      BROKERLIST: "kafka:9092"
      PUBSUB: "zerospike"
      CONTROL: "8100"
      JDBC: "jdbc:mysql://db/rtb4free?user=ben&password=test"
      PASSWORD: "iamspartacus"
    depends_on:
      - kafka
      - zerospike
  db:
    image: ploh/mysqlrtb
    environment:
      - MYSQL_ROOT_PASSWORD=rtb4free
      - MYSQL_DATABASE=rtb4free
      - MYSQL_USER=ben
      - MYSQL_PASSWORD=test
  web:
    image: ploh/rtbadmin_open
    command: bash -c "./wait_for_it.sh db:3306 --timeout=120; bundle exec rails s -p 3000 -b '0.0.0.0' -e development"
    ports:
      - "3000:3000"
    environment:
      - CUSTOMER_NAME=RTB4FREE
      - RTB4FREE_DATABASE_HOST=db
      - RTB4FREE_DATABASE_PORT=3306
      - RTB4FREE_DATABASE_USERNAME=ben
      - RTB4FREE_DATABASE_PASSWORD=test
      - RTB4FREE_DATABASE_NAME=rtb4free
      - ELASTICSEARCH_ENABLE=true
      - ELASTICSEARCH_HOST=elastic1:9200
      - ELASTICSEARCH_KIBANA_URL=http://kibana:5601/
      - RTB_CROSSTALK_REGION_HOSTS={"US" => "crosstalk"}
      - RTB_CROSSTALK_PORT=8100
      - RTB_CROSSTALK_USER=ben
      - RTB_CROSSTALK_PASSWORD=iamspartacus
  elastic1:
    image: ploh/elastic_pwd
    environment:
      - discovery.type=single-node

      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
  logstash1:
    image: ploh/logstash_pwd
    environment:
      - "XPACK_MONITORING_ELASTICSEARCH_URL=http://elastic1:9200"
      - "XPACK_MONITORING_ENABLED=true"
  kibana:
    image: docker.elastic.co/kibana/kibana:6.2.2
    environment:
      - SERVER_NAME=elastic1
      - ELASTICSEARCH_URL=http://elastic1:9200
    ports:
      - "5601:5601"
  simulator:
    image: "jacamars/rtb4free:v1"
    environment:
      BIDDER: "bidder:8080"
      WIN:    "10"
      PIXEL:  "95"
      CLICK:  "2"
      SLEEP:  "100"
    command: bash -c "./wait-for-it.sh bidder:8080 -t 120 && sleep 60;  ./load-elastic -host $$BIDDER -win $$WIN -pixel $$PIXEL -click $$CLICK -sleep $$SLEEP"