After determining your basic requirements, you are now ready to build your DSP. We suggest a base system at first, with a single SSP, then after you get that operational you can then begin to scale.
We recommend building a base system first, using 4 different servers. These servers needed will be the following:
When completed, you can horizontally scale more bidders, and you can also add more servers and provide fault tolerance.
Each of these servers must have Docker running.
It is important to create meaningful hostnames so that you don't get confused at what you are looking at later. It is not useful to name one a 'bidder' and another 'crosstalk' because it is possible to run multiple bidders on multiple servers.
In addition, when you begin to scale horizontally you will be replicating hardware
In general, name your system by its region + its size. For example region EU might have these servers:
There are five Docker stack files. They handle the following services:
To make it easy, we have a Docker container that will generate the 5 stack files for you. Here's an example run:
$docker run REGION-NAME: EU RTB: EU-8c16g-1 PROXY: EU-8c16g-1 KAFKA: EU-8c16g-2 WEB: EU-8c16g-3 ELK: EU-12c32g-1 .... generating .... creating: docker-compose-rtb.yml creating: docker-compose-haproxy.yml creating: docker-compose-kafka.yml creating: docker-compose-web.yml creating: docker-compose-elk.yml Done!
Now that the stack files have been generated for you, now it's time to deploy.
Now we create the Master and worker docker nodes. Follow these steps:
Copy the 5 compose files to EU-8c16g-1. Then type this command:
$docker swarm init $docker network create --driver=overlay --attachable kafka_net $docker network create --driver=overlay --subnet 10.10.9.0/24 --attachable frontend
On each of the other 3 nodes start Docker as a worker node
The first service to start is the Web. This will bring up MySQL, Ruby on Rails, and Crosstalk. At this stage the campaign Manager will be operational. On EU-8c16g-1 execute the following command:
$docker stack deploy -c docker-compose-web.yml
The next thing to bring up is Kafka. This will start Zookeeper addition Kafka. All the other services depend on these. On EU-8c16g-1 execute the following command:
$docker stack deploy -c docker-compose-kafka.yml
The next thing to start is ELK. This will bring up Logstash, Kibana and ElasticSearch On EU-8c16g-1 execute the following command:
$docker stack deploy -c docker-compose-elk.yml
The next set of services to bring up is the bidder(s). This starts the RTB4FREE service and when it comes up, it will be ready for business, once campaigns are loaded. On EU-8c16g-1 execute the following command:
$docker stack deploy -c docker-compose-rtb.yml
Next service to bring up is HA Proxy. Bringing up the Proxy will cause bid requests to be routed to your bidders. On EU-8c16g-1 execute the following command:
$docker stack deploy -c docker-compose-haproxy.yml
At this point the system is operational. However some basic tests should be done to ensure the system is functioning normally.