Logging Primer

Last updated: March 24, 2017

Overview

The bidder can log 5 kinds of data: Requests, Bids, Wins, Pixels, Clicks, Status (not counting the log stored in /var/log/rtb4free.log

The logs can be sent to a file in the XRTB/logs directory or onto a ZeroMQ channel. The logs are setup in the zeromq object in the ~XRTB/Campaigns/payday.json file. Y

ou can use the System Console's Configuration tab to set up where the logs go.. Or you can edit the file directly. Here's an example definition of zeromq in ~XRTB/Campaigns/payday.json:
"zeromq": {
			"bidchannel": "file://logs/bids",
			"responses": "tcp://*:5575&responses",
			"nobid": "",
			"winchannel": "tcp://*:5572&wins",
			"requests": "file://logs/request&time=5",
			"logger": "tcp://*:5574&logs",
			"clicks": "tcp://*:5573&clicks",
			"subscribers": {
				"hosts": ["localhost", "192.168.1.167"],
				"commands": "5580"
			},
			"requeststrategy": 10,
			"status": "file://logs/status&time=30"
		},								
								

Here we can see bids are going to ~XRTB/logs/bids file. wins are going out on a ZeroMQ channel. Notice that the requeststrategy is set to 10. This field can be a string, one of "all", "requests", "bids", "wins". The "all" means log everything. The term "bids" means log bids and wins. The term "wins" means log just wins.

You can also use a number, which means log a random sample of x% of the requests + all bids, all wins, and all wins. Note, pixels are always logged. Using a percentage is useful when you just can't keep up with all the request data. Here you can choose how much of the request data to keep. The number 10 means log a request if you bid, and 10% of all those you don't bid on, plus all the wins and bids.

You can override this percentage by exchange by modifying the Bid Endpoint in the seats object set forth in System Console's Configuration tab.

Log Types

The log types that RTB4FREE generates are requests, bids, wins, pixels, clicks and status

Requests

Bids

Bid sample goes here

Wins

Win sample goes here

Pixels

Pixel sample goes here.

Clicks

Click sample goes here

Status

Status sample goes here

Logging to File

Discuss file logging

Logging to Zeromq

Discuss Zeromq logging.

Logstash

This section deals with using Logstash. On the bidder we use Filebeat to transfer the logs to Logstash.

Filebeat

Discuss filebeat.

Elastic Search

Discuss how to shove data into ELK.

S3

Discuss how to cram stuff into S3

© Ben M. Faul, Peter Loh, RTB4FREE.COM 2016
Email: Ben.Faul@gmail.com