- Iago Quick Start
- Iago Overview
- Iago Architecture Overview
- Implementing Your Test
- Configuring Your Test
- Contributing to Iago
NOTE: This repo has only recently been made public and our velocity is high at the moment. Please join [email protected] for updates and to ask questions.
If you are already familiar with the Iago Load Generation tool, follow these steps to get started; otherwise, start with the Iago Overview and perhaps Iago Philosophy, also known as "Why Iago?". For questions, please contact [email protected].
-
Download and unpack the Iago distribution. We support Scala 2.9 and recommend you clone the latest master: master.
-
Read the documentation.
- Identify your transaction source; see Transaction Requirements and Sources of Transactions for more information.
- In Scala, extend the Iago server's
RecordProcessororThriftRecordProcessorclass, or in Java, extendLoadTestorThriftLoadTest; see Implementing Your Test for more information. - Create a
launcher.scalafile in your Iagoconfigdirectory with the appropriate settings; see Configuring Your Test for more information.
Launch Iago from the distribution with java -jar iago_jar -f your_config. This will create the Iago processes for you and configure it to use your transactions. To kill a running job, add -k to your launch parameters: java -jar iago_jar -f your_config -k.
If you launch your Iago job on your local machine and an old Iago job is still running, it probably won't get far: it will attempt to re-use a port and fail. You want to kill the running job, as described above.
If you build via Maven, then you might wonder "How do I launch Iago 'from the distribution'?" The steps are:
% mvn package -DskipTests % mkdir tmp; cd tmp % unzip ../target/iago-version-package-dist.zip % java -jar iago-version.jar -f config/my_config.scala
Don't assume that you can skip the package/unzip steps if you're just changing a config file. You need to re-package and unzip again.
If you are using Iago as a library, for example, in the case of testing over the Thrift protocol or building more complex tests with HTTP or Memcached/Kestrel, you should instead add a task to your project's configuration. See Configuring Your Test for more information.
Iago is a load generation tool that replays production or synthetic traffic against a given target. Among other things, it differs from other load generation tools in that it attempts to hold constant the transaction rate. For example, if you want to test your service at 100K requests per minute, Iago attempts to achieve that rate.
Because Iago replays traffic, you must specify the source of the traffic. You use a transaction log as the source of traffic, in which each transaction generates a request to your service that your service processes.
Replaying transactions at a fixed rate enables you to study the behavior of your service under an anticipated load. Iago also allows you to identify bottlenecks or other issues that may not be easily observable in a production environment in which your maximum anticipated load occurs only rarely.
Iago can generate service requests that travel the net in different ways and are in different formats. The code that does this is in a Transport, a class that extends ParrotTransport. Iago comes with several Transports already defined. When you configure your test, you will need to set some parameters; to understand which of those parameters are used and how they are used, you probably want to look at the source code for your test's Transport class.
- HTTP: Use FinagleTransport
- Thrift: Use ThriftTransport
- Memcached: Use MemcacheTransport
- Kestrel: Use KestrelTransport
- UDP: Use ParrotUdpTransport
Your service is typically an HTTP or Thrift service written in either Scala or Java.
For replay, Iago recommends you scrub your logs to only include requests which meet the following requirements:
- Idempotent, meaning that re-execution of a transaction any number of times yields the same result as the initial execution.
- Commutative, meaning that transaction order is not important. Although transactions are initiated in replay order, Iago's internal behavior may change the actual execution order to guarantee the transaction rate. Also, transactions that implement
Futureresponses are executed asynchronously. You can achieve ordering, if required, by using Iago as a library and initiating new requests in response to previous ones. Examples of this are available.
Unless you change your configuration's reuseFile parameter, make sure that your sample log has at least 1000 items.
Transactions typically come from logs, such as the following:
- Web server logs capture HTTP transactions.
- Proxy server logs can capture transactions coming through a server. You can place a proxy server in your stack to capture either HTTP or Thrift transactions.
- Network sniffers can capture transactions as they come across a physical wire. You can program the sniffer to create a log of transactions you identify for capture.
In some cases, transactions do not exist. For example, transactions for your service may not yet exist because they are part of a new service, or you are obligated not to use transactions that contain sensitive information. In such cases, you can provide synthetic transactions, which are transactions that you create to model the operating environment for your service. When you create synthetic transactions, you must statistically distribute your transactions to match the distribution you expect when your service goes live.
Iago consists of one or more Iago feeders, which reads your transaction source, and one or more Iago servers, which format and deliver requests to the service you want to test. The feeder contains a Poller object, which is responsible for guaranteeing one minute's worth of transactions in the pipeline to the Iago servers. Metrics are available in logs, and we expect future enhancements to support parsing and visualizing this data.
The Iago servers generate requests to your service. Together, all Iago servers generate the specified number of requests per minute. A Iago server's RecordProcessor object executes your service and maps the transaction to the format required by your service.
The following sections show examples of implementing your test in both Scala and Java. See Code Annotations for the Examples for information about either example.
To implement a load test in Scala, you must extend the Iago server's RecordProcessor class to specify how to map transactions into the requests that the Iago server delivers to your service. The following example shows a RecordProcessor subclass that implements a load test on an EchoService HTTP service:
package com.twitter.example
import org.apache.thrift.protocol.TBinaryProtocol
import com.twitter.parrot.processor.RecordProcessor // 1
import com.twitter.parrot.thrift.ParrotJob // 2
import com.twitter.parrot.server.{ParrotRequest,ParrotService} // 3
import com.twitter.logging.Logger
import org.jboss.netty.handler.codec.http.HttpResponse
import thrift.EchoService
class EchoLoadTest(parrotService: ParrotService[ParrotRequest, HttpResponse]) extends RecordProcessor {
val client = new EchoService.ServiceToClient(service, new TBinaryProtocol.Factory()) // 4
val log = Logger.get(getClass)
def processLines(job: ParrotJob, lines: Seq[String]) { // 5
lines map { line =>
client.echo(line) respond { rep =>
if (rep == "hello") {
client.echo("IT'S TALKING TO US") // 6
}
log.info("response: " + rep) // 7
}
}
}
}To implement a Thrift load test in Scala, you must extend the Iago server's Thrift RecordProcessor class to specify how to map transactions into the requests that the Iago server delivers to your service. The following example shows a ThriftRecordProcessor subclass that implements a load test on an EchoService Thrift service:
package com.twitter.example
import org.apache.thrift.protocol.TBinaryProtocol
import com.twitter.parrot.processor.ThriftRecordProcessor // 1
import com.twitter.parrot.thrift.ParrotJob // 2
import com.twitter.parrot.server.{ParrotRequest,ParrotService} // 3
import com.twitter.logging.Logger
import thrift.EchoService
class EchoLoadTest(parrotService: ParrotService[ParrotRequest, Array[Byte]]) extends ThriftRecordProcessor(parrotService) {
val client = new EchoService.ServiceToClient(service, new TBinaryProtocol.Factory()) // 4
val log = Logger.get(getClass)
def processLines(job: ParrotJob, lines: Seq[String]) { // 5
lines map { line =>
client.echo(line) respond { rep =>
if (rep == "hello") {
client.echo("IT'S TALKING TO US") // 6
}
log.info("response: " + rep) // 7
}
}
}
}To implement a load test in Java, you must extend the Iago server's LoadTest class to specify how to map transactions into the requests that the Iago server delivers to your service. The LoadTest class provides Java-friendly type mappings for the underlying Scala internals. The following example shows a LoadTest subclass that implements a load test on an EchoService HTTP service:
package com.twitter.jexample;
import com.twitter.example.thrift.EchoService;
import com.twitter.parrot.processor.LoadTest; // 1
import com.twitter.parrot.thrift.ParrotJob; // 2
import com.twitter.parrot.server.ParrotRequest; // 3
import com.twitter.parrot.server.ParrotService; // 3
import com.twitter.util.Future;
import com.twitter.util.FutureEventListener;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.jboss.netty.handler.codec.http.HttpResponse
import java.util.List;
public class EchoLoadTest extends LoadTest {
EchoService.ServiceToClient client = null;
public EchoLoadTest(ParrotService<ParrotRequest, HttpResponse> parrotService) {
super(parrotService);
client = new EchoService.ServiceToClient(service(), new TBinaryProtocol.Factory()); // 4
}
public void processLines(ParrotJob job, List<String> lines) { // 5
for(String line: lines) {
Future<String> future = client.echo(line);
future.addEventListener(new FutureEventListener<String>() {
public void onSuccess(String msg) {
System.out.println("response: " + msg);
}
public void onFailure(Throwable cause) {
System.out.println("Error: " + cause);
}
});
}
}
}To implement a Thrift load test in Java, you must extend the Iago server's ThriftLoadTest class to specify how to map transactions into the requests that the Iago server delivers to your service. The ThriftLoadTest class provides Java-friendly type mappings for the underlying Scala internals. The following example shows a ThriftLoadTest subclass that implements a load test on an EchoService Thrift service:
package com.twitter.jexample;
import com.twitter.example.thrift.EchoService;
import com.twitter.parrot.processor.ThriftLoadTest; // 1
import com.twitter.parrot.thrift.ParrotJob; // 2
import com.twitter.parrot.server.ParrotRequest; // 3
import com.twitter.parrot.server.ParrotService; // 3
import com.twitter.util.Future;
import com.twitter.util.FutureEventListener;
import org.apache.thrift.protocol.TBinaryProtocol;
import java.util.List;
public class EchoLoadTest extends ThriftLoadTest {
EchoService.ServiceToClient client = null;
public EchoLoadTest(ParrotService<ParrotRequest, byte[]> parrotService) {
super(parrotService);
client = new EchoService.ServiceToClient(service(), new TBinaryProtocol.Factory()); // 4
}
public void processLines(ParrotJob job, List<String> lines) { // 5
for(String line: lines) {
Future<String> future = client.echo(line);
future.addEventListener(new FutureEventListener<String>() {
public void onSuccess(String msg) {
System.out.println("response: " + msg);
}
public void onFailure(Throwable cause) {
System.out.println("Error: " + cause);
}
});
}
}
}You define your Iago subclass to execute your service and map transactions to requests for your service:
- Import
com.twitter.parrot.processor.RecordProcessor(Scala) orLoadTest(Java), whose instance will be executed by a Iago server. - Import
com.twitter.parrot.thrift.ParrotJob, which contains the Iago server class. - Import
com.twitter.parrot.server.ParrotServiceandcom.twitter.parrot.server.ParrotRequest - Create an instance of your service to be placed under test. Your service is a client of the Iago service.
- Define a
processLinesmethod to format the request and and execute your service. - Optionally, you can initiate a new request based on the response to a previous one.
- Optionally, do something with the response. In this example, the response is logged.
To configure your test, create a launcher.scala file that that creates a ParrotLauncherConfig instance with the configuration parameters you want to set.
There are several parameters to set. A good one to figure out early is transport; that will in turn help you to find out what, e.g., responseType you need.
The following example shows parameters for testing a Thrift service:
import com.twitter.parrot.config.ParrotLauncherConfig
new ParrotLauncherConfig {
distDir = "."
jobName = "load_echo"
port = 8080
victims = "localhost"
log = "logs/yesterday.log"
requestRate = 1
numInstances = 1
duration = 5
timeUnit = "MINUTES" // affects duration; does not affect requestRate
imports = "import com.twitter.example.EchoLoadTest"
responseType = "Array[Byte]"
transport = "ThriftTransport"
loadTest = "new EchoLoadTest(service.get)"
parser = "thrift"
}Note: For a sample configuration file, see config/launcher.scala within the Iago distribution.
You can specify any of the following parameters:
| Parameter | Description | Required or Default Value |
|---|---|---|
jobName |
A string value that specifies the the name of your test. Example: |
Required |
log |
A string value that specifies the complete path to the log you want Iago to replay. The log should be on your local file system. The log should have at least 1000 items or you should change the Example: |
Required |
victims |
A string of comma-separated values that specify the hosts on which to execute the load test. Example: |
Required |
port |
An integer value that specifies the port on which to deliver requests to the Example: |
Required |
distDir |
The subdirectory of your project you're running from, if any. Example: |
"." |
customLogSource |
A string with Scala code that will be put into the Feeder config Example: |
"" |
scheme |
A string value that specifies the scheme portion of a URI. Example: |
http |
header |
A string value that specifies the HTTP Host header. Example: |
"" |
duration |
An integer value that specifies the time to run the test in Example: |
|
timeUnit |
A string value that specifies time unit of the
Example: |
|
maxRequests |
An integer value that specifies the total number of requests to submit to your service. Example: |
1000 |
reuseFile |
A boolean value that specifies whether or not to stop the test when the input log has been read through. Setting this value to true will result in Iago starting back at the beginning of the log when it exhausts the contents. If this is true, your log file should at least be 1,000 lines or more. Example: |
true |
requestRate |
An integer value that specifies the number of requests per second to submit to your service. Example: Note: if using multiple server instances, requestRate is per-instance, not aggregate. |
1 |
loggers |
A List of LoggerFactories; allows you to define the type and level of logging you want Example: import com.twitter.logging.LoggerFactory import com.twitter.logging.config._ |
Nil |
numFeederInstances |
Will bring up the specified number of feeder instances Example: |
1 |
numInstances |
An integer value that specifies the number of Iago servers concurrently making requests to your service. Example: |
1 |
parser |
A string value that specifies how the request is to be interpreted. It is one of the following values:
Example: |
http |
verboseCmd |
A boolean value that specifies the level of feedback from Iago. A value of Example: |
false |
reuseConnections |
A boolean value that specifies whether connections to your service's hosts can be reused. A value of Example: |
true |
doConfirm |
If set to false, you will not be asked to confirm the run. Example: |
true |
hostConnectionCoresize |
Number of connections per host that will be kept open, once established, until they hit max idle time or max lifetime Example: |
1 |
hostConnectionLimit |
Limit on the number of connections per host Example: |
Integer.MAX_VALUE |
hostConnectionIdleTimeInMs |
For any connection > coreSize, maximum amount of time, in milliseconds, between requests we allow before shutting down the connection Example: |
60000 |
hostConnectionMaxIdleTimeInMs |
The maximum time in milliseconds that any connection (including within core size) can stay idle before shutdown Example: |
300000 |
hostConnectionMaxLifeTimeInMs |
The maximum time in milliseconds that a connection will be kept open Example: |
Integer.MAX_VALUE |
maxPerHost |
Maximum number of parrot_server instances per mesos box Example: |
1 |
serverXmx |
Defines heap size. Suggested not to be higher than 8 GB (will cause issues scheduling) Example: |
4000 |
thriftClientId |
If you are making Thrift requests, your clientId Example: |
"" |
createDistribution |
You can use this field to create your own distribution rate, instead of having a constant flow. You will need to create a subclass of RequestDistribution and import it. Example: createDistribution = """createDistribution = {
rate => new MyDistribution(rate)
}""" |
"" |
####Extension Point Parameters
Alternative Use: You can specify the following extension point parameters to configure projects in which Iago is used as both a feeder and server. The Iago feeder provides the log lines to your project, which uses these log lines to form requests that the Iago server then handles:
| Parameter | Description | Required or Default Value |
|---|---|---|
imports |
Imports from this project to Iago Example: If |
import org.jboss.netty.handler.codec.http.HttpResponse |
requestType |
The request type of requests from Iago. Examples:
|
ParrotRequest |
responseType |
The response type of responses from Iago. Examples:
|
HttpResponse |
transport |
The kind of transport to the server, which matches the Example: The Thrift Transport will send your request and give back |
FinagleTransport |
loadTest |
Your processor for the Iago feeder's lines, which converts the lines into requests and sends them to the Iago server. Example: |
new LoadTestStub(service.get) |
Iago is open source, hosted on Github here. If you have a contribution to make, please fork the repo and submit a pull request.