oTree is a Django-based framework for implementing multiplayer decision strategy games. Many of the details of writing a web application are abstracted away, meaning that the code is focused on the logic of the game. oTree programming is accessible to programmers without advanced experience in web app development.
This repository contains the games; the oTree core libraries are here.
[email protected] (you can also add me on Skype by searching this email address; please mention oTree in your contact request)
Please contact me if any part of oTree does not work for you (or is unclear).
- Gregor Muellegger (http://gremu.net/, https://github.com/gregmuellegger)
- Juan B. Cabral
- Alexander Schepanovski (https://github.com/Suor/)
- Alexander Sandukovskiy
- Som Datye
Sign up to be notified about updates to oTree here
- Setup
- Conceptual overview
- Apps
- Templates
- Forms
- Object model and
self - Groups and multiplayer games
- Money and Points
- Treatments
- Rounds
- Trying your app
- Test Bots
- Admin
- Deploying to a server
- Amazon Mechanical Turk
- oTree programming For Django Devs
See the tutorial here
See the installation instructions here.
Or, clone this repo and run:
pip install -r requirements_base.txt
./otree resetdb
./otree runserver
You should see the following output on the command line::
Validating models...
0 errors found
|today| - 15:50:53
Django version |version|, using settings 'settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Now that the server's running, visit http://127.0.0.1:8000/ with your Web
browser.
To ease the learning curve of oTree, we strongly recommend using PyCharm Professional, even though there are many other good editors for Python code. This is because:
- PyCharm has features that make oTree/Django development easier
- oTree has special integration with PyCharm's code completion functionality
- This documentation gives instructions assuming you are using PyCharm
- oTree has been thoroughly tested with PyCharm
If you are a student, teacher, or professor, PyCharm Professional is free. Note: we recommend PyCharm Professional rather than PyCharm Community Edition.
In oTree, the top-level concept is a "Session". This term refers to an event where a group of people spend time taking part in oTree experiments. An example of a session would be:
"On Tuesday at 3PM, 30 people will come to the lab for 1 hour, during which time they will play a trust game, followed by 2 rounds of an ultimatum game, followed by a questionnaire. Participants get paid EUR 10.00 for showing up, plus their payoffs they earn playing the games."
A session can be broken down into what oTree calls "subsessions". These are interchangeable units or modules that come one after another. Each subsession has a sequence of one or more pages the player must interact with. The session in the above example had 4 subsessions:
- Trust game
- Ultimatum game 1
- Ultimatum game 2
- Questionnaire
Each subsession is defined in an oTree app. The above session would require 3 distinct apps to be coded:
- Trust game
- Ultimatum game
- Questionnaire
You can define your session's properties in SESSION_TYPES in settings.py. Here are the parameters for the above example:
{
'name':'my_session',
'participation_fee':10.00,
'app_sequence':['trust', 'ultimatum', 'questionnaire'],
}app_sequence allows you to have a session that consists of multiple apps.
For example, the questionnaire is a separate standalone app, rather than being part of the ultimatum app.
The advantage of this is that you can reuse the same questionnaire app in different session types,
simply by adding it to the end of app_sequence.
In oTree, the terms "player" and "participant" have distinct meanings. The distinction between a participant and a player is the same as the distinction between a session and a subsession.
A participant is a person who takes part in a session. The participant data model contains properties such as the participant's name, how much they made in the session, and what their progress is in the session.
A player is an instance of a participant in one particular subsession. A participant can be player 1 in the first subsession, player 5 in the next subsession, and so on.
In oTree, an app is a folder containing Python and HTML code. When you create your oTree project, it comes pre-loaded with various apps such as public_goods and dictator. A session is basically a sequence of apps that are played one after the other.
From the oTree launcher, click the "Terminal" button. (If the button is disabled, make sure you have stopped the server.) When the console window appears, type this:
./otree startapp your_app_nameThis will create a new app folder based on a oTree template, with most of the structure already set up for you.
Think of this as a skeleton to which you can add as much as you want. You can add your own classes, functions, methods, and attributes, or import any 3rd-party modules.
Then go to settings.py and create an entry for your app in SESSION_TYPES that looks like the other entries.
This is where you store your data models.
Every oTree app needs the following 3 models:
- Player
- Group
- Subsession
A player is part of a group, which is part of a subsession.
For example, let's say you are programming an ultimatum game, where in each two-person group, one player makes a monetary offer (say, 0-100 cents), and another player either rejects or accepts the offer. When you analyze your data, you will want your "Group" table to look something like this:
+----------+----------------+----------------+
| Group ID | Amount offered | Offer accepted |
+==========+================+================+
| 1 | 50 | TRUE |
+----------+----------------+----------------+
| 2 | 25 | FALSE |
+----------+----------------+----------------+
| 3 | 50 | TRUE |
+----------+----------------+----------------+
| 4 | 0 | FALSE |
+----------+----------------+----------------+
| 5 | 60 | TRUE |
+----------+----------------+----------------+
You need to define a Python class that defines the structure of this database table. You define what fields (columns) are in the table, what their data types are, and so on. When you run your experiment, the SQL tables will get automatically generated, and each time users visit, new rows will get added to the tables.
Here is how to define the above table structure:
class Group(otree.models.BaseGroup):
...
amount_offered = models.CurrencyField()
offer_accepted = models.BooleanField()Every time you add, remove, or change a field in models.py, you need to run ./otree resetdb (or, in the launcher, click "Clear Database").
The Constants class is the recommended place to put your app's parameters and other constants (i.e. things that do not vary from player to player)
Here are the required constants:
name_in_urlis an attribute that defines the name this app has in the URLs, which players may see.players_per_group(described elsewhere in the documentation)num_rounds(described elsewhere in the documentation)
Each page that your players see is defined by a Page class in views.py. (You can think of "views" as a synonym for "pages".)
For example, if 1 round of your game involves showing the player a sequence of 5 pages, your views.py should contain 5 page classes.
At the bottom of your views.py, you must have a page_sequence variable that specifies the order in which players are routed through your pages. For example:
page_sequence=[Start, Offer, Accept, Results]Each Page class has these methods and attributes:
oTree automatically passes group, player, subsession, and Constants objects to the template, so you can access them from your template in the following format: {{Constants.payoff_if_rejected}}. If you need to pass any additional variables to the template, you can define a method vars_for_template that returns these variables in a dictionary.
Should return True if the page should be shown, and False if the page should be skipped. Default behavior is to show the page.
For example, if you only want a page to be shown to P2 in each group:
def is_displayed(self):
return self.player.id_in_group == 2The name of the HTML template to display. This can be omitted if the template has the same name as the Page class.
Example:
# This will look inside your app under the 'templates' directory,
# to '/app_name/MyView.html'
template_name = 'app_name/MyView.html'Set to an integer that specifies how many seconds the user has to complete the page. After the time runs out, the page auto-submits.
Example: timeout_seconds = 20
Lets you specify what values should be auto-submitted if timeout_seconds is exceeded, or if the experimenter
moves the participant forward. If this is omitted, then oTree will default to 0 for numeric fields, False for boolean
fields, and the empty string for text/character fields.
This should be a dictionary where the keys are the elements of form_fields, and the values are the values that should
be auto-submitted.
After the player clicks the "Next" button, oTree makes sure that any form fields validate (and re-displays to the player with errors otherwise).
Here you can put anything additional that should happen after form validation. If you don't need anything to be done, it's OK to leave this method blank, or to leave it out entirely.
This is not a method on the Page class, but rather a top-level function in views.py. It is useful when you need certain variables to be passed to multiple pages in your app.
Instead of repeating the same values in each vars_for_template, you can define it in this function.
Your app's templates/ directory will contain the templates for the HTML that gets displayed to the player.
oTree uses Django's [template system] (https://docs.djangoproject.com/en/dev/topics/templates/).
Instead of writing the full HTML of your page, for example::
<!DOCTYPE html>
<html lang="en">
<head>
<!-- and so on... -->You define 2 blocks:
{% block title %}
Title goes here
{% endblock %}
{% block content %}
Body HTML goes here.
{% formfield player.contribution with label="What is your contribution?" %}
{% next_button %}
{% endblock %}You may want to customize the appearance or functionality of all pages in your app (e.g. by adding custom CSS or JavaScript). To do this, edit the file templates/global/Base.html.
To include images, CSS, or JavaScript in your pages, put the following line in your template below the extends block:
{% load staticfiles %}And follow the [instructions here] (https://docs.djangoproject.com/en/dev/howto/static-files/).
oTree comes pre-loaded with the following plugins and libraries.
oTree comes with [Bootstrap] (http://getbootstrap.com/), a popular library for customizing a website's user interface.
You can use it if you want a [custom style] (http://getbootstrap.com/css/), or a [specific component] (http://getbootstrap.com/components/) like a table, alert, progress bar, label, etc. You can even make your page dynamic with elements like [popovers] (http://getbootstrap.com/javascript/#popovers), [modals] (http://getbootstrap.com/javascript/#modals), and [collapsible text] (http://getbootstrap.com/javascript/#collapse).
To use Bootstrap, usually you add a class= attributes to your HTML element.
For example, the following HTML will create a "Success" alert:
<div class="alert alert-success">Great job!</div>oTree comes pre-loaded with HighCharts. You can find examples in the library of how to use it.
To pass data like a list of values from Python to HighCharts, you should first pass it through the otree.common.safe_json() function. This converts to the correct JSON syntax and also uses "mark_safe" for the template.
Example:
>>> a = [0, 1, 2, 3, 4, None]
>>> safe_json(a)
'[0, 1, 2, 3, 4, null]'
oTree comes pre-loaded with jQuery, a JavaScript library that lets you make your pages dynamic. You can include a script and reference the standard $ variable.
oTree comes pre-loaded with KaTeX; you can insert LaTeX equations like this: <span class="latex">1 + i = (1 + r)(1 + \pi)</span>
Since oTree uses Bootstrap for its user interface, your oTree app should work on all major browsers (Chrome/Internet Explorer/Firefox/Safari). When participants visit on a smartphone or tablet (e.g. iOS/Android/etc.), they should see an appropriately scaled down "mobile friendly" version of the site. This will generally not require much effort on your part since Bootstrap does it automatically, but if you plan to deploy your app to participants on mobile devices, you should test it out on a mobile device during development, since some HTML code doesn't look good on mobile devices.
Each page in oTree can contain a form, which the player should fill out and submit by clicking the "Next" button. To create a form, first you should go to models.py and define fields on your Player or Group. Then, in your Page class, you can define form_models to specify he model that this form modifies (either models.Player or models.Group), and form_fields, which is list of the fields from that model.
When the user submits the form, the submitted data is automatically saved back to the field in your model.
You should include form fields by using a {% formfield %} element. You generally do not need to write raw HTML for forms (e.g. <input type="text" id="...">).
The player must submit a valid form before they get routed to the next page. If the form they submit is invalid (e.g. missing or incorrect values), it will be re-displayed to them along with the list of errors they need to correct.
Example 1:
Example 2:
oTree automatically validates all input submitted by the user.t
For example, if you have a form containing a PositiveIntegerField,
oTree will not let the user submit values that are not positive integers, like -1, 1.5, or hello.
Additionally, you can customize validation by passing extra arguments to your model field definition. For example, if you want to require a number to be between 12 and 24, you can specify it like this:
offer = models.PositiveIntegerField(min=12, max=24)If you specify a choices argument, the default form widget will be a select box with these choices instead of the standard text field.
year_in_school = models.CharField(choices=['Freshman', 'Sophomore', 'Junior', 'Senior'])If you would like a specially formatted value displayed to the user that is different from the values stored internally, you can return a list consisting itself of tuples of two items to use as choices for this field. The first element in each tuple is the actual value to be set on the model, and the second element is the human-readable name. For example:
year_in_school = models.CharField(
choices=[
('FR', 'Freshman'),
('SO', 'Sophomore'),
('JR', 'Junior'),
('SR', 'Senior'),
]
)After the field has been set, you can access the human-readable name using get_FOO_display
method, like this:
self.get_year_in_school_display() # returns e.g. 'Sophomore'
If a field is optional, you can do:
offer = models.PositiveIntegerField(blank=True)If you need a form's choices or validation logic to depend on some dynamic calculation, then you can instead define one of the below methods in your Page class in views.py.
def {field_name}_choices(self)
Example:
def offer_choices(self):
return currency_range(0, self.player.endowment)def {field_name}_min(self)
The dynamic alternative to min.
def {field_name}_max(self)
The dynamic alternative to max.
def {field_name}_error_message(self, value)
This is the most flexible method for validating a field.
For example, let's say your form has an integer field called odd_negative, which must be odd and negative: You would enforce this as follows:
def odd_negative_error_message(self, value):
if not (value < 0 and value % 2):
return 'Must be odd and negative'Let's say you have 3 integer fields in your form whose names are int1, int2, and int3, and the values submitted must sum to 100. You would define the error_message method in your Page class:
def error_message(self, values):
if values["int1"] + values["int2"] + values["int3"] != 100:
return 'The numbers must add up to 100'If you need the list of form fields to be dynamic,
instead of form_fields you can define a method get_form_fields(self) that returns
the list.
The full list of form input widgets offered by Django is here.
oTree additionally offers RadioSelectHorizontal and SliderInput.
Custom widgets and hidden fields
It's not mandatory to use the {% formfield %} element; you can write the raw HTML for any form input if you wish
to customize its behavior or appearance.
Just include an <input> element with the same name attribute as the field. For example, if you want a hidden input,
you can do this:
# models.py
my_hidden_input = models.PositiveIntegerField()
# views.py
form_fields = ['my_hidden_input', 'some_other_field']
# HTML template
<input type="hidden" name="my_hidden_input" value="5" id="id_my_hidden_input"/>
Then you can use JavaScript to set the value of that input, by selecting the element by id "id_my_hidden_input".
For simple widgets you can use jQuery; for more complex or custom form interfaces, you can use a front-end framework with databinding, like React or Polymer.
If you want your custom widget's style to look like the rest of the oTree widgets, you should look at the generated HTML from the {% formfield %} tag. You can copy and paste the markup into the template and use that as a starting point for modifications.
In oTree code, you will see the variable self all throughout the code. self is the way you refer to the current object in Python. It is therefore important to understand that the meaning of self is totally different depending on where you are in your code. For example, if you are inside a Page class, self.player.payoff refers to the current player object, but if you are inside the Player class in models.py, self.player.payoff is invalid because self is the player; you instead need to do self.payoff.
oTree's different objects are all connected; here is an example of how to traverse these connections using the "dot" operator.
class Session(...) # this class is defined in oTree-core
def example(self):
# current session object
self
# parent objects
self.session_type
# child objects
self.get_subsessions()
self.get_participants()
class Participant(...) # this class is defined in oTree-core
def example(self):
# current participant object
self
# parent objects
self.session
# child objects
self.get_players()
# in your models.py
class Subsession(otree.models.Subsession):
def example(self):
# current subsession object
self
# parent objects
self.session
# child objects
self.get_groups()
self.get_players()
# accessing previous Subsession objects
self.in_previous_rounds()
self.in_all_rounds()
class Group(otree.models.Group):
def example(self):
# current group object
self
# parent objects
self.session
self.subsession
# child objects
self.get_players()
class Player(otree.models.Player):
def example(self):
# current player object
self
# method you defined on the current object
self.my_custom_method()
# parent objects
self.session
self.subsession
self.group
self.participant
self.session.session_type
# accessing previous player objects
self.in_previous_rounds()
self.in_all_rounds() # equivalent to self.in_previous_rounds() + [self]
# in your views.py
class MyPage(Page):
def example(self):
# current page object
self
# parent objects
self.session
self.subsession
self.group
self.player
# example of chaining lookups
self.player.participant
self.session.session_typeYou can follow pointers in a transitive manner. For example, if you are in the Page class, you can access the participant as self.player.participant. If you are in the Player class, you can access the session type as self.session.session_type.
In oTree, you can define multiplayer interactive games through the use of groups
To do this, go to your app's models.py and set Constants.players_per_group. For example, in a 2-player game like an ultimatum game or prisoner's dilemma, you would set this to 2. If your app does not involve dividing the players into multiple groups, then set it to None. e.g. it is a single-player game or a game where everybody in the subsession interacts together as 1 group. In this case, self.group.get_players() will return everybody in the subsession. If you need your groups to have uneven sizes (for example, 2 vs 3), you can do this: players_per_group=[2,3]; in this case, if you have a session with 15 players, the group sizes would be [2,3,2,3,2,3].
Each player has a numeric field id_in_group. This is useful in multiplayer games where players have different roles, so that you can determine if the player is player 1, player 2, or so on.
Groups have the following methods:
get_players(): returns a list of the players in the group.get_player_by_id(n): Retrieves the player in the group with a specificid_in_group.get_player_by_role(r). The argument to this method is a string that looks up the player by their role value. (If you use this method, you must define therolemethod on the player model, which should return a string that depends onid_in_group.)
Player objects have methods get_others_in_group() and get_others_in_subsession() that return a list of the other players in the group and subsession, respectively.
Wait pages are necessary when one or more players need to wait for another player to take some action before they can proceed. For example, in an ultimatum game, player 2 cannot accept or reject before they have seen player 1's offer.
Wait pages are defined in views.py. If you have a wait page in your sequence of pages, then oTree waits until all players in the group have arrived at that point in the sequence, and then all players are allowed to proceed.
If your subsession has multiple groups playing simultaneously, and you would like a wait page that waits for all groups (i.e. all players in the subsession), you can set the attribute wait_for_all_groups = True on the wait page.
Wait pages can define the following methods:
def after_all_players_arrive(self)
This code will be executed once all players have arrived at the wait page. For example, this method can determine the winner of an auction and set each player's payoff.
def title_text(self)
The text in the title of the wait page.
def body_text(self)
The text in the body of the wait page
For the first round, the players are split into groups of Constants.players_per_group. This matching is random, unless you have set group_by_arrival_time set in your session type in settings.py, in which case players are grouped in the order they start the first round.
In subsequent rounds, by default, the groups chosen are kept the same. If you would like to change this, you can define the grouping logic in Subsession.before_session_starts. For example, if you want players to be reassigned to the same groups but to have roles randomly shuffled around within their groups (e.g. so player 1 will either become player 2 or remain player 1), you would do this:
def before_session_starts(self):
for group in self.get_groups():
players = group.get_players()
players.reverse()
group.set_players(players)A group has a method set_players that takes as an argument a list of the players to assign to that group, in order. Alternatively, a subsession has a method set_groups that takes as an argument a list of lists, with each sublist representing a group. You can use this to rearrange groups between rounds, but note that the before_session_starts method is run when the session is created, before players begin playing. Therefore you cannot use this method to shuffle players depending on the results of previous rounds (there is a separate technique for doing this which will be added to the documentation in the future).
In many experiments, participants play for currency: either virtual points, or real money. oTree supports both scenarios. Participants can be awarded a fixed base pay (i.e. participation fee). In addition, in each subsession, they can be awarded an additional payoff.
You can specify the payment currency in settings.py, by setting REAL_WORLD_CURRENCY_CODE to "USD", "EUR", "GBP", and so on. This means that all currency amounts the participants see will be automatically formatted in that currency, and at the end of the session when you print out the payments page, amounts will be displayed in that currency.
In oTree apps, currency values have their own data type. You can define a currency value with the c() function, e.g. c(10) or c(0). Correspondingly, there is a special model field for currency values: CurrencyField. For example, each player has a payoff field, which is a CurrencyField. Currency values work just like numbers (you can do mathematical operations like addition, multiplication, etc), but when you pass them to an HTML template, they are automatically formatted as currency. For example, if you set player.payoff = c(1.20), and then pass it to a template, it will be formatted as $1.20 or 1,20 €, etc., depending on your REAL_WORLD_CURRENCY_CODE and LANGUAGE_CODE settings.
Note: instead of using Python's built-in range function, you should use oTree's currency_range with currency values. For example, currency_range(c(0), c(0.10), c(0.02)) returns something like:
[Money($0.00), Money($0.02), Money($0.04), Money($0.06), Money($0.08), Money($0.10)]Each player has a payoff field, which is a CurrencyField.
If your player makes money, you should store it in this field.
player.participant.payoff is the sum of the payoffs a participant made in each subsession.
At the end of the experiment, a participant's total profit is calculated by adding the fixed base pay
to the payoff that participant earned as a player in each subsession.
Sometimes it is preferable for players to play games for points or "experimental currency units", which are converted to real money at the end of the session. You can set USE_POINTS = True in settings.py, and then in-game currency amounts will be expressed in points rather than real money.
For example, c(10) is displayed as 10 points. You can specify the conversion rate to real money in settings.py by providing a real_world_currency_per_point key in the session type dictionary. For example, if you pay the user 2 cents per point, you would set real_world_currency_per_point = 0.02.
You can convert a point amount to money using the to_real_world_currency() method,
which takes as an argument the current session
(this is necessary because different sessions can have different conversion rates).
Let's say real_world_currency_per_point = 0.02
c(10) # evaluates to Currency(10 points)
c(10).to_real_world_currency(self.session) # evaluates to $0.20If you want to assign participants to different treatment groups, you can put the code in the subsession's before_session_starts method. For example, if you want some participants to have a blue background to their screen and some to have a red background, you would randomize as follows:
def before_session_starts(self):
# randomize to treatments
for player in self.get_players():
player.color = random.choice(['blue', 'red'])(To actually set the screen color you would need to pass player.color to some CSS code in the template, but that part is omitted here.)
If your game has multiple rounds, note that the above code gets executed for each round. So if you want to ensure that participants are assigned to the same treatment group each round, you should set the property at the participant level, which persists across subsessions, and only set it in the first round:
def before_session_starts(self):
if self.round_number == 1:
for p in self.get_players():
p.participant.vars['color'] = random.choice(['blue', 'red'])Then, in the view code, you can access the participant's color with self.player.participant.vars['color'].
In the above example, players got randomized to treatments. This is useful in a live experiment, but when you are testing your game, it is often useful to choose explicitly which treatment to play. Let's say you are developing the game from the above example and want to show your colleagues both treatments (red and blue). You can create 2 session types in settings.py that have the same keys to session type dictionary , except the treatment key:
SESSION_TYPES = [
{
'name':'my_game_blue',
# other arguments...
'treatment':'blue',
},
{
'name':'my_game_red',
# other arguments...
'treatment':'red',
},
]Then in the before_session_starts method, you can check which of the 2 session types it is:
def before_session_starts(self):
for p in self.get_players():
if 'treatment' in self.session.session_type:
# demo mode
p.color = self.session.session_type['treatment']
else:
# live experiment mode
p.color = random.choice(['blue', 'red'])Then, when someone visits your demo page, they will see the "red" and "blue" treatment, and choose to play one or the other. If the demo argument is not passed, the color is randomized.
In oTree, "rounds" and "subsessions" are almost synonymous. The difference is that "rounds" refers to a sequence of subsessions that are in the same app. So, a session that consists of a prisoner's dilemma iterated 3 times, followed by an exit questionnaire, has 4 subsessions, which consists of 3 rounds of the prisoner's dilemma, and 1 round of the questionnaire.
You can specify how many rounds a game should be played in models.py, in Constants.num_rounds.
Subsession objects have an attribute round_number, which contains the current round number, starting from 1.
Player objects have methods in_previous_rounds() and in_all_rounds() that each return a list of players representing the same participant in previous rounds of the same app. The difference is that in_all_rounds() includes the current round's player.
For example, if you wanted to calculate a participant's payoff for all previous rounds of a game, plus the current one:
cumulative_payoff = sum([p.payoff for p in self.player.in_all_rounds()])Similarly, subsession objects have methods in_previous_rounds() and in_all_rounds() that work the same way.
in_all_rounds() only is useful when you need to access data from a previous round of the same app.
If you want to pass data between subsessions of different app types (e.g. the participant is in the questionnaire and needs to see data from their ultimatum game results),
you should store this data in the participant object, which persists across subsessions. Each participant has a field called vars, which is a dictionary that can store any data about the player. For example, if you ask the participant's name in one subsession and you need to access it later, you would store it like this:
self.player.participant.vars['first name'] = 'Chris'
Then in a future subsession, you would retrieve this value like this:
self.player.participant.vars['first name'] # returns 'Chris'
For session-wide globals, you can use self.session.vars.
This is a dictionary just like participant.vars.
You can launch your app on your local development machine to test it, and then when you are satisfied, you can deploy it to a server.
You will be testing your app frequently during development, so that you can see how the app looks and feels and discover bugs during development. To test your app, run the server in the oTree launcher. You may need to reset the database first.
Click on a session name and you will get a start link for the experimenter, as well as the links for all the participants. You can open all the start links in different tabs and simulate playing as multiple participants simultaneously.
You can send the demo page link to your colleagues or publish it to a public audience.
Once you start playing your app, you will sometimes get a yellow Django error page with lots of details. To troubleshoot this, look at the error message and "Exception location" fields. If the exception location is somewhere outside of your app's code (like if it points to an installed module like Django or oTree), look through the "Traceback" section to see if it passes through your code. Once you have found a reference to a line of code in your app, go to that line of code and see if the error message can help you pinpoint an error in your code. Googling the error name or message will often take you to pages that explain the meaning of the error and how to fix it.
PyCharm has an excellent debugger that you should be using continuously. You can insert a breakpoint into your code by clicking in the left-hand margin on a line of code. You will see a little red dot. Then reload the page and the debugger will pause when it hits your line of code. At this point you can inspect the state of all the local variables, execute print statements in the built-in intepreter, and step through the code line by line.
More on the PyCharm debugger here.
Automated tests are an essential part of building a oTree app. You can easily program a bot that simulates multiple players simultaneously playing your app.
Tests with dozens of bots complete with in seconds, and afterward automated tests can be run to verify correctness of the app (e.g. to ensure that payoffs are being calculated correctly).
This automated test system saves the programmer the effort of having to re-test the application every time something is changed.
oTree tests entire sessions, rather that individual apps in isolation. This is to make sure the entire session runs, just as participants will play it in the lab.
Let's say you want to test the session named ultimatum in settings.py. To test, click the "Terminal" button in the oTree launcher run the following command from your project's root directory:
./otree test ultimatum_gameThis command will test the session, with the number of participants specified in settings.py. For example, num_bots is 30, then when you run the tests, 30 bots will be instantiated and will play concurrently.
To run tests for all sessions in settings.py, run:
./otree testTests are contained in your app's tests.py. Fill out the play_round() method of your PlayerBot (and ExperimenterBot if you have experimenter pages). It should simulate each page submission. For example:
self.submit(views.Start)
self.submit(views.Offer, {'offer_amount': 50})Here, we first submit the Start page, which does not contain a form. The next page is Offer, which contains a form whose field is called offer_amount, which we set to 50. This is a way of automating the task of
Your test bot must simulate playing the game correctly. The bot in the above example would raise an error if the page after Start was called Instructions rather than Offer, or if the field offer_amount was actually called something else. Your test bot is a specification of how you expect your app to work, so when it raises an error, it will alert you that your app is not behaving as intended.
Rather than programming many separate bots, you program one bot that can play any variation of the game. For example, if you have different treatment groups that play different pages, you can branch by checking a variable on the treatment. For example, here is how you would play if one treatment group sees a "threshold" page but the other treatment group should see an "accept" page:
if self.group.threshold:
self.submit(views.Threshold, {'offer_accept_threshold': 30})
else:
self.submit(views.Accept, {'offer_accepted': True})If player 1 in a group sees different pages from player 2, you can define separate methods play_p1() and play_p2() and branch like this:
if self.player.id_in_group == 0:
self.play_p1()
else:
self.play_p2()To get the maximal benefit, your bot should thoroughly test all parts of your code. Here are some ways you can test your app:
-
Ensure that it correctly rejects invalid input. For example, if you ask the user to enter a number that is a multiple of 3, you can verify that entering 4 will be rejected by using the
submit_invalidmethod as follows. This line of code will raise an error if the submission is accepted:self.submit_invalid(views.EnterNumber, {'multiple_of_3': 4}) -
You can put assert statements in the bot's
validate_play()method to check that the correct values are being stored in the database. For example, if a player's bonus is defined to be 100 minus their offer, you can check your program is calculating it correctly as follows:self.submit(views.Offer, {'offer': 30})assert self.player.bonus == 70 -
You can use random amounts to test that your program can handle any type of random input:
self.submit(views.Offer, {'offer': random.randint(0,100)})
Bots can either be programmed to simulate playing the game according to an ordinary strategy, or to test "boundary conditions" (e.g. by entering invalid input to see if the application correctly rejects it). Or yet the bot can enter random input on each page.
If your app has [[Experimenter Pages]], you can also implement the play method on the ExperimenterBot.
oTree comes with an admin interface, so that experimenters can manage sessions, monitor the progress of live sessions, and export data after sessions.
Open your browser to the root url of your web application. If you're developing locally, this will be http://127.0.0.1:8000/.
Create a session in the admin. [TODO: more info]
To launch a session, each participant must open their link. There are 2 options for how to open URLs.
In the admin interface, go to the "global data" section, and copy the "lab link". This is a permanent URL that will last as long as you use the same server [TODO: finish]
Each workstation has a permanent URL that, when clicked, will route the participant to the currently active session.
choose an active session from the dropdown. Then, copy
If you are running your experiment in a lab, you should deploy the links to the target workstations using whatever means is available to you. If you have a tool that can push distinct URLs to each PC in the lab, you can use that. Or you can set up a unique email account for each workstation, and then send the unique links to PCs using a mail merge. Then open the link on each PC.
oTree uses a unique code to identify each participant. However, you can assign each participant a "label" that can be any convenient way to identify them to you, such as:
- Name
- Computer workstation number
- Email address
- ID number
This label will be displayed in places where participants are listed, like the oTree admin interface or the payments page.
You can assign each participant a label by adding a parameter to each start link. For example, if you want to assign a participant the label "WORKSTATION_1", you would take the start link for that participant:
http://[participant's start link]
And change it to:
http://[participant's start link]?participant_label=WORKSTATION_1
Outside of oTree, you can create a script that adds a unique participant_label to each start link as indicated above. Then,
when the link is opened, the oTree server will register that participant label for that participant.
While your session is ongoing, you can monitor the live progress in the admin interface. The admin tables update live, highlighting changes as they occur. The most useful table to monitor is "Session participants", which gives you a summary of all the participants' progress. You can also click the "participants" table of each app to see the details of all the data being entered in your subsessions.
When you first install oTree, The entire admin interface is accessible without a password. However, when you are ready to launch your oTree app, you should password protect the admin so that visitors and participants cannot access sensitive data.
If you are launching an experiment and want visitors to only be able to play your app if you provided them with a start link,
set the environment variable OTREE_AUTH_LEVEL to EXPERIMENT.
If you would like to put your site online in public demo mode where anybody can play a demo version of your game,
set OTREE_AUTH_LEVEL to DEMO. This will allow people to play in demo mode, but not access the full admin interface.
Experiments can be launched to participants playing over the internet, in a similar way to how experiments are launched the lab. Login to the admin, create a session, then distribute the links to participants via email or a website.
In a lab, you usually can start all participants at the same time, but this is often not possible online, because some participants might click your link hours after other participants. If your game requires players to play in groups, you may want to set the group_by_arrival_time key in session type dictionary to True. This will group players in the order in which they arrive to your site, rather than randomly, so that players who arrive around the same time play with each other.
During an experiment, subjects are expected to stay on the given pages/game, instead of browsing irrelevant websites or using other applications. Kiosk mode locks down the oTree pages on a web browser thus allows the subjects to focus. Here we provide some guidelines to initiate Kiosk mode with different browsers/on various systems. In general, Kiosk mode is rather user-friendly so one can easily search online how to use it on specific platforms.
- Go to Setting – Accessibility – Guided Access
- Turn on Guided Access and set a passcode for your Kiosk mode
- Open your web browser and enter your URL
- Triple-click home button to initiate Kiosk mode
- Circle areas on the screen to disable (e.g. URL bar) and activate
There are several apps for using Kiosk mode on Android, for instance: Kiosk Browser Lockdown.
For iOS and Android tablets, Kiosk mode will continue to function after normal restart. However, if subjects enter Android safe mode, the app can be disabled.
- Go to Setting – Users – Add new user
- Create a new user with a desktop shortcut
- Right-click the shortcut and select “Properties”
- In the “Target” filed, add to the end either
--kiosk "http://www.your-otree-server.com"or--chrome-frame --kiosk "http://www.your-otree-server.com" - Disable hotkeys (see here)
- Open the shortcut to activate Kiosk mode
IE on PC See here
There are several apps for using Kiosk mode on Mac, for instance: eCrisper. Mac keyboard shortcuts should be disabled.
At the end of your session, you can open and print a page that lists all the participants and how much they should be paid.
You can download your raw data in text format (CSV) so that you can view and analyze it with a program like Excel, Stata, or R.
Each model field you define can also have a doc= argument.
Any string you add here will be included in the autogenerated documentation file,
which can be downloaded through the data export page in the admin.
Any application can be run so that that debug information is displayed on the bottom of all screens. The debug information consists of the ID in group, the group, the player, the participant label, and the session code. The session code and participant label are two randomly generated alphanumeric codes uniquely identifying the session and participant. The ID in group identifes the role of the player (e.g., in a principal-agent game, principals might have the ID in group 1, while agents have 2).
##Progress-Monitor
The progress monitor allows the researcher to monitor the progress of an experiment. It features a display that can be filtered and sorted, for example by computer name or group. The experimenter can see the progress of all participants, including their current action and taken decisions. Updates are shown as they happen in real time and cells that change are highlighted in yellow. Because the progress monitor is web-based, multiple collaborators can simultaneously open it on several devices on premises or at remote locations.
The session interface is an optional feature convenient in some experiments. In many experimental settings, in addition to monitoring, an experimenter needs to receive instructions or provide input for the experiment. The session interface can instruct an experimenter on what to do next and show text to be read aloud. The session interface can also request input from the experimenter at a speci c point in the session. For example, in an Ellsberg experiment, the experimenter might roll an opaque urn prior to the session; the session interface will remind the experimenter to show the urn to the participants, tell the experimenter when all participants have selected their bets, and instruct her to draw a ball from the urn. It will then ask the drawn color, so that oTree can calculate participants' payoff�s.
oTree can be deployed on your own server, or using a cloud service like Heroku.
If you are not experienced with web server administration, Heroku may be a much simpler option for you, because Heroku automatically handles much of the configuration. Instructions on how to deploy oTree to Heroku are here.
Nevertheless, in various situations it will be preferable to run oTree on your own server. Reasons may include:
- You do not want your server to be accessed from the internet
- You will be launching your experiment in a setting where internet access is unavailable
- You want full control over how your server is configured
oTree runs on top of Django, so oTree setup is the same as Django setup. Django runs on a wide variety of servers, except getting it to run on Windows may require extra work.
The most typical setup will be a Linux server with Apache. The instructions for this setup are here.
If you have been developing your project on your local PC, you should push your oTree folder to your webserver, e.g. with Git. Then, you should make sure your webserver has Python installed (possibly in a virtualenv), and do pip install -r requirements.txt to install all the dependencies. When you are ready to launch the experiment, you should set OTREE_PRODUCTION to 1, to turn off DEBUG mode.
Here are the steps for deploying to Heroku.
Create a free account on Heroku. You can skip the "Getting Started With Python" guide.
Install the Heroku Toolbelt.
This provides you access to the Heroku Command Line utility.
Once installed, you can use the heroku command from your command shell.
From the oTree launcher, click the terminal button to access the command shell. Log in using the email address and password you used when creating your Heroku account:
$ heroku login
Enter your Heroku credentials.
Email: [email protected]
Password:
Authentication successful.
Authenticating is required to allow both the heroku and git commands to operate.
Create an app on Heroku, which prepares Heroku to receive your source code:
$ heroku create
Creating lit-bastion-5032 in organization heroku... done, stack is cedar-14
http://lit-bastion-5032.herokuapp.com/ | https://git.heroku.com/lit-bastion-5032.git
Git remote heroku added
When you create an app, a git remote (called heroku) is also created and associated with your local git repository.
Heroku generates a random name (in this case lit-bastion-5032) for your app, or you can pass a parameter to specify your own app name.
cd to the root directory of your oTree project. Then do:
$ git push heroku master
$ ./otree-heroku resetdb myherokuapp
Now visit the app at the URL generated by its app name. As a handy shortcut, you can open the website as follows:
$ heroku open
If it's a production website, you should set the environment variables (e.g. OTREE_PRODUCTION and OTREE_AUTH_LEVEL), like this:
heroku config:set OTREE_PRODUCTION=1
heroku config:set OTREE_AUTH_LEVEL=DEMOheroku git:remote -a [myherokuapp]oTree is most frequently used with PostgreSQL as the production database. You can create your database with a command like this:
psql -c 'create database django_db;' -U postgres
Then, you should set the following environment variable, so that it can be read by dj_database_url:
DATABASE_URL=postgres://postgres@localhost/django_db
oTree provides integration with Amazon Mechanical Turk (AMT).
You can publish your game to Amazon mechanical Turk directly from oTree's admin interface. Then, workers on mechanical Turk can accept and play your app as an MTurk HIT and get paid a participation fee as well as bonuses they earned by playing your game.
Amazon MTurk requires the support of SSL by the server on which you deploy oTree.
That is you should be able to access your server with the following link https://www.myserver.com.
There are two Out of the Box solutions that oTree supports:
- running your experiment on oTree local SSL server with the following command
./otree runsslserver. This SSL server is fully compatible with oTree and can be used interchangeably with regular server./otree runserver. - deploying oTree on Heroku. Heroku by default support SSL. Deploying to heroku is explained here.
Researchers must have an employer account with AMT, which currently requires a U.S. address and bank account.
To make payments to participants you need to generate
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
here:
On heroku add generated values to your environment variables:
heroku config:set AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID
heroku config:set AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEYYou should look in settings.py for all settings related to Mechanical Turk (do a search for "mturk").
You can edit the properties of the HIT such as the title and keywords,
as well as the qualifications required to participate.
The monetary reward paid to workers is the participation_fee for your session_type.
When you publish your HIT to MTurk, it will be visible to workers.
When a worker clicks on the link to take part in the HIT, they will see the MTurk interface,
with your app loaded inside a frame (as an ExternalQuestion).
Initially, they will be in preview mode, and will see the preview_template you specify in settings.py.
After they accept the HIT, they will see the first page of your session,
and be able to play your session while it is embedded inside a frame in the MTurk worker interface.
The only modification you should make to your app for it to work on AMT is to add a {% next_button %}
to the final page that your participants see. When the participant clicks this button,
they will be directed back to the mechanical Turk website and their work will be submitted.
After workers have completed the session, you can click on the "payments" Tab for your session. Here, you will be able to approve submissions, and also pay the bonuses that workers earned in your game.
The Mechanical Turk Developer Sandbox is a simulated environment that lets you test Human Intelligence Tasks (HITs) prior to publication in the marketplace. This environment is available for both worker and requester.
After publishing the HIT you can test it both as a worker and as a requester
using the links provided on "MTurk" Tab of your session admin panel.
These links will work only locally given that you created
your HIT being on local server(./otree runsslserver).
Games that involve synchronous interaction between participants (i.e. wait pages) can be tricky on mechanical Turk.
First, you should set group_by_arrival_time as True so that participants are assigned to groups in the order in which they arrive,
to minimize unnecessary waiting time.
However, there is still the issue that if one participant drops out then other participants might be stuck on a wait page. One way to mitigate this attrition problem is to use a "lock-in" task. In other words, before your multiplayer game, you can have a subsession that is a single-player app and takes some effort to complete. The idea is that a participant takes the effort to complete this initial task, they are less likely to drop out after that point. Then, the first few participants to finish the lock in task will be assigned to the same group in the next subsession, which is the multiplayer game.
An upcoming feature in oTree that has not yet been implemented is the ability to auto-submit pages if the participant drops out or does not complete the page in time. This should enable the gameplay to proceed even if there is attrition.
- Field labels should go in the template formfield, rather than the model field's
verbose_name. null=Trueanddefault=Noneare not necessary in your model field declarations; in oTree fields are null by default.- On
CharFields,max_lengthis not required.
There are several alternatives for upgrading or reinstalling oTree.
(TODO: when to use which)
- On Windows: Browse to ``%APPDATA%
and delete the folderotree-launcher` - On Mac/Linux: Delete the folder
~/.otree-launcher - Re-download the launcher according to the instructions on http://www.otree.org/download/
Start the launcher and click the "terminal" button to get your console. Then type:
git pull https://github.com/oTree-org/oTree.git master
pip install -r requirements_base.txt
./otree resetdb
Note: you may get merge conflicts if you have modified many files.
Start the launcher and click the "terminal" button to get your console. Then type:
Modify otree-core version number in requirements_base.txt (the latest version is here),
then run:
pip install -r requirements_base.txt