Logging In

If you don't have a username and password, ask the lab manager or Dan. 

Create A new ORSEE User
  • Log in with administrative login (usual login and pasword)
  • Options >> Edit Administrators >> Create New
    • Be sure email address is correct
    • Set new user to to "admin".

Create A New Experiment

In ORSEE, an Experiment is a collection of sessions. For example, you might have a "ConG Prisoner's Dilemma" Experiment, under which there might be several sessions. Organizing things like this helps ensure that you don't re-invite people back to an experiment they have already participated in.  
Click under the left-hand Experiment "Create New" >>
  • Internal Name: should be useful to you in identifying the experiment, e.g. "ConG Prinsoner's Dilemma"
  • Public Name: should be very general. E.g. "Leeps Experiment". This might be seen by subjects, and so shouldn't give any hint as to the experiment. 
  • Email sender address: set to your ucsc email address.
    When invites are sent to our subject pool, they email will say it is coming from this address. You are probably the best person to deal with the occasional reply. Feel free to keep it at leeps if you want to. 
  • The rest is fairly intuitive. 
  • When done, click "Add".
Now when you click on the left-hand "Overview" you will see your newly added experiment to the list of current experiments. 

To get a sense of how these work, click on one of the other experiments that has a large number of Sessions, Invited Subjects and Show-up Subjects. 


Scheduling an Experiment Session

To be clear, a session is a specific date and time for your experiment run. For example, under the Cong Prisoner's Dilemma experiment, I might have a ninety minute session on March 31 2019 from 2:00-3:30 PM. 

Here we will set up the session, set the time, number of participants. After the sessions is set up, we'll invite subjects to sign up. 


Create Session

Under your experiment, in the "Sessions" box, click "Create New"
  • Date: Keep in mind this is Day.Month.Year! (I think at least three grad students - including a European - have messed up a session because they didn't set the date correctly)
  • Time and Expected Duration: be sure to be a little generous with your time. Unless you've done a number of production sessions, think about adding an extra 10 minutes to your session duration.
  • Reserve Participants: unless you're very confident that most of your subjects will show on time, I'd set this to 40% of your required number of participants. 
    Setting the reserve higher: If you set your session at an odd time and date (e.g. late Friday) you might expect a higher percentage of no-shows, and so set the reserve higher. Also, if you had people sign up more than a week before the actual session, I've noticed you have more no-shows since people forgot they signed up. 
    Have Fewer No Shows: You can help ensure more people show up to your session by calling all your subjects the day before to remind them of their commitment to show. 
  • Registration end: if you are scheduling the experiment just a few days (or hours!) before your session date, you should reduce this to 12 hours.  Or perhaps set it to 0 hours. After the "registration end" date and time, no one will be able to sign up for the session. 

Click "Add" when done.... but you're not done yet!
create session

Invite Subjects to Session(s)

Now that you have a session, you can invite subjects to it. 

Assigning Subjects

  • First, we'll assign subjects to the experiment. "Assigning" means we're creating a pool of potential subjects for this experiment. If a person is in this pool, they are able to join your experiment sessions.
    In general, you'll just want to do with the defaults and click 'seach and show'.
    In some cases you might want to exclude and include certain types of subjects (by gender, year of study, experience with other experiments, etc). 
    • In the Experiment page (where you see "basic data", "Sessions", and "Participants" boxes.), click "Assign subjects"
    • Here you select the 'type' of subjects you want to potentially invite to your experiment. In almost all cases, you'll want to go with the default and potentially invite everyone.
    • But you can also create a pool of subjects that, say, participated in a previous experiment. Or only invite male subjects, or exclude feminist studies majors. 
      - Click "search and show"
    • You now see a list of all subjects you are about all to your potential subject pool.  
      - (if for some valid reason you need to exclude people from this list - if you notice duplicates - you can use the check boxes to select subjects and click "assign only marked participants")
      - scroll down and click "Assign all participants in list" to all all.
Back at the Experiment page, you'll now see a large number under Participants >> Assigned Subjects. 


assign subjects

How ORSEE Deals With Your Subjects

It's important to understand how ORSEE manages subjects for your experiment. Your experiment does not automatically have access to all subjects in the ORSEE pool. 

Here is a breakdown:
  • Participants: The full pool of subjects
    • Registered for lab experiments: this is the full pool of people you are permitted to draw on. This number will be in the thousands, but realistically only several hundred will check your email. 
    • Registered for internet experiments: The full pool of people interested in non-lab experiments. 
  • Assigned Subjects: A smaller pool. These are the subjects you've selected to potentially take part in your experiment. There is a distinction drawn between the full pool of subjects and the pool your experiment might use, just in case it's important to you to only invite certain majors, or only invite certain genders, or only invite or exclude subjects that participated in other experiments. 
  • What about recently signed-up subjects? Somewhat annoyingly, if a subject recently signed up on ORSEE, they will not be automatically added to your experiment pool (the assigned subjects list in your experiment). These recently signed up people are the most likely to want to participate - so it's important that you assign them over before you send any invitations.  


Very Important Note
If you are inviting participants to an experiment that hasn't been touched for some time, be sure to assign new subjects to that experiment. 
As people sign up to be participants in ORSEE, these fresh potential recruits will not automatically be added to your experiment!

You will need to assign them. In your Experiment Page, under "Options", click "Assign subjects" 
  >> Select "Search and Show" with default settings 
  >> The number before "participants in result set" is the number of new participants you are about to add to your experiment pool
  >> at the bottom of this page, if you want to add all, click "Assign all participants in list"

Email Invite Subjects to Session

  • Before you send your email, be sure to "Assign Subjects", and add any people that have recently signed up for ORSEE. 
  • Now that we have session(s) we want to invite subjects to, and we have a pool of subjects to invite, we want to send out invitation emails. 
    • Go back to your experiment page  (where you see "basic data", "Sessions", and "Participants" boxes.), click "Send invitations"
    • Subject: leave this as default (or if you change it, it shouldn't give away the type of experiment you're running)
    • Body of message: Subjects receive a personalized email with this text. 
      Note that #fname#, 
      #lname#, and #link# are personalized to a specific individual. 
      Note also that #sessionlist# is list of all sessions this subject is available to participate in (ones that they were 'assigned' to). So, there might be more than one session in the list if an individual is eligible for many sessions.
    • Once it all looks good, click "mail preview" (this won't send anything). 
      • In Mail Preview, verify it all looks good. 
      • Click "back to mail page" to edit or send emails. 
    • When you're ready to send the email, click "Send to all".
      If you're sending out lots of session invitations a week, you might want to click "Send - mail to all who have not got one yet", so as not to overload anyones inbox. 
Unsubscribe:  If you get any complaints, know that the email also has an 'unsubscribe' link in it, that removes them for our ORSEE database. Just tell them to click on that link, and they won't be invited to future sessions. 
Email Timing: A note on how emails are sent. With over 1,000 potential subjects, ORSEE sends emails out in small batches over a long period of time. It perhaps takes several hours to send 1000 emails. This helps ensure email services don't consider leeps as spam (the .edu domain helps too). This means it might take many hours before all emails are sent. So don't expect people to sign up for your session instantly.  

Back on the Experiments page, you can see subjects sign up for sessions (the counts next to each session), and you can see emails going out by the Participants >> Invited Subjects field. 

invite subjects to sessions by email

Monitoring the Status of Your Session Before it Starts

On your experiment page, you'll see all your sessions listed. There, you can see the number of subjects that signed up (needed and reserve). 
In the example to the right, the full required 17 people signed up. 

Clicking on "Registered Subjects", you can see all the people who signed up, plus their contact information. 

Repeats and Grad Students: from experience, you should also make an effort to look closely at who your subjects are. Many people really like participating in leeps experiments and sign up under different emails and slightly different names. If you catch this early, you can contact these people, delete their duplicates, and ensure you only have the subjects you want showing up at session time. 
With grad students, Luba Peterson famously went to a bunch of LEEPS experiments before Ryan and Dan started to recognize her as an econ grad student. 

A quick view for multiple experiments and sessions can be found on the ORSEE calendar. 

on your Experiments page

Dealing with Subjects - Ensuring Attendance, Canceling a Session, etc. 

On your experiment page, you can see all your sessions listed by Date and Time. 

Clicking on "Registered Subjects", you can see all the people who signed up, plus their contact information.

Call Your Subjects the Day Before: The day before your session, you should call your subjects to remind them to show (from hard-won experience, this helps ensure you get enough people to show-up when promised)

Canceling Sessions - Should you need to cancel your session, you may use the "Send bulk mail to listed participants" method (lower right of page, this send an email to this list via leeps@ucsc.edu), but you should also call all these subjects as well. Very likely, some subjects will miss the email and show-up anyway. The lab would like to protect its reputation for professionalism.

Canceling Sessions Last Minute - Sometimes you might have to cancel the session at its start (e.g. not enough subjects showing up, technical problems). You must still pay subjects the show-up. 
Also, be sure to mark these subjects as "shown-up" but not "participated". 
If you want to exclude this group of subjects from future sessions for some reason (perhaps they have gained too much experience before a technical malfunction), simply also mark them as "shown-up" and "participated". 

You session page, (click "Registered Subjects" for the session date and time)

Starting and Finishing An Experiment Session

On the session page, you can see all the subjects who signed up for a particular session. 

Be sure to
: (This is actually really important to do)
  • Shown-Up: Mark with a check all subjects that showed-up.
  • Participated: Subjects who actually took part in the full session
  • Once these are all marked correctly, be sure to click "Change"
  • Shown-up, but didn't participate: for all your extras, they'll receive the show-up fee and asked to leave before the actual session starts. Since they are not marked as participated, they will be invited to this experiment in the future. 
  • Too many no shows: once a person has had "too many" no-shows  (in the orsee mysql db, 'automatic_exclusion_noshows' variable) they are removed from ORSEE automatically and never invited again. 
Suggestion: You might want to just print out this page before the session, and mark everything by hand. And then enter it into ORSEE after the session is over. 

Finishing an Experiment: once you've marked all subjects as shown-up and/or participated, and after the session is over, mark the session as Completed. You're all done with this session. 

Removing a Subject from a Session

If a subjects contact you with sufficient advance time, feel free to remove them from the session list. 

Once they are removed from the session, that opens up a free spot for someone else to sign up

If you feel they gave too little warning, feel free to just mark them as "no show", and they will be handled automatically by ORSEE's too-many-noshows-so-stop-inviting-this-subject system. 

  1. Sign into ORSEE's Admin page
  2. Find the session that canceling subject wants to be removed from
    1. Go to calendar
    2. click on the [participants] link. 
  3. On the session page, find the subject. 
  4. Note the 'session' column. For this subject, set their "session" to "no session", then click "Change (see image)