Magpie Experiments

Introduction

Magpie is a browser-based application to conduct behavioural experiments. If you use our Magpie instance, both your experiment and the data you collect will be stored on our own servers.

Registering your experiment

1. Contact admin@cls.ru.nl with your Ponyland username to gain access to https://magpie.cls.ru.nl
2. Go to https://magpie.cls.ru.nl and register your experiment. After registration, you will see the Edit page of the experiment, which should contain a text block that looks something like this:

experimentId: '2',
serverUrl: 'https://magpie.cls.ru.nl',
socketUrl: 'wss://magpie.cls.ru.nl/socket',

Your experimentId: will probably be a different number. This number will become important later on.

Developing on Ponyland

Now it is time to develop your experiment. You can do this locally on your PC or on Ponyland. Below I will describe a workflow that works on Ponyland. Magpie makes use of NodeJS. This is installed already on Ponyland as a namespace. Just go to your favourite Pony and activate it:

. pathadd nodejs

Now let's set up an example experiment to illustrate how you would deploy and inspect it during development. Navigate to a folder where you want to develop your experiment, and run:

npx magpie-base@latest new "project-name"

This creates a folder with a project-name of your choice. Enter it and run:

npm install

This will install the dependencies required by your project. Just for testing we can now host a development instance of the experiment on the Ponyland server of your choice:

npm run serve

Eventually, you will see something like:

  App running at:
  - Local:   http://localhost:8080

So if you are running this on thunderlane and you open another interactive session on thunderlane, you can see what Magpie is serving if you do:

curl localhost:8080

You will see the HTML that is being served on localhost on thunderlane. However, it is a bit hard to debug your experiment by just inspecting HTML code. So let's make it so you can interact with the development version of your experiment in your local browser.

To achieve this we need to do some port forwarding using SSH. On your local machine open a terminal and execute the command:

ssh -L 8080:localhost:8080 username@pony.science.ru.nl

Adjust username and pony accordingly. Now open your favourite web browser and type localhost:8080 in the address bar. Voila, you will see the development version of your experiment!

Creating your experiment

A word of warning: this section won't teach you how experimental design in Magpie works. Please read the Magpie documentation and reference for that. With that disclaimer out of the way, let's add some content to our example experiment.

Go to your experiment folder and take a look inside the src folder. You will see 3 files: App.vue, magpie.config.js, and main.js. The structure of the experiment is defined in App.vue. Let's replace the code inside of it with this:

<template>
  <!-- The title prop will be used for the browser tab title -->
  <Experiment title="magpie demo">

      <!-- This is the welcome screen -->
      <InstructionScreen :title="'Welcome'">
        This is a sample introduction screen.
        <br />
        <br />
        This screen welcomes the participant and gives general information about
        the experiment.
        <br />
        <br />
        This mock up experiment is a showcase of the functionality of magpie.
      </InstructionScreen>

      <!-- We iterate over our experiment trials -->
      <template v-for="(rating_task, i) in sliderRating">
        <!-- and display a screen with a slider rating task
             using the built-in SliderScreen component -->
        <SliderScreen
            :key="i"
            :option-left="rating_task.optionLeft"
            :option-right="rating_task.optionRight"
            :question="rating_task.question">
        </SliderScreen>
      </template>

      <!-- This screen will ask some optional questions about the
           participant's background, like age, gender etc. -->
      <PostTestScreen />

      <!-- While setting your experiment mode to 'debug' in the magpie config
       this screen will show the results of the current experiment directly. Once you switch to directLink or prolific
       this screen will submit the results to your magpie backend -->
      <SubmitResultsScreen />
  </Experiment>
</template>

<script>
export default {
  name: 'App',
  data() {
    return {
      sliderRating,
    };
  }
};

// Here we define the independent variables for our experiment
// we could also use a separate csv file for this purpose

const sliderRating = [
  {
    question: 'How are you today?',
    optionLeft: 'good',
    optionRight: 'bad'
  },
  {
    question: "What is the weather like?",
    optionLeft: 'good',
    optionRight: 'bad'
  },
  {
    question: "How was your breakfast?",
    optionLeft: 'good',
    optionRight: 'bad'
  }
];
</script>

Save the file and redeploy the experiment using npm run serve. If you inspect this experiment in your browser, you should see a welcome screen, three screens containing a question and a slider, and a final screen with demographic questions.

Deploying an experiment

Using npm run serve to deploy the experiment is really useful for development, but it does not actually store any participant data and it is not accessible at an URL on the internet. To actually deploy a production version of our experiment we need to configure a few files, build the static files for the experiment, and serve those files at an accessible URL.

First we need to adjust the vue.config.js file located in your project folder:

module.exports = {
  configureWebpack: {
    module: {
      rules: [
        {
          test: /\.csv$/,
          loader: 'csv-loader',
          options: {
            dynamicTyping: true,
            header: true,
            skipEmptyLines: true
          }
        }
      ]
    }
  },
  lintOnSave: false,
  pluginOptions: {
    lintStyleOnBuild: false,
    stylelint: {}
  },
  publicPath: './'
};

We also need to adjust magpie.config.js located in your project's src folder such that the results of experiment are sent to the correct database. This is where you need specify the experimentId that was generated when you registered your experiment. Also make sure that serverUrl and socketUrl are set correctly, and that mode is set to directlink. You can also fill in you email address if you want. That way participants can contact you if the experiment crashes. Below you can find an example magpie.config.js configuration.

export default {
  experimentId: '2',
  serverUrl: 'https://magpie.cls.ru.nl',
  socketUrl: 'wss://magpie.cls.ru.nl/socket',
  // this will be used in prolific mode
  completionUrl: 'https://...',
  contactEmail: 'test@random.com',
  // Either 'debug', 'directLink' or 'prolific'
  mode: 'directLink',
  language: 'en'
};

Now we can build the static files for the experiment as follows:

npm run build

This creates a folder dist containing all of the static files you need to deploy your experiment. Now we just need somewhere to put these files that is accessible on the internet. As you may know, Ponyland can already be used to serve static websites. So simply create a folder in /www/lst/live/htdocs/ and copy your static files to it:

cp -r /path/to/your/project/folder/dist/* /www/lst/live/htdocs/project-name/

Now participants should be able to access your experiment at https://cls.ru.nl/project-name. Upon completion of the experiment their data will be sent to the corresponding database that can be inspected on https://magpie.cls.ru.nl. In the web interface of that site, simply click on the Manage Experiments button, and click on Retrieve CSV to download the data collected by your experiment.