Crowdflower Trickery: dynamic tasks

Here at Bizo, we often use crowdflower to improve the quality of our data. In doing so, we’ve come across some cool, but under-documented, tricks. One trick that we’ve particularly found useful is using liquid for designers to dynamically generate crowdflower tasks. Let us take a look at how to do this with a toy example.

Problem:
We are given a list of deserts from a bakery as shown in data.csv bellow . Our task is to determine the following:

1. If the desert is a cake, is it appropriate for:

• Wedding events
• Birthday events
• Casual eating

2. If the desert is NOT a cake, is it appropriate for:

• eating using hands
• eating using forks
• eating using spoons

file: data.csv
"id","name","desert_type"
0,"Peanut Butter Cake","cake"
1,"Strawberry Donut","donut"
2,"Cookies and crème ice cream cake","cake"
3,"Apple Strudle","strudle"
4,"Chocolate Pie","pie"
5,"Red Velvet Cake","cake"


Initial Approach:
One approach is to create two separate jobs: one which will ask workers questions relevant to cakes and one which will ask the workers questions relevant to non-cake deserts. To create these jobs we would first have to break up our data into two data files: one containing data only for cakes (see cakes.csv) and one for everything else (see non-cakes.csv). Then, we would have to specify the appropriate cml code for each job. For our example, the data and cml code will look like:

Job 1: Cake deserts

file: cake.csv
"id","name","desert_type"
0,"Peanut Butter Cake","cake"
2,"Cookies and crème ice cream cake","cake"
5,"Red Velvet Cake","cake"

file: cake.cml
Desert: {{name}}
<cml:checkboxes label="This desert appropriate for">
<cml:checkbox label="wedding events"/>
<cml:checkbox label="birthday events"/>
<cml:checkbox label="casual eating"/>
</cml:checkboxes>


Job 2: Non-cake deserts

file: non-cake.csv
"id","name","desert_type"
1,"Strawberry Donut","donut"
3,"Apple Strudle","strudle"
4,"Chocolate Pie","pie"

file: non-cake.cml

Desert: {{name}}

<cml:checkboxes label="This desert appropriate for">

<cml:checkbox label="eating using hands"/>

<cml:checkbox label="eating using forks"/>

<cml:checkbox label="eating using spoons"/>

</cml:checkboxes>

Approach using Liquid:

Using liquid, we can solve the same problem with just one job rather than two. To do so, we simply embed liquid logic tags into our cml code to dynamically display tasks to users -- the appropriate checkboxes will appear based on the "desert_type" field. Furthermore, we do not need to break up the data.csv file.

cml code

Desert: {{name}}
<cml:checkboxes label="This desert is appropriate for">
{% if desert_type=='cake' %}
<cml:checkbox label="wedding events"/>
<cml:checkbox label="birthday events"/>
<cml:checkbox label="casual eating"/>
{% else %}
<cml:checkbox label="eating using hands"/>
<cml:checkbox label="eating using forks"/>
<cml:checkbox label="casual using spoons"/>
{% endif %}
</cml:checkboxes>