About Crowdbucket

Crowdbucket helps you evaluate multiple documents in a fast and intutive way. It splits the work of evaluation over multiple people and combines the results in the end. This way you can evaluate thousands of documents in a short amount of time.

Features

Upload a csv of items that should be evaluated. You need both a text and a label. The usual workflow is, that you want users to evaluate if the label is fitting to the text. But you can also upload any other kind of data. Like a questionaire where you want to know if the user agrees or disagrees with a statement. Or a list of items that should be sorted in a specific order.

How to use

To use Crowdbucket you need to create an account. After that you can create a new project and upload your data. You can then share the link to your project with your team and they can start evaluating the items. After all items are evaluated you can download the results as a csv file and continue your work with the analysis software of your choice.

Format of Upload Data

The data you upload should be in a specific format. The format is a CSV file with the following columns:

id required, unique, varchar	text required, text	label required, text	control optional, boolean	trap optional, boolean
6638a6ef8b12418ab9da75aa	Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.	Is this a filler text?	false
658dcfbad3c98af6d68d0fa7	At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.	Is this a filler text?	true
658dcfbad3c98af6d68d0fa7	Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.	Is this a filler text?
658dcfbad3c98afj838d0fa7	This is a trap.	Is this a question?		true

id is a requiered field containing your unique ID. Your data will be exported with a field named user_defined_id. You are responsible for selecting a unique ID that you can identify later on in your project.

text is a requiered field containing the text that you want to evaluate. The text can be expanded, but it's better to keep it short.

label is a requiered field containing your label that should be classified. You can either use a different label for each item or the same label for multiple items.

control is an optional column. You only need this if an item should be swiped by every user. You don't need to have this column, but if you include it, mark the control items withtrue (True, TRUE or 1 works as well – every other variable will be interpreted as false).

trap is an optional column. You only need this if you want to include obviously wrong answers, either because you want to check the attention span of the user or to give them a relief to swipe left at least once every few labels.

Download a sample csv file here.

Format of Tutorial Data

The data you upload should be in a specific format. The format is a CSV file with the following columns:

order required, unique, int	text required, text	label required, text	answer required, text	swipe_direction required, text	explanation optional, text	note optional, text	hint optional, text
1	Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.	Is this a filler text?	Swipe right	right	This is obviously a filler text.	This is not latin	Please swipe right, this is the only true answer.
2	At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.	Is this a filler text?	Swipe down	down
3	Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.	Is this a filler text?	Swipe hard left	hard left	Also this is wrong	You did it wrong
4	This is a trap.	Is this a question?	Swipe hard right	hard right	Just checking if you are clicking on our explanation.	Don't fall for it.	OMG you did fall for it.

order is a requiered field containing the order of the tutorial starting at 0 or 1. The items will be shown in this order to the user.

text is a requiered field containing the text that you want to show in your tutorial. The text can be expanded, but it's better to keep it short.

label is a requiered field containing the label as a headline for this tutorial item. You can either use a different label for each item or the same label for multiple items.

answer is a requiered field containting what the user should do with this item. For example "Swipe this item to the right to aggree."

swipe_direction is a requiered field containing the swipe direction that is correct for this item. Only options are "left", "right", "up", "down" for a binary swipe project extended by "hard left" or "hard right" if you are using a likert skala.

explanation is an optional field containing an explanation for the user why you prefer this action.

note is an optional field containing further information for the user. It's more like a tip, for example: "You can always undo a swipe in the project".

hint is an optional field that pops up when a user was swiping in the wrong direction. It should help the user to understand why the swipe was wrong.