Blog

Make Qualtrics survey work with Amazon Mechanical Turk smoothly. Yes, I mean smoothly!

I use Qualtrics to do advanced surveys that requires "drag and drop", "flow logic", etc. And then I use Amazon Mechanical Turk to recruit subjects for really cheap (I usually do $0.15/subject). This post is about how to make the two services work together.

It seems that all other existing approaches require subjects to copy a "confirmation code" here and there. But what I'll show you here is a very smooth process: Mturk workers would see the survey inside of Mturk, they work on it, submit, and get paid (see below):

mturk0.png

A Fun and Satisfying Life

The concluding remark of the the start-up class, ENGR 490, was quite thoughtful: A student's educational objective is to "learn to have a fun and satisfying life".

Things like "starting a successful company", "getting a degree", "becoming rich and famous", etc. are all possible, but not indispensable, means to that end -- "a fun and satisfying life". Never lose your direction on that!

Recommender Module Performance Enhancement & Drupal for Data-intensive Computing

I've always wanted to build a cutting edge recommender system for Drupal as good as what Amazon offers. Google Summer of Code 2009 gave me the first chance to attack this task, and I developed the Recommender API module and helper modules that provides recommendation service based on users browsing history, fivestar ratings, product purchasing history, etc. After 2 years of application in the real world, I received many users feedback concerning performance/scalability issue of the modules, which cannot be fixed under the current PHP implementation -see why here-.

To solve the performance issue, I think the best option is to outsource the complex recommendation computation to Apache Mahout instead of using my own PHP implementation. I have submitted another GSoC application for 2011 to work on this. Hope it will get accepted so that I can get this done.

The second part of my GSoC 2011 application is to build a framework so that 3rd party programs, such as Apache Mahout, can easily exchange data with Drupal for data-intensive computing, such as computing recommendations. More details is discussed in my GSoC 2011 application. I hope this would facilitate more innovations on data-intensive computation with Drupal using 3rd party script/programs.

If you like these ideas, please support my application at http://groups.drupal.org/node/137054.

Drupal rocks, and let's make it rock more :D

Roadmap for the Recommender API module and helper modules

The Recommender API module and a few helper modules were released in 2009 as a result of my Google Summer of Code 2009 project for Drupal. Thanks to users of the modules, I have received many useful feedback and suggestions over the past 1+ year of application.

Below is the roadmap for the next release of Recommender API module, which will be completely re-written.

  1. Outsource the recommendation computation to Apache Mahout. This is to break the PHP performance bottleneck when doing complex matrix calculations. See more details at Issue #816112 and Issue #414570.

  2. Add Views support so that there are more customized ways to show the recommendations. See more details at Issue #673786.

  3. Support Drupal 7. See more details at Issue #910258.

"Related modules" block for Drupal.org -- Past and Future

Our research group at the University of Michigan has been working on the "related modules" block for Drupal.org for more than 2 years now. We have published 2 papers on this project so far:

1) Assessment of Conversation Co-mentions as a Resource for Software Module Recommendation. Will be presented at ACM Recommender System Conference'09

2) Conversation Pivots and Double Pivots. Presented at ACM Computer Human Interaction Conference'08

Pages