[<< | Prev | Index | Next | >>]

Monday, October 05, 2009

AI Challenge: Compete, or Cooperate?

Re. the AI Challenge:

I would like to make this a cooperative challenge instead of competitive one, and am curious to hear people's thoughts about that.

The problem with a competitive contest, especially one composed of a sequence of challenges with most depending on the solutions to prior ones, is that people will not only be keeping their code and ideas tight during a challenge, but also afterward because they won't want to give their competition a leg-up on subsequent challenges.

The problem with that in turn is that the evolution of the ideas becomes separate clonal lines instead of cross-pollinating, which is exponentially worse in terms of rate of progress.

So, my question is: Will more or less people be interested in pursuing this if it is non-competitive? (Let's assume for now there is no substantial monetary prize in either case.) And if less, will it be less as in zero, or not so much less that it isn't still a net win in terms of rate of progress?

The way I imagine a "cooperative challenge" being run is essentially the same as a competitive one, where people/teams make submissions, versions are run against each other and scored and ranked (though by version or submission name instead of by team), and so on, with the main difference being that all submissions must come with source and a reasonably understandable overview of the method employed, and that people are openly encouraged to use each other's ideas and code, provided they properly credit the latter and strongly suggested they properly credit the former. (And of course direct collaboration between teams would be expected and encouraged as well.)

People/teams still get implicit kudos for submitting a version that performs well -- and continue to get kudos if/when their code is used in subsequent submissions. Furthermore, I would expect it to be more fun in the sense that even when you are out of ideas at the moment, your meta-team will still be making progress and moving things forward, and you can jump back in at any time without having fallen behind.

The only thing missing is the competition against each other, which I worry is a primary motivator for many. On the other hand, I wonder if absent a significant monetary reward, those people have already largely been filtered out and those who are left are most motivated to see progress made, and hence would prefer a cooperative arrangement anyway. Also, there still would be some bonified competition in the sense that different teams will prefer different approaches, and they will battle it out on the score board to settle which one actually works best...

Thoughts?

Lastly, what if we had substantial prize money available -- any ideas for allocating prizes to a cooperative community?

[<< | Prev | Index | Next | >>]