The recent conclusion of the netflix competition let me to think about the reason to partecipate or don’t partecipate to AI competition .
The main problem of the competitions is the difficulty to construct a good problem. Is very difficult to give a problem with a small possibility of lose generality , for about every problem there is ad-hoc solution and the winner of such competition probably is not the best solution for a generic problem but only for a small group of problems around the competition problem.
So partecipating to a competition is very easy to go in the direction of developing an “overfitting” solution for the specific problem and this is the reason why I never partecipate to an AI competition.
In the other side a competition like netflix let you to verify in the real how much some solutions work and let you to understand why some strategy get better result than others but it is very important to proceed with caution to make conclusion about bad result.