For DeepMind there were no limits. He has been able to solve such disparate problems that go from the StarCraft video game saga to protein analysis, but now he has found a serious problem; the computer programming.
Google’s AI division is not capable of generate more elaborate codealthough in itself it has merit that it can work without having received basic information about algorithms and programming languages.
Computer programming is highly internalized among humans, and therefore should be a simple operation for an efficient AI system. But it has been causing problems in this regard, and the DeepMind team has not been able to find the key to the error. At first they associated it with a language problemthat is, the challenge description is an expression of what the algorithm should do, but the code is an identical expression albeit in a different language.
Thus, DeepMind AI it has been trained to acquire the description and convert it into an internal representation that would later generate a functional code. At first I was asked to process some of the material on GitHub, some 700GB of codeto which if you add the plain text would represent many more lines per gigabyte, a natural language that can be confusing.
DeepMind fed the results into the system very well organized: problem description, working code, failed code, and test cases used.
a latent problem
More than 40% of the solutions offered by Google’s AI system tended to run out of memory or simply failed to find an efficient solution in a reasonable amount of time, resulting in poor code. The initial idea was to see if any program AlphaCode could pass that initial test, but only 1% did.
The solutions that worked for DeepMind were similar to each other, with the entire set of incorrect answers being randomly distributed. The system identified the ten largest code groups and chose a representative from each group.
But even so, DeepMind was not sufficiently prepared, and the results denote that more than 54% of the computer programmers who would have faced the AI would have defeated it, since it has the level of a programmer with little training and experience.
The training of the AI system was based on more than 2,000 petaflops and took with him 16 times the annual energy budget of an average home. By increasing the number of solutions, the system was able to generate a greater number of correct solutions, following a factor of 10. Thus, we determined that AlphaCode works better, but is much more expensive.
However, and despite its limitations, the system of DeepMind it is working as expected and is doing internal translations between the problem and the solutions, not just throwing out chunked code. AlphaCode will continue to send you data constantly, allowing the new version to improve considerably.
The system devised by DeepMind could generate shorter code snippets that handle specific requirements. Similarly, it is curious that AlphaCode was never given an indication of what constitutes an algorithm. However, it generated functional code using an AI structure similar to that used for language translation.
