Imagen improvising and making music just by drawing! Draw the melody and it will be played. This is what researchers from Japan and Spain wanted to create with JamSketch. Behind the scenes some processing is working on understanding what you want to play. Someone with no prior musical knowledge should be able to enjoy playing music with the JamSketch.
The real-time improvisation support system automatically generates melodies according to the melodic outlines drawn by the users. To make it sound good the researchers also used a lot of existing music pieces and musical knowledge to make a genetic algorithm based on this dataset. They also added an expressive performance model for timing and dynamic transformations. The paper 0101 from NIME2017 goes more into detail of the mathematical formulas and calculations used.
The future work includes evaluation of both user experience and the quality of generated melodies as well as further improvements with larger-scale melody corpus.
Source: https://www.nime.org/proceedings/2017/nime2017_paper0101.pdf
