In 1927, Indian National Congress held the All India Music Conference in Madras. At the end of the conference it was decided that an organisation be formed that helped the cause of music. The prime reason for this conference was E. Krishna Iyer who had played a vital role in reviving the south Indian dance art form - Bharatanatyam.

Lovers of music and other well wishers wanted to stimulate interest in Carnatic music in the South and develop a rich culture. The academy had sound planning from the start with an expert committee consisting of some of the leading musicians and scholars to advise the academy on all technical matters.

The Academy was formally inaugurated on the 18th of August, 1928 by C. P. Ramaswami Iyer, in the Y.M.I.A. Auditorium before a large and distinguished gathering.