Javed debuted by singing a track from the movie Beti No.1, but it didn't help his career much. After that, he sang for several movies including Chameli, Dil Jo Bhi Kahey, Golmaal - Fun Unlimited, Kyun...! Ho Gaya Na, Neal 'n' Nikki and Zindaggi Rocks among others. He was noted for his renedering of the Punjabi-based track "Nagaada Nagaada" in the 2007 movie Jab We Met alongside Sonu Nigam. It's been a long and hard struggle for this 29 year old talented singer. However he was shot to real fame in 2008 with the song "Jashn-E-Bahaara" in the magnum opus Jodhaa Akbar, which is directed by Ashutosh Gowariker and music composed by A R Rahman and also a popular song "Ek Din Teri Rahon Me" composed by Pritam for the movie Naqaab. "Tu Muskura" which he has sung along with Alka Yagnik is from the movie Yuvvraaj, again from Rahman. In 2008, Javed Ali struck a chord with another song composed by Rahman, "Guzarish", that includes guest vocals of Sonu Nigam from Aamir Khan's Ghajini. This song became the most popular song of Ghajini as well as one of the biggest hits of the year. He also sang "Arziyan" from the film Delhi-6, which was critically appreciated. The song 'Mayya Yashoda' from Jhoota Hi Sahi(2010) was also his best efforts in all the way with Chinmayi(A World Tour member with A.R. Rahman). His latest Tamil songs include "Chinnanjirusuga Manasukkul" alongside Bela Shende for the Tamil movie Kunguma Poovum Konjum Puravum composed by Yuvan Shankar Raja and "Siragugal" from the Tamil movie Sarvam which is again composed by Yuvan Shankar Raja.
His songs from movie Tum Mile composed by Pritam were hits, especially the song "Tu Hi Haqeeqat". His latest hits include "Maham Maye" from Komaram Puli and most notably, "Kilimanjaro", from Enthiran which is topping the charts for many continuous weeks. He is now a Gem of A.R.Rahman's World tour. He has also sung a bengali song for the film "Sedin Dekha Hoyechilo" in Tollywood, the music director being Jeet Ganguly later he worked with vivek.