Medeski was born in Louisville, Kentucky, and grew up in Florida. After studying piano as a child (starting when he was five years old), he began as a teenager to perform with musicians such as Mark Murphy and Jaco Pastorius. He attended Pine Crest School. In 1983, after graduating from high school, he began studying piano at the New England Conservatory in Boston, where he performed as a sideman with Dewey Redman, Billy Higgins, Bob Mintzer, Alan Dawson and Mr. Jellybelly. Medeski attributes his early interest in playing improvised music and jazz to listening to Oscar Peterson.
Medeski performs in a range of musical styles, from accessible groove based funk and jazz (such as the MMW albums and A Go Go with John Scofield) to more experimental music (including many of John Zorn's projects: Duras: Duchamp, Interzone, Liber Novus, Nova Express, Dreamachines, Templars: In Sacred Blood, At the Gates of Paradise etc. and collaborations with David Fiuczynski).
Medeski is best known for his work with Medeski Martin & Wood including drummer and percussionist Billy Martin and bassist Chris Wood. The trio formed in New York in 1991 playing gigs at The Village Gate. Since the mid-1990s they have toured steadily in both national and international funk, jazz, and jam music scenes. The band has collaborated with such musicians as Iggy Pop, Trey Anastasio and John Scofield.
Medeski produced and was featured on the Dirty Dozen Brass Band's 1999 release Buckjump. Also in 1999, he played the organ on two tracks of the Morphine album The Night.
In 2000, Medeski became a member of The Word, a bluesy gospel style project with members Robert Randolph (pedal steel) from Robert Randolph and the Family Band, brothers Luther Dickinson (guitar) and Cody Dickinson (drums), and Chris Chew (bass) of North Mississippi Allstars. The band released a self-titled album in 2001 and toured in 2001 and 2002. The band reunited in 2005 for a performance at the Bonnaroo Music Festival, and they embarked on tour again in late 2007.
He has also occasionally performed with Phil Lesh and Friends.
Medeski was a guest on Marian McPartland's Piano Jazz radio program on September 27, 2005. During the show he talked at length about his formal training on the piano, his approach to playing music, and some of the musicians that he admires the most. He performed several songs by Thelonious Monk and other composers, as well as the Medeski, Martin & Wood classic "Bubblehouse". The show was released on compact disc on April 4, 2006.
Medeski was also an inaugural member of the Independent Music Awards' judging panel to support independent artists.
On July 14, 2006, John Medeski & The Itch performed their debut gig at the All Good Music Festival in Masontown, West Virginia. The Itch consists of Eric Krasno (guitar) of Soulive and Adam Deitch (drums) with Medeski playing a B3 Organ. Also in 2006, Medeski performed with The Million Dollar Bashers for the recording of the soundtrack to the Bob Dylan biopic, I'm Not There.
In 2008, Medeski was featured on guitarist Will Bernard's Blue Plate Special album.
In 2016, Medeski formed the supergroup Saudade along with Chino Moreno of Deftones, Team Sleep, Palms, and Crosses; guitarist Dr. Know of Bad Brains; bassist Chuck Doom of Crosses and Team Sleep; and drummer Mackie Jayson of Cro-Mags and Bad Brains.