Berman was born in Chicago, Illinois, the son of Irene (née Marks) and Nathan Berman.
Berman started as a straight actor, receiving his training at the Goodman Theatre in Chicago, honing his acting skills in stock companies in and around Chicago and New York. In the mid-1950s, he became a member of Chicago's Compass Players, which later evolved into The Second City. While performing improvised sketches with Compass, Berman began developing solo pieces, often employing an imaginary telephone to take the place of an on-stage partner.
In a 2012 podcast interview with Marc Maron, 87-year-old Berman accused comedian Bob Newhart of ripping off his improvisational telephone routine style, describing its genesis and saying it was a "very special technique that couldn't really be imitated. It could be stolen. And it was." He continued, "I was coming to work at night and a guy stopped his car, passed me by, and said 'Hey, Shelley! There's a guy [who] stole your act!'" When asked by Maron if it was done maliciously, Berman replied, "Maliciously? He wouldn't do it maliciously. Nobody does that. But he did it to make a living. And he became a star." Berman later added, "I thought it was a rotten thing to do. I thought the agents who sold him - I thought they were just as guilty as everybody else. But, my God, to go into a town and do my show, and the critics saying that I borrowed some stuff from Newhart..."
In 1957, Berman landed his first job as a comedian at Mister Kelly's in Chicago, which led to other nightclub bookings, and a recording contract with Verve Records. His comedy albums would earn him three gold records and he'd win the first Grammy Award for a non-musical recording. He was the first standup comedian to play Carnegie Hall. Berman would go on to appear on numerous TV specials, and all of the major variety shows of the day.
Berman's success as a comedian enabled him to continue with his first love - acting. He starred on Broadway in A Family Affair and continued to do stage work in productions of The Odd Couple, Damn Yankees, Fiddler on the Roof, I'm Not Rappaport, La Cage aux Folles, and Guys and Dolls, among many others. Comedic and dramatic acting roles on television began to come his way, including appearances on episodes of The Twilight Zone, Bewitched, Peter Gunn, The Mary Tyler Moore Show, Adam-12, Emergency!, Brothers, Night Court, MacGyver, L.A. Law, Friends, Walker, Texas Ranger, The King of Queens, Grey's Anatomy, Boston Legal, Hannah Montana, CSI: NY, and Hawaii Five-0 (in 2012). From 2002 to 2009, Berman appeared as Larry David's father on Curb Your Enthusiasm, a role for which he received a 2008 Emmy Award nomination.
Among Berman's film credits are The Best Man (with Henry Fonda), Divorce American Style (with Dick Van Dyke and Debbie Reynolds), Every Home Should Have One (with Marty Feldman), The Last Producer (with Burt Reynolds), Meet the Fockers (with Robert De Niro and Ben Stiller), The Holiday (with Cameron Diaz), and You Don't Mess with the Zohan (with Adam Sandler).
Berman has been married to Sarah Herman since April 19, 1947. The two met while they were studying acting at Chicago's Goodman Theatre.
In the mid-1960s, Berman and wife Sarah adopted two children, son Joshua and daughter Rachel. The Bermans were planning Joshua's bar mitzvah when he was diagnosed with a brain tumor. Joshua died October 29, 1977, age 12.
Berman has authored three books, two plays, several TV pilot scripts, and numerous poems. For over twenty years, Berman taught humor writing in the Master of Professional Writing program at University of Southern California, where he is now a Lecturer Emeritus. He continues to do film and television work, and make personal appearances across the country year-round.
Berman and his wife are both enthusiastic supporters of the Motion Picture and Television Fund (located in Woodland Hills, California), a charitable organization that offers assistance and care to those in the motion picture and television industries with limited or no resources, and contribute their time and resources to the benefit of the facilities and the residents.