Etymology

The historically prior sense of “an upper balcony for black patrons” probably derives from an early association between peanuts and the African slaves who first introduced them to America;  Peanuts have never gotten much respect in America. Their reputation is better than it once was, but if something is “worth peanuts,” we know it’s trifling. Perhaps the root of the problem — so to speak — was that peanuts were first introduced to America by African slaves. During the Civil War, when food supplies were scarce, both Union and Confederate troops started eating these legumes out of necessity, and goobers gradually caught on. By 1870, P. T. Barnum’s vendors were hawking peanuts at his circus. Soon, they were also sold in theaters, where the cheap seats became known as “peanut galleries.” the extended sense of “a source of heckling” most probably derives from the disesteem in which those balconies and their occupants were generally held.

Noun

1. The upper balcony in a racially-segregated venues such as a theatre to which black patrons were restricted.  Peanut gallery was in use in the 1880s, as a synonym for nigger gallery (1840s) or nigger heaven (1870s), the upper balcony where blacks sat, as in segregated theaters.
   • 2001: John E. Kleber, The Encyclopedia of Louisville, page 630 (University Press of Kentucky; ISBN 9780813121000)

     As early as the 1870s, most theaters allowed African Americans to sit in designated areas, while the dress and parquet circles were reserved for whites. A few theaters did not allow blacks at all. In the early 1920s, black leaders protested these “” on the grounds that African Americans paid the same ticket price. A boycott was organized that resulted not only in the closing of the but also closing of the theaters to blacks altogether. It was not until the public accommodations drive in the early 1960s that all theaters were opened to blacks. On May 14, 1963, the Louisville Board of Aldermen passed the public accommodations law that made discrimination in all public facilities illegal.

2. Any source of heckling, unwelcome commentary or criticism, especially from a know-it-all or of an inexpert nature.

   Enough already from the peanut gallery; if you think you can do a better job, go right ahead.