You read about it in his autobiography, and here it is!
During a trip to Long Island, my wife and I made a detour to Far Rockaway and tracked down the house that Feynman grew up in. (Thank you to Joan Feynman for excellent directions!) It is a beautiful house, well kept after all these years. It was very interesting to look at this house, and know so much of its history.
And for the curious traveler, the address is 792 Cornaga Avenue (they changed the street names from Feynman’s time there when it was called New Broadway). Please be nice to the current residents and don’t knock on their door!