Your ancestors did not live in a vacuum, they had neighbors and family. They interacted with people in a variety of ways. One of them may have been through organizations.
Check the area where your ancestor lived for fraternal organizations, religious organizations, ethnic organizations, men's lodges, clubs, political groups and don't forget the women. They may have been part of women's organizations as well. Whether or not there are records is questionable but at the very least you should be able to get some ideas from the newspapers.
Knowing who your ancestors were spending their leisure time with can only add more flesh to your bare bones genealogies.