You are now ready to enter into the world of public records. The first thing you are going to need to do is to figure out where the records you want are. Records may be stored on a town, county or state level. Once you figure out where you ancestor was born, married or died, you need to figure out where the records are held.
I can't tell you where yours will be, there are just too many variables. There is a book, however, called "The Source: A Guidebook to American Genealogy" which your local library may have or be able to get for you and it tells you in every state where the records are stored.
Before you go looking for records, you may want to join a local genealogical society. Some states require that you be a registered genealogist to be allowed access to records, especially birth records that are under 100 years old.
Some places may let you make photo copies of records and other places may require you buy a registered copy. This can start to get expensive but it is worth it to have the hard copies. You want to try to get not only your ancestor but all of their siblings. You never know which record might have that gem of information.
Always treat any books you are looking at respectfully and politeness goes a long way when you are visiting places where people are trying to do their job at the same time.
Before heading off to look for records, check to see if they might be available online. Some states have digitalized their records. Some of them, especially older records are available on Ancestry.com and Familyserch.com