Ragweed season typically starts around August 15 and is caused by both Common (pictured above) and Giant Ragweed. Normally the Goldenrods bloom at the same time and, because they are more noticeable (the yellow flowers in the photo below), many people mistakenly believe they are the cause of their allergic rhinitis.
Only pollen that is light enough to go up one's nose can cause nose allergies. That pollen happens to be flying around because Ragweed is a wind pollinated plant (as are grasses and many trees). Take a look at Ragweed and you will see there are no insects on the drab green flowers. Goldenrod flowers, however, are covered with all manner of bees and other insects. The heavy pollen sticks to their legs and is redistributed amongst the flowers.