These are the fruits (in this case DRUPES actually) of the Dogwood tree. They are inedible, at least to humans.
The Dogwood gets its name from the Old English "Dag", as its firm timber was used to make dags, ie daggers and skewers.
An old legend says that the Dogwood was used to make the cross Jesus was crucified on, as it was larger and straighter in those days.
From that day, Jesus made the Dogwood gnarled and twisted so no further crosses could be made from its wood, and he also changed the flowers into a four-petalled white flower, signifying the cross, with the red(ish) drupes representing his blood.
Nonsense of course, but a nice tale.