Madeline is the leader of the Amazons.
After a one-night stand with Dean, Lydia gives birth to Emma. Within a day the child is the equivalent of an eight year old, and Lydia hands her over to Madeline for training. Madeline takes her to a building where she indoctrinated into the Amazon ways with other young girls.
Charlene Penn, the detective supposedly investigating the murders, is also an Amazon who informs Madeline that the Winchesters are on their trail. Madeline orders the death of Sam, along with Dean.
When Sam and Dean return to the building where the Amazons were based, they have disappeared.