Born in New York City, Miller moved with her family to London, England when she was a year old. She went to Francis Holland Junior School, from the time she was 4 to 11. She attended Heathfield St Mary's School, a boarding school in Ascot, Berkshire and later studied for a year at the Lee Strasberg Institute in New York City
Her father Edward Miller is an American banker turned Chinese art dealer. Her mother Jo Miller is South African of British descent and ran Lee Strasberg's acting academy in London. Miller has a sister, Savannah, a fashion designer; two half-brothers, Charles and Stephen, and a stepsister, Natasha Corrett (the daughter of her father's second wife Kelly Hoppen)