In June 1642, in the Puritan town of Boston, a crowd gathers to witness the punishment of Hester Prynne, a young woman found guilty of adultery. She is required to wear a scarlet "A" ("A" standing for adulteress) on her dress to shame her. She must stand on the scaffold for three hours, to be exposed to public humiliation. As Hester approaches the scaffold, many of the women in the crowd are angered by her beauty and quiet dignity. When demanded and cajoled to name the father of her child, Hester refuses.
As Hester looks out over the crowd, she notices a small, misshapen man and recognizes him as her long-lost husband, who has been presumed lost at sea. When the husband sees Hester's shame, he asks a man in the crowd about her and is told the story of his wife's adultery. He angrily exclaims that the child's father, the partner in the adulterous act, should also be punished and vows to find the man. He chooses a new name – Roger Chillingworth – to aid him in his plan.
The Reverend John Wilson and the minister of Hester's church, Arthur Dimmesdale, question the woman, but she refuses to name her lover. After she returns to her prison cell, the jailer brings in Roger Chillingworth, a physician, to calm Hester and her child with his roots and herbs. He and Hester have an open conversation regarding their marriage and the fact that they were both in the wrong. Her lover, however, is another matter and he demands to know who it is; Hester refuses to divulge such information. He accepts this, stating that he will find out anyway, and forces her to hide that he is her husband. If she ever reveals him, he warns her, he will destroy the child's father. Hester agrees to Chillingworth's terms although she suspects she will regret it.
Following her release from prison, Hester settles in a cottage at the edge of town and earns a meager living with her needlework. She lives a quiet, sombre life with her daughter, Pearl. She is troubled by her daughter's unusual fascination by Hester's scarlet "A". As she grows older, Pearl becomes capricious and unruly. Her conduct starts rumours, and, not surprisingly, the church members suggest Pearl be taken away from Hester.
Hester, hearing rumors that she may lose Pearl, goes to speak to Governor Bellingham. With him are ministers Wilson and Dimmesdale. Hester appeals to Dimmesdale in desperation, and the minister persuades the governor to let Pearl remain in Hester's care...