The memory keeper's daughter by Kim Edwards, 2006.

This book is set in multiple time periods beginning in the 1960s and 1970s and tells the story of the birth of twins to successful couple, David and Norah. The girl twin is born with the obvious physical expression of Down's Syndrome so David, himself a doctor, takes a split second to tell his wife that the second baby had died and sends her away to an institution to be looked after. The person who is given the task is Caroline Gill, an associate of David's, but she decides to keep the child and moves away from the area to claim her as her own.
The lie told by David to his wife festers and, unknown to her and their son, ruins their life together . It changes David's personality so that he begins to grow away from his wife and son. The memory keeper of the title is David who becomes increasingly obsessed with photography and the effort to capture infinitesimal fragments of time, his daughter is Phoebe.
An interesting, and unusual, story about the importance of honesty and the alternate lives that could have been lived which also highlights the different attitudes to Down's Syndrome in the 1960s and 1970s.