IVF is now being used so widely that the risks could outweigh the benefits for some couples, researchers have warned.
Doctors say the evidence underpinning the use of IVF in younger couples with unexplained infertility is weak.
Because the technique has risks for the mother and child, clinics should be more cautious about using it, they argue.
Professor Siladitya Bhattacharya, of the Evidence Based IVF Group at the University of Aberdeen, told Sky News that couples would be better off giving nature more time rather than rushing into a medical solutions.
He said: "There are situations where IVF is probably being used too soon by some couples who would have conceived on their own.
"In couples where the woman is young, or where the duration of infertility is short, awaiting spontaneous conception by doing nothing for two to three years is a reasonable option."
IVF was originally developed for women with fallopian tube disorders and severe male infertility.
But in the UK the proportion of IVF cycles for tubal problems fell from 19% in 2000 to 12% in 2011.
Over the same period the number of cycles for unexplained fertility problems more than trebled from 6,202 to 19,552.
Writing in the British Medical Journal, the researchers also warn that IVF has risks, increasing the chances of a premature birth and congenital malformations.
Mums are also at higher risk of pre-eclampsia from multiple pregnancies.
But Susan Seenan, chief executive of the support group Infertility Network UK, who had a son through IVF, said: "It's not something that anyone would undertake lightly.
"The last thing anyone thinks about when they're trying to conceive and deciding to have a family is that they will have to do it through fertility treatment.
"But unfortunately around one in six couples struggle to conceive and some of them will have to go through IVF treatment.
"Couples are not turning more readily to it, I think it is becoming slightly more easily available to some couples and unfortunately with the society we live in today some couples are leaving it a little bit later before they start trying for a family.
"The older you are, particularly with the female, the more difficult it can be to conceive and the more likely you are to have some fertility problems."
The Human Fertilisation and Embryology Authority said it regularly reviews the latest research on IVF success rates.
A spokeswoman said: "IVF has enabled thousands of women to have a much-wanted family.
"Fertility clinics in the UK are required by law to provide patients with information about the risks involved before treatment."