So more of, you have a flat rate of 5% growth rate. As a colony gets bigger it needs more nurseries to keep that flat rate of 5%. If you don't build more, that rate could drop to 4%, then 3%, etc...
Or, in my opinion it starts at 5% growth rate, and goes down as a colony gets bigger but can exceed the 5% growth up to a maximum of say 15%. Having more nurseries means less scientists and workers available for other tasks namely manning vehicle factories and smelters or researching topics.