SCM cement replacement changes the chemistry of the concrete, some properties are less desirable, and others are beneficial to long term durability. I agree with @Luke-Lombardi and @will.nash, in general the good outweighs the bad when it comes to durability and the inclusion of SCM’s (supplementary cementitious materials).
However, we should not be looking at this in binary terms. SCM’s are just a single variable in the mix and other factor such as water-cementitious ratio and reinforcing cover play a significant role in durability of the solution. Your approach to a durable design will depend on the environment you are in and the properties you are trying to promote.
I would not fear a high volume SCM. Early strength is a real limitation, but even this can me mitigated by blending in more reactive SCM’s or by using more finely ground Portland Cement.
I would recommend you hire a concrete mix professional that is familiar with concrete durability to help maximize cement reduction while achieving the projects durability requirements.