Web Master:
Hey Angela, one of the first things you should do is to ensure that it is well drained enough - the fact that it has not really got going properly indicates that it was never happy and therefore drainage seems the most likely problem.
Oxygen starvation and root rot will cause a tree to drop fruit and leaves as it looses its ability to take up water and nutrients from the ground. Eventually it'll keel over and die.
If you need to dig it up, make sure you have a good sized rootball which needs holding together tightly with hessian. As it's still fairly young its roots won't have developed much. Then fill the hole with water and see how well it drains. It may be that you just have to find a better location, one that is elevated will always help, or even a very large tub like a half wine barrel.