Investors are shying away from precious metals, especially when looking at the latest bullion sales data from the U.S. Mint.