Advertisement Rustic fresh unpeeled potatoes on a wooden desks Storing potatoes in the refrigerator can cause the starches to convert into sugar more quickly, resulting in a gritty texture and an overly sweet taste. Keep them in a cool, dark place like a pantry. Advertisement