Do I need to worry about phosphates in my pool?

You probably don’t need to worry about phosphates. If your water is clear and chlorine is holding, ignore them. Algae needs more than just phosphates to grow, so keep free chlorine at 1-3 ppm and fix low chlorine, bad circulation, or a dirty filter first. Only test for phosphates if you have persistent algae despite good chlorine levels.