The main problem with magic is that it’s a bit like sunlight – magical creatures can’t access it directly as a source of nourishment, just as we can’t access sunlight to power our own growth and development.
We need plants to turn sunlight into food, and then we eat the plants.
Magical creatures see us in a similar way, meaning there’s a whole ecosystem built around us which most people are completely unaware of. At least on this world.