Funnily enough, that is one of the features it already supports (avoiding unlit streets). Another one is avoiding especially hilly routes.

Is "hilly" a simple binary indicator on each street (or section of road) or is there an entire layer of elevation data included in the world data?

In my app, yes, it's a binary flag. But in the underlying data you can get detailed elevation data for every section of road worldwide that is readily usable in open source routing engines. GraphHopper supports STRM and CGIAR (two different data elevation data sources, available globally and free to use).

SRTM is pretty bad, but good enough until height data is easier to work with.