Indeed time is relative. But, the observed distribution of matter in the universe turns out to single out one particular frame of reference (= large scale spacetime coordinate system). Namely the one in which the CMB radiation is isotropic (same in all directions, neither red shifted nor blue shifted). It is with respect to this particular reference frame that the “age of the universe” is defined.