We only need it for one thing: gamma size. Moreover, some bits in the drmModeCrtc will become out-of-date, for instance the current mode, so let's avoid caching the whole struct and only keep what we know won't change.