...
Using the NVIDIA library, 10-bit color encoding is supported for HEVC/H.265 but not for H.264. If the source video is 10-bit, then select “As Source” in the Bit Depth pull down under Advanced options. Source video with 10-bit color can also be down converted and encoded to 8-bit color by selecting “8 BPP”.
Similarly, using the X264 library for H.264 encoding, a source with 10-bit color can be encoded to 10-bit color or can be down converted and encoded to 8-bit color.