Truncation from 'PVOID' to 'BYTE'

I'm trying to de-reference a pointer into a BYTE variable, the buffer im de-referencing is coming in as a byte from the other side, but I am getting an error at compile;

C4305: 'type cast' : truncation from 'PVOID' to 'BYTE'

BYTE nextByte = 0;
PVOID *pBuffInput = pIrp->AssociatedIrp.SystemBuffer;
nextByte = (BYTE) *pBuffInput;

Casting it into an INT or DWORD works fine, what am I missing here?