After reading a lot of docs. (Can't cite a reference. Sorry. I accidentally deleted my browser app data that contain a few site that has a lot of related information)
The answer to this question is "you can"
As long you got the kernel and device blob source code for the phone you're using.
But this would be a tedious task