Do note that there is a HUGE difference between challenging or difficult and overpowered. The last thing that you or your players want to deal with in a game is an overpowered monster.
The most challenging part about homebrewing anything is ensuring that it is properly balanced. Most of the time I will use this to create boss monsters, but you can homebrew just about any sort of monster you can imagine. One of my favorite things about 5e is how simple it is to create your own homebrew monsters.