Not understanding how the compiler searches for includes

So, I'm trying to include C++ header files that are in an adjacent file and I cannot for the life of me figure out how to get the compiler to find them. All I can do is move all of my files into the same folder and then all of the sudden it finds them. Well, I take that back, I can specify the entire file path and it works, but I see people all the time using just

<sub1/filename.h>

when their source is in subdir/sub2/source.cpp and the header is in subdir/sub2/source.h. How do I make this magic happen?