You can open previous location by hitting Ctrl-O. You can prefix the command with a number to go multiple files back. You can also travel forward in “file history” by using Ctrl-I.

There’s a nice article on Vim Wikia with more details on a subject.