When you pass a test for an pilot rating, you receive a "Temporary Airman Certificate" (form 8060-4) that grants you the privileges of that rating. In my case, on Sunday, I used that temporary certificate so that I was legal to fly the airplane back home to Morristown.
In case you're interested in what one looks like, see page 11 of this document on the FAA's web site. Mine was filled out by hand, section IX says "private" and section XII says "airplane single engine land".