Oh I just noticed this in the documentation:
If you copy and paste the request URL from the examples into a web browser, you will get a invalid token error message. Even if you replace yourToken in the request URL with a valid token that you obtained using your credentials, you will still get the invalid token message. Submitting requests to the service directly using a web browser is not supported. You need to construct a similar request in the API you have chosen to build your application.
So, I assume if I setup the request through the Javascript API it will work.