In the first part of this series, we presented how the standard Android online account management framework works and explored how Google account authentication and authorization modules are implemented on Android. In this article we will see how to use the Google credentials stored on the device to log in to Google Web sites automatically. Note that this is different from using public Google API’s, which generally only requires putting an authentication token (and possibly an API key) in a request header, and is quite well supported by the Google APIs Client Library. First, some words on what motivated this whole exercise (may include some ranting, feel free to skip to the next section).
Android developer console API: DIY
Originally developed by Timelappse, and now open source, Andlytics does all the things mentioned above, and more (and if you need yet another feature, consider contributing). So how does it manage to do all of this without an API? Through blood, sweat and a lot of protocol
But there is a bright side to all this: Developer Console v2. It was announced at this year’s Google I/O to much applause, but was only made universally available a couple of weeks ago (sound familiar?). It is a work in progress, but is showing promise. And the best part: it uses perfectly readable (if a bit heavy on
null‘s) JSON to transport data! Naturally, there was much rejoicing at the Andlytics Github project. It was unanimously decided that the sooner we obliterate all traces of GWT, the better, and the next version should use the v2 console ‘API’. Deciphering the protocol didn’t take long, but it turned out that while to log in to the v1 console all you needed was a ClientLogin (see the next section for an explanation) token straight out of Android’s
AccountManger, the new one was not so forgiving and the login flow was somewhat more complex. Asking the user for their password and using it to login was obviously doable, but no one would like that, so we needed to figure out how to log in using the Google credentials already cached on the device. Android browser and Chrome are able to automatically log you in to the developer console without requiring your password, so it was clearly possible. The process is not really documented though, and that prompted this (maybe a bit too wide-cast) investigation. Which finally leads us to the topic of this post: to show how to use cached Google account credentials for single sign-on. Let’s first see what standard ways are available to authenticate to Google’s public services and API’s.
Google services authentication and authorization
The oldest and most widely used till now authorization protocol for installed applications is ClientLogin. It assumes the application has access to the user’s account name and password and lets you get an authorization token for a particular service, that can be saved and used for accessing that service on behalf of the user. Services are identified by proprietary service names, for example ‘cl’ for Google Calendar and ‘ah’ for Google App engine. A (non-exhaustive) list of supported service names can be found in the Google Data API reference. Here are a few Android-specific ones, not listed in the reference: ‘ac2dm’, ‘android’, ‘androidsecure’, ‘androiddeveloper’, ‘androidmarket’ and ‘youngandroid’ (probably for the discontinued App Inventor). The token can be fairly long-lived (up to two weeks), but cannot be refreshed and the application needs to obtain a new token when it expires. Additionally, there is no way to validate the token short of accessing the associated service: if you get an OK HTTP status (200), it is still valid, if 403 is returned you need to consult the additional error code and retry or get a new token. Another limitation is that ClientLogin tokens don’t offer fine grained access to a service’s resources: access is all or nothing, you cannot specify read-only access or access to a particular resource only. The biggest drawback for use in mobile apps though is that ClientLogin requires access to the actual user password. Therefore, if you don’t want to force users to enter it each time a new token is required, it needs to be saved on the device, which poses various problems. As we saw in the previous post, in Android this is handled by GLS and the associated online service by storing an encrypted password or a master token on the device. Getting a token is as simple as calling the appropriate
AccountManger method, which either returns a cached token or issues an API request to fetch a fresh one. Despite it’s many limitations, the protocol is easy to understand and straightforward to implement, so it has been widely used. It has been officially deprecated since April 2012 though, and apps using it are encouraged to migrate to OAuth 2.0, but this hasn’t quite happened yet.
The OAuth 2.0 specification defines 4 basic flows for getting an authorization token for a resource, and the two ones that don’t require the client (in our scenario an Android app) to directly handle user credentials (Google account user name and password), namely the authorization code grant flow and the implicit grant flow, both have a common step that needs user interaction. They both require the authorization server (Google’s) to authenticate the resource owner (the user of the our Android app) and establish whether they grant or deny the access request for the specified scope (e.g., read-only access to profile information). In a typical Web application that runs in a browser, this is very straightforward to do: the user is redirected to an authentication page, then to a access grant page that basically says ‘Do you allow app X to access data Y and Z?’, and if they agree, another redirect, which includes an authorization token, takes them back to the original application. The browser simply needs to pass on the token in the next request to gain access to the target resource. Here’s an official Google example that uses the implicit flow: follow this link and grant access as requested to let the demo Web app display your Google profile information. With a native app things are not that simple. It can either
- use the system browser to handle the permission grant step, which would typically involve the following steps:
- launch the system browser and hope that the user will finish the authentication and permission grant process
- detect success or failure and extract the authorization token from the browser on success (from the window title, redirect URL or the cookie store)
- ensure that after granting access, the user ends up back in your app
- finally, save the token locally and use it to issue the intended Web API request
- embed a
WebViewor a similar control in the apps’s UI. Getting a token would generally involve these steps:
- in the app’s UI, instruct the user what to do and load the login/authorization page
- register for a ‘page loaded’ callback, and check for the final success URL each time it’s called
- when found, extract the token from the redirect URL or the
WebView‘s cookie jar and save it locally
- finally use the token to send the intended API request
Neither is ideal, both are confusing to the user and to implement the first one on Android you might event have to (temporarily) start a Web server (
redirect_uri is set to
http://localhost in the API console, so you can’t just use a custom scheme). The second one is generally preferable, if not pretty: here’s an (somewhat outdated) overview of what needs to be done and a more recent example with full source code. This integration complexity and UI impedance mismatch are the problems that OAuth 2.0 support via the
AccountManager initially, and recently Google Play Services aim to solve. When using either of those, user authentication is implemented transparently by passing the saved master token (or encrypted password) to the server side component, and instead of a
WebView with a permission grant page, you get the Android native access grant dialog. If you approve, a second request is sent to convey this and the returned access token is directly delivered to the requesting app. This is essentially the same flow as for Web applications, but has the advantages that it doesn’t require context switching from native to browser and back, and is much more user friendly. Of course, it only works for Google accounts, so if you wanted to write, say, a Facebook client, you still have to use a
WebView to process the access permission grant and get an authorization token.
Now that we have an idea what authentication methods are available, let’s see if we can use them to access an online Google service that doesn’t have a dedicated API.
Google Web properties single sign-on
- Case 1: you haven’t authenticated to any of the Google properties. If you access, for example,
mail.google.comin that state you will get a login screen originating at
https://accounts.google.com/ServiceLoginwith parameters specifying the service you are trying to access (‘mail’ for Gmail) and where to send you after you are authenticated. After you enter your credentials, you will generally get redirected a few times around the
accounts.google.com, which will set a few session cookies, common (
Domain=.google.com) for all services (always SID and LSID, plus a few more). The last redirect will be to the originally requested service and include an authentication token in the redirected location (usually specified with the
https://mail.google.com/mail/?auth=DQAAA...). The target service will validate the token and set a few more service-specific sessions cookies, restricted by domain and path, and with the
HttpOnlyflags set. From there, it might take a couple of more redirects before you finally land at an actual content page.
- Case 2: you have already authenticated to at least one service (Gmail in our example). In this state, if you open, say, Calendar, you will go through
https://accounts.google.com/ServiceLoginagain, but this time the login screen won’t be shown. The accounts service will modify your SID and LSID cookies, maybe set a few new ones and finally redirect you the original service, adding an authentication token to the redirect location. From there the process is similar: one or more service-specific cookies will be set and you will finally be redirected to the target content.
Logging in using
- Get a ClientLogin token (this we can do via the
- Pass it to
https://www.google.com/accounts/IssueAuthToken, to get a one-time use, short-lived token that will authenticate the user to any service (the so called, ‘ubertoken’)
- Finally, pass the ubertoken to
https://www.google.com/accounts/TokenAuth, to exchange it for the full set of browser cookies we need to do SSO
This outlines the process, but is a little light on the details. Fortunately, those can be found in the Chromium OS source code, as well as a few other projects. After a fair bit of digging, here’s what we uncovered:
- To get the mythical ubertoken, you need to pass the SID and LSID cookies to the
IssueAuthTokenendpoint like this:
- The response will give you the ubertoken, which you pass to the
TokenAuthendpoint along with the URL of the service you want to use:
- If the token check out OK, the response will give you a URL to load. If your HTTP client is set up to follow redirects automatically, once you load it, needed cookies will be set automatically (just as in a browser), and you will finally land on the target site. As long as you keep the same session (which usually means the same HTTP client instance) you will be able to issue multiple requests, without needing to go through the authentication flow again.
The hard way
AccountManagercan only give us authentication tokens, and not cookies, this might seem like a hopeless catch-22 situation. However, while browsing the
authtokenstable of the system’s accounts database earlier, we happened to notice that it actually had a bunch of tokens with type
LSID. Our next step is, of course, to try to request those tokens via the
AccountManagerinterface, and this happens to work as expected:
String sid = am.getAuthToken(account, "SID", null, activity, null, null)
String lsid = am.getAuthToken(account, "LSID", null, activity, null, null)
Having gotten those, the rest is just a matter of issuing two HTTP requests (error handling omitted for brevity):
String TARGET_URL = "https://play.google.com/apps/publish/v2/";
Uri ISSUE_AUTH_TOKEN_URL =
Uri TOKEN_AUTH_URL = Uri.parse("https://www.google.com/accounts/TokenAuth");
String url = ISSUE_AUTH_TOKEN_URL.buildUpon().appendQueryParameter("SID", sid)
HttpPost getUberToken = new HttpPost(url);
HttpResponse response = httpClient.execute(getUberToken);
String uberToken = EntityUtils.toString(entity, "UTF-8");
String getCookiesUrl = TOKEN_AUTH_URL.buildUpon()
HttpGet getCookies = new HttpGet(getCookiesUrl);
response = httpClient.execute(getCookies);
CookieStore cookieStore = httpClient.getCookieStore();
// check for service-specific session cookie
String adCookie = findCookie(cookieStore.getCookies(), "AD");
// fail if not found, otherwise get page content
String responseStr = EntityUtils.toString(entity, "UTF-8");
This lets us authenticate to the Android Developer Console (version 2) site without requiring user credentials and we can easily proceed to parse the result and use it in a native app (warning: work in progress!) from here. The downside is that for this to work, the user has to grant access twice, for two cryptically looking token types (SID and LSID).
Of course, after writing all of this, it turns out that the stock Android browser already has code that does it, which we could have used or at least referenced from the very beginning. Better yet, this find leads us to an yet easier way to accomplish our task.
The easy way
'weblogin:'. If you use it along with the service name and URL of the site you want to access, it will do all of the steps listed above automatically and instead of a token will give you a full URL you can load to get automatically logged in to your target service. This magic URL is in the format shown below, and includes both the ubertoken and the URL of the target site, as well as the service name (this example is for the Android Developer Console, line is broken for readability):
Here’s how to get the
String tokenType = "weblogin:service=androiddeveloper&"
String loginUrl = accountManager.getAuthToken(account,tokenType, false, null, null)
This is again for the Developer Console, but works for any Google site, including Gmail, Calendar and even the account management page. The only problem you might have is finding the service name, which is hardly obvious in some cases (e.g., ‘grandcentral’ for Google Voice and ‘lh2’ for Picasa).
It takes only a single HTTP request form Android to get the final URL, which tells us that the token issuing flow is implemented on the server side. This means that you can also use the Google Play Services client library to issue a
weblogin: ‘token’ (see screenshot below and note that unlike for OAuth 2.0 scopes, it shows the ‘raw’ token type). Probably goes without saying, but it also means that if you happen to come across someone’s
accounts.db file, all it takes to log in into their Google account(s) is two HTTPS requests: one to get the
MergeSession URL, and one to log in to their accounts page. If you are thinking ‘This doesn’t affect me, I use Google two-factor authentication (2FA)!’, you should know that in this case 2FA doesn’t really help. Why? Because since Android doesn’t support 2FA, to register an account with the
AccountManager you need to use an application specific password (Update: On ICS and later, GLS will actually show a WebView and let you authenticate using your password and OTP. However, the OTP is not required once you get the master token). And once you have entered one, any tokens issued based on it, will just work (until you revoke it), without requiring entering an additional code. So if you value your account, keep your master tokens close and revoke them as soon as you suspect that your phone might be lost or stolen. Better yet, consider a solution that lets you wipe it remotely (which might not work after your revoke the tokens, so be sure to check how it works before you actually need it).
As we mentioned above, this is all ClientLogin based, which is officially deprecated, and might be going away soon (EOL scheduled for April 2013). But some of the Android Google data sync feeds still depend on ClientLogin, so if you use it you would probably OK for a while. Additionally, since the
weblogin: implementation is server-based, it might be updated to conform with the latest (OAuth 2.0-based?) infrastructure without changing the client-side interface. In any case, watch the Android Browser and Chormium code to keep up to date.
Google offers multiple online services, some with both a traditional browser-based interface and a developer-oriented API. Consequently, there are multiple ways to authenticate to those, ranging from form-based username and password login to authentication API’s such as ClientLogin and OAuth 2.0. It is relatively straightforward to get an authentication token for services with a public API on Android, either using Android’s native
AccountManager interface or the newer Google Play Services extension. Getting the required session cookies to login automatically to the Web sites of services that do not offer an API is however neither obvious, nor documented. Fortunately, it is possible and very easy to do if you combine the special
'weblogin:' token type with the service name and the URL of the site you want to use. The best available documentation about this is the Android Browser source code, which uses the same techniques to automatically log you in to Google sites using the account(s) already registered on your device.
Moral of the story: interoperability is so much easier when you control all parties involved.