Archive

Audit Accounts, Partition Passwords, Stay Secure

Jan 6, 2014

It’s a new year, so it’s time to start counting days until we hear about the first database breach of 2014 to reveal a few million passwords. Before that inevitable compromise happens, take the time to clean up your web accounts and passwords. Don’t be a prisoner to bad habits.

There’s no reason to reuse a password across any account. Partition your password choices so that each account on each web site uses a distinct value. This prevents an attacker who compromises one password (hashed or otherwise) from jumping to another account that uses the same credentials.

At the very least, your email, Facebook, and Twitter accounts should have different passwords. Protecting email is especially important because so many sites rely on it for password resets.

And if you’re still using the password kar120c I salute your sci-fi dedication, but pity your password creation skills.

Next, consider improving account security through the following steps.

Consider Using OAuth — Passwords vs. Privacy

Many sites now support OAuth for managing authentication. Essentially, OAuth is a protocol in which a site asks a provider (like Facebook or Twitter) to verify a user’s identity without having to reveal that user’s password to the inquiring site. This way, the site can create user accounts without having to store passwords. Instead, the site ties your identity to a token that the provider verifies. You prove your identify to Facebook (with a password) and Facebook proves to the site that you are who you claim to be.

If a site allows you to migrate an existing account from a password-based authentication scheme to an OAuth-based one, make the switch. Otherwise, keep this option in mind whenever you create an account in the future.

But there’s a catch. A few, actually. OAuth shifts a site’s security burden from password management to token management and correct protocol implementation. It also introduces privacy considerations related to centralizing auth to a provider as well as how much providers share data.

Be wary about how sites mix authentication and authorization. Too many sites ask for access to your data in exchange for using something like Facebook Connect. Under OAuth, the site can assume your identity to the degree you’ve authorized, from reading your list of friends to posting status updates on your behalf.

Grant the minimum permissions whenever a site requests access (i.e. authorization) to your data. Weigh this decision against your desired level of privacy and security. For example, a site or mobile app might insist on access to your full contacts list or the ability to send Tweets. If this is too much for you, then forego OAuth and set up a password-based account.

(The complexity of OAuth has many implications for users and site developers. We’ll return to this topic in future articles.)

Two-Factor Auth — One Equation in Two Unknowns

Many sites now support two-factor auth for supplementing your password with a temporary passcode. Use it. This means that access to your account is contingent on both knowing a shared secret (the password you’ve given the site) and being able to generate a temporary code.

Your password should be known only to you because that’s how you prove your identity. Anyone who knows that password — whether it’s been shared or stolen — can use it to assume your identity within that account.

A second factor is intended to be a stronger proof of your identity by tying it to something more unique to you, such as a smartphone. For example, a site may send a temporary passcode via text message or rely on a dedicated app to generate one. (Such an app must already have been synchronized with the site; it’s another example of a shared secret.) In either case, you’re proving that you have access to the smartphone tied to the account. Ideally, no one else is able to receive those text messages or generate the same sequence of passcodes.

The limited lifespan of a passcode is intended to reduce the window of opportunity for brute force attacks. Imagine an attacker knows the account’s static password. There’s nothing to prevent them from guessing a six-digit passcode. However, they only have a few minutes to guess one correct value out of a million. When the passcode changes, the attacker has to throw away all previous guesses and start the brute force anew.

The two factor auth concept is typically summarized as the combination of “something you know” with “something you possess”. It really boils down to combining “something easy to share” with “something hard to share”.

Beware Password Recovery — It’s Like Shouting Secret in a Crowded Theater

If you’ve forgotten your password, use the site’s password reset mechanism. And cross your fingers that the account recovery process is secure. If an attacker can successfully exploit this mechanism, then it doesn’t matter how well-chosen your password was (or possibly even if you’re relying on two-factor auth).

If the site emails you your original password, then the site is insecure and its developers are incompetent. It implies the password has not even been hashed.

If the site relies on security questions, consider creating unique answers for each site. This means you’ll have to remember dozens of question/response pairs. Make sure to encrypt this list with something like the OS X Keychain.

Review Your OAuth Grants

For sites you use as OAuth providers (like Facebook, Twitter, Linkedin, Google+, etc.), review the third-party apps to which you’ve granted access. You should recognize the sites that you’ve just gone through a password refresh for. Delete all the others.

Where possible, reduce permissions to a minimum. You’re relying on this for authentication, not information leakage.

Use HTTPS

Universal adoption of HTTPS remains elusive. Fortunately, sites like Facebook and Twitter have set this by default. If the site has an option to force HTTPS, use it. After all, if you’re going to rely on these sites for OAuth, then the security of these accounts becomes paramount.

Maintain Constant Vigilance

Keep your browser secure. Keep your system up to date. Set a reminder to go through this all over again a year from now — if not earlier.

Otherwise, you risk losing more than one account should your password be exposed among the millions. You are not a number, you’re a human being.

...And They Have a Plan

Aug 8, 2013

No notes are so disjointed as the ones skulking about my brain as I was preparing slides for last week’s BlackHat presentation. I’ve now wrangled them into a mostly coherent write-up.

This won’t be the last post on this topic. I’ll be doing two things over the next few weeks: throwing a doc into github to track changes/recommendations/etc., responding to more questions, working on a different presentation, and trying to stick to the original plan (i.e. two things). Oh, and getting better at MarkDown.

So, turn up some Jimi Hendrix, play some BSG in the background, and read on.

== The Problem ==

Cross-Site Request Forgery (CSRF) abuses the normal ability of browsers to make cross-origin requests by crafting a resource on one origin that causes a victim’s browser to make a request to another origin using the victim’s security context associated with that target origin.

The attacker creates and places a malicious resource on an origin unrelated to the target origin to which the victim’s browser will make a request. The malicious resource contains content that causes a browser to make a request to the unrelated target origin. That request contains parameters selected by the attacker to affect the victim’s security context with regard to the target origin.

The attacker does not need to violate the browser’s Same Origin Policy to generate the cross origin request. Nor does the attack require reading the response from the target origin. The victim’s browser automatically includes cookies associated with the target origin for which the forged request is being made. Thus, the attacker creates an action, the browser requests the action and the target web application performs the action under the context of the cookies it receives – the victim’s security context. An effective CSRF attack means the request modifies the victim’s context with regard to the web application in a way that’s favorable to the attacker. For example, a CSRF attack may change the victim’s password for the web application.

CSRF takes advantage of web applications that fail to enforce strong authorization of actions during a user’s session. The attack relies on the normal, expected behavior of web browsers to make cross-origin requests from resources they load on unrelated origins.

The browser’s Same Origin Policy prevents a resource in one origin to read the response from an unrelated origin. However, the attack only depends on the forged request being submitted to the target web app under the victim’s security context – it does not depend on receiving or seeing the target app’s response.

== The Proposed Solution ==

SOS is proposed an additional policy type of the Content Security Policy. Its behavior also includes pre-flight behavior as used by the Cross Origin Resource Sharing spec.

SOS isn’t just intended as a catchy an acronym. The name is intended to evoke the SOS of Morse code, which is both easy to transmit and easy to understand. If it is required to explain what SOS stands for, then “Session Origin Security” would be preferred. (However, “Simple Origin Security”, “Some Other Security”, and even “Save Our Site” are acceptable. “Same Old Stuff” is discouraged. More options are left to the reader.)

An SOS policy may be applied to one or more cookies for a web application on a per-cookie or collective basis. The policy controls whether the browser includes those cookies during cross-origin requests. (A cross-origin resource cannot access a cookie from another origin, but it may generate a request that causes the cookie to be included.)

== Format ==

A web application sets a policy by including a Content-Security-Policy response header. This header may accompany the response that includes the Set-Cookie header for the cookie to be covered, or it may be set on a separate resource.

A policy for a single cookie would be set as follows, with the cookieName of the cookie and a directive of 'any', 'self', or 'isolate'. (Those directives will be defined shortly.)

Content-Security-Policy: sos-apply=_cookieName_ '_policy_'

A response may include multiple CSP headers, such as:

Content-Security-Policy: sos-apply=_cookieOne_ '_policy_'
Content-Security-Policy: sos-apply=_cookieTwo_ '_policy_'

A policy may be applied to all cookies by using a wildcard:

Content-Security-Policy: sos-apply=* '_policy_'

== Policies ==

One of three directives may be assigned to a policy. The directives affect the browser’s default handling of cookies for cross-origin requests to a cookie’s destination origin. The pre-flight concept will be described in the next section; it provides a mechanism for making exceptions to a policy on a per-resource basis.

Policies are only invoked for cross-origin requests. Same origin requests are unaffected.

'any' – include the cookie. This represents how browsers currently work. Make a pre-flight request to the resource on the destination origin to check for an exception response.

'self' – do not include the cookie. Make a pre-flight request to the resource on the destination origin to check for an exception response.

'isolate' – never include the cookie. Do not make a pre-flight request to the resource because no exceptions are allowed.

== Pre-Flight ==

A browser that is going to make a cross-origin request that includes a cookie covered by a policy of 'any' or 'self' must make a pre-flight check to the destination resource before conducting the request. (A policy of 'isolate' instructs the browser to never include the cookie during a cross-origin request.)

The purpose of a pre-flight request is to allow the destination origin to modify a policy on a per-resource basis. Thus, certain resources of a web app may allow or deny cookies from cross-origin requests despite the default policy.

The pre-flight request works identically to that for Cross Origin Resource Sharing, with the addition of an Access-Control-SOS header. This header includes a space-delimited list of cookies that the browser might otherwise include for a cross-origin request, as follows:

Access-Control-SOS: cookieOne CookieTwo

A pre-flight request might look like the following, note that the Origin header is expected to be present as well:

OPTIONS https://web.site/resource HTTP/1.1
Host: web.site
Origin: https://other.origin
Access-Control-SOS: sid
Connection: keep-alive
Content-Length: 0

The destination origin may respond with an Access-Control-SOS-Reply header that instructs the browser whether to include the cookie(s). The response will either be 'allow' or 'deny'.

The response header may also include an expiration in seconds. The expiration allows the browser to remember this response and forego subsequent pre-flight checks for the duration of the value.

The following example would allow the browser to include a cookie with a cross-origin request to the destination origin even if the cookie’s policy had been 'self’. (In the absence of a reply header, the browser would not include the cookie.)

Access-Control-SOS-Reply: 'allow' expires=600

The following example would deny the browser to include a cookie with a cross-origin request to the destination origin even if the cookie’s policy had been 'any'. (In the absence of a reply header, the browser would include the cookie.)

Access-Control-SOS-Reply: 'deny' expires=0

The browser would be expected to track policies and policy exceptions based on destination origins. It would not be expected to track pairs of origins (e.g. different cross-origins to the destination) since such a mapping could easily become cumbersome, inefficient, and more prone to abuse or mistakes.

As described in this section, the pre-flight is an all-or-nothing affair. If multiple cookies are listed in the Access-Control-SOS header, then the response applies to all of them. This might not provide enough flexibility. On the other hand, simplicity tends to encourage security.

== Benefits ==

Note that a policy can be applied on a per-cookie basis. If a policy-covered cookie is disallowed, any non-covered cookies for the destination origin may still be included. Think of a non-covered cookie as an unadorned or “naked” cookie – their behavior and that of the browser matches the web of today.

The intention of a policy is to control cookies associated with a user’s security context for the destination origin. For example, it would be a good idea to apply 'self' to a cookie used for authorization (and identification, depending on how tightly coupled those concepts are by the app’s reliance on the cookie).

Imagine a Wordpress installation hosted at https://web.site/. The site’s owner wishes to allow anyone to visit, especially when linked-in from search engines, social media, and other sites of different origins. In this case, they may define a policy of 'any' set by the landing page:

Content-Security-Policy: sos-apply=sid 'any'

However, the /wp-admin/ directory represents sensitive functions that should only be accessed by intention of the user. Wordpress provides a robust nonce-based anti-CSRF token. Unfortunately, many plugins forget to include these nonces and therefore become vulnerable to attack. Since the site owner has set a policy for the sid cookie (which represents the session ID), they could respond to any pre-flight request to the /wp-admin/ directory as follows:

Access-Control-SOS-Reply: 'deny' expires=86400

Thus, the /wp-admin/ directory would be protected from CSRF exploits because a browser would not include the sid cookie with a forged request.

The use case for the 'isolate' policy is straight-forward: the site does not expect any cross-origin requests to include cookies related to authentication or authorization. A bank or web-based email might desire this behavior. The intention of isolate is to avoid the requirement for a pre-flight request and to forbid exceptions to the policy.

== Notes ==

This is a draft. The following thoughts represent some areas that require more consideration or that convey some of the motivations behind this proposal.

This is intended to affect cross-origin requests made by a browser.

It is not intended to counter same-origin attacks such as HTML injection (XSS) or intermediation attacks such as sniffing. Attempting to solve multiple problems with this policy leads to folly.

CSRF evokes two sense of the word “forgery”: creation and counterfeiting. This approach doesn’t inhibit the creation of cross-origin requests (although something like “non-simple” XHR requests and CORS would). Nor does it inhibit the counterfeiting of requests, such as making it difficult for an attacker to guess values. It defeats CSRF by blocking a cookie that represents the user’s security context from being included in a cross-origin request the user likely didn’t intend to make.

There may be a reason to remove a policy from a cookie, in which case a CSP header could use something like an sos-remove instruction:

Content-Security-Policy: sos-remove=cookieName

Cryptographic constructs are avoided on purpose. Even if designed well, they are prone to implementation error. They must also be tracked and verified by the app, which exposes more chances for error and induces more overhead. Relying on nonces increases the difficulty of forging (as in counterfeiting) requests, whereas this proposed policy defines a clear binary of inclusion/exclusion for a cookie. A cookie will or will not be included vs. a nonce might or might not be predicted.

PRNG values are avoided on purpose, for the same reasons as cryptographic nonces. It’s worth noting that misunderstanding the difference between a random value and a cryptographically secure PRNG (which a CSRF token should favor) is another point against a PRNG-based control.

A CSP header was chosen in favor of decorating the cookie with new attributes because cookies are already ugly, clunky, and (somewhat) broken enough. Plus, the underlying goal is to protect a session or security context associated with a user. As such, there might be reason to extended this concept to the instantiation of Web Storage objects, e.g. forbid them in mixed-origin resources. However, this hasn’t really been thought through and probably adds more complexity without solving an actual problem.

The pre-flight request/response shouldn’t be a source of information leakage about cookies used by the app. At least, it shouldn’t provide more information than might be trivially obtained through other techniques.

It’s not clear what an ideal design pattern would be for deploying SOS headers. A policy could accompany each Set-Cookie header. Or the site could use a redirect or similar bottleneck to set policies from a single resource.

It would be much easier to retrofit these headers on a legacy app by using a Web App Firewall than it would be trying to modify code to include nonces everywhere.

It would be (possibly) easier to audit a site’s protection based on implementing the headers via mod_rewrite tricks or WAF rules that apply to whole groups of resources than it would for a code audit of each form and action.

The language here tilts (too much) towards formality, but the terms and usage haven’t been vetted yet to adhere to those in HTML, CSP and CORS. The goal right now is clarity of explanation; pedantry can wait.

== Cautions ==

In addition to the previous notes, these are highlighted as particular concerns.

Conflicting policies would cause confusion. For example, two different resources separately define an 'any' and 'self' for the same cookie. It would be necessary to determine which receives priority.

Cookies have the unfortunate property that they can belong to multiple origins (i.e. sub-domains). Hence, some apps might incur additional overhead of pre-flight requests or complexity in trying to distinguish cross-origin of unrelated domains and cross-origin of sub-domains.

Apps that rely on “Return To” URL parameters might not be fixed if the return URL has the CSRF exploit and the browser is now redirecting from the same origin. Maybe. This needs some investigation.

There’s no migration for old browsers: You’re secure (using a supporting browser and an adopted site) or you’re not. On the other hand, an old browser is an insecure browser anyway – browser exploits are more threatening than CSRF for many, many cases.

There’s something else I forgot to mention that I’m sure I’ll remember tomorrow.

I’ll leave you with this quote from the last episode of BSG. Thanks for reading!

Six: All of this has happened before.

Baltar: But the question remains, does all of this have to happen again?

The Resurrected Skull

Jul 1, 2013

It’s been seven hours and fifteen days.

No. Wait. It’s been seven years and much more than fifteen days.

But nothing compares to the relief of finishing the 4th edition of The Anti-Hacker Toolkit. The book with the skull on its cover. A few final edits need to be wrangled, but they’re minor compared to the major rewrite this project entailed.

The final word count comes in around 200,000. That’s slightly over twice the length of Hacking Web Apps. (Or roughly 13,000 Tweets or 200 blog posts.) Those numbers are just trivia associated with the mechanics of writing. The reward of writing is the creative process and the (eventual…) final product.

In retrospect (and through the magnfying lens of self-criticism), some of the writing in the previous edition was awful. Some of it was just inconsistent with terminology and notation. Some of it was unduly sprinkled with empty phrases or sentences that should have been more concise. Fortunately, it apparently avoided terrible cliches (all cliches are terrible, I just wanted to emphasize my distaste for them).

Many tools have been excised; others have been added. A few pleaded to remain despite their questionable relevance (I’m looking at you, wardialers). But such content was trimmed to make way for the modern era of computers without modems or floppy drives.

The previous edition had a few quaint remarks, such as a reminder to save files to a floppy disk, references to COM ports, and astonishment at file sizes that weighed in at a few dozen megabytes. The word zombie appeared three times, although none of the instances were as threatening as the one that appeared in my last book.

Over the next few weeks I’ll post more about this new edition and introduce you to its supporting web site. This will give you a flavor for what the book contains better than any book-jacket marketing dazzle.

In spite of the time dedicated to the book, I’ve added 17 new posts this year. Five of them have broken into the most-read posts since January. So, while I take some down time from writing, check out the archives for items you may have missed.

And if you enjoy reading content here, please share it! Twitter has proven to be the best mechanism for gathering eyeballs. Also, consider pre-ordering the new 4th edition or checking out my current book on web security. In any case, thanks for stopping by.

Meanwhile, I’ll be relaxing to music. I’ve put Sinéad O’Connor in the queue. It’s a beautiful piece. And a cover of a Prince song, which reminds me to put some Purple Rain in the queue, too. Then it’s on to a long set of Sisters of Mercy, Wumpscut, Skinny Puppy, and anything else that makes it feel like every day is Halloween.

Password Interlude in D Minor

Aug 27, 2012

While at least one previous post here castigated poor password security, a few others have tried to approach the problem in a more constructive manner. Each of these posts share fundamental themes:

• Protect the password in transit from the threat of sniffers or intermediation attacks – Use HTTPS during the entire authentication process. HSTS is better. HSTS plus DNSSEC is best.
• Protect the password in storage to impede the threat of brute force guessing – Never store the plaintext version of the password. Store the salted hash, preferably with PBKDF2. Where possible, hash the password in the browser to further limit the plaintext version’s exposure and minimize developers’ temptation or expectation to work with plaintext. Hashing affects the amount of effort an attacker must expend to obtain the original plaintext password, but it offers little protection for weak passwords. Passwords like p@ssw3rd or lettheright1in are going to be guessed quickly.
• Protect the password storage from the threat of theft – Balance the attention to hashing passwords with attention to preventing them from being stolen in the first place. This includes (what should be) obvious steps like fixing SQL injection as well as avoiding surprises from areas like logging (such as the login page requests, failed logins), auditing (where password “strength” is checked on the server), and ancillary storage like backups or QA environments.

Implementing PBKDF2 for password protection requires two choices: an HMAC function and number of iterations. For example, WPA2 uses SHA-1 for the HMAC and 4,096 iterations. A review of Apple’s OS X FileVault 2 (used for full disk encryption) reveals that it relies in part on at least 41,000 iterations of SHA-256. RFC 3962 provides some insight, via example, of how to select an iteration count. It’s a trade-off between inducing overhead on the authentication system (authentication still needs to be low-latency from the user’s perspective, and too much time exposes it to easy DoS) versus increasing an attacker’s work effort.

A more sophisticated approach that I haven’t covered yet is the Secure Remote Password (SRP) protocol. SRP introduces a mechanism for password authentication and key exchange; becoming a more secure way to protect the authentication process from passive (e.g. sniffing) and active (e.g. replay, spoofing) attacks. However, the nature of browser security, especially the idea of crypto implemented in JavaScript, adds some interesting wrinkles to the practical security of SRP – not enough to dismiss SRP, just to understand how DNS, mixed-content, and JavaScript’s execution environment may have adverse effects. That’s a topic for another day.

Finally, sites may choose to avoid password management altogether by adopting strategies like OAuth or OpenID. Taking this route doesn’t magically make password-related security problems disappear. Rather than specifically protecting passwords, a site must protect authentication and authorization tokens; it’s still necessary to enforce HTTPS and follow secure programming principles. However, the dangers of direct compromise of a user’s password are greatly reduced.

The state of password security is a sad subject. Like D minor, which is the saddest of all keys.

Parsing .NET ViewState

Jan 27, 2012

The JavaScript-based parser has been moved to a github repository.

Background on parsing unencrypted ViewState is here followed by part two.

.NET ViewState Byte Sequences

A Spirited Peek into ViewState, Part II

May 25, 2011

Our previous article started with an overview of the ViewState object. It showed some basic reverse engineering techniques to start deconstructing the contents embedded within the object. This article broaches the technical aspects of implementing a parser to automatically pull the ViewState apart.

We’ll start with a JavaScript example. The code implements a procedural design rather than an object-oriented one. Regardless of your design preference, JavaScript enables either method.

The ViewState must be decoded from Base64 into an array of bytes. We’ll take advantage of browsers’ native atob and btoa functions rather than re-implement the Base64 routines in JavaScript. Second, we’ll use the proposed [ArrayBuffer] data type in favor of JavaScript’s String or Array objects to store the unencoded ViewState. Using ArrayBuffer isn’t necessary, but it provides a more correct data type for dealing with 8-bit values.

Here’s a function to turn the Base64 ViewState into an array of bytes in preparation of parsing:

function analyzeViewState(input) {
    var inputLength = input.length;
    var rawViewState = atob(input);
    var rawViewStateLength = rawViewState.length;
    var vsBytes = new Uint8Array(ArrayBuffer(rawViewStateLength));

    for(i = 0; i < rawViewStateLength; ++i) {
        vsBytes[i] = rawViewState.charCodeAt(i);
    }

    if(vsBytes[0] == 0xff & vsBytes[1] == 0x01) { // okay to continue, we recognize this version
        var i = 2;
        while(i < vsBytes.length) {
            i = parse(vsBytes, i);
        }
    }
    else {
        document.writeln("unknown format");
    }
}

The parse function will basically be a large switch statement. It takes a ViewState buffer, the current position in the buffer to analyze (think of this as a cursor), and returns the next position. The skeleton looks like this:

function parse(bytes, pos) {
    switch(bytes[pos]) {
        case 0x64: // EMPTY
            ++pos;
            break;
        default: // unknown byte
            ++pos;
            break;
    }
    return pos;
}

If you recall from the previous article, strings were the first complex object we ran into. But parsing a string also required knowing how to parse numbers. This is the function we’ll use to parse numeric values. The functional approach coded us into a bit of a corner because the return value needs to be an array that contains the decoded number as an unsigned integer and the next position to parse (we need to know the position in order to move the cursor along the buffer):

function parseUInteger(bytes, pos) {
    var n = parseInt(bytes[pos]) & 0x7f;

    if(parseInt(bytes[pos]) > 0x7f) {
        ++pos;
        var m = (parseInt(bytes[pos]) & 0x7f) << 7;
        n += m;

        if(parseInt(bytes[pos]) > 0x7f) {
            ++pos;
            var m = (parseInt(bytes[pos]) & 0x7f) << 14;
            n += m;
        }
    }

    ++pos;

    return [n, pos];
}

With the numeric parser created we can update the switch statement in the parse function:

function parse(bytes, pos) {
    var r = [0, 0];
    switch(bytes[pos]) {
        case 0x02:
            ++pos;
            r = parseUInteger(bytes, pos);
            pos = r[1];
            document.writeln("number: " + r[0]);
            break;
        ...

Next up is parsing strings. We know the format is 0x05, followed by the length, followed by the “length” number of bytes. Now add this to the switch statement:

switch(bytes[pos]) {
    ...
    case 0x05:
        ++pos;
        r = parseUInteger(bytes, pos);
        var size = r[0];
        pos = r[1];
        var s = parseString(bytes, pos, size);
        pos += size;
        document.writeln("string (" + size + "): " +
            s.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;'));
        break;
    ...

The parseString function will handle the extraction of characters. Since we know the length of the string beforehand it’s unnecessary for parseStringto return the cursor’s next position:

function parseString(bytes, pos, size) {
    var s = new String("");

    for(var i = pos; i < pos + size; ++i) {
        s += String.fromCharCode(parseInt(bytes[i], 10));
    }

    return s;
}

We’ll cover two more types of objects before moving on to an alternate parser. A common data type is the Pair. As you’ve likely guessed, this is an object that contains two objects. It could also be called a tuple that has two members. The Pair is easy to create. It also introduces recursion. Update the switch statement with this:

switch(bytes[pos]) {
    ...
    case 0x0f:
        ++pos;
        document.writeln("pair");
        pos = parse(bytes, pos); // member 1
        pos = parse(bytes, pos); // member 2
        break;
    ...

More containers quickly fall into place. Here’s another that, like strings, declares its size, but unlike strings may contain any kind of object:

switch(bytes[pos]) {
    ...
    case 0x16:
        ++pos;
        r = parseUInteger(bytes, pos);
        var size = r[0];
        pos = r[1];
        document.writeln("array of objects (" + size + ")");
        for(var i = 0; i < size; ++i) {
            pos = parse(bytes, pos);
        }
        break;
    ...

From here you should have an idea of how to expand the switch statement to cover more and more objects. You can use this page as a reference. JavaScript’s capabilities exceed the simple functional approach of these previous examples; it can handle far more robust methods and error handling. Instead of embellishing that code, let’s turn our text editor towards a different language: C++.

Diving into C++ requires us to start thinking about object-oriented solutions to the parser, or at least concepts like STL containers and iterators. You could very easily turn the previous JavaScript example into C++ code, but you’d really just be using a C++ compiler against plain C code rather than taking advantage of the language.

In fact, we’re going to take a giant leap into the Boost.Spirit library. The Spirit library provides a way to create powerful parsers using clear syntax. (Relatively clear despite one’s first impressions.) In Spirit parlance, our parser will be a grammar composed of rules. A rule will have attributes related to the data type is produces. Optionally, a rule may have an action that executes arbitrary code.

Enough delay. Let’s animate the skeleton of our new grammar. The magic of template meta-programming makes the following struct valid and versatile. Its why’s and wherefore’s may be inscrutable at the moment; however, the gist of the parser should be clear and, if you’ll forgive some exaltation, quite elegant in terms of C++:

template <typename Iterator> struct Grammar : boost::spirit::qi::grammar<Iterator> {
    Grammar() : Grammar::base_type(start) {
        using boost::spirit::qi::byte_;

        empty = byte_(0x64);
        object = empty | pair;
        pair = byte_(0x0f) >> object >> object;
        version = byte_(0xff) >> byte_(0x01);
        start = version // must start with recognized version
            >> +object; // contains one or more objects
    }

    qi::rule<Iterator> empty, object, pair, start, version;
};

We haven’t put all the pieces together for a complete program. We’ll put some more flesh on the grammar before unleashing a compiler on it. One of the cool things about Spirit is that you can compose grammars from other grammars. Here’s how we’ll interpret strings. There’s another rule with yet another grammar we need to write, but the details are skipped. All it does it parse a number (see the JavaScript above) and expose the value as the attribute of the UInteger32 rule.¹ The following example introduces two new concepts, local variables and actions:

template <typename Iterator> struct String : boost::spirit::qi::grammar<Iterator, qi::locals<unsigned> > {
    String() : String::base_type(start) {
        using boost::spirit::qi::byte_;
        using boost::spirit::qi::omit;
        using namespace boost::spirit::qi::labels;

        start = omit[ (byte_(0x05) | byte_(0x1e))
            >> length[ _a = _1 ] ]
            >> repeat(_a)[byte_] ;
    }

    UInteger32<Iterator> length; qi::rule<Iterator, qi::locals<unsigned> > start;
};

The action associated with the length rule is in square brackets. (Not to be confused with the square brackets that are part of the repeat syntax.) Remember that length exposes a numeric attribute, specifically an unsigned integer. The attribute of a rule can be captured with the _1 placeholder. The local variable for this grammar is captured with _a. Local variables can be passed into, manipulated, and accessed by other rules in the grammar. In the previous example, the value of length is set to _a via simple assignment in the action. Next, the repeat parser takes the value of _a to build the “string” stored in the ViewState. The omit parser keeps the extraneous bytes out of the string.

Now we can put the String parser into the original grammar by adding two lines of code (highlighted in bold). That this step is so trivial speaks volumes about the extensibility of Spirit:

    ...
    object = empty | my_string | pair;
    ...
    start = version // must start with recognized version
        >> +object; // contains one or more objects
}

String<Iterator> my_string;
qi::rule<Iterator> empty, ...

The String grammar introduced the repeat parser. We’ll use that parser again in the grammar for interpreting ViewState containers. At this point the growth of the grammar accelerates quickly because we have good building blocks in place:

    ...
    using boost::spirit::qi::byte_;
    using boost::spirit::qi::repeat;

    container = byte_(0x16)
        >> length [ _a = _1 ]
        >> repeat(_a)[object] ;

    empty = byte_(0x64);

    object = empty | my_string | pair;
    ...
}

String<Iterator> my_string;
UInteger32<Iterator> length;
qi::rule<Iterator, qi::locals<unsigned> > container;
qi::rule<Iterator> empty, ...

This has been a whirlwind introduction to Spirit. If you got lost along the way, don’t worry. Try going through the examples in Spirit’s documentation. Then, re-read this article to see if the concepts make more sense. I’ll also make the sample code available to help get you started.

There will be a few surprises as you experiment with building Spirit grammars. For one, you’ll notice that compilation takes an unexpectedly long time for just a dozen or so lines of code. This is due to Spirit’s template-heavy techniques. While the duration contrasts with “normal” compile times for small programs, I find it a welcome trade-off considering the flexibility Spirit provides.

Another surprise will be error messages. Misplace a semi-colon or confuse an attribute and you’ll be greeted with lines and lines of error messages. Usually, the last message provides a hint of the problem. Experience is the best teacher here. I could go on about hints for reading error messages, but that would be an article on its own.

Between compile times and error messages, debugging rules might seem a daunting task. However, the creators of Spirit have your interests in mind. They’ve created two very useful aids to debugging: naming rules and the debug parser. The following example shows how these are applied to the String grammar. Once again, the change is easy to implement:

    ...
    start = omit[ (byte_(0x05) | byte_(0x1e))
        >> length[ _a = _1 ] ]
        >> repeat(_a)[byte_] ;
}

start.name(“String”); // Human readable name for the rule
debug(start); // Produce XML output of the parser’s activity

UInteger32<Iterator> length;
qi::rule<Iterator, qi::locals<unsigned> > start;
};

As a final resource, Boost.Spirit has its own web site with more examples, suggestions, and news on the development of this fantastic library.

It seems unfair to provide all of these code snippets without a complete code listing for reference or download. Plus, I suspect formatting restrictions may make it more difficult to read. Watch for updates to this article that provide both full code samples and more readable layout. Hopefully, there was enough information to get you started on creating your own parsers for ViewState or other objects.

In the next article in this series we’ll shift from parsing ViewState to attacking it and using it to carry our attacks past input validation filters into the belly of the web app.

A Spirited Peek into ViewState, Part I

May 13, 2011

The security pitfalls of the .NET ViewState object have been well-known since its introduction in 2002. The worst mistake is for a developer to treat the object as a black box that will be controlled by the web server and opaque to the end user. Before diving into ViewState security problems we need to explore its internals. This article digs into more technical language¹ than others on this site and focuses on reverse engineering the ViewState. Subsequent articles will cover security. To invoke Bette Davis: “Fasten your seat belts. It’s going to be a bumpy night.”²

The ViewState enables developers to capture transient values of a page, form, or server variables within a hidden form field. The ability to track the “state of the view” (think model-view-controller) within a web page alleviates burdensome server-side state management for situations like re-populating fields during multi-step form submissions, or catching simple form entry errors before the server must get involved in their processing. (MSDN has several articles that explain this in more detail.)

This serialization of a page’s state involves objects like numbers, strings, arrays, and controls. These “objects” are not just conceptual. The serialization process encodes .NET objects (in the programming sense) into a sequence of bytes in order to take it out of the server’s memory, transfer it inside the web page, and reconstitute it when the browser submits the form.

Our venture into the belly of the ViewState starts with a blackbox perspective that doesn’t rely on any prior knowledge of the serialization process or content. The exploration doesn’t have to begin this way. You could write .NET introspection code or dive into ViewState-related areas of the Mono project for hints on unwrapping this object. I merely chose this approach as an intellectual challenge because the technique can be generalized to analyzing any unknown binary content.

The first step is trivial and obvious: decode from Base64. As we’re about to see, the ViewState contains bytes values forbidden from touching the network via an HTTP request. The data must be encoded with Base64 to ensure survival during the round trip from server to browser. If a command-line pydoc base64 or perldoc MIME::Base64 doesn’t help you get started, a simple web search will turn up several ways to decode from Base64. Here’s the beginning of an encoded ViewState:

/wEPDwUJNzIwNzAyODk0D2...

Now we’ll break out the xxd command to examine the decoded ViewState. One of the easiest steps in reverse engineering is to look for strings because our brains evolved to pick out important words like “donut”, “Password”, and “zombies!” quickly. The following line shows the first 16 bytes that xxd produces from the previous example. To the right of the bytes xxd has written matching ASCII characters for printable values – in this case the string 720702894.

0000000: ff01 0f0f 0509 3732 3037 3032 3839 340f ......720702894.

Strings have a little more complexity than this example conveys. In an English-centric world words are nicely grouped into arrays of ASCII characters. This means that a programming language like C treats strings as a sequence of bytes followed by a NULL. In this way a program can figure out that the bytes 0x627261696e7300 represent a six-letter word by starting at the string’s declared beginning and stopping at the first NULL (0x00). I’m going to do some hand-waving about the nuances of characters, code points, character encodings and their affect on “strings” as I’ve just described. For the purpose of investigating ViewState we only need to know that strings are not (or are very rarely) NULL-terminated.

Take another look at the decoded example sequence. I’ve highlighted the bytes that correspond to our target string. As you can see, the byte following 720702894 is 0x0f – not a NULL. Plus, 0x0f appears twice before the string starts, which implies it has some other meaning:

ff01 0f0f 0509 **3732 3037 3032 3839 340f ......720702894.

The lack of a common terminator indicates that the ViewState serializer employs some other hint to distinguish a string from a number or other type of data. The most common device in data structures or protocols like this is a length delimiter. If we examine the byte before our visually detected string, we’ll see a value that coincidentally matches its length. Count the characters in 720702894.

ff01 0f0f 0509 3732 3037 3032 3839 340f ......720702894.

Congratulations to anyone who immediately wondered if ViewState strings are limited to 255 characters (the maximum value of a byte). ViewState numbers are a trickier beast to handle. It’s important to figure these out now because we’ll need to apply them to other containers like arrays.³ Here’s an example of numbers and their corresponding ViewState serialization. We need to examine them on the bit level to deduce the encoding scheme.

Decimal   Hex     Binary  
1         01      00000001  
9         09      00001001  
128       8001    10000000 00000001  
655321    09ffd9  11011001 11111111 0100111

The important hint is the transition from values below 128 to those above. Seven bits of each byte are used for the number. The high bit tells the parser, “Include the next byte as part of this numeric value.”

LSB           MSB  
10000110 00101011

Here’s the same number with the unused “high” bit removed and reordered with the most significant bits first.

MSB   ...   LSB  
0101011 0000110 (5510, 0x1586)

Now that we’ve figured out how to pick out strings and their length it’s time to start looking for ways to identify different objects. Since we have strings on the mind, let’s walk back along the ViewState to the byte before the length field. We see 0x05.

ff01 0f0f 0509 3732 3037 3032 3839 340f ......720702894.

That’s the first clue that 0x05 identifies a string. We confirm this by examining other suspected strings and walking the ViewState until we find a length byte (or bytes) preceded by the expected identifier. There’s a resounding correlation until we find a series of strings back-to-back that lack the 0x05 identifier. Suddenly, we’re faced with an unknown container. Oh dear. Look for the length field for the three strings:

0000000: 1503 0774 6f70 5f6e 6176 3f68 7474 703a ...top_nav?http:
0000010: 2f2f 7777 772e 5f5f 5f5f 5f5f 5f2e 636f //www._______.co
0000020: 6d2f 4162 6f75 7455 732f 436f 6e74 6163 m/AboutUs/Contac
0000030: 7455 732f 7461 6269 642f 3634 392f 4465 tUs/tabid/649/De
0000040: 6661 756c 742e 6173 7078 0a43 6f6e 7461 fault.aspx.Conta
0000050: 6374 2055 73 ct Us

Moving to the first string in this list we see that the preceding byte, 0x03, is a number that luckily matches the amount of strings in our new, unknown object. We peek at the byte before the number and see 0x15. We’ll call this the identifier for a String Array.

At this point the reverse engineering process is easier if we switch from a completely black box approach to one that references MSDN documentation and output from other tools.

Two of the most common objects inside a ViewState are Pairs and Triplets. As the name implies, these containers (also called tuples) have two or three members. There’s a catch here, though: They may have empty members. Recall the analysis of numbers. We wondered how upper boundaries (values greater than 255) might be handled, but we didn’t consider the lower bound. How might empty containers be handled? Do they have a length of zero (0x00)? Without diverging too far off course, I’ll provide the hint that NULL strings are 0x658 and the number zero (0) is 0x66.

The root object of a ViewState is either a Pair or Triplet. Thus, it’s easy to inspect different samples in order to figure out that 0x0f identifies a Pair and 0x10 a Triple. Now we can descend the members to look for other kinds of objects.

A Pair has two members. This also implies that it doesn’t need a size identifier since there’s no point in encoding “2” for a container that is designed to hold two members. (Likewise “3” for Triplets.) Now examine the ViewState using a recursive descent parser. This basically means that we encounter a byte, update the parsing context based on what the byte signifies, then consume the next byte based on the current context. In practice, this means a sequence of bytes like the following example demonstrates nested Pairs:

0000000: ff01 0f0f 0509 3732 3037 3032 3839 340f ......720702894.
0000010: 6416 0666 0f16 021e 0454 6578 7405 793c d..f.....Text.y<

Version
Pair
 - Member 1: Pair
    - Member 1: String
        “720702894” 
    - Member 2: Pair
       - Member 1: ArrayList (0x16) of 6 elements
           Number 0
           Pair
           ...
       - Member 2: Empty
 - Member 2: Empty

Don’t worry if you finish parsing with 16 or 20 leftover bytes. These correspond to the MD5 or SHA1 hash of the contents. In short, this hash prevents tampering of ViewState data. Recall that the ViewState travels back and forth between the client and server. There are many reasons why the server wants to ensure the integrity of the ViewState data. We’ll explore integrity (hashing), confidentiality (encryption), and other security issues in a future article.

I haven’t hit every possible control object that might sneak into a ViewState. You can find a NULL-terminated string. You can find RGBA color definitions. And a lot more.

This was a brief introduction to the ViewState. It’s necessary to understand its basic structure and content before we dive into its security implications. In the next part of this series I’ll expand the analysis to more objects while showing how to use the powerful parsing available from the Boost.Spirit C++ library. We could even dive into JavaScript parsing for those who don’t want to leave the confines of the browser. After that, we’ll look at the security problems due to unexpected ViewState manipulation and the countermeasures for web apps to deploy. In the mean time, I’ll answer questions that pop up in the comments.