As of March 31st, 2012 Bungie will officially discontinue their API's for the Halo Stats, what this means for Halo Stats Archivier is that it will no longer be able to download any games played after this date (see here for more information). In the future, likely begining with Halo 4 I will add support for downloading games, but at that time I'll also be disabling downloading of Halo 2, 3, ODST, and Reach data. Reach may see a reprival in data should 343 provide the api for this. So stay tuned.

Halo Stats Archiver: Desktop Edition 2.0 (HSA: DE) was built from the ground up scrapping the original code base that HSA: DE 1.0 had matured into. With HSA: DE 1.0 there was a serious flaw, it was hard to get working and once it was working it didn't necessarily stay that way. So with the development of HSA: DE 2.0 I decided to scrap the PHP-Gtk2 framework that HSA: DE 1.0 was built on and switch to my new love, Adobe AIR/Flex 3. Doing this not only made it easier for me to develop the application and have it work on Windows, Mac OSX, and Linux, it also makes it 10x easier to install thanks to the AIR platform. AIR also has another added benefit, design creativity, with AIR (like Flex for the web) not only can you change the look and feel of your controls you can also change the window decorations or the window chrome. With 1.0 there was also an annoying tendency for the application to spontaneously close, I found the cause of this after a bit of trial and error, it seemed that PHP after awhile would simply shutdown. This of course I know coming from a primarily web based background, PHP has a script execution time limit, typically in the default PHP install this is 30s, of course I had upped this but applications do not just randomly close another downfall to developing desktop applications with PHP but with AIR there is no such limit, applications run just like all other application.

HSA: DE 2.0 uses a port of my PHP classes which I used to download and parse the stats pages on Bungie.net and load the information into the database. This caused multiple headaches and long nights but in the end it works just as flawless as version 1.0. Of course HTML scraping has its limitations, all bungie has to do is change a style or remove a tag that I'm using in my scrapping and its all over, everything is bust. Thus is the joys of scrapping a website, however in the past I've generally been able to turn around a fix in a few short days. With Halo 3: ODST this is much different, if you hit a stats page (example) it uses Microsoft's Silverlight (insert cringe here) which put a pritty big wrench in things, you can't scrape Silverlight. Ya sure there are a few things on the stats page that arn't but 90% of it is. But with some investigating I discovered a link to a web service (http://www.bungie.net/api/odst/ODSTService.svc) in the Object tag for displaying the Silverlight stats viewer. This of course references a WSDL which descibes the service which also caused another bit of grief its located at http://bnetweb03.bungie.net/api/odst/ODSTService.svc?wsdl but the domain bnetweb03.bungie.net is not a public domain. However being the web developer that I am, I figured likely its on the same server or cluster as www.bungie.net which resolves to 65.59.234.97. So after adding a few entries to my development machine's hosts file for bnetweb01 to bnetweb09 I managed to get connected to the service using Flex Builder 3's "Import web services" tool which auto generates the supporting classes in ActionScript as well as a copy of the complete WSDL. Success, one step closer, but again without host entries it would only fail with a rather cryptic error message which basically equated to the WebService class receiving an invalid WSDL which is a funny way of saying it couldn't connect to the service. After a bit more digging I found in the base class for connecting to the service it was using bnetweb03.bungie.net, changing it there and in the local copy of the WSDL to www.bungie.net fixed the issue and now things are working properly.

After getting some data back from the service I halted the adl (Air Debug Launcher) so I could look into the result. It was terrifying the amount of information on a game as well as the way it was layed out clearly wasn't supposed to be viewed or used by the public. Which was confirmed in a conversation with Achronos the Bungie.net Overlord (aka the guy who's incharge of the site) the excerpt from the conversation that confirmed my suspicion is bellow.

"As for the WSDL, well, that's a bit of a fudge. That server exists, but you can't see it. The API is not meant for public consumption - at least not at this time (and even if you access it, you're not going to like what you find there). At any rate, you probably can still access it via the domain you can see, it will just require a bit of hacking. But I can't really help you with it, sorry."

But I'm percistant, and like Achronos said with a bit of hacking it was possible to connect to the service, and with quite a bit of head scratching in a few hours I had data coming back into memory and it was unusable. Thanks to runningturtle whome I contacted for more information about how the game waves works I have now. He told me that the SKL property in the game waves is in fact the skulls, in the form of a binary flag. Which means that the uint 4294956828 equates to the skulls Mythic,Tilt,Famine,Catch,ToughLuck,BlackEye,IWHBYD,GruntBirthdayParty, and Cowbell. runningturtle also gave me the break down of which skulls are which number in the binary flag. He didn't mention the skull Blind but i deduced its 2.

Knowing this it took me several hours to actually get the formula for properly generating the list of active skulls in a game, but after a while I managed to come up with the code necessary. It took allot of googling to try and find out how to use binary flags in ActionScript 3, I manged to find an article by Lee Brimelow a Platform Evangelist at Adobe that pointed me in the right direction after that it was pretty simple. BUT for some reason some of the SKL's would be negative, which didn't make sense, so I looked into the GW class that Flex Builder 3 generated via it's Import Web Service tool. It had a datatype of Number for SKL, which I realized was wrong after changing it to uint everything was great and it ended up working as I expected giving me all the skulls active in every game. The resulting code is bellow, note I pass in the initial skulls set at the start of the game, this is more for Campaign games as they do not have waves. But in Firefight games the skulls can all be retrieved from the SKL property in the Game Waves.

/**
 * Gets the skulls turned on over the entire game
 * @param gameData Game Data From the ODST Service
 * @param skulls Initial skulls turned on when the game started
 */
private function getGameSkulls(gameData:Game,skulls:Array):void {
    if(gameData.GameWaves.length==0) {
        //No waves so return
        return;
    }
    
    var skl:uint=gameData.GameWaves[gameData.GameWaves.length-1].SKL;
    //Check that the game didn't end on a bonus round
    if(skl==28) {
        skl=gameData.GameWaves[gameData.GameWaves.length-2].SKL;
    }
    
    //----- FIND SKULLS -----//
    
    //Is Mythic on?
    if(skl&32768 && skulls.indexOf('Mythic')==-1) {
        skulls.push('Mythic');
    }
    
    //Is Tilt on?
    if(skl&16384 && skulls.indexOf('Tilt')==-1) {
        skulls.push('Tilt');
    }
    
    //Is ThunderStorm on?
    if(skl&8192 && skulls.indexOf('ThunderStorm')==-1) {
        skulls.push('ThunderStorm');
    }
    
    //Is Famine on?
    if(skl&4096 && skulls.indexOf('Famine')==-1) {
        skulls.push('Famine');
    }
    
    //Is Fog on?
    if(skl&2048 && skulls.indexOf('Fog')==-1) {
        skulls.push('Fog');
    }
    
    //Is Catch on?
    if(skl&1024 && skulls.indexOf('Catch')==-1) {
        skulls.push('Catch');
    }
    
    //Is ToughLuck on?
    if(skl&512 && skulls.indexOf('ToughLuck')==-1) {
        skulls.push('ToughLuck');
    }
    
    //Is BlackEye on?
    if(skl&256 && skulls.indexOf('BlackEye')==-1) {
        skulls.push('BlackEye');
    }
    
    //Is Iron on?
    if(skl&128 && skulls.indexOf('Iron')==-1) {
        skulls.push('Iron');
    }
    
    //Is IWHBYD on?
    if(skl&16 && skulls.indexOf('IWHBYD')==-1) {
        skulls.push('IWHBYD');
    }
    
    //Is GruntBirthdayParty on?
    if(skl&8 && skulls.indexOf('GruntBirthdayParty')==-1) {
        skulls.push('GruntBirthdayParty');
    }
    
    //Is Cowbell on?
    if(skl&4 && skulls.indexOf('Cowbell')==-1) {
        skulls.push('Cowbell');
    }
    
    //Is Blind on?
    if(skl&2 && skulls.indexOf('Blind')==-1) {
        skulls.push('Blind');
    }
}

With the pending release of Halo: Reach in the fall this year, I am planning on adding support for Halo: Reach games. As for the beta, maybe, but at the time of writing it only has a week left in its life and who knows I might get something done before then. One thing about the stats right now for Reach is they are just html, which does make it more familiar to parse. However not nearly as efficient as getting the stats for Halo 3: ODST, which I only have to scrape the game lists then I use the service to grab the data about the game. One of the other big issues with the Reach stats is that they use AJAX for the game lists. Which is extremely hard to navigate using ActionScript, I've been able to sorta easily replicate the act of clicking on an AJAX button using the HTML component available in Adobe AIR but this obviously isn't very efficient but it seems to work well.

As everyone associated with the Bungie community knows, Halo 2 is officially dead (on the Xbox at least) as Microsoft has switched Xbox Live off for all Original Xbox titles. Right now the Stats are still available for all the Halo 2 games, however they are on their last legs at any moment Bungie could take them down as well. Like the Halo 3 Beta stats at that time they will be lost forever, but for now at least the plan is to release HSA: DE 2.0 with full Halo 2 support. If/when Bungie chooses to remove the Halo 2 stats I will not remove support from HSA: DE, I will however remove support for downloading the game data as this will no longer work anyways. But if you managed to snag your game information before hand... you won't loose it just remember to take advantage of the backup feature in HSA: DE 2.0.

Screenshots
Main Window	Service Record	HSA Running on Mac OS 10.6
HSA Running on Ubuntu 10.04 (x64)	Splash Screen	Stats Graphs
Halo 3 Career Stats (Kills)	Total Stats	Halo 3 Career Stats (Deaths)
Halo 3: ODST Career Stats	Halo 3 Fileshare	Halo 3 Heatmaps
Halo 3 Rank History	Halo 3 Screenshots Viewer	Halo: Reach Career Stats
Halo: Reach Commendations	Game Viewer (Team Game)	Game Viewer - Medals (Team Game)
Game Image Viewer

Tweet