The Blog of Ryan Foss

It's a start!

E3 2012 Predictions

I love, love, love E3 and I can’t help but spout some predictions.

I’m most interested in the three major players and their press conferences.  Nintendo, Sony and Microsoft.  Each planning some big announcements and future plans.  Here’s what I think each major player will have to say.


My biggest interest for E3 2012 is Nintendo’s Wii U console?  Wii sales are essentially dead, 3DS not doing so well…  Can Wii U breath life back or will it show us something we didn’t know we wanted?   Can it show something beyond the Nintendo typical cutsie?   I expect that in typical Nintendo fashion, the Wii U will not focus on tech and be underpowered.  Also, the 3DS needs some life but I’m really meh about it so I don’t care much other that to see what they say about hardware.


I don’t follow Sony very much, but  I think Sony is in a tight spot and needs to react and show what they have to offer.  They have a lot of games to talk about and I expect a lot of Vita Vita Vita talk.   Sony could announce a price drop on the console, in reaction to the $99 Xbox console deal (rip off tactic).  I’ve heard some news about a cloud service, which could be streaming games like OnLive, but I doubt it.

I’m curious what Sony will say about the Move.  Remember the Move?  Will they give it more promotion or is it a dead peripherial?

Although unlikely, its possible Sony might hint at the PS4 in vague ways to let us know they are thinking of the future. I’ve been really adamant that Nintendo’s Wii U would steal the show and the other players would want to evaluate what Nintendo’s up to and how they fair, but Sony could see it as an opportunity.


There will be no new console announcement from Microsoft in my opinion.  They won’t want to take attention from the Kinect, and I suspect they will be very interested in how the Wii U goes.  Expect more dashboard/software announcements, Live crap, Windowz Phone blah, Windows App Store, meh.  I expect Microsoft to suck it this year as they try to further embrace casual.  I think they are going to approach this from a different angle, laying the groundwork for a Windows Phone, Windows 8, and Xbox 720 future.

I am very interested to see if their presentation makes them appear as if they are getting out of the games business.  On the other hand, Microsoft could have some awesome sh!t happening if they can really integrate the next console with Windows 8 and Windows Phone in a non-sucky/typical Microsoft way.  And maybe Zune has a future!

All in all, what they present should be telling for what could be their path for the next console.  Oh, and probably something about Halo.


I think Nintendo and the Wii U will steal the show and give us lots to talk about.  I think Sony will be on the offensive and will really pump some fun at us, showcasing the fun gaming “experience”, as well as promote the Vita in a way to make us want to get it.  I think Microsoft is going to be a flop, focusing on other casual crap we won’t really care about while missing the gamer audience.

Position from Depth

I’ve been doing some stuff at work using our visualization, shaders and some python scripting.  I normally don’t post stuff about work for many reasons, but this project has been a lot of fun and is worth blogging about (and I have permission).  I also want to document what I did as well as address some of the issues I encountered.

Essentially, long story short, we’re doing some human safety systems work where we need to detect where a human is in an environment.  I’m not directly involved with that part of the effort, but the team that is is using some depth cameras (like Kinect in a way) to evaluate the safety systems.  Our role, and mine specifically, is to provide visualization elements to meld reality and simulation and our first step is to generate some depth data for analysis.

We started by taking our Ogre3D visualization and a co-worker got a depth of field sample shader working.  This first image shows a typical view in our Ogre visualization.  The scene has some basic elements in world space (the floor, frame and man) and others in local space (the floating boxes) we can test against.

A sample scene, showing the typical camera view.  The upper-right cut-in is a depth preview.

The next image shows the modifications I made to the depth shader.  Instead of using a typical black and white depth image, I decided to use two channels, the red and green channels.  The blue channel is reserved for any geometry beyond the sensor vision. Black is depth less, essentially no geometry exists there.

Two color channel depth output image.

I decided to use two color channels for depth, to improve the accuracy.  That’s why you see color banding, because I hop around both channels.  If I only used one channel, at 8 bit, that would be 256 colors.  A depth of 10 meters would mean that the accuracy would only be about 4 cm (10.0 m / 256). By using two color channels I’m effectively using 16 bit, for a total of 65536 colors (256 * 256), which increased our accuracy to 1.5 mm (10.0 m / 65536).  In retrospect, perhaps I could have used a 16 bit image format instead.

To do this sort of math its surprisingly easy.  Essentially you find the depth value right from the shader and make it a range of 0 to 1, with 1 being the max depth.  Since we are using two channels, we want the range to be between 0 and 65536, so just take the depth and multiply by 65536.  Determining the 256 values for each channel is pretty easy too using the modulus.  (A quick explanation of a modulus is like 1pm = 13.  It’s when numbers wrap around.  So the modulus of 13 by 12 is 1 for example, as is 25 = 1.  You could also consider it the remainder after division.)  So the red channel is determined by the modulus of depth by 256.  The green channel is done similarly, but in this case is determined by the modulus of depth/256 by 256.

red channel = modulus(depth, 256)
green channel = modulus(depth/256, 256)

Here’s an example.  Lets say the depth is 0.9.  That would result in a color value of 58982.4 (0.9 * 65536).  The red channel color would be the modulus of 58982.4 by 256, which equals 102.  The green channel would be the modulus of 58982.4/256 by 256, which is 230.

With that done, I save out the image representing the depth with two channels as I illustrate above.

Next I calculate the position from the image and depth information.  This particular aspect caused me lots of headaches because I was over-complicating my calculations with unnecessary trigonometry.  It also requires that you know a some basic information about the image.  First off, it has to be symmetric view frustum.  Next, you need to know your field of views, both horizontal and vertical, or at least one and the aspect.  From there its pretty easy, so long as you realize the depth is flat (not curved like a real camera).  Many of the samples out there that cover this sort of thing assume the far clip is the cut off, but in my case I designed the depth to be a function of a specified depth.

I know the depth by taking the color of a pixel in the image and reversing the process I outlined above.  To find the x and y positions in the scene I take a pixel’s image position as a percentage (like a UV coordinate for instance), then determine that position based off the center of the image.  This is really quite easy, though it may sound confusing.  For example, take pixel 700, 120 in a 1000 x 1000 pixel image.  The position is 0.70, 0.12.  The position based on center is 0.40, -0.76.  That means that the pixel is 40% right of center, and down 76% of center.  The easiest way to calculate it is to double the value then minus 1.

pixelx = pixelx * 2 – 1
pixely = pixely * 2 – 1

To find the x and y positions, in local coordinates to the view, its some easy math.

x = tan(horizontal FOV / 2) * pixelx * depth
y = tan(vertical FOV / 2) * pixely * depth
z = depth

This assumes that positive X values are on the right, and positive Y values are down (or up, depending on which corner 0,0 is in your image).  Positive Z values are projected out from the view.

To confirm that all my math was correct I took a sample depth image (the one above) and calculated the xyz for each pixel then projected those positions back into the environment.  The following images are the result.

Resulting depth data to position, from the capture angle.
The position data from depth from a different angle.
The position data from depth from a different angle.

The results speak for themselves.  It was a learning process, but I loved it.  You may notice the frame rate drop in the later images.  That’s because I represent the pixel data with a lot of planes, over 900,000.  It isn’t an efficient way to display the data, but all I wanted was confirmation that the real scene and the calculated positions correspond.

Quickcam Pro Webcam Installed without Crappy Logitech Software

My computer at work went bust (thanks to an IT push) and I needed to have an admin reinstall the Logitech Quickcam Pro drivers. Unfortunately, the little driver for the camera comes with a 85 Mb download including some intrusive software that is always running. Why do these little peripherals have this crap?

Anyway, I was determined not to install this crap again so I scoured for other drivers without much luck. So I tried opening the 85 Mb EXE (downloaded from Logitech) with WinZip and what do you know, its an archive. So I extracted it, plugged in my camera and navigated the hardware wizard to the extracted drivers directory and now my Quickcam Pro 4000 works without their “software”.

A Nearest Color Algorithm

I’ve been developing this software at work to take an image from a video stream (a webcam in this case) and detect if a green laser dot is present. After spending most of my day getting a project to compile and a program to recognize my webcam as a stream, and being distracted by numerous other projects and co-workers, I just wasn’t at the top of my algorithm development game to detect the green dot. Combine that with office lighting and you know my dilemma (fluorescent lighting is often green tinted and throws off a white balance.)

I start by stepping through each pixel in the source and check its color. If you take the pixel’s colors from each channel and find the average you’ve essentially got a gray scale value. This doesn’t work so well since some shiny metal might throw off the detection (like a ring). If I wanted to find the brightest spot, this is a good approach, but I needed to emphasize the green. At first I added a weight to the green channel by simply doubling it. So now the average is calculated as (R+G+G+B)/4, counting the green channel twice. This was better, but also prone to problems.

At the end of the day I had a basic project setup and working with some hit or miss detection. I had invested over an hour into multiple approaches and although my algorithms for color matching were working somewhat, and I knew there had to be a better approach.

What dawned on me later was something I should have noted much earlier. I know that the RGB color concept is essentially a three dimensional array, but since it is color I’m used to thinking of it two dimensionally like you see in an image editing program. Once I visualized it as a 3D space, a big cube made of blocks so to speak, I knew my answer.

You know the shortest route from A to B is a straight line right. This is easy to calculate in 2D using the Pythagorean Theorem as you probably recognize:

a² + b² = c²

This also applies in 3D space, which is exactly what RGB color is. An RGB value is a point in a 3D space. The Theorem applies like this:

a² + b² + c² = d²

The obvious solution was in front of me but I didn’t see it. Euclidean distance would give me exactly what I needed. I needed to treat the color difference as a distance.

r² + g² + b² = d²

For instance, say I wanted to find the closest pixel in the supplied source to a target color RGB 128, 255, 128. I check pixels against my target color by finding their channel distance. So imagine I have a black pixel of 0, 0, 0. My distance is calculated as:

r = pixel_color – target_color = 0 – 128 = -128
g = pixel_color – target_color = 0 – 255 = -255
b = pixel_color – target_color = 0 – 128 = -128

d = sqrt(r² + g² + b²) = 312.7

What if I have a pixel with color 0, 0 255, or a pixel with color 0, 255, 0. Which one is closer to my target? If I calculate the average, they are the same. But by distance, the green pixel is closer to my desired.

How Far We’ve Come

It’s been 40 years!
via Bad Astronomy

I updated my iPod Touch for Free

F! Apple. Why would they charge iPod Touch owners $10 to upgrade to the new 3.0 firmware but its free for iPhones. It’s another example of their smarmyness. You can buy a brand new iPod Touch, get home and find out there’s another $10 fee to get it to be current.

Regardless, I’ve been adamant about waiting until the price dropped to free before upgrading but circumstances required I upgrade. I started working with a developer who’s targeting the new 3.0 and his app wouldn’t install. I went to the internets for options.

I found many sites listing how to upgrade for free which sounds great but the comments are sprinkled with many fails. After a few fails of my own, and a restore or two later, I managed to get the 3.0 Firmware onto my iPod Touch 2G without shelling out $ to the man. Here’s how:

1. Be sure you have iTune 8.2 or newer. This is one of the big problems I suspect since the 3.0 Firmware will not work in older iTunes and the error it gives is confusing.
2. Go here and download 3.0 firmware. If you have a 2G Touch, get the 2G ispw file.
3. Start up iTunes and plug in your iTouch.
4. Select it in the iTunes devices list.
5. Click on “check for update”. It will go to the sales pitch but just click cancel on the borrom left. It will return to the devices page and “check for update” should change to just “update”.
6. Hold shift and click “update”
7. Select the ispw file you downloaded.
8. let it run its course like normal.

This should update the firmware and you wont lose any files.

I’ve only done this with my iPod Touch 2G. If you’re wondering, the 2G Touch has physical volume control on the side and speakers, whereas the 1G has no physical volume control and requires headphones.


ARhrrrr is an Augmented Reality thingy for mobile devices. I can’t wait for this sort of stuff to become more common place.

Bad Advertising

As seen in my Google Reader Slashdot feed, some poorly placed advertising.

Link to Slashdot article.

Can you see me now?

My new phone is truely amazing, especially compared to the pieces of crap phones I’ve had so far. I installed this app called GPS Tracker ( the other day and I turned it on this morning so my wife doesn’t have to call and ask where I am.

Of course, it’s never taken this long to drive to work, almost 2hrs! Proof is in the map!

Interestingly enough, its pretty accurate. If I zoom in to the finish point it actually shows the path I walk from my car to my desk.

Hologram BS

Have you seen this? It’s CNN’s “hologram” technology they used for their election coverage.

I caught a glimpse of it on election day, but CNN isn’t HD, so I watched somewhere else. Their little gimmick didn’t keep me (though the HD gimmick did, hrumm).

This “technology” is stupid and wasteful for this purpose. You’re telling me that the 35 HD cameras recording her for this “hologram” can only make her look low quality, poorly lit with a halo. BS! Come on, are we stupid? It was made to look low quality for effect. What really bugs me is the name, hologram. I’m supposed to believe that this “hologram” is on the stage with whats-his-face, as that is what a hologram is. But no, this is just some fancy overlay BS composite. She isn’t there in reality or in some projected image. He is looking at nothing, looking at where she will be inserted/composited, and only the viewing audience can see her in the scene.

Unimpressed because they called it a hologram. Otherwise, its kinda neat though IMO they dialed down the quality to make it look more Star Wars-ish. I think if they really did what they say with all the quality possible it would have looked as if she was really there, perhaps with a little oddness due to lighting differences and such, but still almost seamless.

Android Me ASAP

I’ve been waiting and waiting for an Android phone to come out, and now I’m waiting to be able to afford one. Although from the sidelines, I had big hopes that Google’s OS for phones would change the market. An open source mobile device OS is something I think that could really make a lot of things possible, and stymie some of the greed and corruption and monopolization of this mobile device market.

Now I hear that Android has a GameBoy emulator, something that you don’t have to jailbreak your phone to get like on the iPhone. Regardless of any legal issues, I think its a shining example of open source OS and Google’s open market concepts for Android development.

Although I think the devices Apple makes are testaments of style and function, I dislike their pricing practices. I think they will soon feel a backlash as other devices start to show up that allow much cooler things to happen without someones pocketbook in mind. (Microsoft, I point at you too FYI!)

All hail Google! Kudos.