New Application to Omniverse NUI list

I can't help myself to browse around the Internet and looking for new cool things coming out. This is something I found as they announced it at Google IO but never found a use for using their API's.

Well now I have. I'm going to add a Google Wave client to Omniverse NUI. If you haven't heard about Google Wave you should have a look at the demo at the end of the post. But why? Well first of all I find the way Google Wave (GW from now on) does communication is unique, innovative and inspiring. It matches exactly what I would like to see happening on Omniverse. Fast instant communication independent on location.

Exactly how I will create this client is as everything else about this project still being planed. And also only on 3:ed place in the priority list. But It might be bumped up depending on how big it grows. The fun part of it is that as soon as I looked at the demo I could come up with at least 5 different UI interfaces that would help communication with Wave and it might become a fundamental part of Omniverse since it is a open protocol. As soon as I get my developer account I will make a deeper dive into the API and decide on what use I have for it and it's priority.

Here is the video I promised, Enjoy. (Ps you can skip the first 5 min cause its most crap talk)

I hate keyboards

I really do hate keyboards. I'm a programmer as a profession so I use the things every day and it's quite annoying to hate your primary work tool. The problem I have is that the qwerty layout hasn't changed since 1874 (about the same time as the patent was filed for the lightbulb!) and I just can't understand why. The reason for that layout was to stop the type bars from jamming. Well it's 2009 now and the type bar problem gone just like my hair.

So why haven't anyone reinvented it? Is it so hard to drop the keyboard in the lake and come up with something better. Here is were I hope touch computing will come and save the world. Touch tables have no place for keyboards. It just don't make sense there. There is no tactile feedback so anyone using finger placement and can look away while typing on a normal keyboard will have no chance to do that. So here we have a perfect time to do something new.

So instead of nagging I will give it a shot myself with Omniverse NUI. I'm not going to sit here and say that I have the solution to all your keyboard problems and with that cure cancer but I think that without to much effort we can improve the keyboard a great deal.

So what is my plan I hear you type. Well I'm still thinking about this and it needs lots of testing but one thing that I'm very sure of and that is that the keyboard should follow you hands and not the other way around. So to make your input the keyboard should form up around your hand and always adapt it's position and composition depending on where and what your typing.

Another key part is form and colour. I have read lots and lots of time that this is the easiest things for a person to detect and remember. So if form and colour is the key components to write instead of letter then I think speed will improve a great deal.

When I finish my concept drawings I will put them up for you to hate or admire. Hopefully not to long now. Still have a box to draw in DirectX before that :D

A map of my mind

I like thinking. I don't do it often or might not be very good at it but some time I get a boost of energy and get lots of ideas. This time I managed to get it onto a mindmap so I won't forget all my ideas and it gives me a good overview of all that needs to be done with this build.

As you can see there is lots of options when building touch-tables. The more I read about other peoples work the more ideas I get and it just spirals into a massive project.

I find that the hardware I'm going to build is less inspiring then the UI that I want to build. The mindmap won't show it but I have atleast 200% more ideas when it comes to the UI then with the hardware design. But I hope that I can draw my prototype table this week so at least I have some surface to work on.

So what is my next goal then. Well apart from all the theory I'm learning about DirectX and how to build the table I have a more concrete goal. At the end of the week I will have a lit 3D cube that I can move with my finger. I will prove it with a video and put some pressure on me to get on with stuff.

I just need to convert our glass table in the living room into a temp touch table and I expect my girlfriend to think that is a GREAT idea :D

Enough rambling. Back to work.

PS, Does anyone know how to make wordpress to make the image link to the full size image. The image is WAY to small to be able to read anything Found it :)

UI Inspiration

As usual when I do different project I find myself planing things that I won't be able to do in a LOOOONG time. UI design for Omniverse is something I'm really looking forward to. I already have lots of ideas, some quite good and some just barking mad, and I'm in the process of writing them down so I don't forget it once I'm good enough at DirectX to be able to prototype some of it. If I get time I will post some of my ideas but right now I'm mostly looking at Anime films and other Sf movies for inspiration. Cause one thing I really don't want to do is to repeat old UI design now that I have the chance to make it EXACTLY how I want it without any constraints. But here is some of the key points the UI should contain

• It should be aware of the environment and be able to simulate key parts of it


• I should be able to use it even if I'm dead tired or drunk. Ease of use is paramount!


• Text should be the LAST form of communication for the UI


• All objects that are related in some way should help the user to interact with them (ie, mark a photo and other photos and maybe editors should move closer to the users hand to assist in further interaction)


• Files and folders are FORBIDDEN! More organic organization. (Haven't figured this one out yet)


• QWERTY Keyboard is banished. And all text input must be able to be done without looking but still without any physical feedback. (Location aware input)

Well that's some of the ideas I have right now. More is coming everyday. My goal it still to keep it simple but it's hard to stop you mind to invent some mad UI that is absolutely useless.

So if anyone has any good UI inspirational films or other media give me a shout. All ideas are good as long as there not stupid :)

Will report on my DirectX progress tomorrow.

The battery that powers evil

After 4 hours of hard goggling, poking, mumbling,coffee drinking and more googleing I figured out what the problem was with the PS3 Eye camera. It was nothing wrong with the driver or with it being Windows 7. The issue was my laptop battery!

My issue was that as I started the PS3 Eye it ran for about 10-15 sec and then just went black as it was disconnected. Well I first suspected that it was cause I use Windows 7 and the driver suffer from Windows-envy. Dual booted my machine and exact same problem again. Looked around at Alex P's blog and found him using a tool called DPC Latency Checker. Ran it and it game me this pretty picture.



Something very wrong is going on here every 15 sec. It spikes and just as it does the camera fails. So I call in my research team (aka www.google.com) and found a forum where a guy/girl said that he/she removed the battery to get rid of the spikes that was caused of the power management system [can't remember where I found it :( so I can't give the credit]. Well. thought it was a long shot but I pulled the battery and tadaaaaaa. Problem solved. Spikes were gone and after testing all camera apps and they worked I found inner joy and floated away to nirvana, well had a coffee.

I will see if I can find a solution that don't involve pulling the battery but at least there is a workaround for now.

That's it for me tonight. Way past my bedtime.

Enjoy life!

First test with PS3 Eye...Fail

So I got my PS3 Eye yesterday. Happy as a spring chicken (I assume they are happy) I jammed it into my laptop and expected the miracles to begin. Well first off I installed a old version of Alex P's PS3 eye driver [v2.0b81109] on my Windows 7 machine. Result? Fail. It ran perfect in all resolutions and framerates but after a few seconds of running it just died.

I poked around some more and found a updated driver [v2.1.0.0130] hidden in Alex P's blog. Loaded it up.. Fail.. Same again. To be fair to Alex it hasn't been tested for Windows 7 so I will install Vista tonight and give it another run. Bit sad start but I know it will work. Just need to give it a helping hand (aka threaten to make it a surveillance camera for my girlfriends chilliplant!)

*Don't worry dear girlfriend. I haven't wasted the money. It will work  :D

Decision time

After doing some serious research on what graphical API to use for my Omniverse NUI project I've decided on DirectX. It's not a easy decision since it goes against many of my ideas on what make a good piece of software. The major problem I have with DirectX is that it's Windows only. I just hate that. I want to be able to pick and choose what OS to run it on as I feel fit.

Since I use this project to not only learn some Hardware hacking and how multi-touch works but also learn C++. So DirectX fits in better. It's written in C++ and not in C style as OpenGL. It has great tutorials that just fits better with my brain then the OpenGL ones. Many will prob disagree with this (even I do sometimes) but I want to learn it so I made my mind up. I might switch to a more portable API later in life (read OpenGL) but for now it's all hands on DirectX.

Sadly enough my laptop only support DirectX 9 and badly at that so I can't try out the nice stuff just yet, I hope to be able to use their Direct2D API since I can do some nice vector graphics in code for the UI. But for now I do all in Direct3D and switch over when I get a new machine. I've just started to go though a nice tutorial at directxtutorial.com starting with DX9 and later jumping on DX10. I like the way everything is explained in the tutorial and give me a quick introduction to all the keywords in DX programming so I can dive deeper in subjects as needed. Recommend it to anyone learning DX.

Now for something completely different. My PlayStation Eye has just arrived! Woohoo! Will give it a run tonight and will post up some screen shots of the test. I really hope my laptop will hold it together with the added resolution and framerate of the camera. I hope I can do my first drawing test with the camera this weekend. I post my progress on that.

Until next time
Kenth

The bugs are here!

First two "Bugs" has been reported by myself now. To give myself a nice easy target I added two feature request on SourceForge. They are as follows:

• Fullscreen desktop twice the size of the screen resolution.


• Image viewer

The desktop is pretty obvious. I need a flat virtual surface to place what ever UI I create on. The size is just to give me a chance to test moving the desktop around so I'm not hindered by the screen size or resolution.

The Image viewer is just a simple view to show images. It should be able to move freely around the desktop and be able to re-size. No physics should be done. Maybe a very simple menu system if I have time. I'm still trying out some menu Ideas and will post them as soon as they are a bit more solid.

Omniverse NUI @ SourceForge

Now Omniverse is registered with SourceForge. Nothing there to get and most likely a long wait until anything happens there. But I will post as soon as something useful happens there.

You find the project with the keyword Omniverse NUI.

Omniverse hardware prototype

My first hardware prototype is done. After lots of hard work and many many minutes (three to be precise) the first multi-touch prototype is done. It might not land me a job at Apple Hardware design department but I've gone for function over design aproach :) Here what you need to reproduce it.

• Cardbord box belonging to a bedlamp

• Old webcam you have lying around (I stole my girlfriends)


• Tracing paper (Atleast that what I think it's called. The paper you use to bake you know?)


• Tape


• Wettowl from local pizza place (optional)

Well I won't insult you by telling you how to put it all together but remember that this will only give you dark blobs so your software need to be able to trace that (CCV does that nicely)

My first test gave me very good results as I mentioned in the last post. When I get my Playstation Eye camera I will try to make a more permanent version for my glass table so I can get some better settings and calibration.

And if you wonder about the yellow wettowl. It's a pillow for the camera cause it cant bend it's neck as I wish. And I like to be nice to my gadgets ofc.

Until next time

Ordering in some hardware

After I've been doing some early testing with a crap camera and a cardboard box to get my first blob detection done. It work very good with CCV running at about 4ms detection time and 30 FPS.

But after some more research I found that the Playstation EYE camera is a very good cheap alternative for Multi-touch tables. The Playstation EYE gives me 60fps @ 640 x 480 and 120fps @ 320x240. So it's been ordered now and I hope to give it a test run this weekend.

I just hope that my poor laptop will be able to cope with such high resolution of uncompressed images. I can hear it shaking already. hehe

Project Omniverse

Off I go,

I have given it a name and made it official. So what is it? Well, It's kind of a desktop replacement for Windows 7.

Here is my description as of now:
A desktop replacement software that is optimized for Natural User Interface (NUI) aka Multi-Touch screens. It should be able to deliver all media available in modern computers in a intuitive way without being fixed with normal Window based systems. As the name suggests it can involve all parts of physics or dimensions as it feels fit just to bring the user a natural way of working with the media.

On this project I will depart from my normal standards where I aim to make it OS independent and hardware independent. But since this is such a big undertaking it will be aimed directly to Windows 7 with Directx 10.1 or higher. I hope I can backtrack on this later but I just don't want to be fighting with API's and spend more time with UI/Hardware design.

The whole project will be open-source and the hardware drawings will be available as I make them.

This will be a learning path for me to get going with C++/DirectX, 3D Math, Electronics and of course LASERS!! :) So you will see lots of beginner mistakes, and wonder why in "insert your religions god name here"'s name he is doing that. Well, don't worry. I enjoy making mistakes cause that's the only way I remember what I've done :)

Bye for now
Kenth Fagerlund

Introduction

Hi,

This blog is my attempt to document all my findings as investigate and try on different projects. Most projects end up getting forgotten and discarded so I hope that I can record most of the research I do here so If I find the time to pick it up again all the work is still available.

A added bonus would be if anyone else found use of it so I will do my best to keep it informative but I wouldn't bet on it :)