My efforts creating multi-touch surfaces

Creative Commons License
Omniverse NUI Blog by Kenth Fagerlund is licensed under a Creative Commons Attribution-No Derivative Works 2.0 UK: England & Wales License.
Showing posts with label directx. Show all posts
Showing posts with label directx. Show all posts

Change is good

| Monday, 10 August 2009

Well as usual I have changed my mind. After a week or so doing some DirectX studying and finally getting to a point where I can make stuff happen on the screen I change my mind. This is so typical and I hate my brain for it but I just can't help it.

I have been researching about different ways of doing the UI for the touch table, reading different papers and so on. And every so often I get some code examples and they are all in OpenGL. That's not enough though to sway me off the wonderful world that is DirectX (Not that wonderful but it's all I know , much like your mom's pancake is the best in the world when you are a kid).
No, the thing that made me drop all that DirectX knowledge that I squeezed into my brain next to all the Mythbusters episodes was the same fact that I fought about when I went with DirectX. It is platform independent. Yes, I still want to be able to run my Omniverse on Linux, OS X and Windows. Not that I plan to support all of the at first but I don't want to port it all. But my second reason was that I actually can use this in my work as a mobile developer since all mobiles with and worth handle OpenGL ES.

So there we have it. Back to the drawing board. Not that big of a setback really since I have learned all the basics of 3D graphics that still apply on OpenGL. So if anyone have any good tutorials on OpenGL 3.0 or later give me a shout (Don't actually shout cause most likely I won't hear it)

Slow progress

| Wednesday, 5 August 2009
I feel that it's been long since last post. My excuse is that I was sick and didn't have the strength to do any Touch-table stuff. But now I'm back. So what am I doing? Well I'm continuing with my DirectX stuff so I can gets some stuff to move on the screen. My new goal is to move some lit 3D cubes with touch by the end of the week.

DirectX is not as hard as I expected but it feels like there is lots of boilerplate that always has to be done to create just the smallest of things. But maybe since I'm just a puppy in the world of 3D programming it feels like I write and write but get nothing done.

As I was sick I spent most my time just in bed dreaming up new and cool way to build this table and still keep it small and energy efficient. So I have some benchmarking to do when I decide on the UI but I hope that the Nvidia ION platform have enough brain capacity to handle the 3D crunching that I need. I have been looking at the Acer ApireRevo that is built on the Nvidia ION and still boost 2GB memory. Really small and little heat generated and the graphics might be good enough since I'm not making a game but a UI. And with Nvidia ION II on the being produced in rumour land It might be good that I take my time to finish this project :)

But really. I will have something to show at the end of the week for sure! :D Youtube here I come :D

UI Inspiration

| Monday, 27 July 2009


As usual when I do different project I find myself planing things that I won't be able to do in a LOOOONG time. UI design for Omniverse is something I'm really looking forward to. I already have lots of ideas, some quite good and some just barking mad, and I'm in the process of writing them down so I don't forget it once I'm good enough at DirectX to be able to prototype some of it. If I get time I will post some of my ideas but right now I'm mostly looking at Anime films and other Sf movies for inspiration. Cause one thing I really don't want to do is to repeat old UI design now that I have the chance to make it EXACTLY how I want it without any constraints. But here is some of the key points the UI should contain

  • It should be aware of the environment and be able to simulate key parts of it

  • I should be able to use it even if I'm dead tired or drunk. Ease of use is paramount!

  • Text should be the LAST form of communication for the UI

  • All objects that are related in some way should help the user to interact with them (ie, mark a photo and other photos and maybe editors should move closer to the users hand to assist in further interaction)

  • Files and folders are FORBIDDEN! More organic organization. (Haven't figured this one out yet)

  • QWERTY Keyboard is banished. And all text input must be able to be done without looking but still without any physical feedback. (Location aware input)


Well that's some of the ideas I have right now. More is coming everyday. My goal it still to keep it simple but it's hard to stop you mind to invent some mad UI that is absolutely useless.

So if anyone has any good UI inspirational films or other media give me a shout. All ideas are good as long as there not stupid :)

Will report on my DirectX progress tomorrow.

Decision time

| Thursday, 23 July 2009


After doing some serious research on what graphical API to use for my Omniverse NUI project I've decided on DirectX. It's not a easy decision since it goes against many of my ideas on what make a good piece of software. The major problem I have with DirectX is that it's Windows only. I just hate that. I want to be able to pick and choose what OS to run it on as I feel fit.

Since I use this project to not only learn some Hardware hacking and how multi-touch works but also learn C++. So DirectX fits in better. It's written in C++ and not in C style as OpenGL. It has great tutorials that just fits better with my brain then the OpenGL ones. Many will prob disagree with this (even I do sometimes) but I want to learn it so I made my mind up. I might switch to a more portable API later in life (read OpenGL) but for now it's all hands on DirectX.

Sadly enough my laptop only support DirectX 9 and badly at that so I can't try out the nice stuff just yet, I hope to be able to use their Direct2D API since I can do some nice vector graphics in code for the UI. But for now I do all in Direct3D and switch over when I get a new machine. I've just started to go though a nice tutorial at directxtutorial.com starting with DX9 and later jumping on DX10. I like the way everything is explained in the tutorial and give me a quick introduction to all the keywords in DX programming so I can dive deeper in subjects as needed. Recommend it to anyone learning DX.

Now for something completely different. My PlayStation Eye has just arrived! Woohoo! Will give it a run tonight and will post up some screen shots of the test. I really hope my laptop will hold it together with the added resolution and framerate of the camera. I hope I can do my first drawing test with the camera this weekend. I post my progress on that.

Until next time
Kenth

Project Omniverse

| Tuesday, 21 July 2009
Off I go,

I have given it a name and made it official. So what is it? Well, It's kind of a desktop replacement for Windows 7.

Here is my description as of now:
A desktop replacement software that is optimized for Natural User Interface (NUI) aka Multi-Touch screens. It should be able to deliver all media available in modern computers in a intuitive way without being fixed with normal Window based systems. As the name suggests it can involve all parts of physics or dimensions as it feels fit just to bring the user a natural way of working with the media.

On this project I will depart from my normal standards where I aim to make it OS independent and hardware independent. But since this is such a big undertaking it will be aimed directly to Windows 7 with Directx 10.1 or higher. I hope I can backtrack on this later but I just don't want to be fighting with API's and spend more time with UI/Hardware design.

The whole project will be open-source and the hardware drawings will be available as I make them.

This will be a learning path for me to get going with C++/DirectX, 3D Math, Electronics and of course LASERS!! :) So you will see lots of beginner mistakes, and wonder why in "insert your religions god name here"'s name he is doing that. Well, don't worry. I enjoy making mistakes cause that's the only way I remember what I've done :)

Bye for now
Kenth Fagerlund