(L.E.M.U.R.) Laboratory of E-Learning & Multimedia

Tranformations

The purpose of using 3D avatars, or animation itself is the ability to do things that you can’t do in real life. In our simulation we will need to be able to change the avatar’s appearance in many ways. Among some of these transformations we are looking into ways of aging the avatar, and controlling their body weight to make them more obese or healthy depending on the situation they are presented with in the simulation.

Below is a an animated picture of one method of achieving this effect. Many graphics applications have morph features that allow an object to change, from one shape to another, as long as it has the same geometry (the points and faces are the same).

Although a few game engines support morphing, this is currently not possible in Unity.

Although the method described above is ideal, as it allows the artist to shape or sculpt the model as needed, such an effect has been achieved in the past (in games and realtime applications) through the use of bone animation.

Below is another picture that shows how this same effect can be achieved with this method. The bones influencing the shape are highlighted in red.

We can also use bump and normal maps to help obtain this aging effect. These could be used to create wrinkles on the character without having build them into the actual model.

A bump map is basically a black and white image that tells the computer to interpret the darker areas as depressions on a surface, and the lighter parts as elevations.

A normal map is similar to the bump map, but it uses 3 colors to give a more 3 dimensional feel of what the object looks like, and how light reacts to its surface.

The following images are some examples of each. The first image shows a bump map implemented on some environment objects in the Unity 3D engine. The second image illustrates a normal map applied on a 3D object.

Avatar Creation

Since I’ve joined the team, I have been focused mainly on getting realistic looking avatars into the simulations. As they are meant to run in real time on the Unity 3D game engine, it is very important that they are not heavy on memory so it doesn’t slow down the computer.
To begin with, I started out with the base model.  Although for this project we will be creating custom models using 3D Max and Maya, these models were obtained online (AXYZ designs) for prototyping purposes.

I found this a very clean and well built model. In this picture you can see how the lines around the knee all converge to where the knee bends. This allows the leg to bend while preserving its natural shape. How the model will “deform” is also very important when building a model for realtime animation.

Next we have to build the texture. Here ,I took a picture of our programmer and fit it to the model. This process called “UV mapping” is basically matching the 3D coordinates of the model to where they would be on a two dimensional image. In order to get the best fit, I often have to go back between Photoshop and the 3D application, making adjustments for both the UV coordinates and the image itself.

Here is the textured model

Now that that we have a textured model, we’re ready to animate him. First we must set the modelup for animation. This process is called “rigging”.

Rather than reshaping the model for each pose, I create an invisible “skeleton” that tells the computer where this model bends and how it bends. I can define how much a certain “bone” influences the 3D object. This skeleton is really what is being animated.

Although I could animate these bones directly, I’ve created a control system that I can attach to this skeleton, which will give me better ways to animate it. This system is called the “rig”.

In this post, I’ve described some of the basic steps in building an avatar. As you can see, especially with the models, this is a very manual process. As our goal is to recreate a 3D model of the user, we are searching for ways of automating these processes wherever possible.
It is very likely that we will have a library of parts from which our characters are assembled. By creating a database of parts, and standardized system of UV mapping the photographs to the models, it will be easier for us to create a variety of avatars.

Notes and Observations

Team Meeting

HHS

This week we began working with Avatars in terms of creating realistic avatars from photographs. The complexities were found in the inability to manipulate the mesh so that we could replicate the persons bone structure. Shown below are images of two test subjects.

Daniel will be looking into evaluating the software Face Gen (http://www.facegen.com/) To see if it will allow us more control over meshes to produce more accurate head shapes and facial bone structures. The software runs about $900, and has an SDK that we may be able to incorporate in our Unity projects.

It was determined in this meeting that there will be two version of the learning module. One will be designed for patients and in addition to the previous post will consist of the following:

• • Basic user interface – extremely user friendly, icon based environment
  • Touch screen
  • Will collect data
  • Will have mini tutorials and help screens throughout
  • Could possibly have a self adjusting interface – the interface adjust itself based on the responses of the user

The second version of the module will be for physicians and will differ from the above by the following:

• • Web based
  • Mouse Driven
  • Computer savvy user interface and environment – fewer hints and tutorials, no adjusting

Carlos Pita began work on the draft story boards which are due tomorrow. Lots of discussion has already gone into how this interface will look and run. Carlos suggested that the interface be a seamless part of the 3D environment, and that we will use the entire 3d environment rather than lock the camera down. Carlos Gomez suggested that interface screens could be actual 3D objects in the room that once activated, would take up the entire screen.

One concern that arose, was addressing the altering of a models body shape in real time. One facet of the project is to allow a model to change over time based on a given set of initial conditions. We would record the initial conditions based on the measurements and BMI of the participant (end user). The issue is that there really is no uniform way to insure consistency in body alterations over many users. It was decided that we may have to incorporate an interface that allows one to adjust the avatar so that an accurate body types can be set initially. In this way, were an avatar required to gain weight only the appropriate areas would be adjusted.

Trigger Tapes

Our initial goals for the trigger tapes was to hire actors and film the entire segment based on scripted events. It has now been determined that the team could produce the videos completely with avatars. The advantages of this are:

• • Opportunity to design entire process and work flow of avatar development
  • Very similar to the work flow proposed for HHS
  • Once completed, will generate exposure in an area that we had not anticipated having access to

We will explore the script, and the possibilities of modeling and rendering in Maya.