CG Education

Converting A Mudbox Model To 3DSMax
By Neil Blevins
Created On: Feb 10th 2007
Updated On: May 26th 2008

Here's a tutorial on how to transfer a model from max to mudbox, export a mesh and a displacement map, and then render it in max using a bunch of different displacement methods (depending on what renderer you want to use).

Here's a list of methods we'll be using to displace the geometry in max...

Scanline Displace Modifier
Mental Ray

Making the Mesh in Max, and Exporting as Obj

Make Your Mesh in max. I created a simple sphere with a radius of 45 and 32 segments.
Make sure the "Generate Mapping Coords" is on. Or if you have a more complex object, create good uvs for the object using the UVW Map modifier and/or the Unwrap UVW modifier.
Export Selected, choose obj, use the following settings...

Importing, Displacing and Exporting from Mudbox

Open Mudbox
File -> Import, choose your obj file.
Subdivide Selection 4 times.
Displace your mesh using various brushes

Step Level Down three times (go to the Level 1 Subdivision Level)
Select The Object in Object List
File -> Export Selection, save as obj
Step Level Up three times (go to the Level 4 Subdivision Level) (You must be at the top level if you want to export the top level displacement map)
Utilities -> Texture Baking -> New Operation, set your settings as below...

A few notes:

Low Resolution Mesh is whatever level you exported your obj from mudbox as. So if you decide to export level 0 instead of level 1, you should choose level 0 in this dropdown.
Subdivision of Good seems to work fine for most cases.
The High Resolution mesh is the top subdivision level of your mesh.
You'll want to create a displacement map. I unchecked the make normal map checkbox, since I'm interested in displacing.
Give your displacement map a name. Don't add an extension (adding .tif is not necessary).
The size of your bitmap. The higher the better, although the higher the more memory the map takes. 1 to 4k is a good value depending on how close you're getting to the mesh, and how much tiny detail you've added.
Make sure a File Type of TIF is set.
Search Distance. This should be the same as the highest value you've displaced your mesh (The default that mudbox picks is generally a good starting place). This mostly involves guesswork, you'll want this number to be as low as possible, but not so low that you miss the peaks of your displacement. Basically the highest and lowest points of your displacement get missed with a search distance that's too small, so you'll see neutral grey dots at your peaks and valleys (see the image below). I generally start small, render some test displacement maps, and I keep increasing it until the artifacts go away...

I've found turning on Smooth UVs tends to give me better results.
Displacement Format, I've changed this to 16bit, but if you don't have too much detail, 8bit may be ok. A higher color depth means a smoother displacement map (since it has more values of grey), and hence a better representation of your displacement, which is why I chose 16bit.
Keep Zero As Middle Grey means that values of 50% are no displacement, white values displace out, and black values displace in.
Turning off Normalize seems to give me the correct results.

When your Texture Bake is complete, notice the following dialog. Write down the numbers that the dialog provides you.

Figure 4a

Importing Back into Max

Import the obj file of level 1 that you created in mudbox.
Convert To Editable Poly
Apply a Smooth Modifier, click on smoothing group 1 to apply a single smoothing group to your entire mesh.
Apply a Material that has a diffuse color of 128,128,128 (mid gray)

Rendering using the Scanline and Displace Modifier

Apply a Turbosmooth Modifier (Turbosmooth takes up way less memory than Meshsmooth, so for high subdivision models, using turbosmooth is a must to conserve memory), set to iterations 3. 3 is the number of iterations between you level 1 mesh and your level 4 displacement map. So if you subdivided your mudbox object to level 6, and exported a level 2 obj file from mudbox, you'd want to set this value to 4.
Apply Displace Modifier to the object.
Strength is set to whatever the difference between your two range values were. In this example, the range was -1.726 to 1.726, so set the strength to 3.45.
Click on Luminance Center, make sure center is at 0.5.
Choose Map, choose Bitmap map type, select your displacement map.
Click on "Use Existing Mapping".
Open an instance of the map inside the material editor, and set the Filtering to None. When the displacement map is calculated inside mudbox, it goes through a filtering process to achieve the final result. Then when the geometry gets displaced in max, that geometry gets filtered as part of the standard renderer antialiasing process. If you turn on the image filtering as well, you're basically filtering the bitmap a 3rd time, which may make your final result blurrier than the result inside mudbox. But in some cases, especially if you have a lot of high contrast detail, you can play with turning this setting to "Pyramidal" or "Summed Area" and see if it helps remove any artifacts. But I've had more luck keeping it off.
Render in Scanline

Rendering using Mental Ray

Under Render, Assign Renderer, Production, choose mental ray Renderer.
Apply a Turbosmooth Modifier (Turbosmooth takes up way less memory than Meshsmooth, so for high subdivision models, using turbosmooth is a must to conserve memory), and set to iteration of 1. I've found that this can sometimes help with a smooth displacement map in mental ray.
Create a mental ray Material, assign it to your object.
Under Surface, choose "Material To Shader".
Under material, choose Standard.
Go back to the top level of your material.
Click on the map beside Displacement, choose 3d Displacement.

Set Extrusion Strength to whatever the difference between your two range values were. In this example, the range was -1.726 to 1.726, so set the strength to 3.45.
Under Extrusion Map, choose Bitmap map type, select your displacement map.

Set the Filtering to None. When the displacement map is calculated inside mudbox, it goes through a filtering process to achieve the final result. Then when the geometry gets displaced in max, that geometry gets filtered as part of the standard renderer antialiasing process. If you turn on the image filtering as well, you're basically filtering the bitmap a 3rd time, which may make your final result blurrier than the result inside mudbox. But in some cases, especially if you have a lot of high contrast detail, you can play with turning this setting to "Pyramidal" or "Summed Area" and see if it helps remove any artifacts. But I've had more luck keeping it off.
Under the output panel, set RGB Offset to -0.5 (this sets a value of grey to be no displacement).

Open the Render Panel, go into the Renderer Tab, under Shadows & Displacement.
Change Edge Length to 1 or lower.
Max Displace should be 3.45 again, or your highest Extrusion Strength if there are multiple displaced objects in the scene.
Under Max Level, place 3. 3 is the number of iterations between you level 1 mesh and your level 4 displacement map. So if you subdivided your mudbox object to level 6, and exported a level 2 obj file from mudbox, you'd want to set this value to 4.

Render