ZOBJ/ZMAP Blender Import Script

[Deleted]

I've been working on this for the past 3-4 days, it's a zobj / zmap import script for blender. I was stuck on trying to attach the armature to the mesh for like 10 hours, which happened to be just a simple mistake...FUUUUUUUUUUUUUUUUUuuuuuuuuuuuuuuuuuuuu!!!!

Anyway, here's some screenies of the progress so far





Current Features:

• Imports all hierarchies within zobj files
  
• Follows branch (0xDE) instructions
  
• Emulates matrix (0xDA / 0xD8) instructions to fill joints
  
• Skins the meshes to armatures, imported models are ready for posing
  
• Pipesync (0xE7) mesh grouping
  

Planned Features:

• Extract textures
  
• Import animations
  

Note that Nintendo stores the models in awkward poses. In the pics above, I had to manually pose them, though once animation importing works, you could just set a frame as the restpose.

I'm trying as hard as possible have this done by the day after tomorrow

[blinx]

SoD, mah boi! This is looking simply fantastic; surely the perfect tool for my needs.
The rule 34 I shall conjure up, it's... exciting to think about, to say the least.

Fantastic work.

[punk7890™]

Wow nice work man!

[Deleted]

thanks for the support guys


anyway, it's like, 6am and I just finished texture dumping. I still have to tackle uv-mapping and animation. I'm trying hard to get it done before tomorrow, if not, I'll at least do an early release that probably won't support zmaps yet


Updated Progress:

• Parses 0x02 (MODIFY_VTX), 0xF2 (SET_TILESIZE), 0xF5 (SET_TILE), 0xFD (SET_TIMG), 0xFE (SET_ZIMG), 0xFF (SET_CIMG), and 0xF0 (LOAD_TLUT) instructions
  
• Dumps textures to tga files. CI4 textures are expanded to CI8; RGBA16 textures are expanded to RGBA32
  

[Deleted]

A little mad at myself, yesterday I took a nap and ended up sleeping all day

This set me back a bit, but I've still been hard at work on this. I ran into a little issue though; apparently blender doesn't let you import normals. I have 2 solutions in mind. The first is a bit complicated and involves creating a normal map and baking it to the model. The problem here is in order to do so, I'll have to write custom stubs for drawing primitive on a plane, and if there are overlapping faces, the output will be terrible.

The second solution is to average the sum of the x, y, and z normals to get a grayscale color and blend that with the vertex colors. It's not 'true' normals, but it would allow for some decent shading. Here's some pics of what I'm talkin about:

Here is what normals look like plainly interpreted as vertex colors


This is the 'averaged' normals; calculated with 0.5 * (((n.x + n.y + n.z) / 3) + 1)


Like I said before, these aren't 'true' normals, they're vertex colors, but it still looks really good in my opinion. Btw, there are some minor defects you can see which I believe is caused by blender's mesh.calc_normals() function. I'll see what I can do about those

[Deleted]

Finally got textures working! (somewhat)
There's only some little quirks left I have to deal with

• I have to scale the uv-coords by the texture scale to correct the uv mapping
  
• I have to find some way of loading files into other segments. Currently, textures that are located in other segments which aren't available show up as black. (what I might do here is just scan the current directory for files named segment_##.zdata)
  
• I'll have to add support for F3 loadblock instructions (eventually) Once I fix the 2 issues stated before, most textures should show up perfectly fine, but there are some that will have 'garbage' like Link's tunic unless I emulate it properly
  

Here's a pic of the progress so far. you can see the right materials are being assigned, the uv mapping just needs to be adjusted.

[Deleted]

lol, that moment when you're getting so close, you can taste the win



As you can probably guess from the above pic, I fixed up the uv-mapping a bit and added support for loading from other
segments. In the example shown above, I'm actually loading from 4 different segments

• 06 - object_link_boy.zobj - (the model I loaded)
  
• 04 - segment_04.zdata - (gameplay_keep)
  
• 08 - segment_08.zdata - (link's eyes ripped via rice's debug plugin)
  
• 09 - segment_09.zdata - (link's mouth ripped via rice's debug plugin)
  

Obviously, there's still some little quirks with the graphics;
ie. I've yet to add support for clamp/mirror/repeat and some of the textures have a little garbage in them.
But almost done!



EDIT:

lol, forgot to take IA channels into account, fixed that real quick, now it looks like, 50x better





EDIT #2:

I tried out some of the others, and they don't seem to be giving me as much trouble as Link




Also, I went ahead and added support for loading maps (Mesh types 0 and 2 only)




I also believe I've figured out why those tiny graphical errors are showing up. I think blender is automatically removing
duplicate vertices, and when reassigning the faces to the originals, it accidentally rotates the uv's and vertex colors.
The can be easily fixed by going into editmode, Object->Face->Rotate UV's / Rotate Colors for now, but eventually
I'll try to correct this problem



EDIT #3 (11:48am):

I fixed the glitchy uv-mapping problem by checking for doubles before extending and manually adding items to the
mesh rather than using blender's built in unpack_face_list function.




~smooth as silk
now, if I can just fix the problem with some of the textures being exported with garbage in them, I'll upload it :3



EDIT #4 (12:24pm):

Fixed IA textures so they have transparency and set it up to auto-enable transparency for textures that have alpha channels

Btw, not sure if mentioned this or not, but I already added support for setting texture clamping on import (though you probably already noticed by the screenshots of link)

[blinx]

Fantastic. Keep it up, or forever be imprisoned in my dungeon.

[Deleted]

Aight, here's the first release of the plugin

download script here


Installation Instructions:

• Unzip, and place folder (with files inside) in "%ProgramFiles%\Blender Foundation\2.XX\scripts\addons"
  
• Run Blender, click on the icon for the 3d-view and change it to user-preferences
  
• Click the Addons tab
  
• Scroll down until you find the Zelda64 importer and check it to enable it
  
• Change back to 3d view and press Ctrl-U to save your preferences
  

Usage:

• Go to file -> import -> Zelda64 (.zobj; .zmap)
  
• Select your file and import
  

Q/A:

• Q. The script doesn't work!!
  
  A. You need Blender 2.60 or higher, and possibly Python 3 in order for it to work
  
  
  
• Q. How can I load data from other segments?
  
  A. You can load data from other segments by creating files in the same directory as your file with the name 'segment_XX.zdata', replacing XX with the segment number in hexadecimal. For example, common uses would be using gameplay_keep as segment_04.zdata and your scene files as segment_02.zdata.
  
  
  
• Q. Why are parts of my model black?
  
  A. If a texture is in a segment that hasn't been loaded, it will be given a black texture. If you're loading a zmap, be sure to put the scene file in the same folder with the name segment_02.zdata.
  Another useful tip, there are some special cases like Link where you'll see textures with names like 08000000.tga and 09000000.tga. If you rip the animated textures for that zobj (aka. eyes, mouth, etc) with Rice's debug plugin, flip it vertically, and create a new segment file using only the image data (no palette) from that texture, it'll import it correctly
  
  
  
• Q. Why is the character all balled up?
  
  A. Nintendo stores the models in awkward poses. However, you should always be able to get a t-pose if you rotate a couple of limbs by 90 or 180 degrees
  
  
  
• Q. There are multiple models?!?
  
  A. Yeah, zobj's can contain multiple hierarchies, and this script imports all of them.
  
  
  
• Q. Why did you choose today as your deadline?
  
  A. Today's my 21st birthday, I wanted something cool to remember it by
  

Note that this is still a work in progress, and I will continue to update it. Have fun!

[Jason777]

You win at the internet.

[Deleted]

Mar 5, 2013 22:59:25 GMT -5 Jason777 said:

You win at the internet.

LOL, thanks

I've still been working on this by the way, I just haven't been posting updates because I've been rewriting the script from scratch. To be honest, I hadn't worked with python in a long time and really jumped into this project; so my script came out looking really nasty and I was very unhappy with it.

The new script is far more efficient in terms of speed, memory consumption, and stability.

• Values that are used multiple times I store as variables rather than recalculating them for every usage
  
• Commonly used functions like splitting address into segments and offsets and checking if addresses are valid have been made into functions
  
• I found out while looking through gbi.h that only tiles 0 and 7 are used (0 being for actual texture data and 7 for palettes), so I now only use a single texture struct instead of 8 tiles
  
• Hierarchies and meshes have been created as a new class types and are much cleaner and easier to work with
  

Some additional changes and additions:

• The plugin is now a single file rather than a folder
  
• More details about importing are logged to the console, such as the location of the hierarchies, how many bones they have, and whether or not the bones have display lists
  
• Texture dumping is now done in scanlines rather than all at once; basically, I parse the image in lines using a single loop (regardless of the texture format). CI4 texture data is expanded to CI8, and all non-CI data is expanded to RGBA32.
  Once a whole scanline is made, it's written it to the file; which brings me to the second change...
  
• ...texture mirroring is now supported. Since images are now dumped in scanlines; after writing one I just check the horizontal clipping type and if it's set to mirror, I reverse the the scanline and write it a second time.
  For vertical mirroring, I check the vertical clipping flag after the entire image is drawn, and if it's set to mirror the function calls itself again with a particular argument that tells it to loop from top-to-bottom instead of bottom-to-top...which brings me to the third change...
  
• ...tga files store their textures upside-down and n64-textures are stored upside-up; to combat this problem before, I just set the y-offset of the tga header to the height of the image and set flag 0x20 in the tga descriptor field; which pretty much just tells it that the origin is the bottom left corner. Now, I actually flip the image when storing it
  

Additional things I'm working on or plan to get to

• Display lists will be kept track of in an array. After the hierarchy has been parsed and the meshes built, the script will enter a second phase in which it scans the entire file for display lists and builds an array of the offsets. It'll then compare the two lists, remove any items from the new list that have already been created when parsing the hierarchy, and finally import all the remaining display lists in the file
  
• Hopefully, add some compatibility for the majora's mask and beta hierarchy formats. (if you have any documentation on these, please share)
  
• Add support for color blending; (primitive and environment colors)
  
• Add support for shiny objects through modifiers
  
• Add support for masks
  
• Add importing options such as
  :x: disabling shading
  :x: using vertex colors instead of normals (or both)
  :x: disabling texture dumping (in case you want to use your own textures)
  :x: disabling texture importing
  :x: only dumping the textures (no mesh or hierarchy)
  :x: disabling the parsing of matrices
  :x: disabling the creation of armature (in case you just want the mesh)
  :x: disabling the creation of the mesh (only get import the armature)
  

[Deleted]

The new script has finally surpassed the old one!



As you can see, all textures are exported correctly, and now there's support for texture mirroring!
Thanks to Sanguinetti for pointing out that the y axis needed to be flipped, and that MM models
actually do work

It's not quite ready for release yet, I'm gonna add some extra import options and possibly get
primitive color blending working first. Key an eye out though! The second release is coming soon

and just a little something extra for you to chew on....

SPOILER: Click to show

EDIT:

Some miscellaneous fixes, (thanks to Bobbo for pointing these out)

• No duplicate vertices
  
• No duplicate materials
  

I was gonna wait until I added either color blending or animation support for the next release,
but what the hell

Zelda64 Import Script v2

Btw, I started a google code page for the scripts here since I'll be updating this often and
possibly be writing an exporter soon

[Deleted]

Update 3 Progress:

• Added support for 0xFA instructions. Primitive color is multiplied with shading.
  Result =

• Added support for 0xFB instruction. Environment color is multiplied with shading.
  I had actually tried to emulate color combining in general (evaluating the combining
  equation set by 0xFC), but fell short. This still should work properly though

EDIT:
Made a few bugfixes to make the script more stable (ie. really complicated zobj's like horses will
now load partially rather than just crash)
Added import options for the following:

• Using vertex colors instead of shading or using neither
  
• Disabling loading from other segments
  
• Disabling primitive and environment colors
  
• Disabling texture exporting
  
• Disabling texture importing
  
• Disabling texture clamping and mirroring

Also been working on animation importing. It's giving me a run for me money, but I'm determined

[Deleted]

Aight, here's v3 of the script: download

Changes:
- Added support for primitive and environment color blending
- Added import options for:
----- disabling loaded from other segments
----- disabling matrices
----- enabling/disabling primitive and environment colors
----- inverting environment color (tempfix)
----- disabling texture export / import
----- disabling texture clamping / mirroring
----- toony uv's (not scaling down the uv's, resulting in solid-colored meshes that look 'toony')

Bugfixes:
- Increased the vertex buffer size to 64 vertices so some models from other utilities will import correctly
- Added some extra error checking code for vertex loading, prevent crashes on models like object_horse.zobj
- Added check to make sure matrix segment is 0x0D, preventing abnormalities like Zelda's hair from being attached to her waist in object_zl2.zobj
- Fixed texture mirroring issues, resulting in better results for importing characters like kafei

Todo:
- Add support for animation
- Fix texture clamping issues that cause small defects in areas like kafei's legs, goron link's arms, and Zora link's neck
- Rewrite matrix handling code to work with actual matrices rather than just limb offsets

and of course, something to sell it all with:

[punk7890™]

Awesome once again!