And once again I return to work!
It seems everyone has tools to retarget missing bitmaps in maxscript, but nobody seems to share how it’s done.
Here’s a really simple solution to the problem using a dab of dotnet and zero error checking, which wouldn’t be very hard to add.
fn findSameNamedAsset asset newDir = ( local filenameOfAsset = (DotNetClass "System.IO.Path").GetFileName asset local di = DotNetObject "System.IO.DirectoryInfo" newDir local files = di.GetFiles filenameOfAsset (DotNetClass "System.IO.SearchOption").AllDirectories files.directoryName ) fn repathMissingBitmaps newDir = ( ATSOps.Visible = true local arr = #() ATSOps.GetFilesByFileSystemStatus #Missing &arr for asset in arr do ( AtsOps.clearSelection() AtsOps.selectFiles #(asset) local newFileName = findSameNamedAsset asset newDir AtsOps.SetPathOnSelection newFileName ATSOps.refresh() ) OK )
And back to work I go!
I’ve been working with windows forms. I was a little annoyed when I kept crashing 3dsMax via outOfMemroy exceptions after loading hundreds of bitmaps. So, I hastily created this little UI to check my current memory standing. It even has a little log. How quaint.
If the rollout is open, you can call HeapTool.registerEvent
If you want to record the memory change from a certain point, you can call heapTool.setMemState() to register the current heap size as the value we wish to compare to. for example:
heapTool.setMemState() if doesFileExist thumbnailLoc then thumbImg = loadNetBitmap thumbnailLoc else thumbImg = loadNetBitmap defaultThumb heapTool.registerEvent "Add dotNet Bitmap" quiet:true
Here’s the entire, copy-pasta’d, raw, unclean, uncaring tool code. It should be noted that the “large number” types that are returned from “heapSize” and “heapFree” are very nastily hacked into integers. It’s not meant to be accurate. moreso, it gives a picture of what actions are costing lots of memory.
Anyway, use at your own risk. (on a side note, man, the formatter for maxscript looks terribad!)
rollout heapTool "Heap Tool" width:320 height:380 ( label lblUsed "Used Memory: " pos:[8,8] width:104 height:16 progressBar pbUsed "" pos:[8,24] width:304 height:16 color:(color 30 10 190) label lblFree "Free Memory: " pos:[8,48] width:104 height:16 progressBar pbFree "Free Memory" pos:[8,64] width:304 height:16 color:(color 30 10 190) button btnUpdate "Update" pos:[8,88] width:80 height:24 button btnGc "gc()" pos:[96,88] width:40 height:24 button btnFree "free bmps" pos:[144,88] width:72 height:24 button btnUndo "clear undo" pos:[224,88] width:64 height:24 listBox lbxMemLog "" pos:[8,120] width:304 height:19 local lastMemState = 0 local eventBuffer = #() fn getFormattedHeapVal = ( local h1 = heapSize as string h1 = (substring h1 1 (h1.count - 1)) as integer local h2 = heapFree as string h2 = (substring h2 1 (h2.count - 1)) as integer #(h1, h2) ) fn setMemState = ( h = getFormattedHeapVal() lastMemState = h - h ) fn updateDialog = ( -- cheat the sheeit out of the values. h = getFormattedHeapVal() lbxMemLog.items += eventBuffer eventbuffer = #() local used = (h as float - h as float) / h as float * 100.0f local free = (h as float / h as float) * 100.0f lastMemState = h - h pbUsed.value = used pbFree.value = free ) fn registerEvent eventName quiet:false= ( h = getFormattedHeapVal() local delta = ((h - h) - lastMemState) local changeWord = " allocated " if delta < 0.0f do changeword = " released " local element = ("\""+ eventName+ "\"" + changeWord + (delta as string) + "\n") if quiet then ( setMemState() eventBuffer += #(element) ) else ( lbxMemLog.items += #(element) updateDialog() ) ) on btnUpdate pressed do updateDialog() on btnGc pressed do (gc(); registerEvent "garbge collect") on btnFree pressed do (freeSceneBitmaps(); registerEvent "Free Bmps") on btnUndo pressed do (clearUndoBuffer(); registerEvent "Clear Undo") on heapTool open do (lastMemState = 0; updateDialog()) ) createDialog heapTool
I’ll admit, I work in a small office and don’t really have access to a large amount of resources, but we make do. What I wanted to try out was using our networked drive as a server for a file system. I didn’t want to sacrifice computers for a server. More importantly, I needed a way to share our assets between the few people who work on characters. And, I didn’t want to rely on a tedious copy/paste system via windows.
So, I thought i’d give Git a go. Git initially presents itself in a very good way. It works over networked drives, is decentralized, and feature-rich enough to allow for mostly-easy manipulation of data back and forth between our central ‘bare’ repository and client machines. Git has a large amount of support from various groups of opinionated programmers. I even saw some people mention using it on TAO. All of these are good signs, so thumbs up, right?
I had read multiple times while doing preliminary research that git was quite difficult to really Grok and this proved itself true very quickly. I constantly found it a struggle to push and pull binary assets between remote repositories without something going wrong: Files missing, files being overwritten, unexplained errors when submitting or syncing assets. The list continues. The net product of me using git was an extra hour or two at the end of the day insuring my intern and I were sync’d up with mostly identical assets.
So, last week we decided to abandon git and sacrifice a computer to act as our SVN server. Linus Torvalds would think i’m stupid.
Git’s framework is both individualist and democratic. It expresses itself as a trust-driven system of dynamically moving branches of a single software expression. Changes and ideas are promoted naturally via developers selecting what’s best in a tree of branches and evolving the software to a better state. Game development simply does not work this way. Game development is dictated, mostly rigid, with goals that we’d like to think are solid and achievable. Movement other than forward towards our imaginary goalposts is advertised as unwise. Independence is not a primary goal in game development, and from my experience, democracy within a game company is mostly ineffective.
But, lets not get too meta. I think git’s problems mainly stem from the lackluster GUIs that layer on top of the git shell experience. Both GitGUI and Git Extensions lack a clear purpose when performing actions that have tangible results in SVN. In Git extensions, where I spent most of my time, everything felt so broad. Changes were submitted as huge chunks of files that had to be merged in their entirety, and any conflicts were simply overwritten instead of delicately sorted out. This hardly does anything to help understand the software.
In hindsight, it probably would have been better to work on asset changes within their own branches that were then moved around the network, and then merged for a final result. This idea would have fit better within the git paradigm, But, is so contradictory to the idea of working with assets as an artist that it’s hard to want to push for a change like this. Considering i’m going to be on the same project for the next year or so, It’s going to be a while before I am ready to give git another attempt.
I’ve spent a lot of time experimenting with multiple ways of saving data from 3dsmax: INI Files, raw text files written and read via maxscript (ew), and some pretty long-winded C# save/load classes. Each had its own pluses and minuses, but it was a chore to make any adjustment to the read/write code when anything about the structure of the data changed because it was all so very static in its functionality. The ideal solution is to allow our data to define our exports in a dynamic and flexible way that’s reusable.
The key to understanding how this works within max is to take a look at the maxscript command “getPropNames”, which allows us to query the names of all properties within a given structure or object. With Maxscript we can call this command using the self-referencing keyword “this” inside of a struct to gather all properties within the current context. Combine this with “getProperty” and “classof” and we have a simple way to query every property name, property value, and property type for most every structure within Maxscript. It is also a possibility to implement this within C# using similar methods, but we’ll focus on maxscript for this post:
struct animalType ( prop1 = "bear", prop2 = "cat", /* lots and lots of properties */ fn exportData = ( local properties = getPropNames this local propValues = for property in properties collect (getProperty this property) local propTypes = for value in propValues collect (classof value) ) )
These three commands give us three parallel arrays of all properties and methods within our structure, and most importantly, enough information to reconstruct the data when we want to re-load it again. This does not however, handle arrays or node references, which complicate things and will be discussed in a further post.
Moving on, we want to be able to write all of this accumulated data to some sort of file to save it. My favorite format this month is XML, so we’ll go ahead and utilize C#’s wonderful XMLWriter class to handle our output. Using our gathered data, We simply initialize our XMLwriter class and then iterate over our arrays, producing a single element for each property within our structure. At less than 40 lines with white space, it’s a pretty small piece of code that handles a lot of scenarios.
/*inside of a struct...*/ fn exportData xmlFile = ( /* get all of our properties and methods*/ local properties = getPropNames this local propValues = for property in properties collect (getProperty this property) local propTypes = for value in propValues collect (classof value) /*build a format class so we indent out output. makes it readable */ local xmlFormatter = dotnetobject "System.Xml.XmlWriterSettings" xmlFormatter.Indent = true /*Instantiate our xml writer class */ local XMLWriter = dotnetclass "System.Xml.XmlWriter" XMLWriter = XMLWriter.Create xmlFile xmlFormatter /*write the start of our document*/ XmlWriter.WriteStartDocument() XmlWriter.WriteStartElement("structDef") /* Iterate over all of our gathered properties, except methods*/ for i = 1 to properties.count where propTypes[i] != MAXScriptFunction do ( /* create an element for each property. save it's type and name as an attribute */ XmlWriter.WriteStartElement("property") XmlWriter.WriteAttributeString "name" (properties[i] as string) XmlWriter.WriteAttributeString "type" (propTypes[i] as string) XmlWriter.WriteString (propValues[i] as string) XmlWriter.WriteEndElement() ) /* write the tail of the xml document */ XmlWriter.WriteEndElement() XmlWriter.WriteEndDocument() /* close the file. finalizes the write and puts the data to the target */ XmlWriter.Close() ),
Running the above code yields xml similar to the example below.
<structDef> <property name="numEyes" type="Integer">6</property> <property name="id" type="String">hypnocat</property> <property name="type" type="String">cat</property> <property name="numLegs" type="Integer">15</property> <property name="averageLegLength" type="Float">70.31</property> </structDef>
When we load this data, are presented with an interesting challenge. Any data that we pull from the XML file is going to be of type string. If we assign all of these raw values straight back to our structure, we’re going to end up with a struct full of strings and lose any previous and possibly important type data regarding our parameters. Luckily, we can use our self-referencing to cast/convert all of our strings to the proper type.
fn loadData xmlFile = ( /*open our xml file */ local xmlReader = dotNetObject "System.Xml.XmlDocument" xmlReader.Load(xmlFile) /* select all xml nodes of type 'property' */ local properties = dotNetClass "System.Xml.XPathNodeList" properties = xmlReader.SelectNodes("structDef/property") /*get the enumerator and iterate over all nodes*/ enum = properties.GetEnumerator() while enum.MoveNext() do ( /*build correct data type*/ local thisPropertyName = enum.current.Attributes.Item.Value local thisPropertyType = enum.current.Attributes.Item.Value local thisPropertyValue = enum.current.InnerText as (classof (getProperty this thisPropertyName)) /*set the property generically*/ setProperty this thisPropertyName thisPropertyValue ) )
The key to preserving our property types is (classof (getProperty this thisPropertyName)). We can’t cast our values to their appropriate type using just the name of the Type as a string, so we work around that limitation by relying on the structure our load function exists within. Using the property name we’ve loaded, we call getProperty to fetch the already-existing value within our struct . This returns a “Type” rather than a string, and can be used to cast using “as”. Clever (or at least, I think so.)
This setup should handle most common struct properties except for arrays and node references, which we’ll handle later. This post is already incredibly long!
I’ve spent all day coding Maxscript and fixing 3d assets at work. Somehow, coming home and spending the evening writing some simple HTML and CSS for my website refresher has been completely satisfying and not tiring whatsoever.
It’s amazing how much different my motivation levels have been now that i’m actually employed and not scrambling for a job.
I think i’ll be condensing my tech art page into a demo reel and some tool demos tomorrow evening. Excited for the unknowns of the future and feeling confident!
On a side note, working with CSS tonight has revealed the lexicon of how CSS is actually used within a website. CSS is actually rather, quite abstractly, SQLish. But most people wont understand this, or i’m just really tired.
I’m working on a new python application at the moment. It’s a timer based productivity software that’s supposed to use the “set egg timer to 25 minutes and WORK” methodology of getting stuff done. Oddly, i’m not actually using this methodology to keep myself motivated – coding is engaging enough as-is to not require such implements.
Anyway, today, the very base of the timing update systems were put into place. I wanted to have a system that watched over multiple objects that kept track of their own elapsed time and updated at specific intervals. This would allow users to setup multiple milestones with unique “work periods” for differing task lengths. To do this, my code uses three specific classes that percolate time data in a semi-downward motion.
At the top of the hierarchy is the overall time-ticking object. The time object sits in looped perpetuity, continually telling the manager class (described later) to update its children at a designated time interval. This small class inherits from the awesome ‘threading’ module, which allows its functionality to be executed in, you guessed it, another thread – this allows the perpetual while loop to not interupt the flow the overall app. The timer can be started by invoking the inherited method threading.start(), which initiates the code within run(). The timer can be then stopped by invoking threading._stop(). Stopping the execution of the thread is a universal break for all of our dependant timers.
using threading using time updateInterval = 1.000 class ticker(threading.Thread): timeManager = None def run(self): if self.timeManager == None: self._stop() print ("time manager not specified") return while True: time.sleep(updateInterval) print("ticking") self.timeManager.tick()
Next is our management function that keeps track of all of our timers. I’m beginning to think that this part of the application is an unnecessary middleman, but at the current state it remains. The task manager keeps a list of timer objects that are then updated at our tick intervals. It iterates over these objects, and calls the assumed-existing function ‘update’, to percolate our time ticks down the chain. the task manager ensures unique names within timerObjectList as to allow searching for specific timers (yet to be implemented).
class taskManager(): totalElapsed = 0.000 timerObjectList =  def __init__(self): print("task manager initialized") def tick(self): self.totalElapsed += updateInterval self.updateTimerObjects() def updateTimerObjects(self): #print("updating timer objects") for timerObj in self.timerObjectList: print("updating timer object " + timerObj.name) timerObj.update() def addTimer(self, newTimerObject): print("adding element") for timerObj in self.timerObjectList: if timerObj.name == newTimerObject.name: print("name not unique") return False print("adding timer object") self.timerObjectList.append(newTimerObject) return True def removeTimer(self, timerName): i = -1 timerToRemove = False for timerObj in self.timerObjectList: i += 1 if timerObj.name == timerName: timerToRemove = i if timerToRemove: self.timerObjectList.remove(timerToRemove) return True else: print("object with name " + timerName + "does not exist") return False
Currently the sloppiest of the bunch is the production timer object. The production timer runs for an allocated amount of time, then pauses (for the user to take a break). The user then resumes the production timer to start a new ‘cycle of production’. Rinse and repeat.
The biggest flaw sofar in the entire system is that the percolation of time downwards from the ticker class an inaccurate representation of real elapsed time. Times collected will vary by micro of milliseconds from the actual time. Eventually the system will collect time by checking time.clock() to gather correct elapsed time.
class productionTimer(): name = "default" updateMe = True; prevUpdateTime = 0.000 elapsed = 0.000 cycleLength = 0.000 pauseLength = 0.000 cycles = 0 def __init__(self, timerLength, timerName): print("placeholder") self.name = timerName self.cycleLength = timerLength def update(self): if (self.updateMe): self.elapsed += updateInterval print("being updated!") if self.remainingTime() <= 0: print("cycle complete!") self.cycles += 1 self.elapsed = 0.000 self.pause() def remainingTime(self): return self.cycleLength - self.elapsed def pause(self): self.updateMe = False print("paused!") def resume(self): self.updateMe = True print("resumed!")
Next on my list is cleanup for these few classes, and the beginnings of a pyQT gui. I’ve been playing with the pyQT module in Maya, and am quickly discovering it to be an awesome GUI creation system. Hats off to it! (whatever that means).
Today I finished up my work on my lava shader. It turned out pretty nice!
I have a narrated demo video of the shader here: http://www.youtube.com/watch?v=pScLYuWuirw
This project was a fantastic learning experience, and it leaves me wanting to spend more time on shaders. It raised some general questions that I’ve not yet had time to tackle as well:
- How can we tie geometry shaders into materials to make interesting wavy effects? water? bubbly surfaces?
- How can we use more information from the mesh’s world to adjust that object’s visual appearance beyond pre-rendered cube maps?
- It is possible to write a maxscript interface for the hlsl shader. Is this effort even worthwhile?
- Why is 3dsmax mapchannel ordering so wonky?
HLSL wise, I still want to spend more time with screen space effects, and take a look at some fun challenges such as color adjustments using LUTs and writing some sort of mood/environment color adjustment schema.
Meanwhile, it’s time for me to transition into an area that i’ve been neglecting for some time: Python.
Today I sat down to convert my messy and convoluted lava shader into a cleaner two-pass variety. Previously, the shader required me to do lots of samples, ifs, lerps and the such. For example, this little snippet of code lerps between the magma and igneous rock layers depending on the value of the alpha. The alpha used vertex paint, and a user-defined float “lavaAmount” to control the amount of magma being shown.
// sample both the rock and the magma diffuse tex float4 rockDiffuse = tex2D(s_IgneousRockTex, vert_in.UV0); // build a lerp value between the igneous tex and the magma tex. //int because we only want 1s and 0s. no floaty values. int lerpValue = clamp((rockDiffuse.a + lavaAmount + vert_in.vColor.r), 0, 1); // finally, build the diffuse texture. float4 diffuseTex = lerp(rockDiffuse, tex2D(s_magmaTex, ( vert_in.UV0 + -(wTime/80)) ), lerpValue);
The normal mapping was equally icky-gross.
// sample and average the normal map and panning map. float3 magmaNormal = (tex2D(s_MagmaNormalMap_surface, (vert_in.UV0 * 0.2 + -(wTime / 80)) ) * 0.5f ) + (tex2D(s_MagmaNormalMap_panning, vert_in.UV0 * 1.2 + (wTime/ 100)) * 0.5f ); float3 rockNormal = tex2D(s_IgneousRockNormal, vert_in.UV0); // If we're on the igneous level, use igneous. else, use magma. float3 nrml = lerpValue < 1 ? rockNormal : magmaNormal; // finally, move that vector to world space. curNormal = mul(nrml * 2 - 1, tanToWorldSpace);
Now with two passes, Igneous rock layer is simply alpha’d out by our previously-defined lerp value. This layer is the second pass, which is drawn over the magma layer.
// sample our igneous rock texture. float4 rockDiffuse = tex2D(s_IgneousRockTex, vert_in.UV0); // build our lerp values. The 1 minus is simply to make semantics of the UI match the value. int lerpValue = (1 - lavaAmount + rockDiffuse.a); // make our 1 -> 0 and our 0 -> 1 lerpValue = !lerpValue; // and then, simply set our values. output_color.rgb = rockDiffuse; output_color.a = lerpValue;
Here’s a video of my magma layer. It incorperates the averaging of the two normals, which are then used to distort the UV channel. This is a first-pass of this effect.
Here’s the code for the distortion and the normal map. After sampling our normal map the code distorts our UV input and then re-samples the normal map to ensure that the normal map warps with the diffuse map.
// build the texture of our normal maps; curNormal = tex2D(s_MagmaNormalMap_surface, vert_in.UV0); curNormal += tex2D(s_MagmaNormalMap_panning, vert_in.UV0 * 1.2 + (wTime/ 100)); //Distort our UVs. the scalar 0.1 is scale value for the size of the distortion. vert_in.UV0 += curNormal.xz * 0.1; //Now re-sample the normal map with distorted UVs to get a normal from the new location; float3 magmaNormal = tex2D(s_MagmaNormalMap_surface, vert_in.UV0); magmaNormal += tex2D(s_MagmaNormalMap_panning, vert_in.UV0 * 1.2 + (wTime/ 100)); //Because we added two normal maps together, divide by two to get the mean of the normals. (average) magmaNormal /= 2.0f; curNormal = mul(magmaNormal * 2 - 1, tanToWorldSpace);
This shader is coming along quite nicely, and the split to two passes makes it quite readable, unlike before.
Recently, I’ve gotten my base Blinn/Phong shader to a state where i’m relatively happy with it. It’s posted to my portfolio on the Tech Art Page. I plan on using this shader in the future as the base for any material-related shaders. For example, today I wrote a very fast ramp shader mod for the blinnphong. This took me little under 30 minutes to implement, and the results are pretty nice!
Also, using distance attenuation on your shader causes your light brightness to increase as it gets closer to the surface of your object. As this lambertian term of brightness is also used as lookup on the ramp texture, you get a neat effect if you don’t clamp it between 1 and 0. Look!:
The ramp texture loops around as the “U” value increases beyond 1, and wraps to the front of the texture. This causes the banding pattern shown in the above video. This issue is fixed by setting your sampler_state AddressU and AddressV to ‘clamp’.
I love this stuff!
Enemcee.com is the portfolio of Nicholas Morgan Covington, a tech artist within the game industry. This blog is a simple place for him to post and show off his daily technical musings
Where does the word enemcee come from? It's a phonetic (probably incorrect) spelling of the writer's initials - N (en) M (em) C (cee). Clever, right? Thought so.