Instantiation Via (Poser) Geometry Swapping ?

[Ken1171]

At least for this specific case. That doesn't seem to be much of a reduction to me.

Thanks for running the test. I would do it exactly as you did, using Task Manager - except for 1 little thing. The thing about instancing is that it might only account for geometry. If you load (or duplicate) the same textured figure, there is a good chance that textures would bog out the RAM more than the geometry. To put this to test, it would be interesting to do the test again without any textures. I just did that, and here are the results:

* 323MB empty scene
* 520MB Loaded Dawn, no textures, no morphs +217MB
* 540MB Duplicated Dawn (2 figures on scene) +20MB <-- Aha! Instancing!
* 574MB Duplicated Dawn again (3 figures on scene) +34MB <-- Instancing!
* 606MB Dusplicated Dawn again (4 figures on scene) +30MB <-- Instancing!

Run this test and see if you get the same thing. I did it in Poser 11.2 here. Might be different in older versions. Remember - instancing is about geometry, not textures. Not sure about morphs, though.

 

[KageRyu]

@kobaltkween , I would love an Instance/Array manager - especially with all of the hours of work I put into some of my more complex scenes (like this ruined city flyover I have been working on for months).
Sadly, and it is one of the reasons I have become more limited active in most forums, there are a lot out in the community that attack any idea thay personally do not have a use for. I even saw it in my time at RDNA in the private vendor forums (not as bad as at some forums...but still present).

Whether it's proper instancing or not on such a tool would be less of a concern to me as it's ability to manipulate arrays and groups of objects , figures, materials, etc... If it allowed for the figures in an array to be independantly posed it would be even more useful (for crowds, flocks, and swarms), even if all of the figures needed to share posing with a little creative setup of multiple arrays it could still cut down on set up and animation time of crowds, groups, flocks, and swarms.

 

[3dcheapskate]

Ok, deleted all textures, used snarly's sceneFixer to remove all detached nodes, saved scene (1 figure), restarted Poser:

PP2014
* 268MB empty scene
* 756MB One figure (loaded saved scene) (+488MB)
* 1193MB Object>Duplicate - 2 figures in scene (+437MB)
* 1616MB Object>Duplicate - 3 figures in scene (+423MB)
2nd/3rd figures appear to each add around 85% of the memory usage of the first

Tried the same in Poser 11
* 266MB empty scene
* 738MB One figure (loaded saved scene) (+472)
* 1181MB Object>Duplicate - 2 figures in scene (+443MB)
* 1623MB Object>Duplicate - 3 figures in scene (+442MB)
2nd/3rd figures appear to each add around 90% of the memory usage of the first

My results appear to show the opposite to yours ?

Here's the triplets. I believe I used 'Spawn Full Body Morph' on the morphed V4, and then for each item of clothing used 'Copy Morphs (from V4)' to copy just that morph and apply it.

 

[Ken1171]

My results appear to show the opposite to yours ?

Maybe the major difference is that you have used morphs, and I haven't, so the test wasn't the same. As far as I know, instancing is about geometry, not textures and morphs. In programs like iClone, we can even bake all morphs into the mesh, freeing resources and making instancing more memory efficient. In Poser, we can't bake anything, but we can use the Grouping Tool to convert the figure, clothing, hair, and props into mesh prop, which would preserve the shape by freezing all the morphs, leaving only geometry. Another way would be to export the complete figure with hair and everything to OBJ, and load that instead. However, the easiest way would be to simply load everything without morphs for the sake of this test.

 

[3dcheapskate]

Very probably - after all you did say "...Not sure about morphs, though ..."

So I decided to try your 'export the full figure (no texture maps) as an OBJ' approach to take the morphs out of the equation.
The OBJ file was 52.7MB, and the results I got are not encouraging:

PP2014
* 285MB empty scene
* 505MB One figure (lOBJ imported) (+220MB)
* 697MB Edit>Duplicate - 2 figures in scene (+192MB)
* 889MB Edit>Duplicate - 3 figures in scene (+192MB)
2nd/3rd figures appear to each add around 87% of the memory usage of the first

Poser 11
* 318MB empty scene
* 568MB One figure (lOBJ imported) (+250MB)
* 766MB Edit>Duplicate - 2 figures in scene (+198MB)
* 965MB Edit>Duplicate - 3 figures in scene (+199MB)
2nd/3rd figures appear to each add around 79% of the memory usage of the first

 

[Ken1171]

That's strange - how come we get such different results in Poser 11? If you have Dawn, you could simplify the tests by loading her CR2, because she doesn't have any morphs by default, not even JCMs. She loads 100% blank, which is awesome for content creators. Mine loads with textures, so I have reset her materials to remove all texture maps, and then move the camera around to settle memory usage. Over here it settles at around 323MB in a blank scene with only bare Dawn loaded with no textures.

After this, all I did was to use Object --> Duplicate a few times until I end up with 4 copies on stage, keeping an eye on Task Manager to see the memory load changes. In my tests, every duplication adds only around 30MB, instead of 217MB. That's 13~21 times less memory (in average) than when I loaded the figure.

I have ran the test again in Poser 11.3, doing like described above using DawnSE.cr2. Here are the new results:

429MB <- Loaded the DawnSE.cr2 and deleted all textures +429MB
449MB <- Object -> Dusplicate +20MB <<-- Instancing!!
481MB <- Object -> Dusplicate +32MB <<-- Instancing!!
518MB <- Object -> Dusplicate +37MB <<-- Instancing!!

These are more or less in the same ballpark as before. It took 21X less memory usage to duplicate DawnSE compared to the first time she was loaded. I have also extended the test by loading a new DawnSE.cr2 from the library, complete with textures (no morphs) to see if it makes a difference. Here is the result.

594MB <- Loaded a new DawnSE.cr2, unchanged with textures (no morphs). +76MB <<-- Instancing!!
568MB <- Deleted all textures from it -26MB, which means +50MB for this new copy with no textures. That's 8X less memory than loading Dawn the first time.

The impact on memory load seems a little more visible when we load a CR2 instead of duplicating existing geometry already on scene. It took 50MB instead of 20-30MB from duplication. And yet, that still seems FAR from how much memory it took to load it the first time (8+ times less).

For the record, I am only observing the Poser.exe task, ignoring the multiple CEF tasks because they are only used to communicate with the HTML5 library. From this new test, duplicating the figure seems to save me about 30MB comparing to loading from the library. It seems like instancing in Poser is more optimized for duplication than from loading from the library. This is new info comparing to my last test.

BTW, I am using Windows 10 Pro x64 auto-updated to the latest version. My results confirm that Poser indeed supports instancing. In my scene, I even made the ground invisible to reduce clutter, and I have a small cube at the center that I use as a light target. There is nothing else on scene, and all 3 lights are spotlights (no large HDRI maps loaded).

I can't say why you got different results, but I was able to replicate the experiment and the results look consistent. Duplicating the first Dawn used 21 times less memory, and that's what instancing is all about.

 

[Karina]

It looks like this issue is getting weirder and weirder:

I loaded *one* standard SASHA-16 with her standard set of morphs plus the Morphs++, and the full set of textures (most are 4000 x 4000 px).
So it's the full figure and not just an empty dev rig.

Task manager tab is "Performance: RAM" which gives you an idea of how much RAM is actually used by the Poser application with all it's bells & whistles:
Memory useage:
5,1 GB

Then I loader another one, straight from the library:
Memory useage:
5.2 GB

So I continued loading more SASHAs.
I stopped after 10 (ten!) copies.
Memory useage:
6,3 GB

Outsourced momory: --> 850 MB constantly, during the whole tests.

Next I looked at the Task Manager's "Process View" tab figures of RAM useage, which should show me the memory useage of the preview port:

1 SASHA: 1.309 MB
2 SASHAs: 1.393,2 MB
10 SASHAs: 2.588,8 MB

Remember that all SASHAs were loaded fully textured and *with* all morphs, directly from the library, by simply clicking in more of her; no duplication was applied!

So there must be definitely going on something under the bonnet of Poser which we don't know about yet.

Poser version is 11.07.339 (yeah, I know...)

Cheers!

K

 

[Ken1171]

It looks like this issue is getting weirder and weirder:

Hey thank you for taking your time and doing the test. I am positive Poser is instancing models when loaded and also when duplicated. In my test from above, duplications seem more memory efficient than loading files, but the memory savings are still significant in either cases. In your test, we can see the instancing is kicking in as well, but when it comes to textures and morphs, I believe it's something else doing it's job under the hood.

I believe it was around version 6 or 7 that Poser finally got a memory manager. I can't remember if it was eFrontier or SMS who did it, but I do know for a fact that before that, loading ANYTHING would instantly ADD to the memory load, until you ran out of it and Poser would crash. Instancing will only handle geometry as far as I know. I believe textures and morphs are handled by the memory manager to avoid duplications that used to crash Poser in the past.

I remember SMS claiming that Poser will NEVER load an entire texture at full resolution. It will instead downscale it for previews, and use buckets during renders to only load a small piece of a texture at any given time. This is what makes it possible to run Poser in a laptop with as little as 2GB RAM. When the memory manager was added, Poser can now handle multiple copies of the same textures and morphs more efficiently, while instancing handles geometry duplication. As far as I know, geometry instancing was only introduced very recently. The memory manager is older, but not much.

 

[Karina]

You may be right here, because I started off with Poser(7) Pro, and the largest scene I did was with 7 iterations of V4, one M4, and a rather large scene with a lot of props and textures.
Each of the V4s had a different body texture.
While other users said you can't have more than 3 or 4 figures in your scene without Poser choking.

I was running XP64 at that time, with 8 GB of RAM and a (I guess? 1.2 GHZ Athlon processor).

So I assume that it's true that Poser since P6 only loads textures in smaller chunks which it can digest, both in preview and during render (though you almost couldn't work in the preview unless you switched to "Hidden Line" mode - and even then it was still awfully sluggish.
Making some not currently needed figures and props invisible finally did the trick.

Since then I'm into "modular" sets, where you can turn off not-needed elements off /on with a single click, rather that like in the olden days, an "all or nothing" option.

I'll run some more tests tomorrow if i have the time (currently busy with my own current project and a bunch of Python scripts).

Especially I want to find out how that old, *very" old scene will use RAM.
If I can still find it... and rebuild it that is...

But tonight it's dozy-dozy time for me. See you tomorrow!

Take care

K

 

[3dcheapskate]

My Dawn has textures too, so I manually deleted all nodes from all her Mat zones and saved as a new CR2.
Restarted Poser 11(.2.272)
(Note*: the first figure was what it first appeared to stabilize at, the second lower figure was what it seemed to restabilize at a bit later. Sowhich stable figure do I use?)
* 320-290MB Deleted La Femme so I have an empty scene
* 375-360MB One figure (loaded my untextured Dawn CR2) (+40 to 85MB ... probably +55 to +70)
* 390-375MB Edit>Duplicate - 2 Dawn figures in scene (+0 to 30MB ... probably +15MB)
* 420-410MB Edit>Duplicate - 3 Dawn figures in scene (+20 to 45MB ... probably +30 to+35)

*This time I looked at the numbers in task manager over a longer period (I was making a cup of tea and doing odd jobs around the house) and realized that they seemed to stabilize at a couple of different values. I definitely wouldn't draw any definite conclusions from the results I'm getting. All I'd be prepared to say is that the 2nd and 3rd figure seem to add slightly less memory use than the first figure does.

 

[phdubrov]

@Ken1171 what I don't get is why so much memory is consumed on first Dawn.
My numbers are more in line with @3dcheapskate
Win7, Poser 11.3
My numbers from ProcessExplorer, PrivateBytes - Working set in megabytes, and actions taken
397 416 1LF (default scene)
538 556 2LF
679 695 3LF
287 297 Empty (construct only)
332 351 1D SE (+45M not 400M!)
354 378 +D from library (+22M)
401 428 both to unimesh (+47M, 23M on 1)
449 479 Duplicate D1 (unimeshed)
482 513 +D from library (+33M)
508 538 4D to unimesh (+26M)
629 658 SL1 4D

 

[Ken1171]

@3dcheapskate I guess we are getting somewhere.

@phdubrov Good point, I have no idea. Maybe I didn't wait long enough for Poser memory load to stabilize?

I suppose what we wanted to check here was if adding another copy of the same model would duplicate the memory load, and by now we can safely claim it doesn't, right? However, in theory, instancing a 2nd (identical) copy of a model shouldn't take his much memory, but we don't know how SMS had it implemented.

Nonetheless, I am from a time where loading a second copy would indeed duplicate the memory load, and that was often 1 more step towards running out of RAM - and crashing Poser. Those days are done now.

 

[3dcheapskate]

Stepping away from the memory consumption for a moment, here's a couple more alternative geometries for the figure posted in the OP. Simply replace the existing 'GeomSwapAlt01.obj' and 'GeomSwapAlt02.obj' with these.

Here's what the OBJs look like:

And here's how they look when used as AltGeoms with the geometry swapping figure in the OP

Texture images and grass shader screenshot attached. OBJs and MC6 in the attached zip

 

[Ken1171]

Looking great! My only suggestion would be to use the Diffuse channel instead of Alternate Diffuse for when you want to render with Superfly.

 

[3dcheapskate]

I use PP2014 (even though I have Poser 11) because I have an aversion to new-fangled !

There was a thread (with lots of input from bagginsbill) where he discovered what you need to do to get translucence working correctly with point/spot/infinite lights (it simply doesn't work with IBL). The key was to use two diffuse nodes with normals forward and duplicate image maps. That's the only reason for using Alt Diffuse in the shader.

Unfortunately I can't find the thread at present. RDNA and/or SM forum I think.

 

[3dcheapskate]

By the way, the Clump1 OBJ has 704 vertices/176 faces and the Clump 2 OBJ has 576v/176f so 38,016 vertices/9,504 faces max for all the grass.

When this is complete I think I'll call it "Who Cares If It's Instancing ?"

 

[Ken1171]

I believe we don't need to fight translucency when rendering with PBR in Superfly. AO is a simpler (and quicker) way to fake GI shadows in Firefly, but with PBR we get the real thing - no need for jumping hoops to fake anything. Since Superfly became available, I have been finding it much simpler to create shaders exactly for that reason. With Firefly we had to fake and simulate lots of things, making shaders more complex than they need to be nowadays.

By the way, the Clump1 OBJ has 704 vertices/176 faces and the Clump 2 OBJ has 576v/176f so 38,016 vertices/9,504 faces max for all the grass.

If you render in Poser with Octane, it comes with built-in tools for instancing and scattering. I remember creating a Poser animation with it where I have populated a whole forest with just 1 tree. In another scene (done in Vue) I have put 10 thousand bunnies on a terrain with color, size, and pose variations. In memory it only needed space for a single one.

Instancing is awesome, and it starts with mesh like you are making here! ^____^

 

[Ken1171]

A bit on the Poser Python side, is there an "easy" way to copy the materials from one prop to another? Or do we have to scan for every node and replicate them on the other prop by hand? I am trying to duplicate a prop in code, and now it's only missing the materials.

 

[phdubrov]

SaveMaterialSet - LoadMaterialSet

actor SaveMaterialCollection LoadMaterialCollection

 

[Ken1171]

Oh, so we can only open/save files from disk (the library)? There is no way to assign a shader node to another? I mean, when we use Object -> Duplicate, it doesn't ask us for a material preset. It duplicates the one from the object itself.