GDC 2023 VFX Roundtable Dialogue Notes

Thanks to your endurance as we’ve compiled all of those notes, and thanks immensely to all of you unimaginable VFX artists (and extra!) that had been capable of be a part of us at GDC this previous 12 months. Right here’s our greatest try at capturing notes on the discussions hosted.

The VFX Roundtables at GDC annually are one thing each really particular, and barely disappointing – they’re the one largest gathering of real-time visible results expertise that I’ve ever seen, bringing with them an pleasure, vitality, and the shared overwhelm of an ever altering and increasing business. On the similar time, there’s merely no solution to seize the depth and number of the conversations that I WISH might occur amongst all of us.

For GDC 2023, I had absolutely the pleasure and superb alternative to co-host with my good pal Jason Keyser (VFX Apprentice). Beneath are our collective greatest efforts to record notes from the discussions.

The Official GDC Itemizing of the Visible Results Roundtable:

The roundtable hosts for the week

The spherical desk discussions are damaged into three days, every with a barely completely different focus: Normal, Inventive, Technical. Every day started with a fast introduction of the subject, a reminder of the worth of our shared neighborhood and participation on realtimevfx.com, our discord, twitter, and so forth, and most significantly for the hour – a floor rule to attempt to preserve every subject to about 5 minutes, and permit everybody an opportunity to contribute and be heard.

Halfway via the session (as soon as individuals loosen up a bit), we might cease and ask for arms raised on three questions (I beloved this, so props to Jason or whomever he had this concept from) – Elevate your hand in case you’re a scholar or simply getting began in your profession in VFX. Elevate your hand in case you’re searching for work as a person or as a enterprise. Elevate your hand in case you’re searching for VFX Artist assist to your undertaking. Now be sure you all meet one another as soon as we finish the session in the present day. Apparently, it felt just like the room was break up roughly into thirds every day.

Normal matters and revisiting floor damaged within the VFX Summit earlier within the week.

Sadly, flight delays and an early name time mixed to stop me (Keith) from with the ability to make today. Happily, Jason Keyser is an unimaginable host and Sandra Femenía kindly took notes. First bullets had been matters / questions requested to the group, sub-bullets are the solutions.

• Matter: Being extroverted whereas working from house
  • Stream on Twitch
  • Be concerned with the net communities
• AI in our Artwork?
  • Just a few individuals stated they’re utilizing AI Extensively
  • Others are utilizing it or seeing it used as reference solely, not a last product
• VFX Jargon
  • Alpha clipping, and so forth having completely different meanings/makes use of from UE to Unity
  • Our particular jargon inflicting challenges with speaking suggestions from misuse or absence of particular key phrases
  • Is there a VFX Dictionary someplace?
    • We should always create a working group to doc /host one thing like this on Realtimevfx.com
  • Examples of variations in language between engines:
    • COD Engine makes use of the title “Runner” for particles that observe different VFX Knowledge. Bungie calls them Mother or father-Baby particles
    • Keith’s addition whereas writing aggregating these notes: A Shader in Unity = Materials in Unreal, whereas a Materials in Unity = a Materials Occasion in Unreal
• Graph Editors?
  • Don’t let artists use graphs as a result of optimization considerations
  • Let artists do stuff on graphs after which have a TA optimize it
  • It relies upon
  • Materials Capabilities are highly effective pals to artists
  • Shader graphs are essential
  • Bungie offers whole freedom for artists and helps them with good tooling and profiling
  • COD doesn’t have a shader editor as a result of they concentrate on an extremely organized manufacturing pipeline with reusability and practical VFX.
• Houdini in Pipeline
  • Frostbite Engine (as used on Battlefield), makes use of houdini A LOT – mesh particles, pyro sims, knowledge transfers, and so forth.
  • Houdini permits customers to create no matter knowledge you want.
  • Houdini is extraordinarily essential, our artists must comprehend it
  • We kinda began utilizing it a number of years in the past, however everybody was hesitant as a result of artists had been intimidated by the technicality of it
  • It’s so essential to construct seamless pipelines experiences between your engine and houdini
  • It permits for procedural approaches, which may scale very properly into manufacturing
• Hiring Portfolios – what’s essential to display?
  • Delicate abilities, like communication and demanding considering are so essential
  • I wish to see curiosity
  • Inventive vs. Technical – present us your artistry first, and your technical capabilities second
    • Additionally preserve optimization/efficiency in thoughts
  • Readability and narrative – we would like one thing that’s clear on display.
• How a lot do you utilize scratchpad in Unreal’s Niagara?
  • I exploit scratchpad with a purpose to save time once I discover myself repeating the identical performance/setup many times
  • An instance of a module I created was to entry g-buffer to erode particles based mostly on display coloration
• Unit testing for VFX
  • Largely take a look at by hand. In Infinity Battle, that they had unit testing with a field to run via all of the map doing a warmth map of framerate, however most locations don’t have an infrastructure like that
  • Some setups document GPU value over to per construct to construct graphs that you could evaluate, however the techniques broke so much and infrequently weren’t usable.
  • Instance was given of making dynamic techniques, akin to Customized LODs relying on the quantity of barrels exploding inside a radius round one another.
• When within the pipeline do you normally begin making VFX optimization and efficiency benchmarking?
  • I counsel to do it on the finish so VFX artists can nonetheless work as an alternative of expending 3 days attempting to optimize issues that would not be shipped on the finish
• Lightning spherical:
  • You should utilize scratchpad to animate electrical energy in such a method that it dynamically interacts with the world round it
  • Documenting your undertaking’s form language in Notion, Milanote or Obsidian to your workforce to test every now and then might prevent numerous iteration/suggestions time throughout manufacturing
  • You should utilize scratchpad to make smoke that rolls outwards
  • Within the tech artist roundtable somebody talked about that they used Houdini for directional harmonics
  • For deformation techniques, you’ll be able to pack data within the UV channels with Houdini
  • In case you compile your shaders with out having Unreal centered, the compilation will run quicker since your system gained’t be rendering Unreal on the similar time

VFX via an Inventive Lens, specializing in attendee questions concerning the creative elements of VFX. This may increasingly embody artwork training (as college students or veterans), artwork path, and extra.

For today, I used to be capable of sit again, take notes, and take part as an viewers member watching Jason work his magic.

Day 2: The Inventive Lens

• For people who use simulations, what are the very best practices that you just’ve discovered through the years whereas artwork directing them?

  • It depends upon what software program you’re utilizing. Artwork administrators will deliver requests to us, and we’ve got to determine how you can translate and iterate’
    • Non harmful pipelines are crucial
    • Are you able to construct in controls whilst you’re initially organising, so that you don’t put your self right into a gap
    • Think about exhibiting a “grid” of choices
  • Begin very coarse/broad, with as low decision and quick of a simulation as attainable – iterate utterly via implementation within the impact in recreation so you’ll be able to perceive what’s working, then progressively add element (and simulation time) to your simulation.
  • Vector Discipline Painter – VR App, permits the person to face inside the form and paint vectors round them and see the house

• What different cool instruments have you ever guys discovered?

  • Substance Designer has turn into very rewarding and helpful
  • SyncSketch – very helpful for opinions
  • klash.studio – we’ve got an area sync sketch competitor right here that claims his software is value as nicely!
  • ToonSquid on Ipad for body by body animation
  • Toon Increase Concord
  • Krita
  • Flip a clip
  • Procreate
  • Tahoma2D/OpenToonz
  • Aseprite
  • ProtoPie
  • Goodnotes has an awesome form software for notes and diagrams on Ipad
  • Obsidian for reference / video boards, collaboration

• How do you identify visible language for phrases for Artwork Directed VFX / Steering – “make it extra magical, much less fantasy is, extra stylized, and so forth”. What vocabulary will we use?

  • Rather a lot has to do with Form language and readability – minor paint overs are lifesavers
  • Having a method information is totally essential so you’ll be able to reference it as a supply of fact – set these guidelines forward of time and replace them when your improvement workforce grows and defines new issues
  • Draw overs are essentially the most useful solution to translate from the spoken work to visible language, if it’s a nonetheless body. Movement is more durable
  • Movement requires individuals to additionally specify the place in time are the factors of suggestions
  • Having a very good workforce lead that works intently between the artwork director and the workforce to iterate and discover the great
  • Documentation / Notes are very useful to permit the whole time to reference collectively – viewers recommended Notion, Milanote and Obsidian

• In case your sources are unfold skinny, is it higher to place extra effort into larger results and unfold skinny, or get protection?

  • It depends upon the undertaking and desires
  • In case you’re in class, and concerned about VFX, it is best to be sure you concentrate on constructing an awesome presentation to your work, quite than spreading for protection
  • It’s value understanding why the consequences are wanted – typically you want the consequences to make moments really feel good, which may usually be achieved with no need a ton of polish
  • Think about trying on the market for property to cowl the simpler and fewer essential issues so you’ll be able to concentrate on the issues that matter essentially the most to your manufacturing
  • Ensure you put within the time to get the connections and gameplay logic in place – that’s usually the place an enormous portion of labor comes from, and good implementation permits you to work extra effectively.

• How do you nail timing in your impact? What about simulated results?

  • Think about the musical beat – make mouth noises and listen to the timing?
  • Timing is all the time contextual – Block within the impact so you’ll be able to see it in context and perceive what’s going to work earlier than you add particulars, particularly with simulations
  • Simulations are crucial to dam within the timing earlier than you add particulars, as a result of it turns into a lot more durable to change
  • Two methods for simulation flip books that will let you retime
    • Optical Stream / Movement Stream / UV Stream that blends between frames of your flip ebook
    • Steady Diffusion is absolutely good at including frames to a flip ebook in case you didn’t get sufficient frames and don’t wish to return to render
  • This 12 months’s VFX Summit centered fairly closely on this topic, so everybody that may entry the vault ought to test them out!
  • Think about taking part in with the timing a body render of your impact – it’s a lot simpler so as to add a body or delete a body to rapidly experiment with timing

• Fast intermission: Elevate your hand in case you’re a scholar or simply getting began in your profession in VFX. Elevate your hand in case you’re searching for work as a person or as a enterprise. Elevate your hand in case you’re searching for VFX artists? The room was roughly third

• When have you learnt in case your impact is completed?

  • “Your impact isn’t carried out, it’s taken away from you”
  • Once I begin noodling on one thing randomly, I step away, after which I can come again later and see it with contemporary eyes.
    • How do you outline noodling?
      • Once I’m not getting a lot affect from what I’m doing, it’s in all probability a good suggestion to step away, shelve it till tomorrow, and so forth.
  • It’s essential to have a very good lead or director to assist make sure that there’s a strong definition of success to really feel an achievement for engaging in the aim and preserve everybody on observe. Maintain the vitality going to the place it’ll have essentially the most affect.
  • FX in video games get shipped when their 80% carried out or when the time runs out
  • We frequently have a lot work to do, so we’ve got to get intimately aware of the phrase “Ok”

• How do you overcome the sensation of not liking your work when everybody else does?

  • If everybody else is saying your work is nice – consider them! You’re so near it, you usually can’t see it.

• How do you encourage everybody or your self to do the sketching section?

  • Have a sketchbook only for the weather/VFX, so there’s a focus when in that ebook to concentrate on VFX issues.
  • In case you can’t discover the time to sketch each impact, taking the time to seek out numerous reference will help fill a few of that want
  • Pinterest, in case you can navigate it, is usually a excellent spot to dive into inspiration
  • Sketching is usually a very helpful apply for exploring and defining the timing, locking within the really feel of the impact earlier than getting misplaced within the simulation
  • You may solely save time whilst you nonetheless have it.
  • If fighting the “I Ought to” simply power your self to place one thing down within the editor and power it to be a “want” – that’s normally the kick wanted to
  • Extremely counsel Elemental Magic books

• How do you’re employed with oversaturated environments the place the lighting is blown out and all the things is vibrant?

  • Think about backing your particle with a reproduction that has a “grey” worth to offer a impartial stage that you could management.
  • Ensure you take a look at your results in a wide range of lighting environments, or no less than within the last lighting context

• What in your workflows creates time and causes the most important complications?

  • Designers – ensuring your artwork path is aligned with the design imaginative and prescient to make sure that you don’t should throw away your work
  • The most important waste of time is constructing an iteration loop to really see your results – guaranteeing that design helps create workflows whilst you’re working collectively is essential
  • Testing for community video games – this looks as if such a ache level – any suggestions?
    • Everybody acknowledges that it is a explicit fragile situation that always causes very unhealthy slowdowns
    • Recording video captures to observe your impact a number of occasions
    • Render Doc is a really great tool to seize complete play throughs and with the intention to discover bulk points
    • Ask your developer workforce to assist assist
    • Can the engineers assist setup take a look at scenes with bots to breed your eventualities for a really quick iteration course of
    • Check gyms for the win!
  • Lightning Spherical –
    • Object based mostly gradients by subtracting world place by native place and utilizing it as a UV
    • There’s a YouTube tutorial for how you can make a holographic show for anybody “Unreal Niagara 3d Hologram Impact”
    • Exploring low cost hacks to create 3d Noises in shaders
    • Use a really vibrant ugly neon inexperienced on property that want cleanup later
    • Relaxed Cone Mapping in Niagara speak – “Approaching Technical Artwork Methods Unconventionally Utilizing Unreal Engine’s Niagara”
    • At all times use Eye Adaption to bear in mind publicity
    • WASDQ in Unreal Materials grid aligns your nodes!
      • Align high: Shift + W
      • Align center: Alt + Shift + W
      • Align backside: Shift + S
      • Align left: Shift + A
      • Align heart: Alt + Shift + S
      • Align proper: Shift + D
      • Straighten Connection(s): Q

  VFX manufacturing seen via a Technical Lens, guiding the dialog via matters akin to graph based mostly impact authoring, impact lighting and optimization, new mesh based mostly methods, VR/AR/XR, and extra.*

  Day 3 was my day to take level, and finally deal with the immense duty of tossing and catching the superior GDC microphone in entrance of a crowd.

  This throwable, squishy microphone is one among my favourite issues from GDC.

  • What new instruments are individuals utilizing of their manufacturing pipelines for real-time VFX?
    • Blender
    • Embergen
    • FluidNinja
    • Maya is taking part in with AI compression of VDB Sequences
  • Is anybody exploring absolutely 3d/volumetric approaches in real-time but?
    • Houdini and Embergen have improbable workflows for VDB exports, however utilizing them within the engine remains to be difficult – a number of plugins / groups are very near having nice options, so it’s close to!
    • VR remains to be difficult/inconceivable to do real-time volumetrics, so there’s nonetheless numerous faking
      • One workforce is monitoring and offsetting the depth from every eye intelligently within the shader to create the phantasm of depth in VR house
  • Does anybody else have any insights into “rear-projection” methods like this or different methods for VR?
    • Just a few individuals within the room are at present engaged on VR tasks
  • How do individuals handle expectations when administrators, managers, and so forth see offline-rendered content material that can’t work at scale in a real-time manufacturing? (For instance – seeing Unreal’s R&D posts about real-time fluids and administrators ask for it of their recreation)
    • We now have to be trustworthy and make them perceive that it’s simply nonetheless not attainable.
    • One individual’s artistic director requested to be proven how you can make VFX so he might know what’s and isn’t attainable, then that turned part of their tradition – that future leaders all needed to perceive sufficient to have context about their asks.
  • Lighting and environments are trying superb in recreation engines, however VFX are usually not fairly there but – that’s inflicting more durable conversations when level-setting expectations
    • Have to push again and handle expectations concerning the limits we nonetheless have with VFX
    • Gen 5 remains to be not capable of do all the things – many of the price range enhancements went to lighting
    • Having a graphics engineer round will help with these conversations, and discover methods to assist with the issues!
      • We might have concepts as VFX artists, however the engineer can present much more context concerning the limitations and causes.
  • Are individuals utilizing out of the field engines, customized engines, or engine extensions?
    • Recreation structure of Unity is superb as-is
    • Graphics developments are greatest carried out customized
    • What we see within the demos isn’t what you get instantly
    • It’s really easy to waste time on all of the cool issues, or attempting to maintain up with issues that don’t matter – it’s good to ask what’s important to your recreation particularly and focus your vitality there.
    • It depends upon the workforce size- some locations can’t spare the sources to create customized options to all of their issues.
    • Customized tech additionally has an maintenance value, particularly in Unity and Unreal – you’ll should spend time upkeeping each time you get a brand new engine model
    • A graphics engineer stated that of their improvement with an in-house engine, they wish to prototype in Unity first to know what is required, then implement the ultimate end in their very own engine.
      • They concentrate on making the tooling as environment friendly as attainable, then the artists are available in on the finish to finalize.
  • Once you see a flowery new factor as an organization, how do you determine whether or not you’ll pursue it?
    • If it’s an unknown, then take the low-hanging fruit to show an concept first
      • Are there efficiency points? Visible points?
      • If it’s third celebration software program, asking how usable it’s, how supported it’s, and so forth
      • Concentrate on the issue you’re attempting to resolve, not the software
    • One firm did month-to-month VFX challenges to get their artwork groups exploring and utilizing new instruments and methods in a secure house, then they may make educated selections for his or her manufacturing
      • May very well be mixed with different disciplines as nicely to make micro recreation jams
    • Instruments is usually a entice
      • Good instruments don’t substitute good artwork
    • Networking is an effective way to find what works for others and get new concepts
  • How do you cope with the fixed updating of software program and the seen frustrations, insecurities, and exhaustion of groups with new software program?
    • [At this point, I interrupted and asked for a show of hands – ‘We don’t talk about this enough, but since we’re all here in the room, I think it’s worth it to call it out: How many of you feel uncomfortable as artists on a regular basis?’ (or something like that) Everyone raised their hands.
    • An instructor for a neurodivergent school mentioned that he has trouble with students who like stability
      • He’s had to really lean into the idea that it’s more important to get the project done than to use the “correct” tool
    • Someone recommended the Tech Art Summit presentation from this year titled “Building a DCC and Project Agnostic Animation Pipeline” by Frieder Erdmann (https://schedule.gdconf.com/session/technical-artist-summit-building-a-dcc-and-project-agnostic-animation-pipeline/891909)
      • The takeaway – everyone has this problem.
    • Film industry has more diverse coverage of specialists within their VFX craft
      • It’s healthy to let people specialize
      • They can learn by other VFX artists next to them
    • Lots of people want to discover new things so they can do their job better
      • It’s just easy to lose focus on the part that matters – art over tools
  • What type of Profiling tools do people use?
    • Pix
    • Zone Console
    • Renderdoc
    • Niagara has great internal profiling tools
      • Does dumps and time graphics, spreadsheets, etc
    • Common things to watch out for / consider when you find frame rate hitches caused by VFX:
      • Overdraw
      • Too many live physics simulations
      • Too many VATS in memory at once (Large textures!)
      • Track down the likely suspects first and toggle them off to verify it’s them.
      • Build best practices along the way, so optimization doesn’t become such a big chore at the end.
  • Poll – How many people are authoring in Unity, Unreal, or Proprietary engines?
    • Pretty even spread throughout the audience
  • How are we solving for lighting on our particles in physically based render pipelines ?
    • Setup a series of rooms that go from the whole EV and lighting spectrum and make it easy to jump between them so you can see your effect in the full spread of lighting environments.
    • Name the particle with the use case intention (low/med/high) to control bloom
      • Work with lighting team to figure out what needs to be adjusted between each
    • Setting particular ranges of exposures for VFX vs. other aspects and authoring to fit within those ranges
    • It’s very challenging to retrofit these systems
  • Don’t forget to also consider color management – ACES is an industry standard for film and it’s starting to creep into games – someone mentioned using an ACES pipeline in Unreal with great results because there is still data in the highs and lows.
    • Someone suggested calibrating your monitors regularly
  • What issues are people having with Lumen in Unreal?
    • There’s no way to control the diffuse directional lighting intensity on your particles, so you’re stuck
    • Different light types create different values of shadows – watch out for this
  • Have any of you encountered positive use cases of AI for VFX? (this conversation immediately became philosophical first)
    • For the past several years, large companies have been betting on previously established IP to ensure profits, so smaller companies have had a difficult time producing enough content to compete – AI might be able to help with this
    • Ethics and jobs are big concerns for everyone
      • The hope is that people will be empowered to iterate faster and focus on better craft and end results
    • Indie devs will be able to do much bigger things than they could before
    • Creativity in how you use the tools will continue to be critical, regardless of the future
  • Are there any good workflows or tools using AI currently in use for VFX?
    • Substance designer gets powered up by AI
      • Realizes you want cleaner lines when warping
    • Stable Diffusion:
      • Take a texture and make it look like a reference image
      • Can it help fill in frames of a flipbook?
    • Generally cleaning up things you already do
    • A solo developer making a custom engine anecdotally described it helping him tremendously to speed up research for functions, etc.
  • Lightning Round: What new tools are you excited about?
    • Fluid Ninja
    • ActionVFX library is being converted to Unreal Engine
    • New material systems for Unreal using realistic lighting properties
    • Frostbite’s new volumetric renderers (new Deadspace uses it)
    • Alembic particles are now supported in Niagara
    • Niagara point caches
    • Using dither to avoid overdraw
    • Signed Distance Fields
    • Sparse Distance Fields
      • Allow to add comparative LOD distances to particles, which can be cheaper than using regular mesh distance fields
      • Depth Fade can be subbed out for distance fields
  • What are some tools in custom pipelines that are amazingly useful?
    • Profiling with filters to find different VFX in engine and suppress all but the one you’re looking at
    • Niagara debug tools
    • Screen recording A LOT
  • Has anyone explored Universal Scene Description (USD) file formats yet? It seems promising for collaboration.
    • There was a very good tech art talk about it this year
  • Maybe FBX is going away?

  As we wrapped up the final day of discussions, I was struck by a few conflicting thoughts. For our industry, I noticed a general theme across the entire conference that was focusing on tech artist solutions rather than engineering solutions. Before anyone throws their coffee at me, let me explain – when I began attending GDC, many of the most exciting innovations on display were paired with extremely programmer-centric presentations and discussions. This year, everything cool and exciting was paired with either 1) instructions on how to procure/install a 3rd party app that will ‘make all your dreams come true’™ or 2)guidance on how you, as a slightly technical artist, can create this solution yourself. I felt that this GDC was filled with artist empowerment.

  The conflict in this thought, however, came in the realization that we are well on our way to an industry where sub-specialization as artists needs to be embraced. We’re very likely to start seeing needs for a team dynamic include an artist that excels in ‘pre-engine’ fluid simulation pipelines, and an artist with houdini tool building expertise, and an artist that knows how to customize Fluid Ninja, another that can write Python integrations in Unreal, and a 5th that builds real-time shaders and blueprints. Each equally important, and exciting, but damn I couldn’t help but feel old and miss the ‘wild west’ days of trying to do it all without killing myself.

  Now, for the Roundtable itself, I had another pair of conflicting thoughts: On one hand, this was an incredible experience – so many talented, diverse artists and developers in one room joining for a shared conversation around our passions, and it was exciting to feel! On the other hand, I wanted more depth, or more useful takeaways out of these conversations. It’s entirely likely that this is just a limitation of the roundtable discussion format- there’s a probably valid argument that the roundtable is supposed to be paired with or spawn further, deeper conversations, but it on its own has to be broad.

  Ultimately though, GDC and our special VFX summit/round table events left me feeling so proud of the incredible community that shows up each year, or each day on our forum, discord, facebook, etc to share, inspire, and mentor. Thanks everyone for participating in these conversations with us.

  And a huge special thanks to:

  Jason Keyser, VFX Apprentice (@keyserito)
  Sandra Femenia, VFX Apprentice (@Saiel)
  Drew Skillman, (@Drew) the OG Roundtable Host who still hosts the prior notes on his website here: Visual Effects Artist Roundtables (2012-2019) — Drew Skillman

  Till next time!

  Keith