GDC 2023 VFX Roundtable Dialogue Notes

Thanks in your endurance as we’ve compiled all of those notes, and thanks immensely to all of you unimaginable VFX artists (and extra!) that have been capable of be part of us at GDC this previous yr. Right here’s our greatest try at capturing notes on the discussions hosted.

The VFX Roundtables at GDC every year are one thing each really particular, and barely disappointing – they’re the one largest gathering of real-time visible results expertise that I’ve ever seen, bringing with them an pleasure, vitality, and the shared overwhelm of an ever altering and increasing business. On the similar time, there’s merely no method to seize the depth and number of the conversations that I WISH might occur amongst all of us.

For GDC 2023, I had absolutely the pleasure and superb alternative to co-host with my good buddy Jason Keyser (VFX Apprentice). Under are our collective finest efforts to listing notes from the discussions.

The Official GDC Itemizing of the Visible Results Roundtable:

The roundtable hosts for the week

The spherical desk discussions are damaged into three days, every with a barely completely different focus: Basic, Creative, Technical. Every day started with a fast introduction of the subject, a reminder of the worth of our shared neighborhood and participation on realtimevfx.com, our discord, twitter, and so forth, and most significantly for the hour – a floor rule to attempt to maintain every matter to about 5 minutes, and permit everybody an opportunity to contribute and be heard.

Halfway by the session (as soon as folks loosen up a bit), we’d cease and ask for fingers raised on three questions (I beloved this, so props to Jason or whomever he had this concept from) – Increase your hand in the event you’re a pupil or simply getting began in your profession in VFX. Increase your hand in the event you’re searching for work as a person or as a enterprise. Increase your hand in the event you’re searching for VFX Artist help in your challenge. Now be sure to all meet one another as soon as we finish the session as we speak. Apparently, it felt just like the room was cut up roughly into thirds every day.

Basic matters and revisiting floor damaged within the VFX Summit earlier within the week.

Sadly, flight delays and an early name time mixed to stop me (Keith) from having the ability to make at the present time. Happily, Jason Keyser is an unimaginable host and Sandra Femenía kindly took notes. First bullets have been matters / questions requested to the group, sub-bullets are the solutions.

• Subject: Being extroverted whereas working from house
  • Stream on Twitch
  • Be concerned with the web communities
• AI in our Artwork?
  • A number of folks stated they’re utilizing AI Extensively
  • Others are utilizing it or seeing it used as reference solely, not a remaining product
• VFX Jargon
  • Alpha clipping, and so forth having completely different meanings/makes use of from UE to Unity
  • Our particular jargon inflicting challenges with speaking suggestions from misuse or absence of particular key phrases
  • Is there a VFX Dictionary someplace?
    • We must always create a working group to doc /host one thing like this on Realtimevfx.com
  • Examples of variations in language between engines:
    • COD Engine makes use of the identify “Runner” for particles that comply with different VFX Information. Bungie calls them Dad or mum-Baby particles
    • Keith’s addition whereas writing aggregating these notes: A Shader in Unity = Materials in Unreal, whereas a Materials in Unity = a Materials Occasion in Unreal
• Graph Editors?
  • Don’t let artists use graphs as a result of optimization issues
  • Let artists do stuff on graphs after which have a TA optimize it
  • It relies upon
  • Materials Features are highly effective associates to artists
  • Shader graphs are essential
  • Bungie offers whole freedom for artists and helps them with good tooling and profiling
  • COD doesn’t have a shader editor as a result of they concentrate on an extremely organized manufacturing pipeline with reusability and reasonable VFX.
• Houdini in Pipeline
  • Frostbite Engine (as used on Battlefield), makes use of houdini A LOT – mesh particles, pyro sims, knowledge transfers, and so forth.
  • Houdini permits customers to create no matter knowledge you want.
  • Houdini is extraordinarily essential, our artists have to understand it
  • We kinda began utilizing it a number of years in the past, however everybody was hesitant as a result of artists have been intimidated by the technicality of it
  • It’s so essential to construct seamless pipelines experiences between your engine and houdini
  • It permits for procedural approaches, which might scale very properly into manufacturing
• Hiring Portfolios – what’s essential to show?
  • Gentle expertise, like communication and significant considering are so essential
  • I wish to see curiosity
  • Creative vs. Technical – present us your artistry first, and your technical capabilities second
    • Additionally maintain optimization/efficiency in thoughts
  • Readability and narrative – we would like one thing that’s clear on display screen.
• How a lot do you employ scratchpad in Unreal’s Niagara?
  • I exploit scratchpad with the intention to save time after I discover myself repeating the identical performance/setup many times
  • An instance of a module I created was to entry g-buffer to erode particles primarily based on display screen shade
• Unit testing for VFX
  • Principally check by hand. In Infinity Struggle, that they had unit testing with a field to run by all of the map doing a warmth map of framerate, however most locations don’t have an infrastructure like that
  • Some setups report GPU value over to per construct to construct graphs that you would be able to examine, however the methods broke rather a lot and infrequently weren’t usable.
  • Instance was given of making dynamic methods, comparable to Customized LODs relying on the quantity of barrels exploding inside a radius round one another.
• When within the pipeline do you often begin making VFX optimization and efficiency benchmarking?
  • I counsel to do it on the finish so VFX artists can nonetheless work as a substitute of expending 3 days attempting to optimize issues that would not be shipped on the finish
• Lightning spherical:
  • You should use scratchpad to animate electrical energy in such a manner that it dynamically interacts with the world round it
  • Documenting your challenge’s form language in Notion, Milanote or Obsidian in your crew to test once in a while might prevent lots of iteration/suggestions time throughout manufacturing
  • You should use scratchpad to make smoke that rolls outwards
  • Within the tech artist roundtable somebody talked about that they used Houdini for directional harmonics
  • For deformation methods, you may pack data within the UV channels with Houdini
  • In the event you compile your shaders with out having Unreal centered, the compilation will run sooner since your system received’t be rendering Unreal on the similar time

VFX by an Creative Lens, specializing in attendee questions concerning the creative points of VFX. This may increasingly embody artwork training (as college students or veterans), artwork route, and extra.

For at the present time, I used to be capable of sit again, take notes, and take part as an viewers member watching Jason work his magic.

Day 2: The Creative Lens

• For those who use simulations, what are the most effective practices that you simply’ve discovered over time whereas artwork directing them?

  • It relies on what software program you’re utilizing. Artwork administrators will carry requests to us, and we have now to determine translate and iterate’
    • Non damaging pipelines are essential
    • Are you able to construct in controls whilst you’re initially establishing, so that you don’t put your self right into a gap
    • Think about exhibiting a “grid” of choices
  • Begin very coarse/broad, with as low decision and quick of a simulation as doable – iterate fully by implementation within the impact in recreation so you may perceive what’s working, then progressively add element (and simulation time) to your simulation.
  • Vector Subject Painter – VR App, permits the person to face inside the form and paint vectors round them and see the house

• What different cool instruments have you ever guys discovered?

  • Substance Designer has turn out to be very rewarding and helpful
  • SyncSketch – very helpful for evaluations
  • klash.studio – we have now an area sync sketch competitor right here that claims his software is price taking a look at as nicely!
  • ToonSquid on Ipad for body by body animation
  • Toon Increase Concord
  • Krita
  • Flip a clip
  • Procreate
  • Tahoma2D/OpenToonz
  • Aseprite
  • ProtoPie
  • Goodnotes has an ideal form software for notes and diagrams on Ipad
  • Obsidian for reference / video boards, collaboration

• How do you determine visible language for phrases for Artwork Directed VFX / Steerage – “make it extra magical, much less fantasy is, extra stylized, and so forth”. What vocabulary will we use?

  • Rather a lot has to do with Form language and readability – minor paint overs are lifesavers
  • Having a method information is completely essential so you may reference it as a supply of fact – set these guidelines forward of time and replace them when your growth crew grows and defines new issues
  • Draw overs are essentially the most useful method to translate from the spoken work to visible language, if it’s a nonetheless body. Movement is more durable
  • Movement requires folks to additionally specify the place in time are the factors of suggestions
  • Having a superb crew lead that works intently between the artwork director and the crew to iterate and discover the great
  • Documentation / Notes are very useful to permit your complete time to reference collectively – viewers instructed Notion, Milanote and Obsidian

• In case your sources are unfold skinny, is it higher to place extra effort into greater results and unfold skinny, or get protection?

  • It relies on the challenge and wishes
  • In the event you’re in class, and desirous about VFX, it’s best to be sure to concentrate on constructing an ideal presentation in your work, relatively than spreading for protection
  • It’s price understanding why the consequences are wanted – typically you want the consequences to make moments really feel good, which might typically be achieved with no need a ton of polish
  • Think about wanting on the market for belongings to cowl the simpler and fewer essential issues so you may concentrate on the issues that matter essentially the most in your manufacturing
  • Be sure to put within the time to get the connections and gameplay logic in place – that’s typically the place an enormous portion of labor comes from, and good implementation permits you to work extra effectively.

• How do you nail timing in your impact? What about simulated results?

  • Think about the musical beat – make mouth noises and listen to the timing?
  • Timing is all the time contextual – Block within the impact so you may see it in context and perceive what’s going to work earlier than you add particulars, particularly with simulations
  • Simulations are essential to dam within the timing earlier than you add particulars, as a result of it turns into a lot more durable to change
  • Two strategies for simulation flip books that mean you can retime
    • Optical Movement / Movement Movement / UV Movement that blends between frames of your flip guide
    • Secure Diffusion is absolutely good at including frames to a flip guide in the event you didn’t get sufficient frames and don’t wish to return to render
  • This yr’s VFX Summit centered fairly closely on this topic, so everybody that may entry the vault ought to test them out!
  • Think about enjoying with the timing taking a look at a body render of your impact – it’s a lot simpler so as to add a body or delete a body to shortly experiment with timing

• Fast intermission: Increase your hand in the event you’re a pupil or simply getting began in your profession in VFX. Increase your hand in the event you’re searching for work as a person or as a enterprise. Increase your hand in the event you’re searching for VFX artists? The room was roughly third

• When have you learnt in case your impact is finished?

  • “Your impact isn’t performed, it’s taken away from you”
  • Once I begin noodling on one thing randomly, I step away, after which I can come again later and see it with contemporary eyes.
    • How do you outline noodling?
      • Once I’m not getting a lot influence from what I’m doing, it’s most likely a good suggestion to step away, shelve it till tomorrow, and so forth.
  • It’s essential to have a superb lead or director to assist be certain there’s a stable definition of success to really feel an achievement for conducting the aim and maintain everybody on observe. Preserve the vitality going to the place it’ll have essentially the most influence.
  • FX in video games get shipped when their 80% performed or when the time runs out
  • We frequently have a lot work to do, so we have now to get intimately accustomed to the phrase “Adequate”

• How do you overcome the sensation of not liking your work when everybody else does?

  • If everybody else is saying your work is nice – consider them! You’re so near it, you typically can’t see it.

• How do you encourage everybody or your self to do the sketching part?

  • Have a sketchbook only for the weather/VFX, so there’s a focus when in that guide to concentrate on VFX issues.
  • In the event you can’t discover the time to sketch each impact, taking the time to seek out lots of reference might help fill a few of that want
  • Pinterest, in the event you can navigate it, generally is a excellent spot to dive into inspiration
  • Sketching generally is a very helpful observe for exploring and defining the timing, locking within the really feel of the impact earlier than getting misplaced within the simulation
  • You possibly can solely save time whilst you nonetheless have it.
  • If battling the “I Ought to” simply pressure your self to place one thing down within the editor and pressure it to be a “want” – that’s often the kick wanted to
  • Extremely counsel taking a look at Elemental Magic books

• How do you’re employed with oversaturated environments the place the lighting is blown out and every little thing is shiny?

  • Think about backing your particle with a reproduction that has a “grey” worth to present a impartial stage that you would be able to management.
  • Be sure to check your results in a wide range of lighting environments, or a minimum of within the remaining lighting context

• What in your workflows creates time and causes the most important complications?

  • Designers – ensuring your artwork route is aligned with the design imaginative and prescient to make sure that you don’t need to throw away your work
  • The most important waste of time is constructing an iteration loop to really see your results – making certain that design helps create workflows whilst you’re working collectively is essential
  • Testing for community video games – this looks like such a ache level – any ideas?
    • Everybody acknowledges that this can be a specific fragile situation that usually causes very dangerous slowdowns
    • Recording video captures to look at your impact a number of occasions
    • Render Doc is a really useful gizmo to seize complete play throughs and so to discover bulk points
    • Ask your developer crew to assist help
    • Can the engineers assist setup check scenes with bots to breed your eventualities for a really quick iteration course of
    • Check gyms for the win!
  • Lightning Spherical –
    • Object primarily based gradients by subtracting world place by native place and utilizing it as a UV
    • There’s a YouTube tutorial for make a holographic show for anybody “Unreal Niagara 3d Hologram Impact”
    • Exploring low cost hacks to create 3d Noises in shaders
    • Use a really shiny ugly neon inexperienced on belongings that want cleanup later
    • Relaxed Cone Mapping in Niagara discuss – “Approaching Technical Artwork Strategies Unconventionally Utilizing Unreal Engine’s Niagara”
    • All the time use Eye Adaption to consider publicity
    • WASDQ in Unreal Materials grid aligns your nodes!
      • Align prime: Shift + W
      • Align center: Alt + Shift + W
      • Align backside: Shift + S
      • Align left: Shift + A
      • Align middle: Alt + Shift + S
      • Align proper: Shift + D
      • Straighten Connection(s): Q

  VFX manufacturing seen by a Technical Lens, guiding the dialog by matters comparable to graph primarily based impact authoring, impact lighting and optimization, new mesh primarily based strategies, VR/AR/XR, and extra.*

  Day 3 was my day to take level, and finally deal with the immense duty of tossing and catching the superior GDC microphone in entrance of a crowd.

  This throwable, squishy microphone is one in every of my favourite issues from GDC.

  • What new instruments are folks utilizing of their manufacturing pipelines for real-time VFX?
    • Blender
    • Embergen
    • FluidNinja
    • Maya is enjoying with AI compression of VDB Sequences
  • Is anybody exploring totally 3d/volumetric approaches in real-time but?
    • Houdini and Embergen have implausible workflows for VDB exports, however utilizing them within the engine remains to be difficult – a number of plugins / groups are very near having nice options, so it’s close to!
    • VR remains to be difficult/unattainable to do real-time volumetrics, so there’s nonetheless lots of faking
      • One crew is monitoring and offsetting the depth from every eye intelligently within the shader to create the phantasm of depth in VR house
  • Does anybody else have any insights into “rear-projection” strategies like this or different strategies for VR?
    • Only some folks within the room are at present engaged on VR tasks
  • How do folks handle expectations when administrators, managers, and so forth see offline-rendered content material that can’t work at scale in a real-time manufacturing? (For instance – seeing Unreal’s R&D posts about real-time fluids and administrators ask for it of their recreation)
    • We’ve got to be sincere and make them perceive that it’s simply nonetheless not doable.
    • One particular person’s artistic director requested to be proven make VFX so he might know what’s and isn’t doable, then that grew to become part of their tradition – that future leaders all needed to perceive sufficient to have context about their asks.
  • Lighting and environments are wanting superb in recreation engines, however VFX will not be fairly there but – that’s inflicting more durable conversations when level-setting expectations
    • Have to push again and handle expectations concerning the limits we nonetheless have with VFX
    • Gen 5 remains to be not capable of do every little thing – many of the funds enhancements went to lighting
    • Having a graphics engineer round might help with these conversations, and discover methods to assist with the issues!
      • We could have concepts as VFX artists, however the engineer can present much more context concerning the limitations and causes.
  • Are folks utilizing out of the field engines, customized engines, or engine extensions?
    • Sport structure of Unity is superb as-is
    • Graphics developments are finest performed customized
    • What we see within the demos isn’t what you get immediately
    • It’s really easy to waste time on all of the cool issues, or attempting to maintain up with issues that don’t matter – you must ask what’s crucial to your recreation particularly and focus your vitality there.
    • It relies on the crew size- some locations can’t spare the sources to create customized options to all of their issues.
    • Customized tech additionally has an repairs value, particularly in Unity and Unreal – you’ll need to spend time upkeeping each time you get a brand new engine model
    • A graphics engineer stated that of their growth with an in-house engine, they wish to prototype in Unity first to know what is required, then implement the ultimate lead to their very own engine.
      • They concentrate on making the tooling as environment friendly as doable, then the artists are available on the finish to finalize.
  • Once you see a elaborate new factor as an organization, how do you determine whether or not you’ll pursue it?
    • If it’s an unknown, then take the low-hanging fruit to show an concept first
      • Are there efficiency points? Visible points?
      • If it’s third social gathering software program, asking how usable it’s, how supported it’s, and so forth
      • Concentrate on the issue you’re attempting to resolve, not the software
    • One firm did month-to-month VFX challenges to get their artwork groups exploring and utilizing new instruments and strategies in a protected house, then they might make educated choices for his or her manufacturing
      • May very well be mixed with different disciplines as nicely to make micro recreation jams
    • Instruments generally is a lure
      • Good instruments don’t change good artwork
    • Networking is an effective way to find what works for others and get new concepts
  • How do you cope with the fixed updating of software program and the seen frustrations, insecurities, and exhaustion of groups with new software program?
    • [At this point, I interrupted and asked for a show of hands – ‘We don’t talk about this enough, but since we’re all here in the room, I think it’s worth it to call it out: How many of you feel uncomfortable as artists on a regular basis?’ (or something like that) Everyone raised their hands.
    • An instructor for a neurodivergent school mentioned that he has trouble with students who like stability
      • He’s had to really lean into the idea that it’s more important to get the project done than to use the “correct” tool
    • Someone recommended the Tech Art Summit presentation from this year titled “Building a DCC and Project Agnostic Animation Pipeline” by Frieder Erdmann (https://schedule.gdconf.com/session/technical-artist-summit-building-a-dcc-and-project-agnostic-animation-pipeline/891909)
      • The takeaway – everyone has this problem.
    • Film industry has more diverse coverage of specialists within their VFX craft
      • It’s healthy to let people specialize
      • They can learn by other VFX artists next to them
    • Lots of people want to discover new things so they can do their job better
      • It’s just easy to lose focus on the part that matters – art over tools
  • What type of Profiling tools do people use?
    • Pix
    • Zone Console
    • Renderdoc
    • Niagara has great internal profiling tools
      • Does dumps and time graphics, spreadsheets, etc
    • Common things to watch out for / consider when you find frame rate hitches caused by VFX:
      • Overdraw
      • Too many live physics simulations
      • Too many VATS in memory at once (Large textures!)
      • Track down the likely suspects first and toggle them off to verify it’s them.
      • Build best practices along the way, so optimization doesn’t become such a big chore at the end.
  • Poll – How many people are authoring in Unity, Unreal, or Proprietary engines?
    • Pretty even spread throughout the audience
  • How are we solving for lighting on our particles in physically based render pipelines ?
    • Setup a series of rooms that go from the whole EV and lighting spectrum and make it easy to jump between them so you can see your effect in the full spread of lighting environments.
    • Name the particle with the use case intention (low/med/high) to control bloom
      • Work with lighting team to figure out what needs to be adjusted between each
    • Setting particular ranges of exposures for VFX vs. other aspects and authoring to fit within those ranges
    • It’s very challenging to retrofit these systems
  • Don’t forget to also consider color management – ACES is an industry standard for film and it’s starting to creep into games – someone mentioned using an ACES pipeline in Unreal with great results because there is still data in the highs and lows.
    • Someone suggested calibrating your monitors regularly
  • What issues are people having with Lumen in Unreal?
    • There’s no way to control the diffuse directional lighting intensity on your particles, so you’re stuck
    • Different light types create different values of shadows – watch out for this
  • Have any of you encountered positive use cases of AI for VFX? (this conversation immediately became philosophical first)
    • For the past several years, large companies have been betting on previously established IP to ensure profits, so smaller companies have had a difficult time producing enough content to compete – AI might be able to help with this
    • Ethics and jobs are big concerns for everyone
      • The hope is that people will be empowered to iterate faster and focus on better craft and end results
    • Indie devs will be able to do much bigger things than they could before
    • Creativity in how you use the tools will continue to be critical, regardless of the future
  • Are there any good workflows or tools using AI currently in use for VFX?
    • Substance designer gets powered up by AI
      • Realizes you want cleaner lines when warping
    • Stable Diffusion:
      • Take a texture and make it look like a reference image
      • Can it help fill in frames of a flipbook?
    • Generally cleaning up things you already do
    • A solo developer making a custom engine anecdotally described it helping him tremendously to speed up research for functions, etc.
  • Lightning Round: What new tools are you excited about?
    • Fluid Ninja
    • ActionVFX library is being converted to Unreal Engine
    • New material systems for Unreal using realistic lighting properties
    • Frostbite’s new volumetric renderers (new Deadspace uses it)
    • Alembic particles are now supported in Niagara
    • Niagara point caches
    • Using dither to avoid overdraw
    • Signed Distance Fields
    • Sparse Distance Fields
      • Allow to add comparative LOD distances to particles, which can be cheaper than using regular mesh distance fields
      • Depth Fade can be subbed out for distance fields
  • What are some tools in custom pipelines that are amazingly useful?
    • Profiling with filters to find different VFX in engine and suppress all but the one you’re looking at
    • Niagara debug tools
    • Screen recording A LOT
  • Has anyone explored Universal Scene Description (USD) file formats yet? It seems promising for collaboration.
    • There was a very good tech art talk about it this year
  • Maybe FBX is going away?

  As we wrapped up the final day of discussions, I was struck by a few conflicting thoughts. For our industry, I noticed a general theme across the entire conference that was focusing on tech artist solutions rather than engineering solutions. Before anyone throws their coffee at me, let me explain – when I began attending GDC, many of the most exciting innovations on display were paired with extremely programmer-centric presentations and discussions. This year, everything cool and exciting was paired with either 1) instructions on how to procure/install a 3rd party app that will ‘make all your dreams come true’™ or 2)guidance on how you, as a slightly technical artist, can create this solution yourself. I felt that this GDC was filled with artist empowerment.

  The conflict in this thought, however, came in the realization that we are well on our way to an industry where sub-specialization as artists needs to be embraced. We’re very likely to start seeing needs for a team dynamic include an artist that excels in ‘pre-engine’ fluid simulation pipelines, and an artist with houdini tool building expertise, and an artist that knows how to customize Fluid Ninja, another that can write Python integrations in Unreal, and a 5th that builds real-time shaders and blueprints. Each equally important, and exciting, but damn I couldn’t help but feel old and miss the ‘wild west’ days of trying to do it all without killing myself.

  Now, for the Roundtable itself, I had another pair of conflicting thoughts: On one hand, this was an incredible experience – so many talented, diverse artists and developers in one room joining for a shared conversation around our passions, and it was exciting to feel! On the other hand, I wanted more depth, or more useful takeaways out of these conversations. It’s entirely likely that this is just a limitation of the roundtable discussion format- there’s a probably valid argument that the roundtable is supposed to be paired with or spawn further, deeper conversations, but it on its own has to be broad.

  Ultimately though, GDC and our special VFX summit/round table events left me feeling so proud of the incredible community that shows up each year, or each day on our forum, discord, facebook, etc to share, inspire, and mentor. Thanks everyone for participating in these conversations with us.

  And a huge special thanks to:

  Jason Keyser, VFX Apprentice (@keyserito)
  Sandra Femenia, VFX Apprentice (@Saiel)
  Drew Skillman, (@Drew) the OG Roundtable Host who still hosts the prior notes on his website here: Visual Effects Artist Roundtables (2012-2019) — Drew Skillman

  Till next time!

  Keith