GDC 2023 VFX Roundtable Dialogue Notes

Thanks in your endurance as we’ve compiled all of those notes, and thanks immensely to all of you unimaginable VFX artists (and extra!) that had been in a position to be part of us at GDC this previous yr. Right here’s our greatest try at capturing notes on the discussions hosted.

The VFX Roundtables at GDC every year are one thing each really particular, and barely disappointing – they’re the one largest gathering of real-time visible results expertise that I’ve ever seen, bringing with them an pleasure, power, and the shared overwhelm of an ever altering and increasing trade. On the identical time, there’s merely no approach to seize the depth and number of the conversations that I WISH might occur amongst all of us.

For GDC 2023, I had absolutely the pleasure and wonderful alternative to co-host with my good pal Jason Keyser (VFX Apprentice). Beneath are our collective finest efforts to listing notes from the discussions.

The Official GDC Itemizing of the Visible Results Roundtable:

The roundtable hosts for the week

The spherical desk discussions are damaged into three days, every with a barely totally different focus: Basic, Inventive, Technical. Every day started with a fast introduction of the subject, a reminder of the worth of our shared neighborhood and participation on realtimevfx.com, our discord, twitter, and so forth, and most significantly for the hour – a floor rule to attempt to hold every matter to about 5 minutes, and permit everybody an opportunity to contribute and be heard.

Halfway by means of the session (as soon as individuals loosen up a bit), we might cease and ask for fingers raised on three questions (I liked this, so props to Jason or whomever he had this concept from) – Elevate your hand if you happen to’re a pupil or simply getting began in your profession in VFX. Elevate your hand if you happen to’re in search of work as a person or as a enterprise. Elevate your hand if you happen to’re in search of VFX Artist assist in your mission. Now ensure you all meet one another as soon as we finish the session right this moment. Apparently, it felt just like the room was cut up roughly into thirds every day.

Basic matters and revisiting floor damaged within the VFX Summit earlier within the week.

Sadly, flight delays and an early name time mixed to stop me (Keith) from having the ability to make this present day. Luckily, Jason Keyser is an unimaginable host and Sandra Femenía kindly took notes. First bullets had been matters / questions requested to the group, sub-bullets are the solutions.

• Subject: Being extroverted whereas working from residence
  • Stream on Twitch
  • Be concerned with the web communities
• AI in our Artwork?
  • A number of individuals stated they’re utilizing AI Extensively
  • Others are utilizing it or seeing it used as reference solely, not a ultimate product
• VFX Jargon
  • Alpha clipping, and so forth having totally different meanings/makes use of from UE to Unity
  • Our particular jargon inflicting challenges with speaking suggestions from misuse or absence of particular key phrases
  • Is there a VFX Dictionary someplace?
    • We must always create a working group to doc /host one thing like this on Realtimevfx.com
  • Examples of variations in language between engines:
    • COD Engine makes use of the identify “Runner” for particles that comply with different VFX Knowledge. Bungie calls them Mum or dad-Baby particles
    • Keith’s addition whereas writing aggregating these notes: A Shader in Unity = Materials in Unreal, whereas a Materials in Unity = a Materials Occasion in Unreal
• Graph Editors?
  • Don’t let artists use graphs as a result of optimization considerations
  • Let artists do stuff on graphs after which have a TA optimize it
  • It relies upon
  • Materials Features are highly effective associates to artists
  • Shader graphs are essential
  • Bungie gives complete freedom for artists and helps them with good tooling and profiling
  • COD doesn’t have a shader editor as a result of they deal with an extremely organized manufacturing pipeline with reusability and practical VFX.
• Houdini in Pipeline
  • Frostbite Engine (as used on Battlefield), makes use of houdini A LOT – mesh particles, pyro sims, knowledge transfers, and so forth.
  • Houdini permits customers to create no matter knowledge you want.
  • Houdini is extraordinarily necessary, our artists have to comprehend it
  • We kinda began utilizing it a number of years in the past, however everybody was hesitant as a result of artists had been intimidated by the technicality of it
  • It’s so necessary to construct seamless pipelines experiences between your engine and houdini
  • It permits for procedural approaches, which may scale very properly into manufacturing
• Hiring Portfolios – what’s necessary to exhibit?
  • Comfortable abilities, like communication and important pondering are so necessary
  • I need to see curiosity
  • Inventive vs. Technical – present us your artistry first, and your technical capabilities second
    • Additionally hold optimization/efficiency in thoughts
  • Readability and narrative – we wish one thing that’s clear on display screen.
• How a lot do you utilize scratchpad in Unreal’s Niagara?
  • I exploit scratchpad in an effort to save time once I discover myself repeating the identical performance/setup time and again
  • An instance of a module I created was to entry g-buffer to erode particles based mostly on display screen shade
• Unit testing for VFX
  • Largely check by hand. In Infinity Battle, they’d unit testing with a field to run by means of all of the map doing a warmth map of framerate, however most locations don’t have an infrastructure like that
  • Some setups file GPU value over to per construct to construct graphs that you could evaluate, however the programs broke quite a bit and sometimes weren’t usable.
  • Instance was given of making dynamic programs, resembling Customized LODs relying on the quantity of barrels exploding inside a radius round one another.
• When within the pipeline do you normally begin making VFX optimization and efficiency benchmarking?
  • I counsel to do it on the finish so VFX artists can nonetheless work as a substitute of expending 3 days making an attempt to optimize issues that would not be shipped on the finish
• Lightning spherical:
  • You should utilize scratchpad to animate electrical energy in such a method that it dynamically interacts with the world round it
  • Documenting your mission’s form language in Notion, Milanote or Obsidian in your workforce to test on occasion might prevent lots of iteration/suggestions time throughout manufacturing
  • You should utilize scratchpad to make smoke that rolls outwards
  • Within the tech artist roundtable somebody talked about that they used Houdini for directional harmonics
  • For deformation programs, you may pack info within the UV channels with Houdini
  • When you compile your shaders with out having Unreal centered, the compilation will run sooner since your system received’t be rendering Unreal on the identical time

VFX by means of an Inventive Lens, specializing in attendee questions in regards to the inventive facets of VFX. This will embody artwork schooling (as college students or veterans), artwork course, and extra.

For this present day, I used to be in a position to sit again, take notes, and take part as an viewers member watching Jason work his magic.

Day 2: The Inventive Lens

• For people who use simulations, what are the most effective practices that you just’ve discovered through the years whereas artwork directing them?

  • It is determined by what software program you’re utilizing. Artwork administrators will deliver requests to us, and now we have to determine learn how to translate and iterate’
    • Non harmful pipelines are essential
    • Are you able to construct in controls when you’re initially organising, so that you don’t put your self right into a gap
    • Think about exhibiting a “grid” of choices
  • Begin very coarse/broad, with as low decision and quick of a simulation as potential – iterate fully by means of implementation within the impact in sport so you may perceive what’s working, then progressively add element (and simulation time) to your simulation.
  • Vector Subject Painter – VR App, permits the consumer to face inside the form and paint vectors round them and see the house

• What different cool instruments have you ever guys discovered?

  • Substance Designer has turn into very rewarding and helpful
  • SyncSketch – very helpful for evaluations
  • klash.studio – now we have a neighborhood sync sketch competitor right here that claims his instrument is value as effectively!
  • ToonSquid on Ipad for body by body animation
  • Toon Increase Concord
  • Krita
  • Flip a clip
  • Procreate
  • Tahoma2D/OpenToonz
  • Aseprite
  • ProtoPie
  • Goodnotes has an amazing form instrument for notes and diagrams on Ipad
  • Obsidian for reference / video boards, collaboration

• How do you identify visible language for phrases for Artwork Directed VFX / Steerage – “make it extra magical, much less fantasy is, extra stylized, and so forth”. What vocabulary will we use?

  • Lots has to do with Form language and readability – minor paint overs are lifesavers
  • Having a method information is totally essential so you may reference it as a supply of fact – set these guidelines forward of time and replace them when your improvement workforce grows and defines new issues
  • Draw overs are probably the most useful approach to translate from the spoken work to visible language, if it’s a nonetheless body. Movement is more durable
  • Movement requires individuals to additionally specify the place in time are the factors of suggestions
  • Having a superb workforce lead that works intently between the artwork director and the workforce to iterate and discover the great
  • Documentation / Notes are very useful to permit your entire time to reference collectively – viewers steered Notion, Milanote and Obsidian

• In case your assets are unfold skinny, is it higher to place extra effort into larger results and unfold skinny, or get protection?

  • It is determined by the mission and desires
  • When you’re in class, and fascinated with VFX, it is best to ensure you deal with constructing an amazing presentation in your work, relatively than spreading for protection
  • It’s value understanding why the results are wanted – generally you want the results to make moments really feel good, which may typically be achieved with no need a ton of polish
  • Think about wanting on the market for property to cowl the simpler and fewer necessary issues so you may deal with the issues that matter probably the most in your manufacturing
  • Be sure to put within the time to get the connections and gameplay logic in place – that’s typically the place an enormous portion of labor comes from, and good implementation permits you to work extra effectively.

• How do you nail timing in your impact? What about simulated results?

  • Think about the musical beat – make mouth noises and listen to the timing?
  • Timing is all the time contextual – Block within the impact so you may see it in context and perceive what’s going to work earlier than you add particulars, particularly with simulations
  • Simulations are essential to dam within the timing earlier than you add particulars, as a result of it turns into a lot more durable to change
  • Two strategies for simulation flip books that help you retime
    • Optical Movement / Movement Movement / UV Movement that blends between frames of your flip e book
    • Secure Diffusion is absolutely good at including frames to a flip e book if you happen to didn’t get sufficient frames and don’t need to return to render
  • This yr’s VFX Summit centered fairly closely on this topic, so everybody that may entry the vault ought to test them out!
  • Think about enjoying with the timing a body render of your impact – it’s a lot simpler so as to add a body or delete a body to rapidly experiment with timing

• Fast intermission: Elevate your hand if you happen to’re a pupil or simply getting began in your profession in VFX. Elevate your hand if you happen to’re in search of work as a person or as a enterprise. Elevate your hand if you happen to’re in search of VFX artists? The room was roughly third

• When have you learnt in case your impact is completed?

  • “Your impact is rarely completed, it’s taken away from you”
  • Once I begin noodling on one thing randomly, I step away, after which I can come again later and see it with contemporary eyes.
    • How do you outline noodling?
      • Once I’m not getting a lot impression from what I’m doing, it’s most likely a good suggestion to step away, shelve it till tomorrow, and so forth.
  • It’s necessary to have a superb lead or director to assist make sure that there’s a stable definition of success to really feel an achievement for carrying out the purpose and hold everybody on monitor. Maintain the power going to the place it’ll have probably the most impression.
  • FX in video games get shipped when their 80% completed or when the time runs out
  • We regularly have a lot work to do, so now we have to get intimately aware of the phrase “Ok”

• How do you overcome the sensation of not liking your work when everybody else does?

  • If everybody else is saying your work is nice – consider them! You’re so near it, you typically can’t see it.

• How do you encourage everybody or your self to do the sketching section?

  • Have a sketchbook only for the weather/VFX, so there’s a focus when in that e book to deal with VFX issues.
  • When you can’t discover the time to sketch each impact, taking the time to search out lots of reference may help fill a few of that want
  • Pinterest, if you happen to can navigate it, is usually a excellent place to dive into inspiration
  • Sketching is usually a very helpful observe for exploring and defining the timing, locking within the really feel of the impact earlier than getting misplaced within the simulation
  • You may solely save time when you nonetheless have it.
  • If fighting the “I Ought to” simply drive your self to place one thing down within the editor and drive it to be a “want” – that’s normally the kick wanted to
  • Extremely counsel Elemental Magic books

• How do you’re employed with oversaturated environments the place the lighting is blown out and all the things is vibrant?

  • Think about backing your particle with a replica that has a “grey” worth to offer a impartial stage that you could management.
  • Be sure to check your results in quite a lot of lighting environments, or at the least within the ultimate lighting context

• What in your workflows creates time and causes the largest complications?

  • Designers – ensuring your artwork course is aligned with the design imaginative and prescient to make sure that you don’t must throw away your work
  • The most important waste of time is constructing an iteration loop to really see your results – making certain that design helps create workflows when you’re working collectively is essential
  • Testing for community video games – this looks as if such a ache level – any ideas?
    • Everybody acknowledges that this can be a specific fragile state of affairs that always causes very dangerous slowdowns
    • Recording video captures to observe your impact a number of instances
    • Render Doc is a really useful gizmo to seize complete play throughs and to be able to discover bulk points
    • Ask your developer workforce to assist assist
    • Can the engineers assist setup check scenes with bots to breed your eventualities for a really quick iteration course of
    • Take a look at gyms for the win!
  • Lightning Spherical –
    • Object based mostly gradients by subtracting world place by native place and utilizing it as a UV
    • There’s a YouTube tutorial for learn how to make a holographic show for anybody “Unreal Niagara 3d Hologram Impact”
    • Exploring low cost hacks to create 3d Noises in shaders
    • Use a really vibrant ugly neon inexperienced on property that want cleanup later
    • Relaxed Cone Mapping in Niagara discuss – “Approaching Technical Artwork Strategies Unconventionally Utilizing Unreal Engine’s Niagara”
    • At all times use Eye Adaption to take note of publicity
    • WASDQ in Unreal Materials grid aligns your nodes!
      • Align prime: Shift + W
      • Align center: Alt + Shift + W
      • Align backside: Shift + S
      • Align left: Shift + A
      • Align middle: Alt + Shift + S
      • Align proper: Shift + D
      • Straighten Connection(s): Q

  VFX manufacturing considered by means of a Technical Lens, guiding the dialog by means of matters resembling graph based mostly impact authoring, impact lighting and optimization, new mesh based mostly strategies, VR/AR/XR, and extra.*

  Day 3 was my day to take level, and in the end deal with the immense duty of tossing and catching the superior GDC microphone in entrance of a crowd.

  This throwable, squishy microphone is one in every of my favourite issues from GDC.

  • What new instruments are individuals utilizing of their manufacturing pipelines for real-time VFX?
    • Blender
    • Embergen
    • FluidNinja
    • Maya is enjoying with AI compression of VDB Sequences
  • Is anybody exploring absolutely 3d/volumetric approaches in real-time but?
    • Houdini and Embergen have unbelievable workflows for VDB exports, however utilizing them within the engine continues to be difficult – a number of plugins / groups are very near having nice options, so it’s close to!
    • VR continues to be difficult/inconceivable to do real-time volumetrics, so there’s nonetheless lots of faking
      • One workforce is monitoring and offsetting the depth from every eye intelligently within the shader to create the phantasm of depth in VR house
  • Does anybody else have any insights into “rear-projection” strategies like this or different strategies for VR?
    • Just a few individuals within the room are presently engaged on VR initiatives
  • How do individuals handle expectations when administrators, managers, and so forth see offline-rendered content material that can’t work at scale in a real-time manufacturing? (For instance – seeing Unreal’s R&D posts about real-time fluids and administrators ask for it of their sport)
    • We’ve got to be sincere and make them perceive that it’s simply nonetheless not potential.
    • One particular person’s inventive director requested to be proven learn how to make VFX so he might know what’s and isn’t potential, then that turned part of their tradition – that future leaders all needed to perceive sufficient to have context about their asks.
  • Lighting and environments are wanting wonderful in sport engines, however VFX are usually not fairly there but – that’s inflicting more durable conversations when level-setting expectations
    • Must push again and handle expectations in regards to the limits we nonetheless have with VFX
    • Gen 5 continues to be not in a position to do all the things – a lot of the price range enhancements went to lighting
    • Having a graphics engineer round may help with these conversations, and discover methods to assist with the issues!
      • We might have concepts as VFX artists, however the engineer can present much more context in regards to the limitations and causes.
  • Are individuals utilizing out of the field engines, customized engines, or engine extensions?
    • Sport structure of Unity is wonderful as-is
    • Graphics developments are finest completed customized
    • What we see within the demos isn’t what you get instantly
    • It’s really easy to waste time on all of the cool issues, or making an attempt to maintain up with issues that don’t matter – you must ask what’s vital to your sport particularly and focus your power there.
    • It is determined by the workforce size- some locations can’t spare the assets to create customized options to all of their issues.
    • Customized tech additionally has an repairs value, particularly in Unity and Unreal – you’ll must spend time upkeeping each time you get a brand new engine model
    • A graphics engineer stated that of their improvement with an in-house engine, they prefer to prototype in Unity first to grasp what is required, then implement the ultimate end in their very own engine.
      • They deal with making the tooling as environment friendly as potential, then the artists are available on the finish to finalize.
  • If you see a flowery new factor as an organization, how do you determine whether or not you’ll pursue it?
    • If it’s an unknown, then take the low-hanging fruit to show an concept first
      • Are there efficiency points? Visible points?
      • If it’s third social gathering software program, asking how usable it’s, how supported it’s, and so forth
      • Give attention to the issue you’re making an attempt to unravel, not the instrument
    • One firm did month-to-month VFX challenges to get their artwork groups exploring and utilizing new instruments and strategies in a secure house, then they might make educated selections for his or her manufacturing
      • Could possibly be mixed with different disciplines as effectively to make micro sport jams
    • Instruments is usually a entice
      • Good instruments don’t change good artwork
    • Networking is an effective way to find what works for others and get new concepts
  • How do you cope with the fixed updating of software program and the seen frustrations, insecurities, and exhaustion of groups with new software program?
    • [At this point, I interrupted and asked for a show of hands – ‘We don’t talk about this enough, but since we’re all here in the room, I think it’s worth it to call it out: How many of you feel uncomfortable as artists on a regular basis?’ (or something like that) Everyone raised their hands.
    • An instructor for a neurodivergent school mentioned that he has trouble with students who like stability
      • He’s had to really lean into the idea that it’s more important to get the project done than to use the “correct” tool
    • Someone recommended the Tech Art Summit presentation from this year titled “Building a DCC and Project Agnostic Animation Pipeline” by Frieder Erdmann (https://schedule.gdconf.com/session/technical-artist-summit-building-a-dcc-and-project-agnostic-animation-pipeline/891909)
      • The takeaway – everyone has this problem.
    • Film industry has more diverse coverage of specialists within their VFX craft
      • It’s healthy to let people specialize
      • They can learn by other VFX artists next to them
    • Lots of people want to discover new things so they can do their job better
      • It’s just easy to lose focus on the part that matters – art over tools
  • What type of Profiling tools do people use?
    • Pix
    • Zone Console
    • Renderdoc
    • Niagara has great internal profiling tools
      • Does dumps and time graphics, spreadsheets, etc
    • Common things to watch out for / consider when you find frame rate hitches caused by VFX:
      • Overdraw
      • Too many live physics simulations
      • Too many VATS in memory at once (Large textures!)
      • Track down the likely suspects first and toggle them off to verify it’s them.
      • Build best practices along the way, so optimization doesn’t become such a big chore at the end.
  • Poll – How many people are authoring in Unity, Unreal, or Proprietary engines?
    • Pretty even spread throughout the audience
  • How are we solving for lighting on our particles in physically based render pipelines ?
    • Setup a series of rooms that go from the whole EV and lighting spectrum and make it easy to jump between them so you can see your effect in the full spread of lighting environments.
    • Name the particle with the use case intention (low/med/high) to control bloom
      • Work with lighting team to figure out what needs to be adjusted between each
    • Setting particular ranges of exposures for VFX vs. other aspects and authoring to fit within those ranges
    • It’s very challenging to retrofit these systems
  • Don’t forget to also consider color management – ACES is an industry standard for film and it’s starting to creep into games – someone mentioned using an ACES pipeline in Unreal with great results because there is still data in the highs and lows.
    • Someone suggested calibrating your monitors regularly
  • What issues are people having with Lumen in Unreal?
    • There’s no way to control the diffuse directional lighting intensity on your particles, so you’re stuck
    • Different light types create different values of shadows – watch out for this
  • Have any of you encountered positive use cases of AI for VFX? (this conversation immediately became philosophical first)
    • For the past several years, large companies have been betting on previously established IP to ensure profits, so smaller companies have had a difficult time producing enough content to compete – AI might be able to help with this
    • Ethics and jobs are big concerns for everyone
      • The hope is that people will be empowered to iterate faster and focus on better craft and end results
    • Indie devs will be able to do much bigger things than they could before
    • Creativity in how you use the tools will continue to be critical, regardless of the future
  • Are there any good workflows or tools using AI currently in use for VFX?
    • Substance designer gets powered up by AI
      • Realizes you want cleaner lines when warping
    • Stable Diffusion:
      • Take a texture and make it look like a reference image
      • Can it help fill in frames of a flipbook?
    • Generally cleaning up things you already do
    • A solo developer making a custom engine anecdotally described it helping him tremendously to speed up research for functions, etc.
  • Lightning Round: What new tools are you excited about?
    • Fluid Ninja
    • ActionVFX library is being converted to Unreal Engine
    • New material systems for Unreal using realistic lighting properties
    • Frostbite’s new volumetric renderers (new Deadspace uses it)
    • Alembic particles are now supported in Niagara
    • Niagara point caches
    • Using dither to avoid overdraw
    • Signed Distance Fields
    • Sparse Distance Fields
      • Allow to add comparative LOD distances to particles, which can be cheaper than using regular mesh distance fields
      • Depth Fade can be subbed out for distance fields
  • What are some tools in custom pipelines that are amazingly useful?
    • Profiling with filters to find different VFX in engine and suppress all but the one you’re looking at
    • Niagara debug tools
    • Screen recording A LOT
  • Has anyone explored Universal Scene Description (USD) file formats yet? It seems promising for collaboration.
    • There was a very good tech art talk about it this year
  • Maybe FBX is going away?

  As we wrapped up the final day of discussions, I was struck by a few conflicting thoughts. For our industry, I noticed a general theme across the entire conference that was focusing on tech artist solutions rather than engineering solutions. Before anyone throws their coffee at me, let me explain – when I began attending GDC, many of the most exciting innovations on display were paired with extremely programmer-centric presentations and discussions. This year, everything cool and exciting was paired with either 1) instructions on how to procure/install a 3rd party app that will ‘make all your dreams come true’™ or 2)guidance on how you, as a slightly technical artist, can create this solution yourself. I felt that this GDC was filled with artist empowerment.

  The conflict in this thought, however, came in the realization that we are well on our way to an industry where sub-specialization as artists needs to be embraced. We’re very likely to start seeing needs for a team dynamic include an artist that excels in ‘pre-engine’ fluid simulation pipelines, and an artist with houdini tool building expertise, and an artist that knows how to customize Fluid Ninja, another that can write Python integrations in Unreal, and a 5th that builds real-time shaders and blueprints. Each equally important, and exciting, but damn I couldn’t help but feel old and miss the ‘wild west’ days of trying to do it all without killing myself.

  Now, for the Roundtable itself, I had another pair of conflicting thoughts: On one hand, this was an incredible experience – so many talented, diverse artists and developers in one room joining for a shared conversation around our passions, and it was exciting to feel! On the other hand, I wanted more depth, or more useful takeaways out of these conversations. It’s entirely likely that this is just a limitation of the roundtable discussion format- there’s a probably valid argument that the roundtable is supposed to be paired with or spawn further, deeper conversations, but it on its own has to be broad.

  Ultimately though, GDC and our special VFX summit/round table events left me feeling so proud of the incredible community that shows up each year, or each day on our forum, discord, facebook, etc to share, inspire, and mentor. Thanks everyone for participating in these conversations with us.

  And a huge special thanks to:

  Jason Keyser, VFX Apprentice (@keyserito)
  Sandra Femenia, VFX Apprentice (@Saiel)
  Drew Skillman, (@Drew) the OG Roundtable Host who still hosts the prior notes on his website here: Visual Effects Artist Roundtables (2012-2019) — Drew Skillman

  Till next time!

  Keith