{"id":39,"date":"2025-11-23T10:38:35","date_gmt":"2025-11-23T10:38:35","guid":{"rendered":"https:\/\/lwlkc.xyz\/?p=39"},"modified":"2025-11-23T10:38:38","modified_gmt":"2025-11-23T10:38:38","slug":"audio-reactive-real-time-image-generation-with-touchdesigner-or-exploring-liminal-space","status":"publish","type":"post","link":"https:\/\/lwlkc.xyz\/?p=39","title":{"rendered":"Audio Reactive Real-Time Image Generation With TouchDesigner or Exploring Liminal Space"},"content":{"rendered":"\n<p>I&#8217;m sure it started with YouTube. I&#8217;d been interested in playing around with visuals to accompany music, in particular live jamming, for awhile a tutorial somewhere must have grabbed my attention and set me on the path of experimenting with audio reactive real-time image generation.<br><br>I had seen the tutorial below, which is pretty cool,<br><\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"iPad drawing into StreamDiffusion - Touchdesigner Tutorial\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/sNKisEli3o0?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>but ended up starting to make stuff with <a href=\"https:\/\/derivative.ca\/\" target=\"_blank\" rel=\"noreferrer noopener\">TouchDesigner<\/a> and <a href=\"https:\/\/dotsimulate.com\/tools\" data-type=\"link\" data-id=\"https:\/\/dotsimulate.com\/tools\" target=\"_blank\" rel=\"noreferrer noopener\">StreamDiffusionTD<\/a> by following this one from Scott Mann a little while later.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"TouchDesigner &amp; StreamDiffusion: Real-Time Audio-Reactive AI Art - TouchDesigner Tutorial 216\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/M2U1hvTqzU0?list=PL3vPUwUEY6Y9BSIhqWVWkyzt4JrhUuVH2\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>Some time later, I ended up making this, and it was freaking fun!<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"20250828 streamdiffusion audio reactive   butterflies next loop\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/tdMQaaXvSfY?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>Since that first video I&#8217;ve done a bunch <a href=\"https:\/\/youtube.com\/playlist?list=PLaM29qbpKoLNhmUzEqZ-ldIR_teyndwjp&amp;si=eo8TUfKErOrMgWt6\" data-type=\"link\" data-id=\"https:\/\/youtube.com\/playlist?list=PLaM29qbpKoLNhmUzEqZ-ldIR_teyndwjp&amp;si=eo8TUfKErOrMgWt6\" target=\"_blank\" rel=\"noreferrer noopener\">more<\/a> and started using a hosted GPU backend via the <a href=\"https:\/\/daydream.live\/\" data-type=\"link\" data-id=\"https:\/\/daydream.live\/\" target=\"_blank\" rel=\"noreferrer noopener\">Daydream<\/a> API. You can play around with Daydream in a browser <a href=\"https:\/\/playground.daydream.live\/?inputMode=3d&amp;denoise=11&amp;prompt=blooming+flowers&amp;seed=42&amp;guidance_scale=1&amp;negative_prompt=blurry%2C+low+quality%2C+flat%2C+2d&amp;normalize_prompt_weights=1&amp;3d_shape=torus-knot\" data-type=\"link\" data-id=\"https:\/\/playground.daydream.live\/?inputMode=3d&amp;denoise=11&amp;prompt=blooming+flowers&amp;seed=42&amp;guidance_scale=1&amp;negative_prompt=blurry%2C+low+quality%2C+flat%2C+2d&amp;normalize_prompt_weights=1&amp;3d_shape=torus-knot\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a> and it&#8217;s pretty wild. Highly recommend going and having a play.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>Some of my early experiments were a bit frenetic, with rapidly changing images, but I still find them mesmerizing to watch, like zooming into a fractal. For whatever reason, this one has gotten the most views. <\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"20250906 drift lwlkc+friends - Techno Circus ea_lwl-max\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/Jwvd7kPRoKI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>It&#8217;s still using the same structure, a spectrogram mirror and doubled, to drive StreamDiffusion.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"644\" height=\"524\" src=\"https:\/\/lwlkc.xyz\/wp-content\/uploads\/2025\/11\/Screenshot-2025-11-23-at-8.51.51-PM.png\" alt=\"\" class=\"wp-image-40\" srcset=\"https:\/\/lwlkc.xyz\/wp-content\/uploads\/2025\/11\/Screenshot-2025-11-23-at-8.51.51-PM.png 644w, https:\/\/lwlkc.xyz\/wp-content\/uploads\/2025\/11\/Screenshot-2025-11-23-at-8.51.51-PM-300x244.png 300w\" sizes=\"auto, (max-width: 644px) 100vw, 644px\" \/><\/figure>\n\n\n\n<p>I made some variations on how the spectrogram was used and ended up with things like this where things are looking a little more geometric.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"20251018 raising lungta lwlkc+friends hop   Techno Circus ea lwl max  alt01\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/dR_6naHZJVk?start=188&#038;feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>I quite like this one, where things dipped into a hand drawn feel at times<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"20250220 lwlkc fire brigade hop ea lwl max\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/F9544XsC-Ws?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>and this one, as it&#8217;s a slower and really has that quality of earlier image models that I find interesting, where there is a lot of &#8216;not quite&#8217; going on, a &#8216;almost-becoming&#8217;.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"20251116 intonation phase lwlkc hop ea lwl max\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/2J2jSkSFIb8?list=PLaM29qbpKoLNhmUzEqZ-ldIR_teyndwjp\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>There&#8217;s a new version of StreamDiffusionTD out but it&#8217;s crashing on my mac but it should be significantly higher quality once it&#8217;s working. Quite excited for that.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;m sure it started with YouTube. I&#8217;d been interested in playing around with visuals to accompany music, in particular live jamming, for awhile a tutorial somewhere must have grabbed my attention and set me on the path of experimenting with audio reactive real-time image generation. I had seen the tutorial below, which is pretty cool, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-39","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/posts\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=39"}],"version-history":[{"count":2,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/posts\/39\/revisions"}],"predecessor-version":[{"id":42,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=\/wp\/v2\/posts\/39\/revisions\/42"}],"wp:attachment":[{"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lwlkc.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}