site logo

Stage 1 live stream audio visualization tool

 This is a feature introduced in Ver 0b166.
 By chance, I saw on Youtube an editing movie of a song with a circular spectrum spreading out and compared it with V's Liver's live streaming of the song frame, and I thought, 「Wouldn't it be fun if you could easily use something like this effect on a live stream?」 」 I came up with the idea and created it.
 0b167 introduced hardware acceleration and achieved overwhelming performance improvements.
 Hardware acceleration is only introduced in the 64-bit version and is available as standard in Windows 10.

 Publishing videos using this function・Distribution can be used free of charge for both commercial and non-commercial purposes as long as credit is given.
 Example credit: 「Real-time audio visualizer: thilmera7 stage1」
 Credit can be placed in the media summary section, etc.
 Please contact us for commercial use without credit.

 Images (still images) such as screenshots can be used free of charge for both commercial and non-commercial purposes, and crediting is not required.


What is stage 1?

 「Stage 1」 visualizes the audio on the PC and displays figures that respond to the content on the screen in real time.

 It is basically a derivative of the sound analyzer function of thilmera, but has the following characteristics.

・It can be used as a window capture source because it is targeted as a window capture such as OBS.
・A transparent window function called layered allows you to create beautiful alpha blends for screen captures such as OBS.
Since it is independent from the main processing of・ thilmera, you can specify dedicated settings, dedicated speed and audio source.
・In addition to the settings on the device properties screen, you can also perform quick settings on the menu that opens by right-clicking on the stage 1 screen (or task tray icon).

 Please note that audio output directly via ASIO from a DAW etc. is not supported. (Can I do it by loopbacking with the audio device...?)

 This stage 1 has not been actually tested in live streaming using Live2D etc., so it may not work due to playback/microphone and drawing delays, etc.
 I just tried developing it because it looks interesting, but I don't really know if there's a demand for it or if it's actually a feature that's already out there that I don't need, so if anyone actually wants to use it for streaming and can give me feedback, I'd be happy to do it. I will respond within the range. (Same as thilmera basic development support)
 By the way, the audio-related devices the developer has on hand are Babyface Pro FS, Sound Blaster Z, and onboard.
 I have confirmed that it is possible to react to the "input audio" by adding a microphone and looping it back on the Babyface Pro (FS) mix.

 The theoretical value of processing delay is less than 26.6ms of 10+16.6ms at 60fps of hardware drawing.


Common options

・Either output audio (desktop audio) or input audio (microphone). Or mix both.
・Adjust the volume (reaction size) for each output audio and input audio.
・Specify FPS (frames per second). Specify between 10 and 30. The software calculation for 0b166 was quite difficult even at 20fps, but from 0b167 onwards, with the introduction of hardware acceleration, it became possible to achieve overwhelming speeds.
Specifying the decay rate → Speed ​​that gradually decreases from the peak. Set the settings by feeling.
Square root → You will be able to pick up pitches that are close to prominent pitches and pitches that are not so prominent.
Auto peak → The height of the most prominent range is averaged as the maximum value, so if you turn it on, one point will stand out more easily, and if you turn it off, it will respond as a whole.
・Specify the audio visualization range in Khz. The standard is 3Khz. If it deviates too much, it will cover the higher ranges, but it will become difficult to understand the difference in the lower ranges.
FFT type → Basically, increasing the number will make the protrusion of the reaction smoother.
Backmost window → Setting to fix the window to the back to prevent it from accidentally coming to the front during streaming. (0b167)
Band High resolution → Setting to increase the spectrum resolution introduced from 0b167.
No frame wait → Fast mode without asynchronous frame wait introduced from 0b167. Hardware acceleration is enabled and only version7scan be used. 60fps (hardware dependent) or 100fps.


Bandwidth High Resolution Option

 The resolution level is determined by the number of bits, and the displayed value is b11 or b13. b11 is 11 bits, so 2048 samples. 4048 samples in b12. 8192 samples in b13.
 The appropriate value will vary depending on the hertz number of the original audio, but in general, b12 to b13 is recommended in a developer's environment.
 b14 or higher resolution is a little too high. b11 or below seems too low.
 Also, the higher this setting value is, the longer the target period of the original audio data becomes, resulting in slower instantaneous responses and slower decay, so higher values ​​are not necessarily better.
 If you specify a value that is just at the edge of the resolution level, the level may change frequently due to buffer equalization and increase/decrease due to the acquisition timing, so it is not a good idea to set it to the edge of the resolution level.



Circle 1

 This is an audio visualization stage that is generally called the audio spectrum effect.

Starting position angle → By default, specify the angle starting from the top.
Counterclockwise → Specify the order of clockwise or counterclockwise pitch of the sound.
Starting position movement animation → If it is a positive number in units of 1/1000°, it moves to the right. Negative numbers move the starting position to the left. Specifying 0 will reset it.

Circle 1 side

 This is an audio visualization stage that divides Circle 1 into left and right sides for "output audio and input audio" and displays the two separately in a half-moon shape.

Sparkle

 This is a voice visualization stage that looks like a sparkler and was created as a byproduct of the assumption that Circle 1 was being created.
 The response is based on each range, so unless the sound raises all the ranges, it won't spread out as a whole, so it's unclear whether it's useful as a decoration.

Circle 2 (0b167)

 This is an audio visualization stage that displays wavy waveform data in a circular shape.
 Since it is not a spectrum but a waveform data, the position is not determined by the pitch; the higher the pitch, the finer the pitch, and the lower the pitch, the smoother the mountain.

Straight 1 (0b167)

 This is a horizontal straight audio visualization stage, which is Spectrum's most famous shape.
 You can optionally change the centerline base and underline base.


screenshot

screenshot

Demo movie

https://www.youtube.com/watch?v=tlESvD8-sHU (0b171)
© 2001-2024 Gakuto Matsumura:弦生ささと (thilmera7gmail.com) Privacy Policy