July 26, 2021•1,073 words
[en] 8 Minute read about music performance |
Spleeter x Traktor Stems
Spleeter is a Deezer research project and a source separation library that sparked new interest in audio source separation. Soon after its release a command line utility to generate Traktor Stems with it was created. Its great, but not easy to use.
NUO Stems seemed like a convenient way of transforming my collection with Spleeter into stems. After running for ~8h it had split up all tracks and reassembled them into ".stem.m4a" files with most metadata transferred that played in Traktor. The option to transfer cue points was neat. At a first glance, everything was fine. Activly using the generated tracks for a while now, ill share my experience with it.
Spleeter seems to be trained poorly on electronic music. As soon as a track has an synthetic element that spans across all frequencies, Spleeter just looses its shit. Try anything with noise or a great synth buildup and the speration will absolutly ruin it. Frequency bands are lost, reverb is sucked away, the result sounds awful. The lossy AAC encoding does the rest and kicks an already ruined track into abysmal quality deterioration.
Quick genre performance review
The cleaner the mix, the better Spleeter works and its really good with acoustic elements. Synth pads, noise, mangled samples and heavily processed vocals are usually messed up. If you play Techno, Rave, House or Drum and Bass, its impossible to have a full stem library, ~40% is crap. Clean organica tracks work well enough to blend them nicely in the afterhours. HipHop/Rap tracks and beats usually work well enough to mix them freely.
NUO Stems problems
Not all metadata is transferred: "Label" and "Catalog nr." where usually missing. If the track isn't already in Traktor but has correct metadata embedded, this data is not transferred at all.
Lossy AAC tracks sound usually WAY worse than their lossy FLAC or ALAC counterparts when they are pitchshifter or timestreched. This intersects with the quality loss due to spleeter and makes timestretching those stems an audible pain.
The m4a container supports ALAC, windows does, traktor does, and most media players do. I dont get why NUO Stems doesn't.
When moving the library, I encountered that Traktor messes up my FLAC and stem.mp4 variants, which results in the stem variant beeing flagged as broken, because traktor trys to load the previously discovered FLAC file with the stem.mp4 metadata into a stem deck. The preview player works.
The sanity check of the library does NOT catch this, so I was left fixing half of my library for HOURS and still having some tracks that wont play. After manually relocating them, so that they work again, the beatgrid is usually messed up and needs a second run of analyzing.
Cue points and beatgrids have to be independently maintained, I think a way of simply linking/syncing stem and non-stem versions would be really great.
As discussed earlier, Stems aren't the best choice for techno people without a production background. If you do make your own tunes, there is a second way: creating your own stems is easy.
Mastering for Traktor Stems
I mastered the first track of a new project in the STEM format, and it was an absolute pain. The NI Stems Creator preview will sound like you're in the middle of a war, unless your individual tracks are inaudibly quiet. Even -12db will is too loud, WTF. The export sounds fine, you just have no way of quickly previewing your dynamics settings.
Im no mastering expert with years of experience. I work with ozone and insight to earball educated guesses. That works well enough and produces good results, but transferring that to stems is impossible. The submix must be cleaner, less elements work better and more intensive EQing is a must. I'd say this forces you to generally make better mixes and masters. But: Simply taking an old track, creating four groups and duplicating the previous mastering chain four times will not sound very well.
Mashups / Edits from Spleeter
Tried it, but again: this is really hit or miss. Especially for Techno its usually a miss, because separation isn't so clean. With iZotope RX and some time the results can be greatly improved. If you have the v8 Standard, Spleeter is included as Music Rebalance. If you dont, Spleeter GUI is a great little windows app, Spleet it a little hacky (but working) app for macOS.
RX v8 Advanced includes the Spectral Recovery module, which does a decent job of quickly fixing the higher frequencies of not-so-clean spleeter seperations.
If you simply want to overlay some edits and rebalance the levels on the fly, Ozones music rebalance does a clean job without the need for bouncing/rendering.
Traktor Stems are a really nice feature, but the currently available tooling is lacking. I'd love to see Native Instruments take the initiative to embrace the use, since multitrack/stem DJing seems to be more popular again (take a look at recent updates in Virtual DJ and Algoriddim djay for mac). In my opinion pre seperated stems are better than live generated ones, because they're easier on the CPU in live situations and very reliable. But again, they need proper official support and good tooling to take off.
Spleeter, the technology responsible for the new rise of stems, is neat. But without a good set of tools and some time, the output is very rough and not great for bootlegs. Id argue that with a good ear and decent knowledge of sound design, you can rebuild things pretty fast while having fun. Spleeter separation and proper cleanup will take some time and it's a mundane and tedious work.
Traktor Stems with Spleeter source separation are fun and I'll keep using them for sure. But they are FAR from what I expected, and sadly not in a good way.